ruby - Using delayed_job for Mechanize script in Rails (NoMethodError: undefined method for nil:NilClass) -
i’m going preface saying i’m relative beginner, , while may not understand lot yet, more willing learn. hope can one, have been trying figure out long time , still stuck. know long 1 - wouldn't throw out here unless had put in many days of trying figure out myself first.
i using mechanize run script in rails app fill out several webforms. worked locally. however, need use delayed_job in order run time-intensive script in background. now, developing locally (it on heroku) until problem fixed.
so installed delayed_job_active_record gem, , have been trying finagle working. know although form takes in of parameters successfully, nothing getting inserted sql table. it’s not picking on object’s id. it's not recognizing object's class (it sees nilclass), considers regular ruby methods (such .strip) unknown. (the console's output showing of toward bottom).
i’m doing wrong. painfully obvious - don't have skills yet figure out. i’d forever indebted if can me figure out i’m doing wrong here.
first, here code:
model:
require 'digest' class jumper < activerecord::base after_save |runscript| runscript.delay.scrape end validates :myuserid, :presence => true validates :mypass, :presence => true validates :mydate, :presence => true, :numericality => true validates :mymonth, :presence => true, :numericality => true validates :myyear, :presence => true, :numericality => true validates :mylist, :presence => true attr_accessor :myuserid, :mypass, :mydate, :mymonth, :myyear, :mylist def scrape agent = mechanize.new page = agent.get('http://mywebsite.org/') myform = page.form_with(:name => 'signinform') myuserid_field = myform.field_with(:name => "email") myuserid_field.value = myuserid mypass_field = myform.field_with(:name => "password") mypass_field.value = mypass page = agent.submit(myform, myform.buttons.first) mylistarray = mylist.strip.split(/[\s]+/) mylistfinal = mylistarray.map{|l| l[0..-5].sub(/(.*)\./,'\1').gsub('.','/')}.uniq mylistfinal.each |doi| url ='http://mywebsite=' + doi + '&role=some_info#some_dates' page = agent.get("http://mywebsite.org/submit") page = agent.get("#{url}") entryform = page.form_with(:name => 'submit') entryform.field_with(:name => 'month').options[("#{mymonth}").to_i].select entryform.field_with(:name => 'day').options[("#{mydate}").to_i].select entryform.field_with(:name => 'year').options[("#{myyear}").to_i].select page = agent.submit(entryform, entryform.button_with(:name => 'continue')) end end end`
controller:
def index @doilists = doilist.all respond_to |format| format.html # index.html.erb format.json { render json: @doilists } end end def show @doilist = doilist.find(params[:id]) respond_to |format| format.html # show.html.erb format.json { render json: @doilist } end end def new @doilist = doilist.new respond_to |format| format.html # new.html.erb format.json { render json: @doilist } end end def create @jumper = jumper.new(params[:jumper]) if @jumper.save flash[:notice] = "sucessfully submitted information." redirect_to @jumper else render :action => 'new' end end`
gemfile:
source 'https://rubygems.org' gem 'rails' group :development gem 'sqlite3', '1.3.5' end group :assets gem 'sass-rails', '~> 3.2.5' gem 'coffee-rails', '~> 3.2.2' gem 'uglifier', '>= 1.2.3' end gem 'jquery-rails', '2.0.2' gem 'mechanize' group :production gem 'pg', '0.12.2' end gem 'delayed_job_active_record' gem 'daemons' gem 'thin'`
procfile:
web: bundle exec rails server thin -p $port -e $rack_env worker: bundle exec rake jobs:work`
script/delayed_job:
require file.expand_path(file.join(file.dirname(__file__), '..', 'config', 'environment')) require 'delayed/command' delayed::command.new(argv).daemonize rails_env=production script/delayed_job start --exit-on-complete`
here output when run foreman in terminal:
11:33:30 web.1 | started post "/myjumper" 127.0.0.1 @ 2013-05-02 11:33:29 -0400 11:33:30 web.1 | delayed::backend::activerecord::job load (1.2ms) select "delayed_jobs".* "delayed_jobs" ((run_at <= '2013-05-02 15:33:30.437882' , (locked_at null or locked_at < '2013-05-02 11:33:30.438065') or locked_by = 'delayed_job host:myroot pid:7081') , failed_at null) order priority asc, run_at asc limit 5 1:33:30 web.1 | processing jumperscontroller#create html 11:33:30 web.1 | parameters: {"utf8"=>"✓", "jumper"=>{"myuserid"=>"email@gmail.com", "mypass"=>"mypassword123”, "mylist"=>"listitem", "mymonth"=>"4", "mydate"=>"30", "myyear"=>"1"}, "commit"=>"submit entry"} 11:33:30 web.1 | (0.1ms) begin transaction 11:33:30 web.1 | sql (21.4ms) insert "jumpers" ("created_at", "mydate", "mylist", "mymonth", "mypass", "myuserid", "myyear", "updated_at") values (?, ?, ?, ?, ?, ?, ?, ?) [["created_at", thu, 02 may 2013 15:33:30 utc +00:00], ["mydate", nil], ["mylist", nil], ["mymonth", nil], ["mypass", nil], ["myuserid", nil], ["myyear", nil], ["updated_at", thu, 02 may 2013 15:33:30 utc +00:00]] 11:33:30 web.1 | sql (0.5ms) insert "delayed_jobs" ("attempts", "created_at", "failed_at", "handler", "last_error", "locked_at", "locked_by", "priority", "queue", "run_at", "updated_at") values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [["attempts", 0], ["created_at", thu, 02 may 2013 15:33:30 utc +00:00], ["failed_at", nil], ["handler", "--- !ruby/object:delayed::performablemethod\nobject: !ruby/activerecord:jumper\n attributes:\n id: 68\n mylist: \n created_at: 2013-05-02 15:33:30.920857000 z\n updated_at: 2013-05-02 15:33:30.920857000 z\n myuserid: \n mypass: \n mymonth: \n mydate: \n myyear: \nmethod_name: :scrape\nargs:\n- :jumper\n"], ["last_error", nil], ["locked_at", nil], ["locked_by", nil], ["priority", 0], ["queue", nil], ["run_at", thu, 02 may 2013 15:33:30 utc +00:00], ["updated_at", thu, 02 may 2013 15:33:30 utc +00:00]] 11:33:30 web.1 | (10.7ms) commit transaction 11:33:30 web.1 | redirected `http://mylocalhost:5000/myjumper/68` 11:33:30 web.1 | completed 302 found in 74ms (activerecord: 33.1ms) 11:33:31 web.1 | started "/show/68" 127.0.0.1 @ 2013-05-02 11:33:30 -0400 11:33:31 web.1 | processing jumperscontroller#show html 11:33:31 web.1 | parameters: {"id"=>"68"} 11:33:31 web.1 | jumper load (0.3ms) select "jumpers".* "jumpers" "jumpers"."id" = ? limit 1 [["id", "68"]] 11:33:34 worker.1 | [worker(host:myroot pid:10470)] jumper#scrape failed nomethoderror: undefined method `strip' nil:nilclass - 0 failed attempts 11:33:34 worker.1 | [worker(host:myroot pid:10470)] 1 jobs processed @ 0.8520 j/s, 1 failed ...
it looks you've got undefined's. example, where's mymonth
?
i want give tip simplifying code while i'm here:
instead of:
myuserid_field = myform.field_with(:name => "mstrservlet.email") myuserid_field.value = myuserid
try just:
myform['mstrservlet.email'] = myuserid
that's not solve problem, maybe closer simple enough solve.
and finally, know heroku current media darling, should consider might not best fit particular project.
Comments
Post a Comment