Ruby on Rails

Dear All,

Fairly new to rails and Heroku, so could be doing something wacky - do
let me know if you think my code practice is off, even if unrelated to
this error, I'd like to learn!

I'm using Rails 3.0.0, ruby 1.8.7, and 'sqlite3-ruby', '1.2.5', :require
=> 'sqlite3'.

I've got an application that goes off and scrapes websites for event
information, to do this I'm using:

nokogiri - for scraping
date - to help with parsing
chronic - to help recognise dates
open-uri - to open the url

all of these are just gem 'gem-name', so latest unspecified versions.

The code that does this scraping is in the ApplicationController class -
a bad place for it?
I've just got it working as a background process using delayed_job, I
have a class called EventSearcher with the required 'perform' method.

My error occurs when I upload to Heroku, if I make changes to the app,
but NOT to ApplicationController,
do $git add . / git commit .... / git push heroku master / heroku
workers 1 / heroku open
then the process runs, appears successful in the logs, but returns no
events

The error can be solved I've discovered by making small changes to
Application controller, such as adding the line
logger.debug"Changing AppController again"
then doing $git add . / git commit .... / git push heroku master /
heroku workers 1 / heroku open
this time the process runs, appears in logs to take longer, and returns
the scraped info as required.

Any ideas why this should be??

I have tried solving this myself by putting in 'logger.debug" commands,
but these don't show on my heroku logs, any idea how to get them to show
up?

I've put the code below, for ApplicationController, EventSearcher, and
part of SuggestedEvents it's a bit of a mess I know, but runs fine on
local machine.

-------------- Application Controller

class ApplicationController < ActionController::Base

require 'date'
require 'nokogiri'
require 'open-uri'
require 'chronic'

protect_from_forgery
include SessionsHelper

def authenticate
deny_access unless signed_in?
end

def find_all_events

@event_searches = EventSearch.all

@event_searches.each do |es|

do_search(es)

end
end

def find_one_band_events(event_id)

es = EventSearch.where(:id => event_id)[0]
do_search(es)
end

def do_search(event_search)

logger.debug"Changing AppController again"
logger.debug"in do search"
url = event_search.urlOne

if validate(url)

logger.debug"Passed Validate URL"

doc = Nokogiri::HTML(open(url))

#search the doc with the saved css search strings
if !(event_search.eventDateCSS.empty?)
startDate = doc.css(event_search.eventDateCSS) else eventDate = 0
end
if !(event_search.eventNameCSS.empty?)
eventName = doc.css(event_search.eventNameCSS) else eventName = 0
end
if !(event_search.eventLocationCSS.empty?)
location = doc.css(event_search.eventLocationCSS) else location = 0
end
if !(event_search.eventTimeCSS.empty?)
time = doc.css(event_search.eventTimeCSS) else time = 0 end
if !(event_search.priceCSS.empty?)
price = doc.css(event_search.priceCSS) else price = 0 end
if !(event_search.descriptionCSS.empty?)
description = doc.css(event_search.descriptionCSS) else description
= 0 end
i=0

if eventDate != 0
while i < startDate.length # find all events on that page going
through each case of start date

SuggestedEvent.new do |@savedEvent|

if eventDate != 0
@savedEvent.eventDate = date_from_string(startDate[i].content)
logger.debug(startDate[i].content)
end

test = SuggestedEvent.where(:bandName => event_search.bandName,
:eventDate => @savedEvent.eventDate)

if !test[0]

@savedEvent.bandName = event_search.bandName
if eventName != 0
@savedEvent.eventName = remove_whitespace(eventName[i].content)
end
if time != 0
@savedEvent.doorsOpen = Chronic.parse(time[i].content) end
if price != 0
@savedEvent.price = price[i].content end
if description !=0
@savedEvent.description =
remove_whitespace(description[i].content) end

if location != 0
savedVenueName = paramMatcher('Venue','venueName',
remove_whitespace(location[i].content))
if savedVenueName
@savedEvent.venue = savedVenueName[0]
@savedEvent.city_location = savedVenueName[1]
@savedEvent.longitude = savedVenueName[2]
@savedEvent.latitude = savedVenueName[3]
else
@newVenue = Venue.new
@newVenue.venueName = remove_whitespace(location[i].content)
@newVenue.old_venue_name = @newVenue.venueName
@newVenue.save
@savedEvent.venue = remove_whitespace(location[i].content)
end #of if
else
@savedEvent.venue = 'No saved venue found'
end #of location !=
@savedEvent.save
end # of if !test
end # of .new do
i += 1
end # of while

end #of if eventDate !=0

end # of if url passes

end # of def

# get the date from whatever string gets thrown
def date_from_string(date)

# should remove st rd nd and th

firstParse = Chronic.parse(date)
r1 = /[a-zA-Z]/

#daY Less than 12, assume chronic wrong for Britain, as long as also no
characters such as December,
#where it would be right
if firstParse.day <= 12 and !r1.match(date)

#swap month with day
firstParse = firstParse.change(:day => firstParse.month, :month =>
firstParse.day)

end #of if

return firstParse

end #of def

# remove whitespace and other characters such as newline

def remove_whitespace(dirty_name)

return dirty_name.split(' ').join(" ")

end

#find names in strings from the net
def paramMatcher(modelString, param, cssScrapedString)

case modelString
when 'Venue'
venues = Venue.all
i=0
venues.each do |v|

if v.venueName.scan(cssScrapedString)[0] or
cssScrapedString.scan(v.venueName)[0]

return [v.venueName, v.city_location, v.longitude, v.latitude]
end

end # of .each
end # of case

return nil
end # of def param Matcher

def validate(url)
begin
uri = URI.parse(url)
if uri.class != URI::HTTP
return false
end
rescue URI::InvalidURIError
return false
end
return true
end

end # of class

----------------- Event Searcher

class EventSearcher < ApplicationController

attr_accessor :all_events
attr_accessor :one_event_id

def perform

logger.debug"in perform"

if all_events == 1 # searches all band names

logger.debug"in perform, all_events == 1"

find_all_events

end

if one_event_id != nil #searches just one band name using id

logger.debug"in perform, one_event != nil"
logger.debug(one_event_id)
find_one_band_events(one_event_id)

end

end # Of Perform

end #of class

-------------- Suggested Events

class SuggestedEventsController < ApplicationController

#.... other functions removed for brevity....

def search_all_events

logger.debug"Sugg E search all events"
#the instance of the object that gets put on the queue
@event_searcher = EventSearcher.new
@event_searcher.one_event_id = nil
@event_searcher.all_events = 1
Delayed::Job.enqueue @event_searcher

flash[:notice] = "Finding All events, this takes a while, refresh
after 10 mins"
redirect_to suggested_events_path

end

end

--
Posted via http://www.ruby-forum.com/.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.