...So you aren’t caching the API data? Hanging my head, I had to answer with a No. But I decided to look into that option, as it would remove the direct coupling of my site and the APIs, speeding things up a bit in the process. While you may or may not have noticed, the main page has been an arbitrary hit-or-miss the last couple of weeks. Sometimes it’s up, taking it’s normal 4-5 seconds to load, and other times (when the delicious response data is randomly empty) it would either hang or instantly die (hence, the direct coupling).
Yesterday after church, I finally sat down to work out the kinks. I was fed up with the slow loading and down time. Using the suggestions Chris proposed, I was able to get things going. I’ve never used the Marshal library before, so I didn’t know what to expect; but it turned out to be extremely straightforward. I also wanted to use pure Ruby (no Rails) in my script, so I had to tweak it a little.
I exported the inner-workings of the original methods that grabbed the flickr and delicious data, and put that processing in a Ruby script located in my script folder. As expected, a few things were no longer accessible, including the to_date method. I just used the Ruby libraries directly (such as Date.parse in place of to_date to convert the ISO 8601 format). And all was well.
So the script processes the API data and Marshal.dump’s it to the file system. Those previous methods now simply look for the cached file and Marshal.load it back in, rather than dealing with the API responses directly. Not only do I like this solution a lot more, I’ve cut the load time of the main page from 4-5 seconds down to < 1 second (checked against one of those web page analyzers).
So the core of all of this is working; however, I do still have a problem: automating the script to run on a 30 minute schedule. I have a cron job for this, but apparently it isn’t working. Here’s the cron:
*/30 * * * * /usr/bin/ruby ~/rpheath.com/site/script/api_cache.rb >/dev/null 2>&1
The job is listed there when I check with crontab -l, but I don’t think it’s doing anything. Once I figure that out, I’ll be good to go. Oh, and thanks to Chris for the idea.
01
Chris on Mon Feb 26 at 02:53PM
Despite how vain it sounds, I guessed that I was the “wise man” when I saw the title in my RSS reader :-p I find it odd that the cron isn’t working… have you tried the script directly (exactly as you have it above) ? Maybe it’s a permissions issue (or you may need to specify the full path rather than using ~).
02
Lee on Mon Feb 26 at 05:47PM
It is working great. Nice job.
03
Ryan on Mon Feb 26 at 05:59PM
@Chris -
I can execute the script directly by typing:
I took off the
>/dev/null 2>&1and set up the email variable, and never got any emails with errors or anything, so apparently it’s just not running. Do you know of a way to easily test cron jobs? Is there a simple “test” job I can put in mycrontabfile that would let me know if the jobs are executing?Just in case this matters, I have my jobs in
cron_jobs.txtand anytime I add a new one or change a current one, I runcrontab cron_jobs.txt. I can then list them out, and they show up—just seems like they’re not running.