Thursday, October 25, 2007

acts_as_solr, Capistrano, and testing


I've recently deployed acts_as_solr and the Apache Solr server for full text searching in a new app I am building. The plugin works great, is easy to setup, but can cause some havoc with your deployment strategy if you don't pay attention.

For one thing, out of the box the plugin stores volatile information (tmp files, the indices used for searching, and log files) within the plugin itself, because it comes with its own version of Solr pre-installed. That makes it easy to get started and play with Solr but is not acceptable for a production environment (because every time you redeploy your code you'll be destroying and re-creating those directories within the plugin).

The Java Underworld

Unfortunately this means you'll have to descend from the mighty, glistening tower of Ruby into the Java underworld to setup a Tomcat servlet running Solr. This tutorial does a great job of explaining the process which is pretty easy.

Starting and Stopping Solr

I also have added some Capistrano tasks to start and stop the Solr server during deployments to lessen the chance that the index will be corrupted:
namespace :solr do

task :start, :roles => :app do
run "cd #{latest_release} && #{rake} solr:start RAILS_ENV=production 2>/dev/null"

task :stop, :roles => :app do
run "cd #{latest_release} && #{rake} solr:stop RAILS_ENV=production 2>/dev/null"

task :restart, :roles => :app do

For Capistrano to be able to run the above tasks, I also had to patch the start:solr rake task that comes with acts_as_solr to use backticks instead of exec:

`java{ENV['RAILS_ENV']} -Djetty.port=#{SOLR_PORT} -jar start.jar`


acts_as_solr will greatly slow down any tests that use fixtures (because the indices get updated every time the fixture data gets added or removed). Ideally you would mock the Solr server in unit and functional testing (unless you're directly testing your app's interaction with Solr), but I haven't found a way to mock Solr before your tests run, when fixtures are being added. One way to go might be to create something like mailtrap for Solr requests. The other thing that might help is installing a modification of Solr called background-solr which defers all the index save and destroy requests and performs them in batches -- the idea being that during testing, maybe you would just avoid calling that batch update method for most of your tests, except where you are testing interaction with Solr (as in an integration test).

Building the Index

This is really great rake task to help build the Solr index. It's much faster and more effective that doing it manually for each of your models.


Dr Nic said...

Did you have any further thoughts on mocking/testing acts_as_solr?

Mike said...

Unfortunately, no. I've just accepted that my integration tests are going to be slower (that's the one place where I'm still using fixtures for this particular project).

One idea would be to mock out or otherwise intercept the after_save and after_delete hooks that acts_as_solr mixes into the model, but that didn't seem to work for me (can't recall what the hangup was).

Adam said...

If you want to disable Solr in tests simply put this in your test_helper.rb

class ActsAsSolr::Post

def self.execute(request)




Luke Francl said...

I've started a fork or acts_as_solr that moves the indexes, pid files, and logs to more Rails-like locations. This is configurable, so you can put them where you want, or restore the old behavior.

Thanks for the tip about turning off Solr for tests, Adam!