Monday, April 5, 2010

How Twitter Integrates New Architecture

I've been experimenting with the Cassandra datastore lately and came across an interesting interview with Ryan King discussing Twitter's use of Cassandra.  I thought this tidbit was worth sharing even if you don't care about Cassandra:
A philosophical note here — our process for rolling out new major infrastructure can be summed up as “integrate first, then iterate”. We try to get new systems integrated into the application code base as early in their development as possible (but likely only activated for a small number of people). This allows us to iterate on many fronts in parallel: design, engineering, operations, etc.
He also describes in detail how they used that philosophy to migrate Twitter status updates (the toughest part of their app to scale) to Cassandra.  This also reminded me of Flickr's Feature Flags ... all of which can be used to support continuous deployment, something I've become very interested in after encountering it in the lean startup literature.

I'm planning to writeup my findings on using Cassandra with Ruby sometime soon if there's interest.