Twice
Source: http://github.com/kvogt/twice
I wrote this program because Ruby on Rails is hard to scale. Or at least it was. I don’t know if this is still the case. Regardless, Twice does some really cool things and does many of them faster than Rails, so it’s worth checking out.
When my cofounder Emmett Shear and I sat down to brainstorm ways to make our website faster, we tried to imagine what kind of system could serve pages in just a few milliseconds. The only think we could come up with to hit that kind of performance involved rendering as much of a page as possible directly out of memory – or even the whole page. We ended up designing a system that caches about 99% of a fully rendered page and renders the other 1% on the fly. The end result is that about 95% of Justin.tv’s half a billion monthly dynamic requests are served in under 10ms.
In-memory caching is certainly not a new concept, and Rails actually has some pretty good “partial” caching as well as full page caching. But Justin.tv’s load profile didn’t mesh well with these caching strategies, so we had to try something else. Since our visitors all show up at the exact same moment and create traffic spikes 10x higher than normal, there is no time to spend rendering complex templates or running database queries. Yet somehow we still need to customize every page for logged in users and update pages to reflect changes as the content expires. Twice was written to protect our application servers during these load spikes but also maintain a customized feel to each user .
Twice is a proxy server that sits between a hardened webserver like nginx or Apache and one or more application servers. First, the webserver proxies incoming requests to Twice. Twice inspects the HTTP headers and checks to see if the page has been rendered and is stored in memory. If the page exists, Twice copies the page out of memory and customizes the rendered page for the user who requested it. If the page isn’t found in memory, Twice proxies the request on to the application servers and caches the result if possible.
Here are some of the more interesting features of Twice:
- Cache backends supported: memcached (via pylibmc), memcached (via python), python dictionary
- Fetch a user session from a database, cache it, and render user attributes into a page
- Fetch memcache keys from a “backend” memcache and cache them in the Twice cache
- Request pileup prevention by using soft expiration
- Scales horizontally to any number of cache processes and memcached servers
- Explicit expiration across all cache processes without any IPC overhead
- OOM killer and file descriptor limit discovery
- Geoip, remote ip, memcache key, etc. can be rendered into a page or header
- Short-cache certain response codes
- Hash pages based on browser language header
Twice is open sourced under the MIT license.
Leave a comment