RailsConf 2008 – Saturday Morning Summary

May 31 2008

Jeremy Kemper Keynote

Rails 2

Jeremy’s keynote was billed as the technical talk that DHH didn’t give. He started by saying that Rails 2 is all about working with resources. Rails2 introduced a lot of features involving a unified way of referencing an object, throughout the DOM, in the code itself, etc. He also mentioned that Rails2 dropped a lot of ‘fat’, pulled deprecated features, improved performance, etc and talked about making choices about what needed to be kept and what could be shed.

Then he moved to Rails 2.1, which has been in development over the last 6 months with 1400 contributors, 1600 patches, & 9000 comments. The core team wasn’t able to keep up, so they moved to Git, Github, & Lighthouse. Git provides the excellent ability to fork your own version of Rails and offers many ‘gateways’ to committing code, so that the core team members are no longer the sole ‘gatekeepers’.

Then Jeremy referenced the move to Lighthouse. He only mentioned it very shortly, which I think is unusual. Whereas the move to github was a natural, obvious, and powerful evolution of managing the github core, I think it still stands to be seen how things will work out with Lighthouse. Lighthouse is fairly new and has a limited featureset compared to something like Trac. I haven’t really heard any personal experiences about if Lighthouse is truly better for the core team or not, so if you have any insights, I’d love to hear them.

There are many new features of Rails 2.1:

  • Merging migrations. The new UTC based migrations and out-of-order migrations are one of the biggest new features in Rails 2.1 This is definitely going to be one of the biggest benefits to developers and is an exciting new feature.
  • But it’s not the MOST exciting new feature. That honor goes to the topic Jeremy discussed next: Time Zone support. Rails 2.1 now supports setting a timezone at the start of a request and all the times you see throughout the request will be given in that timezone. This is something I hope will eventually be moved to Ruby core because it is so mind-blowing in terms of how easy it makes handling time zones.
  • Jeremy also very quickly previewed the new change_table syntax. Then he moved on to a live demo (Where else will you find a conference with 2,000 attendees and the keynote speaker writes live code?). He demonstrated all these new features on a blank app with very few hiccups. Pretty cool.
  • Some news to me was that memcache-client is now bundled with Rails 2.1 for handling caching. This is an awesome improvement that will allow you to easily use memcache (including expiring cache) globally throughout Rails very easily.
  • Dirty checking & partial updates – You can know what DB columns have changed and only changed columns will be sent to the DB with an UPDATE. Jeremy noted that there is a nice side effect of cleaner logs. He also noted that there is a danger with partial updates where validations may involve the entire object, but if you only update part of the object, then concurrent updates may cause an invalid record. He recommended using optimistic locking for any cases like that.
  • Smart :inclues. This is about avoiding joins when two seperate lookups are faster (because with a join, you’ll likely return many duplicates: e.g. If you have 1000 messages that belong to only 3 users, all 1000 message records will return all the user data for each of those 1000 rows. So basically 2000 records get returned. Seperate the queries instead of joining and you get 1003 records).

Jeremy demonstrated Rails running on Rubinius, jRuby, ruby-1.9; which got some applause. He mentioned some off-the-cuff performance data of 20-60% increases with Rails using 1.9: not the ‘golden cow’ to worship, but decent none the less. He also said that mongrel, MySQL, Postregres, etc are all supporting or very very close to supporting 1.9.

Overall, a great new set of features that we’ve been reading about quite a bit recently. Of course, it was no surprise when Jeremy announced (to big applause) that Rails 2.1 is released as of today.

RubyConf2008

Rich Kilmer

Rich got up to note that RubyConf2008 has announced their venue: The Orlando, FL Omni Hotel & Resort. RubyConf2007 had some problems where a lot of us wanted to hang out at night, play werewolf, code, etc to the point where I actually wound up renting a conference room at a hotel down the block so we could be there into the night (thanks to all those who threw in a few dollars to cover the cost).

He made it clear that the priority was to have a space that we could basically use 24 hours a day. The resort has FOUR restaurants on the property, DS3 line on premise with 5mbps wireless everywhere, a pool that will be open all night for us, easy access to Disneyworld, and a firepit with chairs in a circle for what I believe will be some awesome werewolf. November 6-7-8. I’ll see you there!!!!

ESI

Aaron Batalion

Aaron’s talk is another presentation dealing with scaling Rails for large numbers of users. But he addresses a much different area. Instead of scaling hardware, the database, app servers, etc., he is talking about about what you can do inside your Rails app to optimize your views and handle more traffic on the same hardware you already have. I consider this as being pretty important because this should be one of your first steps in scaling your site before you start replicating and sharding your database.

Aaron starts by addressing a real issue with many web apps: Most pages are very personalized for each user. If you are using restful_authentication, then you are basically guaranteed to require custom content for each user. He proposes using Edge Side Includes (ESI) in order to avoid using your mongrels all together (as much as possible).

For those who aren’t familiar with ESI: It is an a set of XML tags that allow you to build a complex HTML by breaking it down into several smaller pieces, some of which can be cached long term (general content) and others that are more specialized to an individual request. For Rails developers: think cached render :partial that is assembled into a whole document by the ESI server instead of by Rails.

Aaron showed a great example of a page with some general content and a sidebar with some custom content for each user that demonstrated how the fragment caching works.

So why should we care? Rails already has a several methods of fragment caching (typically with memcache) that provides these services.

Well, ESI has a number of extra benefits:

  • Personalized Full Page Caching – Often times, the entire page can be cached for a given user
  • Slow/Broken Dependencies – By breaking up a page into several pieces that are assembled outside Rails, we can concurrently load those pieces. The request is faster (to the user, in exchange for more load on the server) and if a request should fail, ESI supports try/fail that will load a backup page instead.
  • Application Sharding – Different pieces of the app can be served by completely separate instances
  • Polyglot Assembly – An extention of sharding: ESI allows you to use multiple backends for different fragments and these fragments don’t need to be provided by the same app. Think of it as similar to RPC/SOAP requests from remote services giving you back whole fragments.
  • Cached New User Experience – The cache can be specific to users (using cookies and URL params), including brand new users. So your non-user pages are basically fully cached
  • Inline Invalidation – Easily invalidate caches when updates are made
  • FragmentFu – You get view helpers and other methods that will generate the ESI tags you need. Even better…you get a special respond_to format.fragment so you can use the same view and render both a full page or just a fragment depending on the request. This isn’t available yet, but Aaron says it’s coming.
  • Deployment – There are open-source, commerical, and CDN options:
    • Mongrel-esi
    • Squid 3.0 (partial support)
    • Varnish (basic esi support)
    • Commercial Oracle web cache
    • Commercial – Akamai – Most complete implementation
  • Pros and Cons -
    • Cons
      • YAGNI – Since ESI is really only well supported by the ‘big dogs’ and that means big money.
      • Cache invalidation
      • Lack of mature open source
      • Cost of deployment
    • Pros
      • Concurrent execution
      • Efficient execution – Use Rails only when you need it. Putting pieces together is not what Rails should be doing.
      • Conditional/ AB testing
      • Forces you to split one ‘action’ into smaller restful pieces
      • Syndication for Free – You’ve already built it by splitting up the page
      • Geographically Distributed

Aaron’s presentation will be available from http://blog.hungrymachine.com

Overall, Aaron provided a good talk with some code examples to really help people understand how ESI can be used. Unfortunately, it’s obvious that the technology is still pretty commercial and there’s not a lot of open-source options. This means that only the largest of the large will need to worry about trying to use it. Anything smaller, will likely just use normal fragment caching.

6 responses so far

  1. Thanks for the excelent coverage! =)

  2. Great summary. Very glad you’re doing this since I’m not there.

  3. Drew,

    Thanks for the great writeup of my ESI talk. You’re conclusion is correct.. This is primarily used by the big boys… but MongrelESI lets you play with these ideas, and is production ready and scalable for most applications.

    Thanks,

    Aaron Batalion
    http://blog.hungrymachine.com

  4. one of the benefits of using lighthouse for rails is that one of the core team (technoweenie) is a lead developer for lighthouse. so if they run into problems like they did with trac, it will be much easier to address

  5. [...] http://drewblas.com/2008/05/31/railsconf-2008-saturday-morning-summary/  [...]

  6. [...] put together a great set of blog posts covering: Friday morning, Friday afternoon, Friday evening, Saturday morning, Saturday afternoon, Saturday evening, Sunday morning and Sunday afternoon. He even put together a [...]

Leave a Reply