RailsConf 2008 – Surprise of the Day: Maglev

Definitely the surprise of the day for me was attending the session on MagLev. Before today, I’d never heard of MagLev or Gemstone. On a whim, I walked into this talk instead of watching another one on deploying (which I feel I have enough experience in).

To sum up the MagLev product: it is a Ruby Virtual Machine (I know, I know, not another one…but stick with me!). This virtual machine adds a shared object cache so that separate instances of a Ruby app will share a state and this shared object state is persistant beyond the VM instances.

Avi showed a great demo of two VM instances sharing data. Then we got to see some preliminary performance data that showed an order of magnitude or TWO increase over MRI 1.8. So far so good!

The audience got into a discussion about the future plans for the system and we came out with a few critical things:

  • The Gemstone VM supports ACID transactions and will support some indexing functions to allow for fast data lookup. It became very obvious that the plan is to actually replace ActiveRecord in Rails and instead just use a huge persistent cached object store for data. It’s a terrific idea. We all know that relational databases have an inherent (and probably insurmountable) problem with scaling especially with complex joins and querys. In the same way that Amazon SimpleDB, Google App Engine, CouchDB, etc all limit your ability to do complex joins in exchange for a much more feasible scaling plan, so too could this system provide fast, efficient, and limitlessly scalable data storage and lookup.
  • The product is nowhere near ready. Ah yes…there had to be a downside. Although the product has made great strides (thanks mostly to Ruby’s similarity with Smalltalk and the existance of the Gemstone Smalltalk VM), it still has a ways to go. It can’t run all the Ruby benchmarks, can’t pass the Ruby specs, and definitely can’t run Rails. But they’re getting there. In a year from now, I expect this to be a big talk of the town.
  • This kind of development effort doesn’t come cheap. Gemstone is a commercial/enterprise oriented company, and as such it’s clear they have plans to make money from what they’re building. However, they made it very clear that portions of this system will be open source, other parts will be free, and the rest will be cheap or free for small to medium sized use. It will likely be similar to the Smalltalk VM, which is free for the first 4GB of storage. My personal feelings on the matter is: good for them. It costs money to develop software this advanced and they have every right to commercialize it to make as much profit as possible, same as I do every day to put food on the table. I applaud them for releasing a substantial portion of what they build for free (speech and/or beer), and I hope to be the first in line to pay when it comes time.

“Screw the databases, why would you want to use a relational database” – Chad Fowler

RailsConf 2008 – Friday Afternoon Summary

The afternoon was pretty crazy and I didn’t have a lot of time to take detailed notes. Luckily, the presenters will be posting their slides for all to see. What I have instead are just personal comments on what I found throughout the day:

Tune-Up

I met up with Brian from FiveRuns, who I got to know quite well at RubyConf last year. He introduced me to the new application that they just released yesterday called TuneUp. It is a Rails plugin that does development analysis on your pages to let you quickly find any glaring performance mistakes you may make while coding. It also looks to have a neat function that will let you upload a performance analysis for all to see (and hopefully people will help you fix). You can bet I’ll be blogging in detail about the app when I have a chance to review it in detail

Crud is not spelled with an ‘S’

Steve Midgley

I won’t go into much detail on Steve’s talk. It was pretty decent, but he was only able to get through half of it. It had good content and actual code to work with. Steve was nice enough to post his detailed slides at http://misuse.org/science so I won’t put bullet points here, especially since he only got through about half the presentation. If you’ve got the time, it’s certainly with it to read his slides.

Flexible Scaling
TJ Murphy

TJ is the creator of Warbook, a facebook based game written in Ruby on Rails. His talk starts by discussing the pains he went through.

He talked about moving from being a developer to being a sysadmin as he worked to scale his application. Several notes really hit home as I’ve experienced them as well when scaling with Rails. TJ basically said that when first starting with sysadmin-ing, he’d boot up an EC2 instance and would just google for everything he needed to do until things worked. A very familiar sentiment indeed.

He also gave a lot of info about how he sets up his stack and partitions different services to different instances. Much have what he said directly conflicted with the earlier EngineYard presentation. Overall, I’d have to say I agreed more with the approach and stack management style that TJ presented as opposed to EngineYard, but both methods are valid and will work for someone looking to scale.

TJ also noted that he’ll be posting his talk and notes, however, I don’t have a link for it.

RailsConf 2008 – Friday Evening Summary

Meet the Sun you don’t know

Charlie Nutter

Charlie opened with defining Enterprise Ruby as something that is sold by salespeople using “Steaks and Strippers” (third time today I’ve heard that phrase from a speaker). He spent a lot of time introducing a ton of great things that Sun is doing to help and support developers and a number of great open source packages that Sun funds in part or in whole. He also mentioned starting to blog and releasing code to take back Enterprise Ruby for developers. That really hit home because that’s exactly what I’m going for on this blog. So thanks to Charlie for caring about community above all else.

Ruby Hero Awards

Next were the guys from RailsEnvy to present the new Ruby Hero Awards. They gave out 3 awards for library and program developers and 3 awards for people who provide great support, editorial, and educational content. I’ll list the winners here:

Developers:

Editorialists:

  • Ilya Grigorik – Definitely one of the best Ruby programming bloggers out there. I read his posts religiously and reference them constantly. I aspire for my blog to be half as good as his (which I’m doing very poorly at given how little code is on my site so far)
  • Yehuda Katz
  • Ryan Bates – Probably the best known name of all the winners. Ryan puts out the weekly Railscast screencasts.

David Heinemeier Hansson

The Great Surplus

DHH started by talking about how there is a surplus of productivity when developing with Rails. Ruby on Rails is still a very small community compared to other platforms/frameworks/etc out there, but we can produce so much more and we can turn that productivity into additional revenue. He hit on a terrific point that the reason our community is so great because Rails developers have realized that we are all trying to ‘climb the same mountain’. None of us are all that special and the more we can help each other the better. We as developers have realized how to ‘cede flexibility’. He put it very concisely that “People like choices a lot better than actually having to choose”. What we want is something that is standardized and works without having to evaluate options.

He discussed the paradigm of ‘Convention over Configuration’. I constantly hear about how Rails __forces__ you into its way of thinking. DHH makes it clear that Darn Right! “Make the choice once and then move on”. These are choices on things that don’t really matter. So Rails has been nice enough to pick one for you that is good enough. He goes on to say that Rails lets you ‘pick your choices when you care, but otherwise it picks it for you’. “We offer everything you need as one coherent package” This makes a great comparison to Linux and Ubuntu. Linux used to require a system user/administrator to choose what software package they want to use for everything. Vi/Emacs, Gnome/KDE, the list goes on. Ubuntu is, although not the first, certainly the best at giving you a system where you have to make zero choices before getting exactly what you’re looking for: a system that just works.

David continued by addressing that “the surplus won’t last forever”. This advantage can’t be held forever. He proposed several ways that might occur:

  1. The mainstreem copies Rails – Not very likely he says. It’s been tried and has failed a lot.
  2. Dramatic alternative arrives – Possible, but by the time we know it, it’s probably too late.
  3. Rails becomes mainstream – DHH is obviously very supportive with Rails NOT becoming mainstream. By staying

“If you’re a company competing with another company: You want them to be using crappy tools”

So we know that the surplus can’t last forever. What can we do? Give up? Turn crappy? Run at 110% forever to keep up? No, instead David proposes using the surplus we have know to our advantage as much as possible. He equates it to the current explosion in Dubai. I think this is a great comparison. Dubai is using its huge surplus of cash from oil on building an ‘instant’ modern day metropolis. In the same way, David says, we must invest the surplus that we have now in ourselves. By doing so, the surplus will last longer. And when it runs out we will survive thanks to that investment.

So how do we invest in ourselves using this surplus? DHH proposes:

  • Recharge Tangentially – Do something else besides sitting in front of your computer all day. 37 Signals is supporting their employees in doing things besides programming (including one employee who is getting their pilot’s license…I wish MY company would have subsidized my flight training! Heck, the AIR FORCE wouldn’t even subsidize my flight training!).
  • Sleep More – Pretty self explanitory. For my part, I swear by 8.5-9 hours of sleep per night. No matter what. The last time I DIDN’T have that many hours in bed for more than one night in a row was 7 months ago while travelling to a funeral. And I believe this is one of the prime reasons that I believe I can do awesome ‘peak’ programming work for 4-6 hours a day.
  • Read Paper – It’s important to expain your body of knowledge. On life in general. On other talents. On basically anything. DHH specifically recommended The Secrets of Consulting – Gerald M Weinberg
  • Program Less – If you do the other items in the list, this one becomes pretty easy. If you cut in half the amount of time you can spend on a project, then it becomes painfully obvious if you are using your time as effectively as you possibly can.
  • Start from Scratch – Being stuck on the same project
  • Share – This is the one that is probably the greatest strength of the Rails community. You can only gain knowledge by sharing. You can not lose knowledge.

Finally, DHH discussed the four day work week that 37 Signals recently implemented. It’s been written about fairly extensively, but he summed it up very well:
“The amount of time you put in has a very weak correlation to the amount of productivity you get out”

I feel really good that I’ve so far done quite well at ‘spending my surplus’ appropriately in the past year or two. But there’s always ways to improve and I got some great ideas from his presentation. I hope you all took it to heart as well.

RailsConf 2008 – Friday Morning Summary

RailsConf Friday morning. I’ll write out as best I can about the talks I attend. I’ll bullet-point most of the talks and type out some of my own thoughts as time allows. Things here are moving faster than ever and I can only write so much.

The wireless network is crunching under the load, which is no surprise and happens at every geek conference out there. The screens keep showing a graph of saturated activity on their 14mbps connection. However, after some difficulties connecting, it turns out to be pretty responsive, even with a few thousand people in one room pounding on their laptops. Rich Kilmer mentioned on Twitter that Lost cut out here in Portland for about ten minutes, so Chad begged everyone not to download Lost on the conference network.

Chad’s Intro
Chad made his introductions and gave some history about RubyCOnf & RailsConf. The quote of the morning is “RubyConf2005 maxed out with 200 people, and we didn’t really know a lot of them, and when we started talking to them, it turned out, a lot of them were these ‘Rails’ people…”. His history and experiences with previous Ruby/Rails conferences and meeting capacity is a terrific insight into the behind the scenes to these events. I too was curious about how it could be that RailsConf2008 didn’t sell out and what that means. I’m glad to know that the reason is because Chad’s getting smarter at running them and not because Rails is dying.

Chad has a letter about conferences being like school in the schwag-bag and he also talked about it during his welcome speech. He encouraged people to participate and interact with one another. As he put it: “How do we keep RailsConf weird”. I think this is a great sentiment that will keep the community tight-knit for some time to come. Chad is also talked about rejected speaking proposals (of which mine was one of them).

Joel Spolsky

Great Software

  • Make People Happy
  • Obsess over Aesthetics
  • Observe the Culture Code


Make People Happy

  • Demonstrated logging into windows…automatic updates come up, let’s look at them, okay, let’s install them. 18 progress bars later…
  • Give users agency/control of their environment.
  • Put the user in control.
  • Positive Feedback.
  • We all know about AJAX requests. What do they do? They give your user instant feedback about what is going on.

My thoughts:
-It’s important to note that AJAX CAN be a great tool to provide instant feedback but only when done properly. This includes simple things like a loading widget, highlights when something new is loaded, etc. These are all things that are typically provided by the browser itself in a typical request, but you lose that functionality with AJAX and must replicate it yourself. If you don’t provide a replacement, then you lose the instant feedback. And worse, there is no intermediate feedback in the request at all, which will quickly lead people to click the same link a million times or more.

Obsess over Aestetics

  • Aestetics can overcome a lot of flaws
  • Samsung Blackjack vs Apple iPhone
  • Monument Buildings in France vs US
  • Lipstick vs Guts
  • Skins – If the underlying functionality is heinous, then it can’t be made to look beautiful.
  • Modernist vs decadence

Observe the Culture Code

  • Safety by design
  • Sedans vs SUVs – The Culture Code
  • Enterprise vs web 2.0?
  • Ruby: love beauty happines
  • Python: sadness pain hurt
  • Java: death destruction nuclear war

Misattribution

  • Go to the movies and you have to go to the bathroom but are stuck because of fat people: You will say you hate the movie
  • Go on a date and go to a coffee house: The girl will be drinking coffee that raises her heartbeat and will think she’s on a great date.
  • Go to a speech and the presenter shows pictures of Angelina Jolie and makes lots of jokes (Just like Joel) and you’ll think you loved the presentation.

My thoughts:
A very succinct demonstration that what should be the number one priority is to MAKE YOUR USERS FEEL GOOD. There are practical methods, such as positive feedback and putting the user in control. There are aesthetic methods (making it look pretty makes your users feel good). And there are cultural methods to help you users associate better. Do whatever it takes to make your users feel good.

Overall a good talk. Not mind blowing, but good and thought provoking.

PS Joel Spolsky speaking reminds me of an Eddie Izzard act….anybody else agree?

Entrepreneurs on Rails
Dan Benjamin

The room was absolutely packed. This was one very popular presentation. About half way through about half the audience was forced to leave to clear the aisles in case of a fire.

Know Why – You need to have a reason for building an business. Maybe the reason is “There is a large demand that I can meet”, maybe it’s “It’s cool” (although that’s probably not a good reason). But you need to know why you are doing what you’re doing.

Set Goals – These should be metrics about productivity, revenue, customer base, etc. By setting goals, you can
Have an Exit Strategy
Why Create a Company

  • Seperate entity – A business has it’s own bank account, etc
  • Liability
  • Taxes
  • Working with other companies is easier – Companies want to avoid overhead associated with hiring individuals. They can ‘outsource’ to your company and all they have to do is cut a single check.
  • Raising Money
  • Percieved Credibility – There is a perception that if you are just one guy writing code, you are less credible and less reliable

Types of businesses

  • Create a Brand – This is a great way to bring you into the community. Contribute to open source projects, be active on mailing lists, etc. But this isn’t enough. You need to have a simple and straight forward message associated with your name or company. Examples: thoughtbot.com scoutapp.com
  • Target the right market – Markets have different sizes
  • You will spend 40% of your time marketing. Count on it.
  • Making it Work – Getting started is easy. Ideas are cheap. Making it work is the tough part.
  • Feast or Famine – How did he manage it? Took any job for very little money.
  • Working from Home – He has a place in his house dedicated to work. His office is just for work.

Getting Work

  • RFP – Used to be the RFPs were public and you downloaded them and then wrote and submitted your proposal. Now you must request the RFP before being given a chance to submit a proposal
  • Proposal – Dan usually writes 30-40 page proposals that covers a project in extreme detail
  • Contract – It’s very important that your contract lay out everything in detail.
  • Functionality Outline – The more detailed your specifications are, the better you can support your case when a client says “it’s supposed to do X, not Y”
  • Getting the Money – Businesses like to pay with Net 30, Net 60, or beyond. This can be a big problem for a small entrepenuership where cash flow is critical. Write into your contract that payment must be Net 0 or Net 15.
  • Terms of Service – This limits your liability when things go wrong (ala Twitter). Don’t promise or let people think you’re promising things you can’t deliver
  • Privacy Policy – Again this limits your liability. You need to disclose what you’re doing with user data.

Raise money
Why you might want to raise money

  • Payroll
  • Advertising
  • Equipment
  • Lengthy Development Cycle
  • Office space

Kinds of Funding

  • Personal Financing
  • “Love Money”
  • Equity Funding – Venture, Angels, IPO’s
  • Debt Financing

Getting Out
Eventually you will not be involved anymore. Planning on how to wrap things up or hand it over to someone else.

Overall, a good presentation for any independent developers or people looking to start a business. Nothing really Rails specific, but obviously this is information that a lot of people need.

Hosting Woes

Ezra, et al

Problems

  • active_record “find(:all).each do |leak|” Development with 20 records. Production with millions of records. No limits, no indexes.
  • Not up to par with MySQL – Relink back to prior posts
  • If you have a foreign key, it needs an index
  • plugins
    • ferret – Corrupt indexes, out of control index size. Move to ultra/sphix. Ultrasphix now supports delta indexing. No reason not to move.
    • image science + monit: monit likes to clear ENV variables
    • hodel 3000 – Way too much logging means a total data overload. Switch your log leve

    Handling traffic

    • Digg – 10,000+ visitors, very few signups
    • TechCrunch – 1000 visitors, almost all signups
    • Today Show / Fox News – hundreds of thousand of visitors… 2000+ connections

    How well can you code:
    – File io: Log to different disks, don’t be reading & writing to disk simultaneously

    Deployment

    eycap – gems.engineyard.com
    gem source -a gems.engineyard.com

    Mongrel alternatives are good: ebb & thin. But you don’t need to worry about the difference unless you’re doing 1000+ requests / s. Look at your code first.

    Questions:

    • What happened with nginx: “This presentation was about problems that we’ve had problems with. We haven’t had any problems with nginx – It is perfect”
    • Passenger/mod_rails: “It runs on apache, so we haven’t worked with it much, but it’s interesting and is doing some good things”
    • Memory footprint of different app servers to run more mongrels: “YOU NEVER NEED MORE THAN 3 OR 4 MONGRELS PER CPU. Adding a ton of extra RAM to run more mongrels doesn’t do any good.”
    • Do static files need to be local or can they be shared between cluster elements: “It’s a very good thing to share the static files so that cached and uploaded files are immediately available to the entire cluster”
    • Best way to handle background jobs: “BackgroundRb was naively developed, now background job (bj) is the best method that we use”
    • Do you use keep-alive with nginx: “Not sure, we haven’t tested it. But one great thing is to use many hostnames for asset servering so that the browser will simultaneously load from many different places, even if all the hostnames map to the exact same machine/IP”
    • What is the best way to distribute the workload across servers (especially virtualization): “The typical physical hosting of distributing web, app, db servers isn’t really needed for virtualization. Each slice has its own nginx, mongrels, & memcached.”
    • SLA’s/Uptime: “Engineyard has 3 9’s of total APP TIME availability (approaching 4 9’s).”

    Overall, another good talk. I’m very much interested in deploying/hosting/operating Rails sites, so I was glad that most of my opinions on how to run Rails sites was validated by what the EngineYard team had to say.

    RailsConf 2008 – Thursday PM

    Well, RailsConf has begun. It’s going to be a fun time and I’ll be posting pictures, thoughts, and anything else that comes up throughout the week/end.

    I arrived Thursday afternoon and managed to get over to the convention center in time for the Birds of a Feather talks. I sat in on a great discussion about building viable SAAS products and selling them for real profit (as opposed to getting bought by google without ever having a business model). It was incredibly productive and we got pretty in-depth into targeting niche markets and controlling IP ownership (the surprising concensus was that IP ownership isn’t all that important. Ideas are a dime a dozen…implementing them is the critical part). Overall a terrific kickoff for me at Railsconf

    Comparison of Rails monitoring apps: FiveRuns vs NewRelic RPM vs Scout App

    Monitoring your production Rails application is a very important part of deploying and operating a web app. There are several more general solutions that work very well: Nagios, Munin, etc. As of late, however, several Rails specific options have come into common use. I’d like to discuss the three big players here:

    FiveRuns RM-Manage

    The FiveRuns client has been out for about a year and offers a terrific suite of monitoring: Server load/memory, MySQL queries, Rails errors, etc. As of version 2.0 (which is in open beta is and will be released for customers during RailsConf) it also supports monitoring your mongrels. It works great, but can get a pretty expensive ( They don’t publish their prices, but I’m paying $30/server ). It is an good choice for most users.

    NewRelic RPM

    I’ve been beta testing the NewRelic RPM service for the past few months. It’s a decent service, it’s very easy to install, but is very limited. It will monitor your server load/memory, slow queries, etc as will all the other monitoring tools. But beyond that it doesn’t offer much. You are limited to graphing only a 24 hour period of data, so you can’t see any kind of long term trends.  They have an amazing backend system for collecting data and their site is the fastest and most responsive I’ve ever seen.  Once they get their UI front-end featureset to match their amazing data collection system, they’re going to be awesome.  As of today, they opened to the general public and released their pricing. It is based on the number of mongrel/thin instances no matter how many servers (at least that’s my understanding). For a small to medium app running up to 40 mongrels (which would probably be 2-4 servers), you’ll wind up paying $250 / month, as compared to $60-120 for FiveRuns. Overall, given the limited functionality and hefty price, I can’t yet recommend NewRelic. I hope to see it grow and quickly add more features to change my mind.

    Scout App

    The third option is Scout App. It offers the same suite of monitoring features as the others, but goes a step further by offering a huge range of additional plugins that will allow you to customize its functionality and easily set up extra functionality such as restart dying mongrels. You can also write your own plugins. To add another scoop on this already monstrous sundae, Scout App is the cheapest of all. It will run you only $29.00 / month for four servers.

    If I had to pick one service to recommend, it would likely be Scout. They provide just about everything you can ask for out of a monitoring app at the lowest price point. If anybody else has experience with these services, please add your own comments!

    Help me out, Pragmatic!

    I’m an avid Pragmatic Bookshelf reader.  I probably own 30%-40% of their book titles.  The writing is always great, editing is even better, and they’re often the first to print on some great up-and-coming topics.

    I saw recently that they are offering ‘Friday-series’ PDF books for $9 and I found one on using memcached that I decided to buy.  I’m pretty good with memcached and use it quite regularly, but there are always new things to learn and I figured this would make a great reference if nothing else.  I also thought it would make for great reading on the plane trip during my upcoming trip to RailsConf (call me old-fashioned, but I’d rather carry the physical version rather than read it off my laptop, especially while traveling).

    I purchased the PDF, downloaded it, and when I went to print it I found that the background of every page is tan!  I bought it with the specific purpose of printing, but that’s a waste of a LOT of ink to print a solid background on every page.  I went through a very lengthy process to try and fix this so I could have a nice hard-copy:

    1. Adobe Professional has a great accessibility tool that will re-color the background.  It worked perfectly to remove the background and leave the rest of the document intact.  Unfortunately, despite what the print-preview shows, the pages still print with the tan background.
    2. Adobe Pro also has a tool called ‘Remove background’ but it only works on backgrounds added by Adobe Pro.  Not the case with this PDF, so that doesn’t work.
    3. I searched the web for removing backgrounds from PDFs but found very little.  I tried xpdf but it couldn’t handle this file and any watermark removal tools I found were sub-par.
    4. I e-mailed Pragmatic asking if there was any way of getting a white-background version.  Obviously a long shot, but I figured I had to at least ask.  To their credit, I received a response from Dave Thomas in a matter of minutes.  This is what he had to say:

    To be honest, the Friday’s are really not formatted for printing in many different ways–the font is optimized for screen viewing, and there’s tons of white space. It never really occurred to us that someone would print one.

    Really, I have no problem with the font or the white space, but I found it VERY surprising that it never occurred to them that someone would print one.  I could have sworn that the purpose of a PDF was to provide a format that would display and print consistently across platforms.  To be honest, I can’t find a PDF I’ve bought that I haven’t printed.   Thankfully Peepcode PDFs (which I own almost all of) are B&W and print terrifically.

    In the end, I printed it with the tan background and I’m thankful I ordered new ink cartridges last week that should be arriving very soon.  Hopefully, the Pragmatic guys will think twice about a colored background with their future PDFs.

    Why I switched from Mozy to Carbonite

    I’ve been using Mozy for my personal backup solution for nearly a year and a half. It is a great service that basically relieved me from ever worrying about my backups. It is a great service, but lately it has really gone down hill.

    Starting about six months ago, I started having trouble where Mozy would just fail to backup for several days at a time. I’d get a message that it had been 3 or more days since the last successful backup. I’d have to manually start a bac kup several times before it would actually upload some files. After contacting Mozy support, it appeared that their latest auto-update had failed and my configuration and whatnot was corrupt. After a total uninstall, reinstall, and complete re-upload of all my data, things went back to normal for awhile.

    Recently, I’ve been having a new problem. Whenever I would use Windows explorer, and would try to cut/copy/paste files around my hard drive, explorer.exe would crash. One day, while cleaning out files, I had at least 20 crashes in an hour. When I dug deeper I found the crash to be caused by mozy.dll. This time, a complete uninstall/reinstall didn’t fix the problem.

    The straw that broke the camel’s back is when I opened the mozy status app and discovered that my backups were almost a month behind! And I wasn’t even getting notified that my backups had failed! Obviously, this is no longer worry free.

    Switching to SugarSync
    I happened to get my hands on an invite to SugarSync, so I decided to give that a try. It sounded like it would the answer to all my needs. Not only would it automatically backup all my files, but I could also sync between several computers (a function previously filled by FolderShare). Unfortunately, after trying it out for a few days I discovered two major issues:

    1. Files would fail to upload / sync regularly. I’d have to wait days for a file that I created on my desktop to show up on my laptop. That doesn’t bode well for the backups either.
    2. Many files would have issues with read locks. I’d constantly get error messages from files that were being updated but were still open (specifically: Quickbooks files & KeePass files). Both programs create temp lock files and keep the databases locked open while in use. SugarSync just couldn’t handle this and I was extremely worried about data corruption.

    Switching to Carbonite
    After about two months of these issues, I decided to try Carbonite. I switched my syncing back to Foldershare and installed the client. So far it has been terrific. I’ve got a laundry list of great pros:

    1. It has no problem handling locked open files
    2. It’s fast (uploads much faster than Mozy or SugarSync)
    3. It uploads changed files almost instantly (it doesn’t even seem to wait until night-time or anything)
    4. It has explorer integration to quickly decide what to backup (Mozy has integration for restoring, but backup/status)
    5. Best of all: It adds little dots to backed up folders in explorer. Yellow dots: files not yet backed up. Green dots: Everything a-ok.

    The price is comparable to Mozy and I think will be worth every cent. Here’s to Carbonite!



    Bookmark and Share

    How to scale Rails

    So we established previously that Rails really can scale!  Now it’s time to look at how to do it.  I’ll list here the basic steps and then follow up with a post on each of these:

    1. Of course, Memcached helps a ton!
    2. Start with a master database and add two or three slave databases.  Split your reads from your writes. Rails has a great plugin for this that makes it about as easy as it can be (certainly easier than PHP). This will move you to being able to handle 15 or so app servers.
    3. Denormalize some of your data to reduce query intensity.
    4. Shard your data. Put statistics/usage data on a totally separate set of DB boxes from your user’s data.  If you really are the next MySpace,  check out AsterData for offloading some of that work.
    5. Switch to a commercial DB setup. If you’re still desperate to expand, then a more robust DB is probably in order. The great part: Rails is database agnostic (at least far more so than PHP or ASP) and will happily switch without too much effort.

    Deploying Rails Applications by Ez

    I
    recently received Ezra’s new book "Deploying Rails Applications" in
    the mail.  It is a terrific reference and I will be following up shortly with an in-depth review.

    I was quite delighted to find that I had already done even the most advanced tasks regarding scaling at
    least once or twice (e.g. load balanced app servers, clustered MySQL,
    Memcached, CDN for static files, & extensive
    benchmarking/profiling/optimization).