Sorting a LOT of data
Very interesting article on how Google sorts data – the punchline being that they sorted 1 petabyte (which is 1000 terabytes, or 1 million gigabytes – don’t even try to comprehend it – it’s not possible) in 6 hours and two minutes. Trust me when I tell you – that is absolutely amazing. Sorting is really the heart of what Google does, and to be able to do it that fast in astounding. And on only 4000 computers!
The other interesting part of this story is how they actually STORED that much sorted data. Easy! Just put it on 48,000 hard drives. I’m not kidding. Read the article.