Google has fessed up: apparently it looks like a bunch of shipping containers, since 2005. And it does not use the usual rack stuff or centralised backup power. See Google uncloaks once-secret server for a decent overview, plus pictures.
I think there’s quite a lof of good information in there, with important lessons. Just a few:
- It’s not fancy-brand stuff, they’re simple Gigabyte mainboards
- An open frame rather than enclosures.
- 12V battery on the frame, rather than a centralised UPS. More energy efficient, and no single point of failure.
- Heatsinks, not lots of fans (mechanical stuff fails). There’ll be airflow-creating foo for the whole container, but since it’s a single enclosed space there’s neat separation between hot/cold already. Efficient!
- Two regular SATA harddisks
So, it’s mostly commodity stuff in a custom frame and and other minor tidbits. There’s 1160 servers per container, and by the looks of it it should work out pretty cheap to buy, build and operate.
The energy efficiency has more aspects… if you use less power, you’re also generating less heat that then needs to be got rid of. Cool machines are more reliable and live longer, and even fractions of a degree can make a significant difference.
You can see it all from a monetary perspective, but it also makes sense in terms of maintenance effort, waste/disposal and other environmental impact aspects. Of course these containers still eat a lot of electricity, but there’s a lot in there and it will be way “greener” than most data centers.