What's switching: Nine takeaways from Hot Interconnects
Like two tin cans, communication channels are getting pushed past what the string can handle, in addition to bearing the brunt of poor engineering decisions, according to the speakers at the annual Hot Interconnects conference in California.
Facebook detailed its next-generation switch, Wedge, a high-profile response to the hyper growth of big datacenters. Others lambasted shortcomings of Ethernet, Wi-Fi, InfiniBand, last-mile links, and communications silicon in general.
Silicon photonics is on the near horizon to help fill a few of the gaps, they said. But it brings its own challenges.
Fittingly, the event was held on the headquarters campus of Google, one of many cloud computing giants whose 100MW datacenters are stretching networks to the breaking point. "The constant topic is: What's the most cost effective, high performance way to build these networks?" said Andy Bechtolsheim, chairman of Arista Networks and serial entrepreneur.
The requirements in requests-for-quotes from datacenters have leapt a hundredfold over the last eight years, he said (see above). That's because many datacenters have taken an approach to building flat, fast networks so applications can be blissfully unaware of the details of the network.
Arista specialises in building such nets based on the latest, fastest merchant switch chips it can get, mainly from Broadcom. The approach currently can put nearly a million servers on a single network using 10Gbit/s and 40Gbit/s connections (see below).
The job is software intensive. Ninety per cent of Arista's 600 engineers are writing software, Bechtolsheim said. It's only going to get worse with companies such as Google calling for ways to get within milliseconds the precious data about traffic patterns, currently inaccessible on those chips.
Facebook tries to merge switch, server
Inside Facebook's Wedge: The green card holds a Broadcom switch chip, the red one an Atom-based x86 server SoC.
Facebook decided it couldn't wait for companies like Arista to come out with new switches, so it will build its own. The Wedge switch (above), already being tested in production networks, will become a design Facebook will contribute to its Open Compute Project, an open-source hardware initiative.
"We wanted to get agility because we are changing our requirements in a three-month cycle," far faster than vendors like Arista and Broadcom can field new products, said Yuval Bachar, a former Cisco engineering manager, now working at Facebook.
The company's datacenters are approaching a million-server milestone, Bachar said. Today it uses 10Gbit/s links from its top-of-rack servers, but it will need to upgrade in six to eight months, he said. The Wedge sports up to 32 40G ports.
The most interesting thing about Wedge is its use of a small server card, currently using an x86 SoC. However it could be replaced with an ARM SoC or "other programmable elements," Bachar said.
Facebook's Wedge brings server and switch elements together in one box.
|Related Articles||Editor's Choice|