Ethernet woes: Bandwidth demands, costs

The new dynamic is forcing network engineers to consider starting work on a terabit Ethernet standard before they have finished the 100Gbit spec. It's also got them scratching their heads about how they will get paid for their efforts.

The pressure has become so intense a network engineer for the popular Facebook Web site said his company has considered building its own network equipment while it waits for vendors to deliver boxes that go beyond today's 10G Ethernet speeds.

"One of our top network customers said they need to spend 30 to 60 percent more on capex but their revenue is only growing three percent a year," said a senior engineering manager with Cisco Systems, speaking at an open forum hosted by the Ethernet Alliance. "The whole industry economics are changing," he said.

A senior engineer from Deutsche Telekom agreed that bandwidth needs are growing much faster the revenue. He called for a terabit Ethernet effort to start soon, and said he is also managing initiatives to drive down the cost of line cards and other network components.

Donn Lee, a network engineer for Facebook, said his company's rapidly expanding network could use multiple 100G Ethernet systems today if he could get his hands on them. The standard is in a late stage of being finalized by the IEEE 802.3ba group.

Bandwidth requirements more than double every year for the cluster aggregation network in Facebook's data centers, Lee said. The clusters are used to compute data for the estimated 300 million users of the free Web service.

"Anybody in the top 25 Websites has this problem," said Lee. "We are so busy building out networks, we don't have time to do these presentations," he told an audience of more than 100 network systems and component engineers.

"A 16 terabit switch would be ideal, but it doesn't exist today," Lee said. "The biggest switch I can buy today handles about two terabits," he said.

1 • 2