At Alibaba’s Apsara cloud computing conference in Hangzhou, China today, the company’s CTO Jeff Zhang unveiled an AI inference accelerator chip for the cloud which he claimed offers ten times the compute power of today’s GPUs.

The chip, named Hanguang 800 after a legendary sword from ancient China, was described by Zhang as “the world’s most powerful AI inference chip.” Peak performance on ResNet50-v1 inference reached 78,563 images per second, with peak power efficiency of 500 images per second per watt. It is built on the TSMC 12nm process.

Alibaba is already using Hanguang 800 chips in its cloud infrastructure, to handle image processing, product search, translation and personalised recommendations, the company said. Using the device, the company’s Pailitao service, where users upload pictures of items and search for matching products, had its performance efficiency increased by a factor of 12. This service handles one billion uploaded images each day, requiring an hour to process using the company’s GPU-based infrastructure. Hanguang 800 infrastructure processed the same amount of images in 5 minutes.

While the company said it had “no immediate plans to sell the chip as a standalone product,” a Hanguang 800-based AI cloud service is already available to Alibaba cloud customers.

Jeff Zhang unveils the in-house developed Hanguang 800 AI inference processor for the cloud (Image source: Alibaba)
Jeff Zhang unveils the in-house developed Hanguang 800 AI inference processor for the cloud (Image source: Alibaba)

In his keynote speech, Alibaba CTO Jeff Zhang said that Alibaba’s differences from traditional chip development houses are a strength rather than a weakness.

“I think [Hanguang 800] is the first large chip developed by an internet company,” Zhang said. “This is among the first steps in our launch journey. Alibaba has the confidence and the capability to do things that chip developers can do and cannot do. I think chip houses can do hardware, but I think we can do better with regard to software because that's traditionally been our strong suit.”

“With the speed of an internet company, we've succeeded in completing the chip project from design to verification within just a year,” he added. “Alibaba will become a company that has footings in both hardware and software, and we're very confident in what we're going to achieve with that strategy.”

Damo Academy
This is not Alibaba’s first foray into chip design.

The company’s research institute, Damo Academy, focuses on developing technologies for blockchain, quantum computing and autonomous driving as well as machine learning. Zhang said that Alibaba holds the most blockchain patents out of any company in the world, while Damo recently set up its first quantum computing lab, and has also developed autonomous driving technology to level L4.

Alibaba’s semiconductor unit, T-Head, was spun out of Damo Academy in September 2018. The name “T-Head” is a nickname for honeybadger (literally: flat-headed brother), the division’s mascot. Honeybadgers are known to be both vicious and tenacious in the wild.

T-Head’s first launch was the XuanTie 910, a heavy-duty edge processor based on the open-source hardware instruction set RISC-V (whether Hanguang 800 is also based on RISC-V is unknown). T-Head has also developed Wujian, its SoC development platform. In his speech, Zhang said Alibaba “hopes to open source its chip development capabilities in the future, and intends to build a bigger team for that purpose.”