Google’s Tensor Processing Unit (TPU), first deployed in 2015, provides services today for more than one billion people and provides more than an order of magnitude improvement in performance and performance/W compared to contemporary platforms. Inspired by the success of the first TPU for neural network inference, Google developed multiple generations of machine learning supercomputers for neural network training that allow near linear scaling of ML workloads running on TPUv2 and TPUv3 processors. TPUs extend research frontiers and benefit a growing number of Google services.

Social networks have been deeply woven into our everyday life. These Internet platforms host a plethora of real-time services to keep people connected, provide customized information to users, and preserve information transparency. Underneath their infrastructure, the adoption of Machine Learning (ML) techniques is rapidly becoming omnipresent in both datacenters and end users’ devices, steering a rich feature set to enhance the effectiveness of users’ communication and to improve the quality of online experiences. Meanwhile, to achieve these objectives, ML can consume enormous computing resources and require meticulous resource design, provisioning, and management. In this talk, I will share our vision on the computation technologies and acceleration strategy at Facebook’s infrastructure, in particular, the state-of-the-art of our machine learning approaches on production-scale DNN-based personalized recommendation models for content ranking. Moreover, I will discuss the computing and privacy challenges lying ahead from the perspective of a social network service provider.

We have seen many domain specific architectures (DSAs) for AI computation but only a few of them has gone into mass production. Despite of the quality and yield issue, complete and easy-to-use software stack that maps application onto hardware is a major challenge. This talk presents our work on designing the software stack for our DPU on FPGA. Several different interfaces are designed to allow fast development for developers in different level, while neural network pruning and quantization are embedded. Extendible instruction set is designed to tackle rare neural network architectures. Backend optimization techniques are also introduced. Our work has been used as Xilinx Vitis AI, the unified software stack recently announced by Xilinx.