RE: Habanalabs Open-Source TPC LLVM compiler and SynapseAI Core library
From: Michael Zuckerman
Date: Sun Sep 12 2021 - 03:41:21 EST
Add @Tzachi Cohen
-----Original Message-----
From: Oded Gabbay <ogabbay@xxxxxxxxxx>
Sent: Friday, 10 September 2021 10:27
To: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
Cc: Michael Zuckerman <mzuckerman@xxxxxxxxx>; Doron Singer <dsinger@xxxxxxxxx>; Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>; Dave Airlie <airlied@xxxxxxxxx>; Daniel Vetter <daniel.vetter@xxxxxxxx>; Jason Gunthorpe <jgg@xxxxxxxx>; Linux-Kernel@Vger. Kernel. Org <linux-kernel@xxxxxxxxxxxxxxx>
Subject: Habanalabs Open-Source TPC LLVM compiler and SynapseAI Core library
[Some people who received this message don't often get email from ogabbay@xxxxxxxxxx. Learn why this is important at http://aka.ms/LearnAboutSenderIdentification.]
Hi Greg,
Following our conversations a couple of months ago, I'm happy to tell you that Habanalabs has open-sourced its TPC (Tensor Processing Core) LLVM compiler, which is a fork of the LLVM open-source project.
The project can be found on Habanalabs GitHub website at:
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHabanaAI%2Ftpc_llvm&data=04%7C01%7Cmzuckerman%40habana.ai%7C2380c349fa96487b7c6b08d9742c6fd6%7C0d4d4539213c4ed8a251dc9766ba127a%7C0%7C0%7C637668556484231215%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rp%2FIMKAAi%2FAFjjs3GM0cV4ViFn1bkA9nhAq632QB0TQ%3D&reserved=0
There is a companion guide on how to write TPC kernels at:
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.habana.ai%2Fen%2Flatest%2FTPC_User_Guide%2FTPC_User_Guide.html&data=04%7C01%7Cmzuckerman%40habana.ai%7C2380c349fa96487b7c6b08d9742c6fd6%7C0d4d4539213c4ed8a251dc9766ba127a%7C0%7C0%7C637668556484241172%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UdmYc%2B71EsEMAtzm6kV0Yu53brrZNdUSIEDQ%2F0vKhA8%3D&reserved=0
The guide details the TPC compute engine's architecture, how to write TPC kernels using the TPC-C language, etc.
In addition, we have written a reference implementation of the SynapseAI API, called SynapseAI Core, and released its code under the MIT license to the open-source community at:
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHabanaAI%2FSynapseAI_Core&data=04%7C01%7Cmzuckerman%40habana.ai%7C2380c349fa96487b7c6b08d9742c6fd6%7C0d4d4539213c4ed8a251dc9766ba127a%7C0%7C0%7C637668556484241172%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sadSWIBvATo2oVvSWgzGujnYYxcfGMEUEDgnfsmy4AU%3D&reserved=0
SynapseAI Core contains all the necessary building blocks to run Deep Learning training on Gaudi, although not as optimized as the closed-source library.
The project repository contains a couple of TPC kernels that implement basic DL operators. These kernels can serve as an example of how to implement more complex operators.
To work with the Gaudi device, the library calls the Habanalabs kernel driver uAPI through the already open-source hl-thunk library at:
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHabanaAI%2Fhl-thunk&data=04%7C01%7Cmzuckerman%40habana.ai%7C2380c349fa96487b7c6b08d9742c6fd6%7C0d4d4539213c4ed8a251dc9766ba127a%7C0%7C0%7C637668556484241172%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VocvcnvoLz36biT58GxWTn%2FxiUD%2BqG2WRD9EEOEDNb8%3D&reserved=0
Moreover, the library contains a few tests (and more will follow soon) that demonstrate how to use the SynapseAI API to run workloads which utilize the TPC engines on Gaudi devices. We provided a short readme that explains how to build and run the included tests.
It is important to note we provided all the necessary APIs to connect this library to any Deep Learning frameworks by writing appropriate backends in the frameworks and by writing more TPC kernels to implement the different operators.
Once the driver(s) for the Gaudi NIC ports will be upstreamed, this library may be used together with IBverbs to perform training on multiple Gaudi devices.
Thanks,
Oded