CMake: Define FunctionsUnderstanding the function definition with CMake will help us understand usual function calls.Jan 5, 2023Jan 5, 2023
Compare Deep Learning Toolkits: Theano, TensorFlow, TensorFlow 2.0, PyTorch, and JAXMy recent work on PyTorch Distributed and TorchRec requires me to learn PyTorch 2.0. At the same time, I am learning JAX and XLA from Alpa…Oct 2, 2022Oct 2, 2022
PyTorch DistributedDataParallel InternalsThanks to Shen Li, Yanli Zhao, Alban Desmaison, and Wanchao Liang for teaching me the details about DistributedDataParallel, aka, DDP, the…Sep 30, 20222Sep 30, 20222
Published inThe StartupWhat I Have Done Wrong Leading Baidu’s Open-Source Deep Learning System PaddlePaddleIn late 2016, I was assigned the tech lead of PaddlePaddle, the open-source deep learning system of Baidu.Nov 5, 2020Nov 5, 2020
NLP in SQL: Word VectorsIt is a common bias that SQL cannot handle unstructured data like text. However, this is not true. In this article, we explain how to…Feb 7, 20201Feb 7, 20201
