SINGA

Apache SINGA is an Apache Top Level Project, focusing on distributed training of deep learning and machine learning models. Various example deep learning models are provided in SINGA repo on Github and on Google Colab. SINGA supports data parallel training across multiple GPUs (on a single node or across different nodes). SINGA supports various popular optimizers including stochastic gradient descent with momentum, Adam, RMSProp, and AdaGrad, etc. SINGA records the computation graph and applies the backward propagation automatically after forward propagation. The optimization of memory are implemented in the Device class. SINGA supports loading ONNX format models and saving models defined using SINGA APIs into ONNX format, which enables AI developers to use models across different libraries and tools. SINGA supports the time profiling of each of the operators buffered in the graph. Half precision is supported to bring benefits.