We are very excited today to open-source Sony’s neural network libraries, a software that helps the workflows of deep learning research, development and production.
Neural networks are the core ingredients of deep learning models. Deep learning has first received huge attention in 2012, when an image classification model accomplished a great leap in image recognition, winning against other models with a large gap, in the ImageNet Large Scale Visual Recognition Challenge. Nowadays, deep learning is widely used in many applications as an essential tool, not only as a pattern recognition algorithm, but also as a tool capable of modeling black-box systems.
The architectures of deep learning models vary at a wide range, in various aspects; from small to large, from feed-forward to recurrent, from unsupervised to supervised and so on. The research moves forward very rapidly, and new frameworks, or building blocks, of neural networks are invented on a daily basis. Also, while current deep learning frameworks with high-performance computers or clusters equipped with GPUs, the demand for running deep learning architectures on smartphones, IoT devices, and home consumer products are rising rapidly.
At Sony, we have been working on research and development of deep learning since 2010. The core libraries we are open-sourcing today is the third generation of our internally developed deep learning framework, which has re-birthed many times, each time reflecting our experiences and knowledge of deep learning research, development, and production.
The core libraries provide the following functionalities and features.
- CUDA compatibility.
- A Python API that maximizes the flexibility of the design of neural network models, and encourages fast prototyping and experimentation.
- Support for both static and dynamic computation graphs. The static computation graph is more efficient in both speed and memory, while the dynamic graph is more flexible in designing models.
- Various built-in neural network modules, such as functions, operators and optimizers. The modular software architecture of our libraries allows developers to add new modules for rapid prototyping of new research ideas, products or services.
- A portable, lightweight C++11 core to run on a variety of platforms. The framework is tested on Linux (Ubuntu 16.04) and Windows (8, 10).
- High speed and memory efficiency. The computation graph engine allows safe in-place computation and memory sharing, which drastically reduces memory consumption. The design allows execution of forward and backward propagation with less computation overheads.
- Decoupled implementation of interfaces for various computation schemes, allowing a plug-in style development scheme for each computation scheme. Developers can focus on their special implementation of the actual computation of function operators, rather than the computation graph engine and its memory optimization etc.
A more detailed explanation and usage of the library features at this blog soon.
Our libraries provide essential functionalities of deep learning research, development and deployment. We are ambitious to push the boundaries of artificial intelligence, both in the field of advanced researches and cutting-edge applications, through development of our library.
We thank all the researchers and developers around the world involved in the deep learning research, and everyone at Sony involved in this open source project.
Links:
