We have released Neural Network Libraries 1.5.0! So many exciting features and models have been added, including transformer, weight normalization, Pix2PixHD, ESR-GAN, MAML, etc!
Also, please note that Python 2 will no longer be supported from v1.5.0, as we announced before.
Spotlight
Transformer
Transformer (Vaswani et al., NIPS 2017)
Transformer is a highly popular model based on multi-head attention mechanism, and has been shown to be powerful in a variety of machine learning tasks, particularly in natural language processing and speech recognition. It also provides a foundation upon which many of the recent state-of-the-art models, such as BERT, are built. Tutorial on how to use transformers with Neural Network Libraries is coming soon, so stay tuned!
Weight Normalization
Weight normalization (Salimans and Kingma, NIPS 2016)
Weight normalization is a reparametrization technique for the weight parameters in neural networks that speeds up the optimization.
Pix2PixHD
Pix2PixHD (Wang et al., CVPR 2018) performs high-resolution image-to-image translation. For example, you can convert semantic segmentation maps to photorealistic images as in example below:
| Input | Output | |
|---|---|---|
 |
 |
|
MAML
MAML (Model-Agnostic Meta-Learning) (Finn et al., ICML 2017) is a seminal work in meta-learning, which seeks to train a model that learns how to learn, thus quickly adapting to new tasks with limited samples.
ESRGAN inference
ESRGAN (Wang et al, ECCV 2018 Workshop) performs an enhanced single image super-resolution. Inference can be performed with Neural Network Libraries by converting the pre-trained weights available in pytorch, which also can be done using our code.
Layers
- WarpByFlow function
- Implement batch_det
- BatchNorm backward in test mode
- Add batchinv function.
- Allow clip_by_value to get scalar values as threshold
- Orthogonal Initializer
- Add CUDA implementation for warp_by_flow function.
Utilities/Format Conversion
- Remove binary-only option from pip in Dockerfile.
- Add opset 10,11 support for ONNX Importer.
- Update API LEVEL
- Improve ONNX Exporter for SNPE.
- Add opset 10,11 support for ONNX Exporter.
- Remove redundant Reshape operations.
- Add ONNX op support for onnx importer:
- Add NNabla func support for onnx exporter:
- Add multi datasets save&load testcase
- Remove binary-only option from pip in Dockerfile.
Examples
- Add pretrained models to StarGAN
- Add save_nnp and checkpoint script.
- Removed unnecessary options grad_only=False in MAML and some modifications
- nnp checkpoint in deeplabv3plus
- checkpoint and nnp for imagenet
Bug Fix/Documentation
- [Fix] Add Missing Functions/Solvers in Doc.
- Fix unaligned word data.
- Fix nnabla.proto save problem
- Fix and improve debug assertions in ndi helper functions
- Fix coding error.
- Add basic/advanced usage of backward
- Added guidelines for auto format
- Add quantization document
- Suppress unused variable warnings at global_average_pooling.cpp
- added formated code
