Released Neural Network Libraries 1.5.0!

Tuesday, January 28, 2020


Posted by shin

We have released Neural Network Libraries 1.5.0! So many exciting features and models have been added, including transformer, weight normalization, Pix2PixHD, ESR-GAN, MAML, etc!
Also, please note that Python 2 will no longer be supported from v1.5.0, as we announced before.



Transformer (Vaswani et al., NIPS 2017)
Transformer is a highly popular model based on multi-head attention mechanism, and has been shown to be powerful in a variety of machine learning tasks, particularly in natural language processing and speech recognition. It also provides a foundation upon which many of the recent state-of-the-art models, such as BERT, are built. Tutorial on how to use transformers with Neural Network Libraries is coming soon, so stay tuned!

Weight Normalization

Weight normalization (Salimans and Kingma, NIPS 2016)
Weight normalization is a reparametrization technique for the weight parameters in neural networks that speeds up the optimization.


Pix2PixHD (Wang et al., CVPR 2018) performs high-resolution image-to-image translation. For example, you can convert semantic segmentation maps to photorealistic images as in example below:

Input Output


MAML (Model-Agnostic Meta-Learning) (Finn et al., ICML 2017) is a seminal work in meta-learning, which seeks to train a model that learns how to learn, thus quickly adapting to new tasks with limited samples.

ESRGAN inference

ESRGAN (Wang et al, ECCV 2018 Workshop) performs an enhanced single image super-resolution. Inference can be performed with Neural Network Libraries by converting the pre-trained weights available in pytorch, which also can be done using our code.


Utilities/Format Conversion


Bug Fix/Documentation​