We have released Neural Network Libraries v1.19.0!

Spotlight

Memory usage optimization

We have made the following optimizations to reduce memory usage during training of arbitrary networks:
– all functions delete input data unnecessary for backward computation after running forward computation.
– initial values of gradient data automatically created by the graph engine at the beginning of backward computation are deleted immediately after use.
– memory sharing (inplace) is performed separately for data and gradients, so that gradient data can be deleted more aggressively during backward.

References: 1, 2, 3

Bugfix: grad computation of linear functions

We have noticed implementation mistakes with our higher-order grandient descent added in v1.18.0, and have fixed the problem.

fix nnp_graph load hang up

We have noticed some cases of failing to load nnp files when using pre-trained models, and have fixed the problem.

JSI-GAN

JSI-GAN is a model that performs super-resolution and inverse tone mapping simultaneously, converting low-resolution, low dynaimc range image to high-resolution and high dynamic range. The model consists of three task-specific subnets; namely, image reconstruction (IR) subnet that reconstructs coarse HR HDR image, detail restoration (DR) subnet that restores details to be added on the coarse image, and a local contrast enhancement (LCE) subnet that restores the local contrast.

Bugfix

• Bug fix: Correct default arguments of F.norm_normalization
• Make data_iterator_simple simpler
• fix the problem when save repeated float arguments
• Fix error of saving model and enable variable_batch_size.
• fix overflow in reshape
• fix nnp_graph load hang up
• Fix numerical instability for log_sigmoid and softplus (CPU / GPU)
• remove inplace_grad() related code
• Bugfix: grad computation of linear functions
• fix nnp refactor memory problem
• Fix issues of the converter.
• Change workspace limit dtype from int to int64_t
• fix conv segment fault error

Build

• remove conda (CPU / GPU)
• remove conda from dockerfile
• Fix download failure on Windows

Core

• clears input buffer not needed for after propagation (CPU / GPU)
• Clear grad after propagation

Documentation

• Update nnabla converter document for release nnabla
• Update nnabla converter document for release nnabla

Format Conversion

• Support export to tensorflow frozen graph again.

Layers

• F.cumsum and F.cumprod (CPU / GPU)
• fused binary operator (CPU / GPU)

Utilities

• Add stop_exhausted option for data_iterator

Examples

• remove conda from dockerfile
• Add Zooming Slow-mo Colab
• Add JSI-GAN example
• Stylegan2 inference support for multiple datasets
• stylegan2 distillation pretrained weights url
• Data Cleansing with Storage-efficient Approximation of Influence Functions