Released Neural Network Libraries v1.27.0!

Wednesday, April 13, 2022


Posted by Takuya Yashima

We have released Neural Network Libraries v1.27.0!
Along with our update, NNabla NAS has also been updated to v0.12.0. Furthermore, nnabla-browser, a tool for visualizing Neural Network Libraries, has been released! We will shortly upload a blog post introducing each of these updates!


LAMB Solver (CPU / GPU)

We have implemented LAMB, a solver designed for layerwise adaptive large batch optimization. LAMB has been used in a number of benchmark tasks that require large batch sizes, such as BERT or ResNet-50 training. We have corrected definition of SGDW, and updated the Solver base class design to support fusion of weight decay operation into update operation.

Add batch cholesky function (CPU / GPU)

We have implemented batch Cholesky function. We have referred to this paper for implementation details. Note that, while cholesky decomposition requires only the upper/lower part of the matrix, the backward of this function returns symmetric matrix for convenience.

Speedup SortCuda with cub::DeviceSegmentedRadixSort

We have replaced the sort library in SortCUDA from Thrust to CUB, achieving about 25 times better optimization.

[XAI] Eigencam example

We have implemented colab interactive demo for Eigen-CAM, which computes and visualizes the principle components of the learned features/representations from the convolutional layers.

Name Notebook Task
Eigen-CAM Open In Colab Visual Explanation




Format converter