We have released Neural Network Libraries v1.27.0!
Along with our update, NNabla NAS has also been updated to v0.12.0. Furthermore, nnabla-browser, a tool for visualizing Neural Network Libraries, has been released! We will shortly upload a blog post introducing each of these updates!
Spotlight
LAMB Solver (CPU / GPU)
We have implemented LAMB, a solver designed for layerwise adaptive large batch optimization. LAMB has been used in a number of benchmark tasks that require large batch sizes, such as BERT or ResNet-50 training. We have corrected definition of SGDW, and updated the Solver base class design to support fusion of weight decay operation into update operation.
Add batch cholesky function (CPU / GPU)
We have implemented batch Cholesky function. We have referred to this paper for implementation details. Note that, while cholesky decomposition requires only the upper/lower part of the matrix, the backward of this function returns symmetric matrix for convenience.
Speedup SortCuda with cub::DeviceSegmentedRadixSort
We have replaced the sort library in SortCUDA from Thrust to CUB, achieving about 25 times better optimization.
[XAI] Eigencam example
We have implemented colab interactive demo for Eigen-CAM, which computes and visualizes the principle components of the learned features/representations from the convolutional layers.
| Name | Notebook | Task |
|---|---|---|
| Eigen-CAM |  |
Visual Explanation |
