We have Released Neural Network Libraries v1.14.0! We now support CUDA 11 / cuDNN 8 and we have also added new models for speech synthesis, as well as new solver!

Spotlight

Support CUDA 11 and cuDNN 8

We now support CUDA 11 and cuDNN 8!
CUDA 11 supports new generation of NVIDIA Ampere GPU architecture, and introduces new tensor cores to enhance mixed precision across different data types including TF32.
cuDNN 8 also introduces new and enhanced features, such as optimized grouped convolutions and 3D convolutions, operation descriptors, etc.

Speech Synthesis models: Tacotron1 & 2 / WaveGlow

We have added new speech synthesis models! Tacotron 1 and Tacotron 2 are text-to-speech synthesis system that synthesizes speech directly from a sequence of characters.

WaveGlow is a flow-based network capable of generating high quality speech from mel spectrograms, trained by maximizing the likelihood of the training data.

Along with vision and language, speech has also become an important field for which deep learning can be applied, and we will continue to add new models for speech recognition!

AdaBelief solver

AdaBelief is an optimizer proposed in this paper, aiming to achieve fast convergence of adaptive models, while retaining good generalization of stochastic gradient descent (SGD), as well as training stability, by adapting the stepsize according to the “belief” in current gradient direction.

Layers

• Norm and NormNormalization (CPU / CUDA)
• Add Adabelief solver proposed by Juntang Zhuang (CPU / CUDA)
• Add ScatterAdd Function (CPU / CUDA)
• Inplace for unary/binary arithmetic functions (CPU / CUDA)

Build

• Limit onnx version to 1.6.0
• Change libarchive location (CPU / CUDA)
• Build with chocolatey environment. (CPU / CUDA)
• Change Dockerfile for runtime.
• Improve building Dockerfile
• Update Dockerfile for CUDA10.2
• update Dockerfile to install nnabla_tf_converter (CPU / CUDA)

• separate tf converter from nnabla (CPU / CUDA)

• remove python3.5

• Upgrade pip version.
• feat: update api_level version from nnabla

Format Conversion

• Fix missing parameters from nnp to nnb.
• Enabla GRU for c-runtime.
• fix error when no .protobuf file and parameter [] operator.

Examples

• Tacotron
• Tacotron2
• WaveGlow

Documentation

• update cuda cudnn table
• fix document error for to_dlpack

Bugfix

• Fix norm api level and function id
• Fix issue that BatchMatmul adds new features to cause the converter t…
• Use the new h5py v3 data access method.
• Add missing format character in function parameter check.
• Fix the docs related to the output maps
• BugFix: Index of height and width in the channel-last of RandomErase function
• Limit max threads to values per kernel function and device.