We have released Neural Network Libraries 1.3.0! Multi-head attention layer and Deep Q learning (DQN) example have been added!

Spotlight

MultiHead Attention

Multi-head attention layer provides a building block component for transformer, which has become a state-of-the-art model for many tasks such as language modeling. It is also being actively applied to a variety of computer vision tasks.

Deep Q-Learning

DQN is one of the most widely used deep reinforcement learning (RL) algorithms. Check this blog post for more details!

Layers​

• Min-max quantization
• Use thrust async sort and custom make_seqeuence for faster operation.
• Update FlipCuda and RandomFlipCuda implementation

Utilities

• enable down-version by –api option

Examples

• Add PSM-net training example
• DeepLabV3+: Fix default image size arguments
• Example of Min-max quantization
• An example of fine-tuning YOLOv2 model on smaller dataset
  ​

Bug Fix / Documentation

• Fix min-max-quantize
• Added missing imports in imagenet model doc
• Update nnabla converter support status doc.

NOTE

Asynchronous sort support in CUDA 10.1 is known to cause segmentation fault due to an improper concurrency. This issue will be handled by the next release.