From v1.30.0, we started supporting CUDA 11.4 extension.
Please find packages at https://pypi.org/project/nnabla-ext-cuda114.
OpenMPI 4.1.3 is included in multi-gpu docker images with CUDA 11.4 wheel packages, and can be found at https://hub.docker.com/r/nnabla/nnabla-ext-cuda-multi-gpu/tags.
Please note that CUDA 10.2 extension is no longer supported.
We have prepared wheel packages with all necessary CUDA and cuDNN libraries, for both Linux and Windows platform.
The default versions of the packages are CUDA 11.0.3 and cuDNN 8. The names of the packages have
Users can use these wheel packages directly on a GPU-enabled device without installing CUDA toolkit and cuDNN library. Note that a GPU driver that supports the default CUDA version of the packages is still necessary.
If you would like to enable multi-GPU execution with these wheel packages on Linux, you only need to install OpenMPI on a multi-GPU-enabled device. NCCL is not required as it is already packed in the packages.
pip install -U pip pip install https://nnabla.org/whl/nnabla_ext_cuda_alllib/nnabla_ext_cuda_alllib-1.30.0-cp39-cp39-manylinux_2_17_x86_64.whl
Other wheels are listed in https://nnabla.org/install/#all-in-one_list .
We introduce cuTensor-based implementation to acceralate
transpose in CUDA, especially in inner 2D transposition (ex.
transpose(..., axis=(0, 1, 3, 2))).
As an example of actual use case,
transpose used in
channel last layout to
channel first layout conversion (and vice versa) can now be processed faster than before.
Previous cudnn error messages did not point out what actually caused the error. This improvement shows the error code and navigates users to the CUDNN API guide to identify the possible causes of the error.
Redundant error message is suppressed so that the error message becomes more informative.
RuntimeError: target_specific error in cudnn_set_tensor_nd_descriptor_force_dim /home/gitlab-runner/builds/jmdP2aBr/1/nnabla/builders/all/nnabla-ext-cuda/src/nbla/cuda/cudnn/cudnn.cpp:40 Failed `status == CUDNN_STATUS_SUCCESS`: NOT_SUPPORTED
Caught exception "target_specific error in TestBody nnabla-ext-cuda/src/nbla/cuda/test/test_nbla_cuda_utils.cpp:26 CUDNN_STATUS_ALLOC_FAILED occured in `FakeCudnnFunc()`. Please see CUDNN API documentation for the cause.
Prejudice Remover Regularizer has been added for fairness to mitigate data bias in machine learning.
In addition to tabular version, an image version is also available.
We also provide Colab tutorial that enables training with bias mitigation.
|Prejudice Remover Regularizer||Mitigate the model bias with Prejudice Removal Technique|
|Prejudice Remover Regularizer for Images||Mitigate the model bias with Prejudice Removal Technique for Images|
Training code for CLIP is now available on top of the inference script we previously published.
Users can distribute large mini-batches required for contrastive learning across multiple GPUs.
- fix training attribute problem of module
- Fix QNN recorder zero check
- Fix QNN param exists check
- fix proto to info.proto in load.py
- disable dl_nccl_generate when nccl_disabled
- Update nvidia/cuda base image
- Set to use openib by default
- fix crash for coding error in concatenate
- CenterNet code cleanup
- [XAI] update eigencam notebook
- [XAI ]resolve conflict
- Integrate flatc to converter
- Skip FFT/IFFT pytests for win32+CUDA114
- Update scipy I/F from 1.9.x
- allow selection of python to be used when running build_cpplib.bat
- Add benchmark for F.transpose()
- add empty targz and merge cuda wheel
- Fix CUDA_VERSION_MINOR to various condition
- remove flatc build from Dockerfile
- add shape() function and importer
- Implementation of onnx import of quantize resize scatter compress shape
- add onnx exporter groupnormalization
- pack flatc binary files for windows/linux/mac into python wheel
- update version dependency of file format converter
- [Fairness] Prejudice Remover Regularizer for images (part2)
- [Fairness] Rejection Option-based classification on image data(part2)
- Add conditional adm examples
- Bug fixes of OFA(Once-for-All) and add tutorials for OFA
- Fixed missing regularizer arguments
- Bug fixes of OFA DynamicBN layer
- Add proper extra_repr to all OFA modules.
- Bug fixes and refactoring of OFA utilities
- Fix parameters not being inherited.
- Fixed OFA searcher to get correct net parameters with keys
- fixed jobs.sh
- Update imagenet.yaml
- f-string compatibility with Python version < 3.8
- Fix mismatching order of params of AvgPool
- Add the functions to save checkpoint and restart from it
- Get regulizers out of the optimizers dictionary.
- Extract elastic-nn-related codes to common packages for better code reuse
- Support nnabla.no_grad etc.
- Implementation of mixed operations fast mode
- Add OFA-Xception search space
- Add OFA-ResNet50
- [OFA] Update Model Zoo README with results on imagenet
- updated docs, readme and tutorials and gave timestamps to the config logs
- save arch.json when save-nnp option is specified.
- add image augmentation for csv dataloader
- add no_image_normalization param in executor when saving nnp
- Update DALI Pipeline to the latest version for the ImageNet dataset
- Enable save nnp function for OFA searcher
- Switch to hydra to handle configurations