We have released Neural Network Libraries v1.18.0!

Higher-order gradients for functions are now available, and we have also implemented SLE-GAN!

# Spotlight

## Higher-Order Gradients (GPU)

We have revamped our current double-backward APIs by constructing advanced gradient computation graph with existing functions, enabling **n-th order gradients** (any order gradients or infinite times gradients)!

You can use higher-order gradients as following:

```
x = nn.Variable.from_numpy_array(np.random.randn(2, 2)).apply(need_grad=True)
x.grad.zero()
y = F.sin(x)
def grad(y, x, n=1):
dx = [y]
for _ in range(n):
dx = nn.grad([dx[0]], [x])
return dx[0]
dnx = grad(y, x, n=10)
dnx.forward()
print(np.allclose(-np.sin(x.d), dnx.d))
dnx.backward()
print(np.allclose(-np.cos(x.d), x.g))
```

Users can also see the registry, and register the grad function of their own implementation as following:

```
# Show the registry status
from nnabla.backward_functions import show_registry
show_registry()
# Register the grad function of a function
register("<FancyFunction>", <fancy_function>_backward)
```

## SLE-GAN

We have also implemented SLE-GAN from ICLR 2021, a light-weight generative model for high-fidelity image synthesis, which enables efficient few-shot training from around 100 training samples, with roughly half the size of StyleGAN2’s parameters.

### Layers

### Utilities

- Support conversion tflite to nnabla.

release-note-utility - Fix to support utf-8 enconding with BOM
- Support BatchNorm1d for ONNX importer.
- Add feature for nnabla_cli
- Add OOC option to nnabla_cli train
- Support comment columns in dataset CSV files

### Examples

### Bugfix / Documentation

- Remove redundant input for some funcs.
- fix -1 axis error and grad not found
- fix coding error for mutex setting
- improve grad() with filter out dead parameters
- Increase gpu speed threshold
- Fix nnabla Nan error in training.
- fix: solve the error that pynvml cannot solve the path to nvml.dll on windows
- fix: solve the error that pynvml cannot solve the path to nvml.dll on windows
- Load bias grad only if convolution has bias.
- fix key error of checking GPU
- Bug fix
- dali latest version