Released Neural Network Libraries v1.39.0 and nnabla-rl v0.15.0!

Thursday, May 30, 2024


Posted by Tomonobu Tsujikawa

We have released Neural Network Libraries v1.39.0!
In addition, nnabla-rl has also been updated to v0.15.0 with this release!
Please see “Spotlight” for important changes.


nnabla: Add mlflow autolog to monitors

We have added MLFlow support for nnabla’s monitors.
The Monitors automatically save data as MLflow metrics while training.
Here is a quick example (train.py):

import nnabla.autolog as autolog
import mlflow
from nnabla.monitor import Monitor, MonitorSeries


# Start autolog

# Create monitor
monitor = Monitor('./')
metric_monitor = MonitorSeries("metric", monitor, 1)

with mlflow.start_run():
    # Training-loop
    for i in range(10):
        # Do something here
        metric_monitor.add(i, i * 2)
        # Do something here

# Stop autolog

The below monitors are supported:

  • MonitorSeries
  • MonitorTimeElapsed
  • MonitorImage
  • MonitorImageTile

Make sure mlflow is installed beforehand:

$ pip install mlflow

To automatically save parameters as an artifact, pass with_save_parameters=True.

Check training runs locally through MLflow Tracking:

$ python train.py
$ mlflow ui --port 8080

Visit http://localhost:8080/

Alternatively, if you have an MLflow Tracking server already:

$ export MLFLOW_TRACKING_URI=http://your-tracking-server:port
$ python train.py

Visit http://your-tracking-server:port/

nnabla-rl: Support Gymnasium

We now support Gymnasium environments in this new version of nnabla-rl.
You can now use the Gymnasium environments by importing Gymnasium2GymWrapper.
Here is an example code to run DDPG on Gymnasium pendulum environment:

import gymnasium as gym
from nnabla_rl.environments.wrappers.gymnasium import Gymnasium2GymWrapper

env = gym.make('Pendulum-v1')
# Wrap the environment
env = Gymnasium2GymWrapper(env)

# Now, you can use the Gymnasium environment with nnablaRL algorithms!
config = A.DDPGConfig(start_timesteps=200, gpu_id=0)
ddpg = A.DDPG(train_env, config=config)
ddpg.train(train_env, total_iterations=10000)

A bunch of new algorithms are added to nnabla-rl!

We are excited to announce the addition of the following new algorithms to nnabla-rl:
HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation(HyAR)
Sample-Efficient Reinforcement Learning by Breaking the Replay Ratio Barrier(SRSAC)
AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control(AMP)

Algorithm highlight: AMP

AMP (Adversarial Motion Priors for Stylized Physics-Based Character Control) is a reinforcement learning-based method for generating motion. The motions generated are not only natural but also effectively achieve the given tasks.
We have successfully reproduced the results of the original AMP paper. For more details, see our AMP reproduction code.






Format Converter