We have released Neural Network Libraries v1.39.0!
In addition, nnabla-rl has also been updated to v0.15.0 with this release!
Please see “Spotlight” for important changes.
Spotlight
nnabla: Add mlflow autolog to monitors
We have added MLFlow support for nnabla’s monitors.
The Monitors automatically save data as MLflow metrics while training.
Here is a quick example (train.py):
import nnabla.autolog as autolog
import mlflow
from nnabla.monitor import Monitor, MonitorSeries
mlflow.set_experiment("a-quick-example")
# Start autolog
autolog.autolog(with_save_parameters=True)
# Create monitor
monitor = Monitor('./')
metric_monitor = MonitorSeries("metric", monitor, 1)
with mlflow.start_run():
# Training-loop
for i in range(10):
# Do something here
metric_monitor.add(i, i * 2)
# Do something here
# Stop autolog
autolog.stop_autolog()
The below monitors are supported:
- MonitorSeries
- MonitorTimeElapsed
- MonitorImage
- MonitorImageTile
Make sure mlflow is installed beforehand:
$ pip install mlflow
To automatically save parameters as an artifact, pass with_save_parameters=True.
Check training runs locally through MLflow Tracking:
$ python train.py
$ mlflow ui --port 8080
Visit http://localhost:8080/
Alternatively, if you have an MLflow Tracking server already:
$ export MLFLOW_TRACKING_URI=http://your-tracking-server:port
$ python train.py
Visit http://your-tracking-server:port/
nnabla-rl: Support Gymnasium
We now support Gymnasium environments in this new version of nnabla-rl.
You can now use the Gymnasium environments by importing Gymnasium2GymWrapper.
Here is an example code to run DDPG on Gymnasium pendulum environment:
import gymnasium as gym
from nnabla_rl.environments.wrappers.gymnasium import Gymnasium2GymWrapper
env = gym.make('Pendulum-v1')
# Wrap the environment
env = Gymnasium2GymWrapper(env)
# Now, you can use the Gymnasium environment with nnablaRL algorithms!
config = A.DDPGConfig(start_timesteps=200, gpu_id=0)
ddpg = A.DDPG(train_env, config=config)
ddpg.train(train_env, total_iterations=10000)
A bunch of new algorithms are added to nnabla-rl!
We are excited to announce the addition of the following new algorithms to nnabla-rl:
– HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation(HyAR)
– Sample-Efficient Reinforcement Learning by Breaking the Replay Ratio Barrier(SRSAC)
– AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control(AMP)
Algorithm highlight: AMP
AMP (Adversarial Motion Priors for Stylized Physics-Based Character Control) is a reinforcement learning-based method for generating motion. The motions generated are not only natural but also effectively achieve the given tasks.
We have successfully reproduced the results of the original AMP paper. For more details, see our AMP reproduction code.
nnabla
BugFix
- fix bug of network expander for duplicated name
- bugfix for optimizer state recovery
- fix atomicInc called too many times causes rewind
Utilities
- save executors in graph_def
- Allow to set the parameter file format
- Add code organization framework
- add mlflow save and load model support
Build
- add include cctype to avoid error in specified platform
- fix build errors to upgrade cython (CPU / GPU)
- limit matplotlib version for windows
Format Converter
- Faster Split when input share is large on onnx importer
- Add test when input shape changes
- limit tensorflow version lower than 2.15
Document
