Tutorials for using the Unity plugin

[Balint-H]

The Unity plugin is not only a great for constructing MuJoCo scenes visually and efficiently, but also enables the use of MuJoCo with the beginner-friendly control learning toolkit, ML-Agents. However, if someone is only familiar with either MuJoCo or Unity, I think a couple of examples of the interaction of the two could go a long way to get someone started.

Unity has a set of in-editor interactive tutorial authoring tools, which gave me the idea to create a step-by-step guide for importing and editing a MuJoCo scene, and preparing it for RL with ML-Agents. Currently this is only done for one example (a cartpole), but I'm planning to also add one for locomotion. Here's the link for the WIP project: https://github.com/Balint-H/mj-unity-tutorial . Feel free to contribute if you have ideas for suitable content! Particularly, it would be good to prepare the project so it is easier to use on different OSs.

I'd appreciate it if someone has feedback for a project like this. If you've used the plugin before, were there any quirks or important details you wish you had help figuring out? Are there any specific type of learning environments that you think would be a useful example? Who do you think would be the target audience for example scenes like this?

If you considered trying the plugin but decided against it, what was that convinced you otherwise?

[juanmed]

I would be interested in a tutorial that shows a basic example on how to use the Unity plug-in for rendering only, as described here. Ideally a C++ renderer that connects with unity, if not a python script that does the same.