Release v0.5 (#1202)

.gitignore

@@ -1,29 +1,30 @@
-/unity-environment/[Ll]ibrary/
-/unity-environment/[Tt]emp/
-/unity-environment/[Oo]bj/
-/unity-environment/[Bb]uild/
-/unity-environment/[Bb]uilds/
-/unity-environment/[Pp]ackages/
-/unity-environment/[Uu]nity[Pp]ackage[Mm]anager/
-/unity-environment/Assets/AssetStoreTools*
-/unity-environment/Assets/Plugins*
-/unity-environment/Assets/Gizmos*
+/UnitySDK/[Ll]ibrary/
+/UnitySDK/[Tt]emp/
+/UnitySDK/[Oo]bj/
+/UnitySDK/[Bb]uild/
+/UnitySDK/[Bb]uilds/
+/UnitySDK/[Pp]ackages/
+/UnitySDK/[Uu]nity[Pp]ackage[Mm]anager/
+/UnitySDK/Assets/AssetStoreTools*
+/UnitySDK/Assets/Plugins*
+/UnitySDK/Assets/Gizmos*
 
 # Tensorflow Model Info
 /models
 /summaries
-python/models
-python/summaries
+
+# Training environments
+/envs
 
 # Environemnt logfile
-*unity-environment.log
+*UnitySDK.log
 
 # Visual Studio 2015 cache directory
-/unity-environment/.vs/
+/UnitySDK/.vs/
 
 # Autogenerated VS/MD/Consulo solution and project files
-/unity-environmentExportedObj/
-/unity-environment.consulo/
+/UnitySDKExportedObj/
+/UnitySDK.consulo/
 *.csproj
 *.unityproj
 *.sln
@@ -40,7 +41,7 @@ python/summaries
 *.pidb.meta
 
 # Unity3D Generated File On Crash Reports
-/unity-environment/sysinfo.txt
+/UnitySDK/sysinfo.txt
 
 # Builds
 *.apk
@@ -51,10 +52,10 @@ python/summaries
 *.x86
 
 # Tensorflow Sharp Files
-/unity-environment/Assets/ML-Agents/Plugins/Android*
-/unity-environment/Assets/ML-Agents/Plugins/iOS*
-/unity-environment/Assets/ML-Agents/Plugins/Computer*
-/unity-environment/Assets/ML-Agents/Plugins/System*
+/UnitySDK/Assets/ML-Agents/Plugins/Android*
+/UnitySDK/Assets/ML-Agents/Plugins/iOS*
+/UnitySDK/Assets/ML-Agents/Plugins/Computer*
+/UnitySDK/Assets/ML-Agents/Plugins/System*
 
 # Generated doc folders
 /docs/html
@@ -75,5 +76,20 @@ python/summaries
 *.eggs*
 *.gitignore.swp
 
+# VSCode hidden files
+*.vscode/
 
 .DS_Store
+.ipynb_checkpoints
+
+# pytest cache 
+*.pytest_cache/
+
+# Ignore compiled protobuf files.
+ml-agents-protobuf/cs
+ml-agents-protobuf/python
+ml-agents-protobuf/Grpc*
+
+# Ignore PyPi build files.
+dist/
+build/

CODE_OF_CONDUCT.md

@@ -67,7 +67,8 @@ members of the project's leadership.
 
 ## Attribution
 
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
-available at https://www.contributor-covenant.org/version/1/4/code-of-conduct/
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 1.4, available at
+https://www.contributor-covenant.org/version/1/4/code-of-conduct/
 
 [homepage]: https://www.contributor-covenant.org

CONTRIBUTING.md

@@ -1,54 +1,56 @@
 # Contribution Guidelines
 
-Thank you for your interest in contributing to the ML-Agents toolkit! We are incredibly
-excited to see how members of our community will use and extend the ML-Agents toolkit.
-To facilitate your contributions, we've outlined a brief set of guidelines
-to ensure that your extensions can be easily integrated.
+Thank you for your interest in contributing to the ML-Agents toolkit! We are
+incredibly excited to see how members of our community will use and extend the
+ML-Agents toolkit. To facilitate your contributions, we've outlined a brief set
+of guidelines to ensure that your extensions can be easily integrated.
 
-### Communication
+## Communication
 
-First, please read through our [code of conduct](CODE_OF_CONDUCT.md), 
-as we expect all our contributors to follow it.
+First, please read through our [code of conduct](CODE_OF_CONDUCT.md), as we
+expect all our contributors to follow it.
 
-Second, before starting on a project that you intend to contribute
-to the ML-Agents toolkit (whether environments or modifications to the codebase), 
-we **strongly** recommend posting on our 
-[Issues page](https://github.com/Unity-Technologies/ml-agents/issues) and
-briefly outlining the changes you plan to make. This will enable us to provide
-some context that may be helpful for you. This could range from advice and 
-feedback on how to optimally perform your changes or reasons for not doing it.
+Second, before starting on a project that you intend to contribute to the
+ML-Agents toolkit (whether environments or modifications to the codebase), we
+**strongly** recommend posting on our
+[Issues page](https://github.com/Unity-Technologies/ml-agents/issues)
+and briefly outlining the changes you plan to make. This will enable us to
+provide some context that may be helpful for you. This could range from advice
+and feedback on how to optimally perform your changes or reasons for not doing
+it.
 
 Lastly, if you're looking for input on what to contribute, feel free to
 reach out to us directly at [email protected] and/or browse the GitHub
 issues with the `contributions welcome` label.
 
-### Git Branches
+## Git Branches
 
-Starting with v0.3, we adopted the 
+Starting with v0.3, we adopted the
 [Gitflow Workflow](http://nvie.com/posts/a-successful-git-branching-model/).
-Consequently, the `master` branch corresponds to the latest release of 
+Consequently, the `master` branch corresponds to the latest release of
 the project, while the `develop` branch corresponds to the most recent, stable,
 version of the project.
 
 Thus, when adding to the project, **please branch off `develop`**
 and make sure that your Pull Request (PR) contains the following:
+
 * Detailed description of the changes performed
-* Corresponding changes to documentation, unit tests and sample environments 
-(if applicable)
+* Corresponding changes to documentation, unit tests and sample environments (if
+  applicable)
 * Summary of the tests performed to validate your changes
 * Issue numbers that the PR resolves (if any)
 
-### Environments
+## Environments
 
-We are also actively open to adding community contributed environments as 
-examples, as long as they are small, simple, demonstrate a unique feature of 
-the platform, and provide a unique non-trivial challenge to modern 
+We are also actively open to adding community contributed environments as
+examples, as long as they are small, simple, demonstrate a unique feature of
+the platform, and provide a unique non-trivial challenge to modern
 machine learning algorithms. Feel free to submit these environments with a
-PR explaining the nature of the environment and task. 
+PR explaining the nature of the environment and task.
 
-### Style Guide
+## Style Guide
 
-When performing changes to the codebase, ensure that you follow the style
-guide of the file you're modifying. For Python, we follow 
-[PEP 8](https://www.python.org/dev/peps/pep-0008/). For C#, we will soon be
-adding a formal style guide for our repository.
+When performing changes to the codebase, ensure that you follow the style guide
+of the file you're modifying. For Python, we follow
+[PEP 8](https://www.python.org/dev/peps/pep-0008/).
+For C#, we will soon be adding a formal style guide for our repository.

Dockerfile

@@ -122,14 +122,11 @@ RUN apt-get update && apt-get -y upgrade
 # xvfb is used to do CPU based rendering of Unity
 RUN apt-get install -y xvfb
 
-
-ADD python/requirements.txt .
-RUN pip install --trusted-host pypi.python.org -r requirements.txt
-
-WORKDIR /execute
-COPY python /execute/python
+COPY ml-agents /ml-agents
+WORKDIR /ml-agents
+RUN pip install .
 
 # port 5005 is the port used in in Editor training.
 EXPOSE 5005
 
-ENTRYPOINT ["python", "python/learn.py"]
+ENTRYPOINT ["mlagents-learn"]

README.md

@@ -1,76 +1,86 @@
 <img src="docs/images/unity-wide.png" align="middle" width="3000"/>
 
+<img src="docs/images/image-banner.png" align="middle" width="3000"/>
+
 # Unity ML-Agents Toolkit (Beta)
 
-**The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source Unity plugin 
-that enables games and simulations to serve as environments for training
-intelligent agents. Agents can be trained using reinforcement learning,
-imitation learning, neuroevolution, or other machine learning methods through
-a simple-to-use Python API. We also provide implementations (based on
-TensorFlow) of state-of-the-art algorithms to enable game developers
-and hobbyists to easily train intelligent agents for 2D, 3D and VR/AR games.
-These trained agents can be used for multiple purposes, including
-controlling NPC behavior (in a variety of settings such as multi-agent and
-adversarial), automated testing of game builds and evaluating different game
-design decisions pre-release. The ML-Agents toolkit is mutually beneficial for both game
-developers and AI researchers as it provides a central platform where advances
-in AI can be evaluated on Unity’s rich environments and then made accessible
-to the wider research and game developer communities. 
+**The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source
+Unity plugin that enables games and simulations to serve as environments for
+training intelligent agents. Agents can be trained using reinforcement learning,
+imitation learning, neuroevolution, or other machine learning methods through a
+simple-to-use Python API. We also provide implementations (based on TensorFlow)
+of state-of-the-art algorithms to enable game developers and hobbyists to easily
+train intelligent agents for 2D, 3D and VR/AR games. These trained agents can be
+used for multiple purposes, including controlling NPC behavior (in a variety of
+settings such as multi-agent and adversarial), automated testing of game builds
+and evaluating different game design decisions pre-release. The ML-Agents
+toolkit is mutually beneficial for both game developers and AI researchers as it
+provides a central platform where advances in AI can be evaluated on Unity’s
+rich environments and then made accessible to the wider research and game
+developer communities.
 
 ## Features
+
 * Unity environment control from Python
 * 10+ sample Unity environments
 * Support for multiple environment configurations and training scenarios
-* Train memory-enhanced Agents using deep reinforcement learning
+* Train memory-enhanced agents using deep reinforcement learning
 * Easily definable Curriculum Learning scenarios
-* Broadcasting of Agent behavior for supervised learning
+* Broadcasting of agent behavior for supervised learning
 * Built-in support for Imitation Learning
-* Flexible Agent control with On Demand Decision Making
+* Flexible agent control with On Demand Decision Making
 * Visualizing network outputs within the environment
 * Simplified set-up with Docker
+* Wrap learning environments as a gym
 
 ## Documentation
 
-* For more information, in addition to installation and usage
-instructions, see our [documentation home](docs/Readme.md).
-* If you have
-used a version of the ML-Agents toolkit prior to v0.4, we strongly recommend 
-our [guide on migrating from earlier versions](docs/Migrating.md).
+* For more information, in addition to installation and usage instructions, see
+  our [documentation home](docs/Readme.md).
+* If you are a researcher interested in a discussion of Unity as an AI platform, see a pre-print of our [reference paper on Unity and the ML-Agents Toolkit](https://arxiv.org/abs/1809.02627). Also, see below for instructions on citing this paper.
+* If you have used a version of the ML-Agents toolkit prior to v0.5, we strongly
+  recommend our [guide on migrating from earlier versions](docs/Migrating.md).
 
-## References
+## Additional Resources
 
 We have published a series of blog posts that are relevant for ML-Agents:
-- Overviewing reinforcement learning concepts
-([multi-armed bandit](https://blogs.unity3d.com/2017/06/26/unity-ai-themed-blog-entries/)
-and [Q-learning](https://blogs.unity3d.com/2017/08/22/unity-ai-reinforcement-learning-with-q-learning/))
-- [Using Machine Learning Agents in a real game: a beginner’s guide](https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/)
-- [Post](https://blogs.unity3d.com/2018/02/28/introducing-the-winners-of-the-first-ml-agents-challenge/) announcing the winners of our
-[first ML-Agents Challenge](https://connect.unity.com/challenges/ml-agents-1)
-- [Post](https://blogs.unity3d.com/2018/01/23/designing-safer-cities-through-simulations/)
-overviewing how Unity can be leveraged as a simulator to design safer cities.
+
+* Overviewing reinforcement learning concepts
+  ([multi-armed bandit](https://blogs.unity3d.com/2017/06/26/unity-ai-themed-blog-entries/)
+  and
+  [Q-learning](https://blogs.unity3d.com/2017/08/22/unity-ai-reinforcement-learning-with-q-learning/))
+* [Using Machine Learning Agents in a real game: a beginner’s guide](https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/)
+* [Post](https://blogs.unity3d.com/2018/02/28/introducing-the-winners-of-the-first-ml-agents-challenge/)
+  announcing the winners of our
+  [first ML-Agents Challenge](https://connect.unity.com/challenges/ml-agents-1)
+* [Post](https://blogs.unity3d.com/2018/01/23/designing-safer-cities-through-simulations/)
+  overviewing how Unity can be leveraged as a simulator to design safer cities.
 
 In addition to our own documentation, here are some additional, relevant articles:
-- [Unity AI - Unity 3D Artificial Intelligence](https://www.youtube.com/watch?v=bqsfkGbBU6k)
-- [A Game Developer Learns Machine Learning](https://mikecann.co.uk/machine-learning/a-game-developer-learns-machine-learning-intent/)
-- [Explore Unity Technologies ML-Agents Exclusively on Intel Architecture](https://software.intel.com/en-us/articles/explore-unity-technologies-ml-agents-exclusively-on-intel-architecture)
+
+* [Unity AI - Unity 3D Artificial Intelligence](https://www.youtube.com/watch?v=bqsfkGbBU6k)
+* [A Game Developer Learns Machine Learning](https://mikecann.co.uk/machine-learning/a-game-developer-learns-machine-learning-intent/)
+* [Explore Unity Technologies ML-Agents Exclusively on Intel Architecture](https://software.intel.com/en-us/articles/explore-unity-technologies-ml-agents-exclusively-on-intel-architecture)
 
 ## Community and Feedback
 
-The ML-Agents toolkit is an open-source project and we encourage and welcome contributions.
-If you wish to contribute, be sure to review our 
-[contribution guidelines](CONTRIBUTING.md) and 
+The ML-Agents toolkit is an open-source project and we encourage and welcome
+contributions. If you wish to contribute, be sure to review our
+[contribution guidelines](CONTRIBUTING.md) and
 [code of conduct](CODE_OF_CONDUCT.md).
 
 You can connect with us and the broader community
 through Unity Connect and GitHub:
+
 * Join our
-[Unity Machine Learning Channel](https://connect.unity.com/messages/c/035fba4f88400000)
-to connect with others using the ML-Agents toolkit and Unity developers enthusiastic
-about machine learning. We use that channel to surface updates
-regarding the ML-Agents toolkit (and, more broadly, machine learning in games).
-* If you run into any problems using the ML-Agents toolkit, 
-[submit an issue](https://github.com/Unity-Technologies/ml-agents/issues) and
-make sure to include as much detail as possible.
+  [Unity Machine Learning Channel](https://connect.unity.com/messages/c/035fba4f88400000)
+  to connect with others using the ML-Agents toolkit and Unity developers
+  enthusiastic about machine learning. We use that channel to surface updates
+  regarding the ML-Agents toolkit (and, more broadly, machine learning in
+  games).
+* If you run into any problems using the ML-Agents toolkit,
+  [submit an issue](https://github.com/Unity-Technologies/ml-agents/issues) and
+  make sure to include as much detail as possible.
 
 For any other questions or feedback, connect directly with the ML-Agents
 team at [email protected].
@@ -84,8 +94,14 @@ of the documentation to one language (Chinese), but we hope to continue
 translating more pages and to other languages. Consequently,
 we welcome any enhancements and improvements from the community.
 
-- [Chinese](docs/localized/zh-CN/)
+* [Chinese](docs/localized/zh-CN/)
 
 ## License
 
 [Apache License 2.0](LICENSE)
+
+## Citation
+
+If you use Unity or the ML-Agents Toolkit to conduct research, we ask that you cite the following paper as a reference:
+
+Juliani, A., Berges, V., Vckay, E., Gao, Y., Henry, H., Mattar, M., Lange, D. (2018). Unity: A General Platform for Intelligent Agents. *arXiv preprint arXiv:1809.02627.* https://github.com/Unity-Technologies/ml-agents.