[Docs] Usage (#88)

matteobettini · web-flow · commit dbb11e4cc9e6 · 2024-03-04T10:26:08.000Z
* amend

* fig

* codecov
diff --git a/.github/workflows/python-app.yml b/.github/workflows/python-app.yml
@@ -51,5 +51,7 @@ jobs:
     - if: matrix.python-version == '3.10'
       name: Upload coverage to Codecov
       uses: codecov/codecov-action@v3
+      env:
+        CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
       with:
         fail_ci_if_error: false
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -42,6 +42,7 @@
     "torch": ("https://pytorch.org/docs/master", None),
     "torchrl": ("https://pytorch.org/rl", None),
     "tensordict": ("https://pytorch.org/tensordict", None),
+    "benchmarl": ("https://benchmarl.readthedocs.io/en/latest/", None),
 }
 intersphinx_disabled_domains = ["std"]
 
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -2,15 +2,16 @@
 VMAS
 ====
 
+.. figure:: https://raw.githubusercontent.com/matteobettini/vmas-media/master/media/vmas_scenarios_more.gif
+   :align: center
+
+
+:github:`null` `GitHub <https://github.com/proroklab/VectorizedMultiAgentSimulator>`__
+
 Docs are currently being written, please bear with us.
 
 Forthcoming doc sections:
 
-- Usage
-    - Notebooks
-    - Installing
-    - Running
-    - Citing
 
 - Concepts (explanation of all the lib features) (divide in basic, advanced and medium)
     - Input and output spaces
@@ -33,6 +34,19 @@ Forthcoming doc sections:
 
 - Full package reference with docstrings for all public functions
 
+.. toctree::
+   :maxdepth: 1
+   :caption: Using
+
+   usage/notebooks
+   usage/installation
+   usage/running
+   usage/citing
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Concepts
+
 
 .. toctree::
    :maxdepth: 1
diff --git a/docs/source/usage/citing.rst b/docs/source/usage/citing.rst
@@ -0,0 +1,16 @@
+Citing
+======
+
+If you use VMAS in your research please use the following BibTeX entry:
+
+
+.. code-block:: bibtex
+
+    @inproceedings{bettini2022vmas,
+      title = {VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning},
+      author = {Bettini, Matteo and Kortvelesy, Ryan and Blumenkamp, Jan and Prorok, Amanda},
+      year = {2022},
+      booktitle = {Proceedings of the 16th International Symposium on Distributed Autonomous Robotic Systems},
+      publisher = {Springer},
+      series = {DARS '22}
+    }
diff --git a/docs/source/usage/installation.rst b/docs/source/usage/installation.rst
@@ -0,0 +1,51 @@
+Installation
+============
+
+
+Install from PyPi
+-----------------
+
+You can install `VMAS <https://github.com/proroklab/VectorizedMultiAgentSimulator>`__ from PyPi.
+
+.. code-block:: console
+
+   pip install vmas
+
+Install from source
+-------------------
+
+If you want to install the current main version (more up to date than latest release), you can do:
+
+.. code-block:: console
+
+   git clone https://github.com/proroklab/VectorizedMultiAgentSimulator.git
+   cd VectorizedMultiAgentSimulator
+   pip install -e .
+
+
+Install optional requirements
+-----------------------------
+
+By default, vmas has only the core requirements.
+Here are some optional packages you may want to install.
+
+Training
+^^^^^^^^
+
+You may want to install one of the following training libraries
+
+.. code-block:: console
+
+   pip install benchmarl
+   pip install torchrl
+   pip install "ray[rllib]"==2.1.0 # We support versions "ray[rllib]<=2.2,>=1.13"
+
+Logging
+^^^^^^^
+
+You may want to install the following rendering and logging tools
+
+.. code-block:: console
+
+   pip install wandb
+   pip install opencv-python moviepy
diff --git a/docs/source/usage/notebooks.rst b/docs/source/usage/notebooks.rst
@@ -0,0 +1,9 @@
+Notebooks
+=========
+
+In the following you can find a list of  :colab:`null` Google Colab notebooks to help you learn how to use VMAS:
+
+- :colab:`null` `Using a VMAS environment <https://colab.research.google.com/github/proroklab/VectorizedMultiAgentSimulator/blob/main/notebooks/VMAS_Use_vmas_environment.ipynb>`_. Here is a simple notebook that you can run to create, step and render any scenario in VMAS. It reproduces the ``use_vmas_env.py`` script in the ``examples`` folder
+- :colab:`null` `Using VMAS in BenchMARL (suggested) <https://colab.research.google.com/github/facebookresearch/BenchMARL/blob/main/notebooks/run.ipynb>`_.  In this notebook, we show how to use VMAS in BenchMARL, TorchRL's MARL training library
+- :colab:`null` `Using VMAS in TorchRL <https://colab.research.google.com/github/pytorch/rl/blob/gh-pages/_downloads/a977047786179278d12b52546e1c0da8/multiagent_ppo.ipynb>`_. In this notebook, `available in the TorchRL docs <https://pytorch.org/rl/tutorials/multiagent_ppo.html>`_, we show how to use any VMAS scenario in TorchRL. It will guide you through the full pipeline needed to train agents using MAPPO/IPPO.
+- :colab:`null` `Using VMAS in RLlib <https://colab.research.google.com/github/proroklab/VectorizedMultiAgentSimulator/blob/main/notebooks/VMAS_RLlib.ipynb>`_. In this notebook, we show how to use any VMAS scenario in RLlib. It reproduces the ``rllib.py`` script in the ``examples`` folder.
diff --git a/docs/source/usage/running.rst b/docs/source/usage/running.rst
@@ -0,0 +1,36 @@
+Running
+=======
+
+To use the simulator, simply create an environment by passing the name of the scenario
+you want (from the ``scenarios`` folder) to the :class:`vmas.make_env` function.
+The function arguments are explained in the documentation. The function returns an environment
+object which you can step and reset.
+
+.. code-block:: python
+
+    import vmas
+
+    # Create the environment
+    env = vmas.make_env(
+        scenario="waterfall", # can be scenario name or BaseScenario class
+        num_envs=32,
+        device="cpu", # Or "cuda" for GPU
+        continuous_actions=True,
+        max_steps=None, # Defines the horizon. None is infinite horizon.
+        seed=None, # Seed of the environment
+        n_agents=3  # Additional arguments you want to pass to the scenario
+    )
+    # Reset it
+    obs = env.reset()
+
+    # Step it with deterministic actions (all agents take their maximum range action)
+    for _ in range(10):
+        obs, rews, dones, info = env.step(env.get_random_actions())
+
+Here is a python example on how you can execute vmas environments.
+
+.. python_example_button::
+   https://github.com/proroklab/VectorizedMultiAgentSimulator/blob/main/vmas/examples/use_vmas_env.py
+
+The `Concepts` documentation contains a series of sections that
+can help you get familiar with further VMAS functionalities.
diff --git a/vmas/examples/use_vmas_env.py b/vmas/examples/use_vmas_env.py
@@ -11,44 +11,6 @@
 from vmas.simulator.utils import save_video
 
 
-def _get_random_action(agent: Agent, continuous: bool, env):
-    if continuous:
-        actions = []
-        for action_index in range(agent.action_size):
-            actions.append(
-                torch.zeros(
-                    agent.batch_dim,
-                    device=agent.device,
-                    dtype=torch.float32,
-                ).uniform_(
-                    -agent.action.u_range_tensor[action_index],
-                    agent.action.u_range_tensor[action_index],
-                )
-            )
-        if env.world.dim_c != 0 and not agent.silent:
-            # If the agent needs to communicate
-            for _ in range(env.world.dim_c):
-                actions.append(
-                    torch.zeros(
-                        agent.batch_dim,
-                        device=agent.device,
-                        dtype=torch.float32,
-                    ).uniform_(
-                        0,
-                        1,
-                    )
-                )
-        action = torch.stack(actions, dim=-1)
-    else:
-        action = torch.randint(
-            low=0,
-            high=env.get_agent_action_space(agent).n,
-            size=(agent.batch_dim,),
-            device=agent.device,
-        )
-    return action
-
-
 def _get_deterministic_action(agent: Agent, continuous: bool, env):
     if continuous:
         action = -agent.action.u_range_tensor.expand(env.batch_dim, agent.action_size)
@@ -123,7 +85,7 @@ def use_vmas_env(
             if not random_action:
                 action = _get_deterministic_action(agent, continuous_actions, env)
             else:
-                action = _get_random_action(agent, continuous_actions, env)
+                action = env.get_random_action(agent)
             if dict_actions:
                 actions.update({agent.name: action})
             else:
diff --git a/vmas/simulator/environment/environment.py b/vmas/simulator/environment/environment.py
@@ -3,7 +3,7 @@
 #  All rights reserved.
 import random
 from ctypes import byref
-from typing import Callable, Dict, List, Optional, Tuple, Union
+from typing import Callable, Dict, List, Optional, Sequence, Tuple, Union
 
 import numpy as np
 import torch
@@ -197,6 +197,21 @@ def step(self, actions: Union[List, Dict]):
             dones: Tensor of len 'self.num_envs' of which each element is a bool
             infos : List on len 'self.n_agents' of which each element is a dictionary for which each key is a metric
                     and the value is a tensor of shape '(self.num_envs, metric_size_per_agent)'
+
+        Examples:
+            >>> import vmas
+            >>> env = vmas.make_env(
+            ...     scenario="waterfall",  # can be scenario name or BaseScenario class
+            ...     num_envs=32,
+            ...     device="cpu",  # Or "cuda" for GPU
+            ...     continuous_actions=True,
+            ...     max_steps=None,  # Defines the horizon. None is infinite horizon.
+            ...     seed=None,  # Seed of the environment
+            ...     n_agents=3,  # Additional arguments you want to pass to the scenario
+            ... )
+            >>> obs = env.reset()
+            >>> for _ in range(10):
+            ...     obs, rews, dones, info = env.step(env.get_random_actions())
         """
         if isinstance(actions, Dict):
             actions_dict = actions
@@ -352,6 +367,76 @@ def get_agent_observation_space(self, agent: Agent, obs: AGENT_OBS_TYPE):
                 f"Invalid type of observation {obs} for agent {agent.name}"
             )
 
+    def get_random_action(self, agent: Agent) -> torch.Tensor:
+        """Returns a random action for the given agent.
+
+        Args:
+            agent (Agent): The agent to get the action for
+
+        Returns:
+            torch.tensor: the random actions tensor with shape ``(agent.batch_dim, agent.action_size)``
+
+        """
+        if self.continuous_actions:
+            actions = []
+            for action_index in range(agent.action_size):
+                actions.append(
+                    torch.zeros(
+                        agent.batch_dim,
+                        device=agent.device,
+                        dtype=torch.float32,
+                    ).uniform_(
+                        -agent.action.u_range_tensor[action_index],
+                        agent.action.u_range_tensor[action_index],
+                    )
+                )
+            if self.world.dim_c != 0 and not agent.silent:
+                # If the agent needs to communicate
+                for _ in range(self.world.dim_c):
+                    actions.append(
+                        torch.zeros(
+                            agent.batch_dim,
+                            device=agent.device,
+                            dtype=torch.float32,
+                        ).uniform_(
+                            0,
+                            1,
+                        )
+                    )
+            action = torch.stack(actions, dim=-1)
+        else:
+            action = torch.randint(
+                low=0,
+                high=self.get_agent_action_space(agent).n,
+                size=(agent.batch_dim,),
+                device=agent.device,
+            )
+        return action
+
+    def get_random_actions(self) -> Sequence[torch.Tensor]:
+        """Returns random actions for all agents that you can feed to :class:`step`
+
+        Returns:
+            Sequence[torch.tensor]: the random actions for the agents
+
+        Examples:
+            >>> import vmas
+            >>> env = vmas.make_env(
+            ...     scenario="waterfall",  # can be scenario name or BaseScenario class
+            ...     num_envs=32,
+            ...     device="cpu",  # Or "cuda" for GPU
+            ...     continuous_actions=True,
+            ...     max_steps=None,  # Defines the horizon. None is infinite horizon.
+            ...     seed=None,  # Seed of the environment
+            ...     n_agents=3,  # Additional arguments you want to pass to the scenario
+            ... )
+            >>> obs = env.reset()
+            >>> for _ in range(10):
+            ...     obs, rews, dones, info = env.step(env.get_random_actions())
+
+        """
+        return [self.get_random_action(agent) for agent in self.agents]
+
     def _check_discrete_action(self, action: Tensor, low: int, high: int, type: str):
         assert torch.all(
             (action >= torch.tensor(low, device=self.device))

Original file line number	Diff line number	Diff line change
`@@ -42,6 +42,7 @@`
`42`	`42`	`"torch": ("https://pytorch.org/docs/master", None),`
`43`	`43`	`"torchrl": ("https://pytorch.org/rl", None),`
`44`	`44`	`"tensordict": ("https://pytorch.org/tensordict", None),`
	`45`	`+ "benchmarl": ("https://benchmarl.readthedocs.io/en/latest/", None),`
`45`	`46`	`}`
`46`	`47`	`intersphinx_disabled_domains = ["std"]`
`47`	`48`