Skip to content

Commit abf7b67

Browse files
matteobettinimatteobettini
authored
[Test] Better tests and multi OS (#89)
* amend * amend * [CI] Rerun tests * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * amend * faster tests * faster tests
1 parent 685b7f1 commit abf7b67

26 files changed

Lines changed: 732 additions & 523 deletions

‎.github/unittest/install_dependencies.sh‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#
2+
# Copyright (c) 2024.
3+
# ProrokLab (https://www.proroklab.org/)
4+
# All rights reserved.
5+
#
6+
7+
8+
python -m pip install --upgrade pip
9+
python -m pip install flake8 pytest pytest-cov tqdm matplotlib
10+
python -m pip install cvxpylayers # Navigation heuristic
11+
12+
pip install -e .

‎.github/workflows/pre_commit.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
2+
name: pre-commit
3+
4+
5+
on:
6+
push:
7+
branches: [ $default-branch , "main" , "dev" ]
8+
pull_request:
9+
branches: [ $default-branch , "main" ]
10+
11+
permissions:
12+
contents: read
13+
14+
15+
jobs:
16+
pre-commit:
17+
runs-on: ubuntu-latest
18+
strategy:
19+
matrix:
20+
python-version: [ "3.10" ]
21+
steps:
22+
- uses: actions/checkout@v3
23+
- name: Set up Python ${{ matrix.python-version }}
24+
uses: actions/setup-python@v3
25+
with:
26+
python-version: ${{ matrix.python-version }}
27+
- uses: pre-commit/action@v3.0.1

‎.github/workflows/python-app.yml‎

Lines changed: 0 additions & 57 deletions
This file was deleted.

‎.github/workflows/tests-linux.yml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
2+
name: pytest-linux
3+
4+
on:
5+
push:
6+
branches: [ $default-branch , "main" , "dev" ]
7+
pull_request:
8+
branches: [ $default-branch , "main" ]
9+
10+
permissions:
11+
contents: read
12+
13+
jobs:
14+
build:
15+
16+
runs-on: ${{ matrix.os }}
17+
strategy:
18+
fail-fast: false
19+
matrix:
20+
os: [ ubuntu-latest ]
21+
python-version: ["3.9", "3.10"]
22+
steps:
23+
- uses: actions/checkout@v3
24+
- name: Set up Python ${{ matrix.python-version }}
25+
uses: actions/setup-python@v3
26+
with:
27+
python-version: ${{ matrix.python-version }}
28+
- name: Install dependencies
29+
run: |
30+
sudo apt-get update
31+
sudo apt-get install python3-opengl xvfb
32+
bash .github/unittest/install_dependencies.sh
33+
- name: Test with pytest
34+
run: |
35+
xvfb-run -s "-screen 0 1024x768x24" pytest tests/ --doctest-modules --junitxml=junit/test-results.xml --cov=. --cov-report=xml --cov-report=html
36+
37+
- if: matrix.python-version == '3.10'
38+
name: Upload coverage to Codecov
39+
uses: codecov/codecov-action@v3
40+
env:
41+
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
42+
with:
43+
fail_ci_if_error: false

‎.github/workflows/tests-mac.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
2+
3+
name: pytest-mac
4+
5+
on:
6+
push:
7+
branches: [ $default-branch , "main" , "dev" ]
8+
pull_request:
9+
branches: [ $default-branch , "main" ]
10+
11+
permissions:
12+
contents: read
13+
14+
jobs:
15+
build:
16+
17+
runs-on: ${{ matrix.os }}
18+
strategy:
19+
fail-fast: false
20+
matrix:
21+
os: [ macos-latest ]
22+
python-version: ["3.9", "3.10"]
23+
24+
steps:
25+
- uses: actions/checkout@v3
26+
- name: Set up Python ${{ matrix.python-version }}
27+
uses: actions/setup-python@v3
28+
with:
29+
python-version: ${{ matrix.python-version }}
30+
- name: Install dependencies
31+
run: |
32+
bash .github/unittest/install_dependencies.sh
33+
- name: Test with pytest
34+
run: |
35+
pytest tests/ --doctest-modules --junitxml=junit/test-results.xml --cov=. --cov-report=xml --cov-report=html

‎.github/workflows/tests-windows.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
2+
3+
name: pytest-windows
4+
5+
on:
6+
push:
7+
branches: [ $default-branch , "main" , "dev" ]
8+
pull_request:
9+
branches: [ $default-branch , "main" ]
10+
11+
permissions:
12+
contents: read
13+
14+
jobs:
15+
build:
16+
17+
runs-on: ${{ matrix.os }}
18+
strategy:
19+
fail-fast: false
20+
matrix:
21+
os: [ windows-latest ]
22+
python-version: ["3.9", "3.10"]
23+
24+
steps:
25+
- uses: actions/checkout@v3
26+
- name: Set up Python ${{ matrix.python-version }}
27+
uses: actions/setup-python@v3
28+
with:
29+
python-version: ${{ matrix.python-version }}
30+
- name: Install dependencies
31+
run: |
32+
bash .github/unittest/install_dependencies.sh
33+
- name: Test with pytest
34+
run: |
35+
pytest tests/ --doctest-modules --junitxml=junit/test-results.xml --cov=. --cov-report=xml --cov-report=html

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -121,8 +121,8 @@ pip install torchrl
121121
# Logging
122122
pip installl wandb
123123

124-
# Save renders
125-
pip install opencv-python moviepy
124+
# Rendering
125+
pip install opencv-python moviepy matplotlib
126126

127127
# Tests
128128
pip install pytest pyyaml pytest-instafail tqdm
@@ -410,9 +410,9 @@ To create a fake screen you need to have `Xvfb` installed.
410410
## TODOS
411411

412412
- [ ] Reset any number of dimensions
413-
- [ ] Improve test efficiency and add new tests
414413
- [ ] Implement 1D camera sensor
415414
- [ ] Implement 2D birds eye view camera sensor
415+
- [X] Improve test efficiency and add new tests
416416
- [X] Implement 2D drone dynamics
417417
- [X] Allow any number of actions
418418
- [X] Improve VMAS performance

‎codecov.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
coverage:
2+
status:
3+
patch: off

‎docs/source/usage/installation.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,4 +48,4 @@ You may want to install the following rendering and logging tools
4848
.. code-block:: console
4949
5050
pip install wandb
51-
pip install opencv-python moviepy
51+
pip install opencv-python moviepy matplotlib

‎tests/test_scenarios/test_balance.py‎

Lines changed: 30 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,55 +1,60 @@
1-
# Copyright (c) 2022-2023.
1+
# Copyright (c) 2022-2024.
22
# ProrokLab (https://www.proroklab.org/)
33
# All rights reserved.
4-
import unittest
4+
5+
import pytest
6+
import torch
57

68
from vmas import make_env
79
from vmas.scenarios import balance
810

911

10-
class TestBalance(unittest.TestCase):
12+
class TestBalance:
1113
def setup_env(
1214
self,
15+
n_envs,
1316
**kwargs,
1417
) -> None:
15-
super().setUp()
1618
self.n_agents = kwargs.get("n_agents", 4)
1719

1820
self.continuous_actions = True
19-
self.n_envs = 15
2021
self.env = make_env(
2122
scenario="balance",
22-
num_envs=self.n_envs,
23+
num_envs=n_envs,
2324
device="cpu",
2425
continuous_actions=self.continuous_actions,
2526
# Environment specific variables
2627
**kwargs,
2728
)
2829
self.env.seed(0)
2930

30-
def test_heuristic(self):
31+
@pytest.mark.parametrize("n_agents", [2, 5])
32+
def test_heuristic(self, n_agents, n_steps=50, n_envs=4):
33+
self.setup_env(
34+
n_agents=n_agents, random_package_pos_on_line=False, n_envs=n_envs
35+
)
36+
policy = balance.HeuristicPolicy(self.continuous_actions)
3137

32-
for n_agents in [2, 5, 6, 10]:
33-
self.setup_env(n_agents=n_agents, random_package_pos_on_line=False)
34-
policy = balance.HeuristicPolicy(self.continuous_actions)
38+
obs = self.env.reset()
3539

36-
obs = self.env.reset()
37-
rews = None
40+
prev_package_dist_to_goal = obs[0][:, 8:10]
3841

39-
for _ in range(100):
40-
actions = []
41-
for i in range(n_agents):
42-
obs_agent = obs[i]
42+
for _ in range(n_steps):
43+
actions = []
44+
for i in range(n_agents):
45+
obs_agent = obs[i]
46+
package_dist_to_goal = obs_agent[:, 8:10]
4347

44-
action_agent = policy.compute_action(
45-
obs_agent, self.env.agents[i].u_range
46-
)
48+
action_agent = policy.compute_action(
49+
obs_agent, self.env.agents[i].u_range
50+
)
4751

48-
actions.append(action_agent)
52+
actions.append(action_agent)
4953

50-
obs, new_rews, dones, _ = self.env.step(actions)
54+
obs, new_rews, dones, _ = self.env.step(actions)
5155

52-
if rews is not None:
53-
for i in range(self.n_agents):
54-
self.assertTrue((new_rews[i] >= rews[i]).all())
55-
rews = new_rews
56+
assert (
57+
torch.linalg.vector_norm(package_dist_to_goal, dim=-1)
58+
<= torch.linalg.vector_norm(prev_package_dist_to_goal, dim=-1)
59+
).all()
60+
prev_package_dist_to_goal = package_dist_to_goal

0 commit comments

Comments
 (0)