Test launcher based on pytest (#366)

Author: svalat

.gitignore

@@ -41,6 +41,7 @@ test/**/*.pyc
 test/**/*.dat
 test/**/cmake_packages*
 test/**/
+idefix-tests.junit.xml
 
 # machine specific cache and hidden files
 .*

.pre-commit-config.yaml

@@ -43,6 +43,7 @@ repos:
     rev: v6.1.2
     hooks:
       - id: inifix-format
+        files: ^(test/).*\.(ini)$ # want to skip pytest.ini
 
   - repo: https://github.com/Lucas-C/pre-commit-hooks
     rev: v1.5.5

doc/source/testing.rst

@@ -71,9 +71,17 @@ How tests are driven (testme scripts)
 Each test directory contains a small Python "testMe" driver that uses the helper Python
 class documented in the repository:
 
+- See the test launcher documentation: :doc:`test.py <testing/testLauncher>`
 - See the test helper documentation: :doc:`idfxTest <testing/idfxTest>`
 
-That helper (idfxTest) is responsible for:
+The test launcher (test.py) is responsible for:
+
+- Loading all the test definitions by search all the ``testme.json`` files in the ``test``
+  directory.
+- Calling the :doc:`idfxTest <testing/idfxTest>` helper to run the particular test.
+- Generate reports about success and failures.
+
+The helper (idfxTest) is responsible for:
 
 - parsing TESTME_OPTIONS-like flags (precision, MPI, CUDA, reconstruction, vector potential, etc.),
 - calling configure / compile / run,
@@ -108,10 +116,12 @@ Relevant files
 
 - Workflow entry point: .github/workflows/idefix-ci.yml
 - Reusable jobs: .github/workflows/idefix-ci-jobs.yml
+- Test launcher documentation: :doc:`test launcher <testing/testLauncher>`
 - Test helper documentation: :doc:`idfxTest <testing/idfxTest>`
 
 .. toctree::
    :maxdepth: 2
    :caption: Contents:
 
+   testing/testLauncher.rst
    testing/idfxTest.rst

doc/source/testing/idfxTest.rst

@@ -74,6 +74,9 @@ The constructor parses command-line arguments using ``argparse``. These options
    * - ``-Werror``
      - ``Werror``
      - Treat compiler warnings as errors.
+   * - ``-ccache``
+     - ``ccache``
+     - Enable usage of ccache to build the tests and reduce the build time.
 
 Main Methods
 ------------