[Documentation] Add technical concepts and Mermaid diagrams to documentation (#970)

* Extend documentation with technical concepts and Mermaid diagrams

- Add docs/concepts.md explaining internal architecture, class hierarchy, and execution flow.
- Include Mermaid class and sequence diagrams.
- Add sphinxcontrib-mermaid to documentation dependencies and configuration.
- Update docs/_toc.yml to include the new concepts page.

Co-authored-by: jan-janssen <3854739+jan-janssen@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix sphinxcontrib-mermaid extension import path

* Documentation in jupyter notebook

* Remove unused images

* no more images

* Add execution flow in table of contents

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: jan-janssen <3854739+jan-janssen@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Author: 3 people

.ci_support/environment-docs.yml

@@ -9,6 +9,7 @@ dependencies:
 - numpy
 - openmpi
 - sphinx
+- sphinxcontrib-mermaid
 - sphinx_rtd_theme
 - cloudpickle =3.1.2
 - h5py =3.16.0

.readthedocs.yml

@@ -15,7 +15,6 @@ build:
       - pip install . --no-deps --no-build-isolation
       - "cp README.md docs"
       - "cp notebooks/*.ipynb docs"
-      - "cp -r notebooks/images docs"
       - "jupyter-book config sphinx docs/"
 
 # Build documentation in the docs/ directory with Sphinx

README.md

@@ -160,6 +160,7 @@ as hierarchical job scheduler within the allocations.
   * [License](https://executorlib.readthedocs.io/en/latest/5-developer.html#license)
   * [Modules](https://executorlib.readthedocs.io/en/latest/5-developer.html#modules)
   * [Interface Class Hierarchy](https://executorlib.readthedocs.io/en/latest/5-developer.html#interface-class-hierarchy)
+  * [Execution Flow](https://executorlib.readthedocs.io/en/latest/5-developer.html#execution-flow)
   * [Test Environment](https://executorlib.readthedocs.io/en/latest/5-developer.html#test-environment)
   * [Communication](https://executorlib.readthedocs.io/en/latest/5-developer.html#communication)
   * [External Libraries](https://executorlib.readthedocs.io/en/latest/5-developer.html#external-libraries)

docs/_config.yml

@@ -15,6 +15,7 @@ launch_buttons:
 
 sphinx:
   extra_extensions:
+  - 'sphinxcontrib.mermaid'
   - 'sphinx.ext.autodoc'
   - 'sphinx.ext.napoleon'
   - 'sphinx.ext.viewcode'