Merge branch 'main' into Py314u

Author: jan-janssen

.ci_support/environment-docs.yml

@@ -15,4 +15,5 @@ dependencies:
 - jupyter-book =1.0.0
 - mpi4py =4.1.1
 - python =3.12
+- pydantic =2.12.4
 - pyzmq =27.1.0

.ci_support/environment-integration.yml

@@ -17,7 +17,8 @@ dependencies:
 - ipython =9.9.0
 - matplotlib =3.10.8
 - networkx =3.6.1
+- pydantic =2.12.4
 - pygraphviz =1.14
-- pysqa =0.3.4
+- pysqa =0.3.5
 - pyzmq =27.1.0
 - qe =7.5

.ci_support/environment-mpich.yml

@@ -11,6 +11,7 @@ dependencies:
 - ipython =9.9.0
 - mpi4py =4.1.1
 - networkx =3.6.1
+- pydantic =2.12.4
 - pygraphviz =1.14
-- pysqa =0.3.4
+- pysqa =0.3.5
 - pyzmq =27.1.0

.ci_support/environment-old.yml

@@ -10,6 +10,7 @@ dependencies:
 - h5py =3.6.0
 - networkx =2.8.8
 - ipython =7.33.0
+- pydantic =2.5.3
 - pygraphviz =1.10
 - hatchling =1.27.0
 - hatch-vcs =0.4.0

.ci_support/environment-openmpi.yml

@@ -11,6 +11,7 @@ dependencies:
 - ipython =9.9.0
 - mpi4py =4.1.1
 - networkx =3.6.1
+- pydantic =2.12.4
 - pygraphviz =1.14
-- pysqa =0.3.4
+- pysqa =0.3.5
 - pyzmq =27.1.0

.ci_support/environment-win.yml

@@ -11,5 +11,6 @@ dependencies:
 - ipython =9.9.0
 - mpi4py =4.1.1
 - networkx =3.6.1
+- pydantic =2.12.4
 - pygraphviz =1.14
 - pyzmq =27.1.0

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.15.0
+    rev: v0.15.1
     hooks:
       - id: ruff
         name: ruff lint

binder/environment.yml

@@ -15,5 +15,5 @@ dependencies:
 - mpi4py =4.1.1
 - networkx =3.6.1
 - pygraphviz =1.14
-- pysqa =0.3.3
+- pysqa =0.3.5
 - pyzmq =27.1.0

docs/trouble_shooting.md

@@ -42,18 +42,26 @@ The resource dictionary parameter `resource_dict` can contain one or more of the
 * `threads_per_core` (int): number of OpenMP threads to be used for each function call
 * `gpus_per_core` (int): number of GPUs per worker - defaults to 0
 * `cwd` (str/None): current working directory where the parallel python task is executed
-* `openmpi_oversubscribe` (bool): adds the `--oversubscribe` command line flag (OpenMPI and SLURM only) - default False
+* `cache_key` (str): Rather than using the internal hashing of executorlib the user can provide an external `cache_key`
+  to identify tasks on the file system. The initial file name is going to be `cache_key` + `_i.h5` and the final file
+  name is going to be `cache_key` + `_o.h5`.
+* `num_nodes` (int): number of compute nodes used for the evaluation of the Python function. 
+* `exclusive` (bool): boolean flag to reserve exclusive access to selected compute nodes - do not allow other tasks to 
+  use the same compute node. 
+* `error_log_file` (str): path to the error log file, primarily used to merge the log of multiple tasks in one file.
+* `run_time_limit` (int): the maximum time the execution of the submitted Python function is allowed to take in seconds.
+* `priority` (int): the queuing system priority assigned to a given Python function to influence the scheduling.
 * `slurm_cmd_args` (list): Additional command line arguments for the srun call (SLURM only)
 
 For the special case of the [HPC Job Executor](https://executorlib.readthedocs.io/en/latest/3-hpc-job.html) 
 the resource dictionary parameter `resource_dict` can also include additional parameters define in the submission script
 of the [Python simple queuing system adatper (pysqa)](https://pysqa.readthedocs.io) these include but are not limited to: 
-* `run_time_max` (int): the maximum time the execution of the submitted Python function is allowed to take in seconds.
 * `memory_max` (int): the maximum amount of memory the Python function is allowed to use in Gigabytes. 
 * `partition` (str): the partition of the queuing system the Python function is submitted to. 
 * `queue` (str): the name of the queue the Python function is submitted to. 
 
-All parameters in the resource dictionary `resource_dict` are optional. 
+All parameters in the resource dictionary `resource_dict` are optional. When `pydantic` is installed as optional 
+dependency the `resource_dict` is validated using `pydantic`.
 
 ## SSH Connection
 While the [Python simple queuing system adatper (pysqa)](https://pysqa.readthedocs.io) provides the option to connect to

pyproject.toml

@@ -53,13 +53,17 @@ graphnotebook = [
 ]
 mpi = ["mpi4py==4.1.1"]
 cluster = [
-    "pysqa==0.3.4",
+    "pysqa==0.3.5",
     "h5py==3.15.1",
 ]
+validation = [
+    "pydantic==2.12.4",
+]
 all = [
     "mpi4py==4.1.1",
-    "pysqa==0.3.4",
+    "pysqa==0.3.5",
     "h5py==3.15.1",
+    "pydantic==2.12.4",
     "pygraphviz==1.14",
     "networkx==3.6.1",
     "ipython==9.9.0",