diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..bf5dabd --- /dev/null +++ b/.gitignore @@ -0,0 +1,22 @@ +# cmake generated files +CMakeCache.txt +CMakeFiles +build +Makefile +cmake_install.cmake +idefix +idefix.*.log +generated +git-state.txt +cmake_packages + +# data generated by the run +*.vtk +*.dmp + +# jupyter tmp files +.ipynb_checkpoints + +# python local venv +.venv + diff --git a/AdvancedSetup/solution1/read_problem.ipynb b/AdvancedSetup/solution1/read_problem.ipynb index a056990..e31fa77 100644 --- a/AdvancedSetup/solution1/read_problem.ipynb +++ b/AdvancedSetup/solution1/read_problem.ipynb @@ -11,9 +11,8 @@ "%matplotlib widget\n", "import os\n", "import sys\n", - "# These path are specific to this Jureca tutorial\n", - "user=os.getenv(\"USER\")\n", - "sys.path.append(\"/p/project1/training2437/tera_day2/\"+user+\"/idefix/idefix.src/\")\n", + "# Possibly adapt the idefix path depending on where you cloned the sources\n", + "sys.path.append(\"../../../idefix.src\")\n", "from pytools.vtk_io import readVTK\n", "import matplotlib.pyplot as plt\n", "import numpy as np" @@ -76,7 +75,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.14" + "version": "3.12.3" } }, "nbformat": 4, diff --git a/Readme.ipynb b/Readme.ipynb index fbcb417..def0863 100644 --- a/Readme.ipynb +++ b/Readme.ipynb @@ -9,60 +9,9 @@ "\n", "\n", "# About this tutorial\n", - "This tutorial is provided as a github repository and is mirrored on Jureca for easier access. It is part of the [Toward Exascale-Ready Astrophysics workshop](https://indico3-jsc.fz-juelich.de/event/169/) and has been prepared by Geoffroy Lesur (geoffroy.lesur@univ-grenoble-alpes.fr)\n", + "This tutorial is provided as a github repository. It has been prepared by Geoffroy Lesur (geoffroy.lesur@univ-grenoble-alpes.fr).\n", "\n", - "In this tutorial, you will learn how to use idefix on various architectures. Here, we will do everything through a Jupyter notebook opened on Jureca on the `dc-gpu` partition, but you can also do most of the tutorial on the CPU of your laptop.\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "id": "5e87953c-007b-4506-8183-62227f5fad5d", - "metadata": {}, - "source": [ - "# 1- Deployment on Jureca\n", - "\n", - "Log in to https://jupyter-jsc.fz-juelich.de/\n", - "\n", - "Open a lab environment with\n", - "\n", - "- Lab Config:\n", - " - System: JURECA\n", - " - Project: training2437\n", - " - Partition: dc-gpu\n", - " - Reservation: tera_day2\n", - "- Resources (opens once dc-gpu is selected)\n", - " - Nodes: 1\n", - " - GPUs: 4\n", - " - Runtime: 90\n", - "Kernels and extensions: keep defaults\n", - "\n", - "\n", - "First open a new console on your Jupyter notebook. We then clone the idefix Github repository. Since we don't have direct access to the internet, we use a small script to copy the sources and the tutorial from a shared directory:\n", - "\n", - "```shell\n", - "source /p/project1/training2437/tera_day2/idefix/deploy.sh\n", - "```\n", - "\n", - "This will put everything into `/p/project1/training2437/$USER/tera_day2/idefix`\n", - "\n", - "The deploy script already set up the environement (module and environement variable). If you loose connection and need a new console, you can reload the environement:\n", - "\n", - "```shell\n", - "source /p/project1/training2437/tera_day2/idefix/env.sh\n", - "```\n", - "\n", - "You should now open this notebook on your Jupyter instance on Jureca, in\n", - "\n", - "`/p/project1/training2437/$USER/tera_day2/idefix/tutorial/Readme.ipynb`\n" - ] - }, - { - "cell_type": "markdown", - "id": "75842bfd-0baa-4953-9657-278840d02ec8", - "metadata": {}, - "source": [ - "In Jupyter, we need to load Idefix environement:" + "In this tutorial, you will learn how to use idefix on various architectures. Here, we will do everything through a Jupyter notebook. Except for some GPU specific parts you can also do most of the tutorial on the CPU of your laptop." ] }, { @@ -76,9 +25,8 @@ "%matplotlib widget\n", "import os\n", "import sys\n", - "# These path are specific to this Jureca tutorial\n", - "user=os.getenv(\"USER\")\n", - "sys.path.append(\"/p/project1/training2437/\"+user+\"/tera_day2/idefix/idefix.src/\")\n", + "# Possibly adapt the idefix path depending on where you cloned the sources\n", + "sys.path.append(\"../idefix.src\")\n", "from pytools.vtk_io import readVTK\n", "import matplotlib.pyplot as plt\n", "import numpy as np" @@ -89,24 +37,17 @@ "id": "83936685-a050-428e-ac85-93f3c1b973c8", "metadata": {}, "source": [ - "and we're good to go!\n", - "\n", - "## Optionnal: Deploy on your machine\n", + "## Deploy on your machine\n", "\n", - "
\n", - "Click here to deploy Idefix on your laptop.\n", "
\n", - "Optionnally, you can play around with this tutorial on your laptop/machine. In this case you can clone this tutorial and idefix source code on your machine, so that you can directly use these source files and test what you are doing. In the directory of your choice (this requires an internet access):\n", + "You can play around with this tutorial on your laptop/machine. In this case you can clone this tutorial and idefix source code on your machine, so that you can directly use these source files and test what you are doing. In the directory of your choice (this requires an internet access):\n", "\n", "```shell\n", "git clone --recurse-submodules https://github.com/idefix-code/idefix.git idefix.src\n", "export IDEFIX_DIR=$PWD/idefix.src\n", - "git clone https://github.com/idefix-code/tutorial.git\n", + "git clone --branch master https://github.com/idefix-code/tutorial.git\n", "cd tutorial\n", - "git checkout Jureca\n", "```\n", - "The last line allows you to reach the dedicated tutorial for Jureca.\n", - "\n", "For conveniance, we set the `IDEFIX_DIR` environment variable to the absolute path of the root directory of idefix (as above). \n", "\n", "If you intend to use the python script provided in this tutorial, best is to deploy a python environement with everything already set up. \n", @@ -117,9 +58,7 @@ "python3 -m venv ./env\n", "source env/bin/activate\n", "pip install -r python_requirements.txt\n", - "```\n", - "\n", - "
\n" + "```" ] }, { @@ -156,9 +95,12 @@ "\n", "## Run an example\n", "\n", - "launch the executable\n", + "Launch the executable depending on your running location with either :\n", "\n", "```shell\n", + "# On your laptop\n", + "./idefix\n", + "# On a slurm based cluster\n", "srun -n 1 ./idefix\n", "```\n", "\n", @@ -190,20 +132,14 @@ "make -j 8\n", "```\n", "\n", - "if your build is successful, you can now try to launch idefix with automatic domain decomposition. On a Jureca node (using Slurm):\n", - "\n", - "```shell\n", - "srun -n 4 ./idefix\n", - "```\n", - "\n", - "
\n", - "(optional) On your laptop:\n", + "if your build is successful, you can now try to launch idefix with automatic domain decomposition. Depending on your run location it can be either :\n", "\n", "```shell\n", + "# on your latop\n", "mpirun -np 4 ./idefix\n", - "```\n", - "
\n", - "\n" + "# on a slurm cluster\n", + "srun -n 4 ./idefix\n", + "```\n" ] }, { @@ -214,7 +150,7 @@ "# 3- Configuration, compilation, run on GPUs\n", "## First tests\n", "\n", - "For this first test, we are going to run a simple Orszag-Tang test problem on a single GPU. First cd to the right directory\n", + "For this first test, we are going to run a simple Orszag-Tang test problem on a single GPU. First `cd` to the right directory\n", "\n", "```shell\n", "cd $IDEFIX_DIR/test/MHD/OrszagTang\n", @@ -240,9 +176,12 @@ "\n", "### Running the code on GPUs\n", "\n", - "You then simply launch the executable using srun:\n", + "You then simply launch the executable depending on your running location with either :\n", "\n", "```shell\n", + "# Run locally\n", + "./idefix\n", + "# Run in a slurm cluster.\n", "srun -n 1 ./idefix\n", "```\n", "\n", @@ -251,7 +190,7 @@ "\n", "### Multi-GPUs runs\n", "\n", - "Idefix can run on multiple GPUs (it's been tested on +4000 GPUs simultaneously). This requires an MPI installation compatible with Cuda (e.g. GPU-aware OpenMPI). If you have loaded the environement in [Getting Started](README.md), you should be able to compile a GPU version of Idefix with parallelisation support.\n", + "Idefix can run on multiple GPUs (it's been tested on +4000 GPUs simultaneously). This requires an MPI installation compatible with Cuda (e.g. GPU-aware OpenMPI). If you have loaded the required environement into your cluster about your network and GPU stack, you should be able to compile a GPU version of Idefix with parallelisation support.\n", "\n", "You should first configure the code with CMake adding `-DIdefix_MPI=ON` to the command line and compile. \n", "\n", @@ -263,6 +202,7 @@ "If the compilation succeeds, then you can run a multi-GPU simulation with 4 gpus:\n", "\n", "```shell\n", + "# On a slurm cluster\n", "srun -n 4 ./idefix\n", "```\n", "\n", @@ -328,12 +268,11 @@ "\n", "### Configure the code, build and run it\n", "\n", - "Follow the instruction in the [Getting Started](../GettingStarted/README.md#compile-an-example) section. Note that you can run your setup on a CPU or an GPU, choosing the right configuration with `Cmake`.\n", + "Follow the instruction in the [Getting Started](#compilation) section. Note that you can run your setup on a CPU or an GPU, choosing the right configuration with `Cmake`.\n", "\n", "### Check the outputs\n", "\n", - "To visualize the flow, you may use Paraview or visit to open the files generated by Idefix. On Jureca, you can open the notebook `read_problem.ipynb` in `SimpleSetup/problem1` and execute it to visualize the result directly.\n", - "\n" + "To visualize the flow, you may use Paraview or visit to open the files generated by Idefix. You can also open the notebook `read_problem.ipynb` in `SimpleSetup/problem1` and execute it to visualize the result directly." ] }, { @@ -344,7 +283,7 @@ "outputs": [], "source": [ "# Load the last VTK file produced by Idefix\n", - "V=readVTK(\"/p/project1/training2437/\"+user+\"/tera_day2/idefix/tutorial/SimpleSetup/problem1/data.0005.vtk\")" + "V=readVTK(\"./SimpleSetup/problem1/data.0005.vtk\")" ] }, { @@ -358,7 +297,7 @@ "for field in V.data.keys():\n", " plt.figure(figsize=(10,4))\n", " plt.pcolormesh(V.x,V.y,V.data[field][:,:,0].T)\n", - " plt.title(field+ \" @ t=%f\"%V.t)\n", + " plt.title(field+ \" @ t=%f\"%V.t[0])\n", " plt.colorbar()\n", " plt.gca().set_aspect('equal')" ] @@ -383,7 +322,7 @@ "source": [ "plt.figure(figsize=(10,4))\n", "plt.pcolormesh(V.x,V.y,wz[:,:,0].T)\n", - "plt.title(r\"$\\omega_z$ @ t=%f\"%V.t)\n", + "plt.title(r\"$\\omega_z$ @ t=%f\"%V.t[0])\n", "plt.colorbar()\n", "plt.gca().set_aspect('equal')" ] @@ -397,7 +336,7 @@ "\n", "Try to compile the code for a single GPU (without MPI). The initial resolution is $256\\times 64$. Edit `idefix.ini` and try to increase the resolution to $512\\times 128$ and $1024\\times 256$ and look at the final performances. You should observe a strong dependence on the problem size. This is because for GPU computing to be efficient, you need to keep busy the 1000s of compute core of your GPU. This typically means that you need at least about 1 million cells on modern GPUs for Idefix to reach its full efficiency.\n", "\n", - "These are the typical performances measured on Jureca on a single Nvidia A100 GPU:\n", + "These are the typical performances measured on a single Nvidia A100 GPU:\n", "- $256\\times 64$: 6.1e7 cell/s\n", "- $512\\times 128$: 2.2e8 cell/s\n", "- $1024\\times 256$: 5.9e8 cell/s\n", @@ -506,7 +445,7 @@ "source": [ "# Read output file after 3 orbits\n", "\n", - "V=readVTK(\"/p/project1/training2437/\"+user+\"/tera_day2/idefix/tutorial/AdvancedSetup/problem1/data.0003.vtk\")\n", + "V=readVTK(\"./AdvancedSetup/problem1/data.0003.vtk\")\n", "# define cartesian coordinates from radius (xl) and azimuth (yl) in the VTK file\n", "x=V.xl[:,None]*np.cos(V.yl[None,:])\n", "y=V.xl[:,None]*np.sin(V.yl[None,:])\n", @@ -546,7 +485,7 @@ "source": [ "# Read output file after 3 orbits\n", "\n", - "V=readVTK(\"/p/project1/training2437/\"+user+\"/tera_day2/idefix/tutorial/AdvancedSetup/problem1/data.0003.vtk\")\n", + "V=readVTK(\"./AdvancedSetup/problem1/data.0003.vtk\")\n", "# define cartesian coordinates from radius (xl) and azimuth (yl) in the VTK file\n", "x=V.xl[:,None]*np.cos(V.yl[None,:])\n", "y=V.xl[:,None]*np.sin(V.yl[None,:])\n", @@ -646,7 +585,7 @@ "## Problem1: a CPU segmentation fault\n", "\n", "### Base run\n", - "The first problem is a simple 1D shock tube problem. This can be compiled and run on your laptop or on Jureca.\n", + "The first problem is a simple 1D shock tube problem. This can be compiled and run on your laptop or on a cluster.\n", "\n", "```shell\n", "cd $IDEFIX_ROOT/tutorial/Debugging/problem1\n", @@ -850,7 +789,7 @@ "\n", "## Problem 4: a low performance bug.\n", "\n", - "Let's move to problem 4, which is again a planet-disk interraction problem. This can be compiled and run *on your laptop* or on the Jureca cluster, but let's focus for now on the GPU version on the Jureca cluster (you can try to do the exercise on your laptop). First go to the right directory\n", + "Let's move to problem 4, which is again a planet-disk interraction problem. This can be compiled and run *on your laptop* or on the on a cluster, but let's focus for now on the GPU version on the a cluster (you can try to do the exercise on your laptop). First go to the right directory\n", "\n", "```shell\n", "cd $IDEFIX_ROOT/tutorial/Debugging/problem4\n", @@ -897,14 +836,6 @@ "

\n", "\n" ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a09fb8a4-9723-4671-babb-44b51dd98274", - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": { @@ -923,7 +854,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.14" + "version": "3.12.3" } }, "nbformat": 4, diff --git a/SimpleSetup/problem1/read_problem.ipynb b/SimpleSetup/problem1/read_problem.ipynb index d9e721c..fee0ba8 100644 --- a/SimpleSetup/problem1/read_problem.ipynb +++ b/SimpleSetup/problem1/read_problem.ipynb @@ -11,9 +11,8 @@ "%matplotlib widget\n", "import os\n", "import sys\n", - "# These path are specific to this Jureca tutorial\n", - "user=os.getenv(\"USER\")\n", - "sys.path.append(\"/p/project1/training2437/tera_day2/\"+user+\"/idefix/idefix.src/\")\n", + "# Possibly adapt the idefix path depending on where you cloned the sources\n", + "sys.path.append(\"../../../idefix.src\")\n", "from pytools.vtk_io import readVTK\n", "import matplotlib.pyplot as plt\n", "import numpy as np" @@ -42,7 +41,7 @@ "for field in V.data.keys():\n", " plt.figure(figsize=(10,4))\n", " plt.pcolormesh(V.x,V.y,V.data[field][:,:,0].T)\n", - " plt.title(field+ \" @ t=%f\"%V.t)\n", + " plt.title(field+ \" @ t=%f\"%V.t[0])\n", " plt.colorbar()\n", " plt.gca().set_aspect('equal')" ] @@ -67,10 +66,18 @@ "source": [ "plt.figure(figsize=(10,4))\n", "plt.pcolormesh(V.x,V.y,wz[:,:,0].T)\n", - "plt.title(r\"$\\omega_z$ @ t=%f\"%V.t)\n", + "plt.title(r\"$\\omega_z$ @ t=%f\"%V.t[0])\n", "plt.colorbar()\n", "plt.gca().set_aspect('equal')" ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eb4a74e5-b88b-4be3-bcb9-a7c7d9086e74", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { @@ -89,7 +96,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.14" + "version": "3.12.3" } }, "nbformat": 4, diff --git a/SimpleSetup/solution1/read_problem.ipynb b/SimpleSetup/solution1/read_problem.ipynb index d9e721c..fee0ba8 100644 --- a/SimpleSetup/solution1/read_problem.ipynb +++ b/SimpleSetup/solution1/read_problem.ipynb @@ -11,9 +11,8 @@ "%matplotlib widget\n", "import os\n", "import sys\n", - "# These path are specific to this Jureca tutorial\n", - "user=os.getenv(\"USER\")\n", - "sys.path.append(\"/p/project1/training2437/tera_day2/\"+user+\"/idefix/idefix.src/\")\n", + "# Possibly adapt the idefix path depending on where you cloned the sources\n", + "sys.path.append(\"../../../idefix.src\")\n", "from pytools.vtk_io import readVTK\n", "import matplotlib.pyplot as plt\n", "import numpy as np" @@ -42,7 +41,7 @@ "for field in V.data.keys():\n", " plt.figure(figsize=(10,4))\n", " plt.pcolormesh(V.x,V.y,V.data[field][:,:,0].T)\n", - " plt.title(field+ \" @ t=%f\"%V.t)\n", + " plt.title(field+ \" @ t=%f\"%V.t[0])\n", " plt.colorbar()\n", " plt.gca().set_aspect('equal')" ] @@ -67,10 +66,18 @@ "source": [ "plt.figure(figsize=(10,4))\n", "plt.pcolormesh(V.x,V.y,wz[:,:,0].T)\n", - "plt.title(r\"$\\omega_z$ @ t=%f\"%V.t)\n", + "plt.title(r\"$\\omega_z$ @ t=%f\"%V.t[0])\n", "plt.colorbar()\n", "plt.gca().set_aspect('equal')" ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eb4a74e5-b88b-4be3-bcb9-a7c7d9086e74", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { @@ -89,7 +96,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.14" + "version": "3.12.3" } }, "nbformat": 4, diff --git a/VisualisationAndPost/readVTK/Direct_VTK_visualisation.ipynb b/VisualisationAndPost/readVTK/Direct_VTK_visualisation.ipynb index 85d7fe2..16e6931 100644 --- a/VisualisationAndPost/readVTK/Direct_VTK_visualisation.ipynb +++ b/VisualisationAndPost/readVTK/Direct_VTK_visualisation.ipynb @@ -45,9 +45,8 @@ "%matplotlib widget\n", "import os\n", "import sys\n", - "# These path are specific to this Jureca tutorial\n", - "user=os.getenv(\"USER\")\n", - "sys.path.append(\"/p/project1/training2437/tera_day2/\"+user+\"/idefix/idefix.src/\")\n", + "# Possibly adapt the idefix path depending on where you cloned the sources\n", + "sys.path.append(\"../../../idefix.src/\")\n", "from pytools.vtk_io import readVTK" ] }, @@ -321,7 +320,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.14" + "version": "3.12.3" } }, "nbformat": 4, diff --git a/python_requirements.txt b/python_requirements.txt new file mode 100644 index 0000000..2dceb94 --- /dev/null +++ b/python_requirements.txt @@ -0,0 +1,2 @@ +jupyter >= 1.1.1 +ipympl >= 0.10.0 diff --git a/strip-notebooks.sh b/strip-notebooks.sh new file mode 100755 index 0000000..89d26ac --- /dev/null +++ b/strip-notebooks.sh @@ -0,0 +1,7 @@ +#!/bin/bash + +for tmp in $(git ls-files | grep ".ipynb") +do + echo "- Stripping ${tmp}..." + jupyter nbconvert --ClearOutputPreprocessor.enabled=True --ClearMetadataPreprocessor.enabled=False --to=notebook "${tmp}" --output $(basename ${tmp}) +done