Adding Appendix section

sandeepmittal · sandeepmittal · commit e1d3012fd6f9 · 2021-06-23T18:46:17.000-04:00
diff --git a/sphinx/source/appendix/appendix.rst b/sphinx/source/appendix/appendix.rst
@@ -0,0 +1,6 @@
+
+This is dedicated to Installation and Usage section. Description of various parameters used during Chimbuko's Install and Usage are described here.
+
+.. include:: appendix_install.rst
+.. include:: appendix_instrument_with_tau.rst
+.. include:: appendix_usage.rst
diff --git a/sphinx/source/appendix/appendix_install.rst b/sphinx/source/appendix/appendix_install.rst
@@ -0,0 +1,33 @@
+*************
+Installation
+*************
+
+Gtest Installation
+~~~~~~~~~~~~~~~~~~
+
+.. code:: bash
+
+   apt-get install libgtest-dev
+   cd /usr/src/gtest
+   cmake CMakelists.txt
+   make
+   cp *.a /usr/lib
+
+ADIOS2 Build Location
+~~~~~~~~~~~~~~~~~~~~~
+
+.. code:: bash
+
+   ADIOS2_INSTALL_DIR=/gpfs/alpine/world-shared/csc143/jyc/summit/sw/adios2/devel/gcc
+
+GoogleTest on Summit
+~~~~~~~~~~~~~~~~~~~~
+
+.. code:: bash
+
+   cd ${GTEST_SOURCE_DIR}
+   wget https://github.com/google/googletest/archive/release-1.10.0.tar.gz
+   tar -xvzf release-1.10.0.tar.gz
+   cd ${GTEST_BUILD_DIR}
+   CC=gcc CXX=g++ cmake -DCMAKE_INSTALL_PREFIX=${GTEST_INSTALL_DIR} ${GTEST_SOURCE_DIR}/googletest-release-1.10.0
+   make install
diff --git a/sphinx/source/appendix/appendix_instrument_with_tau.rst b/sphinx/source/appendix/appendix_instrument_with_tau.rst
@@ -0,0 +1,12 @@
+*************************
+Instrumentation with TAU
+*************************
+
+Environment Variables TAU
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- **TAU_MAKEFILE=${PATH_TO_TAU_MAKEFILE}** : Ensure the Makefile is one that supports ADIOS2; these contain "adios2" in the filename
+- **TAU_ADIOS2_PERIODIC=1** : This tells Tau to use stepped IO
+- **TAU_ADIOS2_PERIOD=${PERIOD}** : This is the period in microseconds(?) between IO steps. A value of **PERIOD=100000** is a common choice, but a larger value may be necessary if Chimbuko is not able to keep pace with the data flow.
+- **TAU_ADIOS2_ENGINE=SST** : The engine should be SST for an online analysis in which Chimbuko and the application are running simultaneously. Chimbuko also supports offline analysis whereby an application is run without Chimbuko and the engine is set to **BPfile**.
+- **TAU_ADIOS2_FILENAME=${STUB}** : Here ${STUB} becomes the first component of the ADIOS2 filename, eg for "${STUB}=tau-metrics" and an application "main", the filename will be "tau-metrics-main". This filename is needed to start Chimbuko; in online mode the file exists only temporarily and is used to instantiate the communication, whereas for offline mode the filename becomes the location where the trace dump is written.
diff --git a/sphinx/source/appendix/appendix_usage.rst b/sphinx/source/appendix/appendix_usage.rst
@@ -0,0 +1,44 @@
+*********
+Usage
+*********
+
+Visualization Variables
+~~~~~~~~~~~~~~~~~~~~~~~
+
+- **${VIZ_PORT}** : The port to assign to the visualization module
+- **${VIZ_DATA_DIR}**: A directory for storing logs and temporary data (assumed to exist)
+- **${VIZ_INSTALL_DIR}**: The directory where the visualization module is installed
+
+Parameter Server Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- **PSERVER_NT** : The number of threads used to handle incoming communications from the AD modules
+- **PSERVER_LOGDIR** : A directory for logging output
+- **VIZ_ADDRESS** : Address of the visualization module (see above).
+- **PROVDB_ADDR**: The address of the provenance database (see above). This option enables the storing of the final globally-aggregated function profile information into the provenance database.
+- **PSERVER_ALG** : Set AD algorithm to use for online analysis: "sstd" or "hbos". Default value is "hbos".
+
+Note that all the above are optional arguments, although if the **VIZ_ADDRESS** is not provided, no information will be sent to the webserver.
+
+AD Variables
+~~~~~~~~~~~~
+
+- **RANKS** : The number of MPI ranks that the application will be run on
+- **ADIOS2_ENGINE** : The ADIOS2 communications engine. For online analysis this should be **SST** by default (an alternative, **BP4** is discussed below)
+- **ADIOS2_FILE_DIR** : The directory in which the ADIOS2 file is written (see below)
+- **ADIOS2_FILE_PREFIX** : The ADIOS2 file prefix (see below)
+- **PSERVER_ADDR**:  The address of the parameter server from above.
+- **PROVDB_ADDR**:  The address of the provenance database from above.
+- **NSHARDS**: The number of provenance database shards
+
+Additional AD Variables
+~~~~~~~~~~~~~~~~~~~~~~~
+
+- **-prov_outputpath** : The directory in which the provenance data will be output. This can be used in place of or in conjunction with the provenance database. An empty string (default) disables this output.
+- **-outlier_sigma** : The number of standard deviations from the mean function execution time outside which the execution is considered anomalous (default 6)
+- **-anom_win_size** : The number of events around an anomalous function execution that are captured as contextual information and placed in the provenance database and displayed in the visualization (default 10)
+- **-program_idx** : For workflows with multiple component programs, a "program index" must be supplied to the AD instances attached to those processes.
+- **-rank** : By default the data rank assigned to an AD instance is taken from its MPI rank in MPI_COMM_WORLD. This rank is used to verify the incoming trace data. This option allows the user to manually set the rank index.
+- **-override_rank** : This option disables the data rank verification and instead overwrites the data rank of the incoming trace data with the data rank stored in the AD instance. The value supplied must be the original data rank (this is used to generate the correct trace filename).
+- **-ad_algorithm** : This is an option which sets AD algorithm to use for online analysis: "sstd" or "hbos". Default value is "hbos".
+- **-hbos_threshold** : This is the threshold to control density of detected anomalies used by HBOS algorithm. Its value ranges between 0 and 1. Default value is 0.99
diff --git a/sphinx/source/index.rst b/sphinx/source/index.rst
@@ -10,7 +10,7 @@ Chimbuko: Performance Analysis
 - This library is a part of `Chimbuko <https://codarcode.github.io/Chimbuko>`_, a workflow-level scalable performance trace analysis tool.
 - Funded by the `Exascale Computing Project (ECP) <https://www.exascaleproject.org/>`_, U.S. Department of Energy
 
-Related Github repositories 
+Related Github repositories
 
 - `Chimbuko <https://github.com/CODARcode/Chimbuko>`_
 - `Performance Analysis <https://github.com/CODARcode/PerformanceAnalysis>`_
@@ -24,20 +24,24 @@ Related Github repositories
 
 .. toctree::
    :caption: Installation and Usage
-	     
+
    install_usage/install_usage
 
 .. toctree::
    :caption: IO Schema
-	     
+
    io_schema/schema
-   
+
+.. toctree::
+   :caption: Appendix
+
+   appendix/appendix
+
 .. toctree::
    :caption: API For Developers
 
    api/api
 
-
 Indices and tables
 ==================
 
diff --git a/sphinx/source/install_usage/install.rst b/sphinx/source/install_usage/install.rst
@@ -2,10 +2,7 @@
 Installation
 ************
 
-For x86 systems we provide pre-built docker images users can quickly start
-with thier own TAU instrumented applications (See `Chimbuko docker <https://codarcode.github.io/Chimbuko/installation/docker.html>`_) .
-
-Otherwise, first, download (or clone) **Chimbuko** AD module.
+For x86 systems we provide pre-built docker images users can quickly start with their own TAU instrumented applications (See `Chimbuko docker <https://codarcode.github.io/Chimbuko/installation/docker.html>`_). Otherwise, first, download (or clone) **Chimbuko** AD module.
 
 .. code:: bash
 
@@ -42,20 +39,12 @@ If libfabric is pre-installed on the user's system it should be indicated to Spa
 
    spack load mochi-sonata
 
-To build test cases, users need to install gtest.
-
-.. code:: bash
-
-   apt-get install libgtest-dev
-   cd /usr/src/gtest
-   cmake CMakelists.txt
-   make
-   cp *.a /usr/lib
+To build test cases, users need to install gtest. See Instructions to installs gtest `here <../appendix/appendix_install.html>`_.
 
 Finally, to build the AD module
 
 .. code:: bash
-	  
+
    cd ${AD_SOURCE_DIR}
    ./autogen.sh
    cd ${AD_BUILD_DIR}
@@ -64,10 +53,10 @@ Finally, to build the AD module
 
 Here **--with-perf-metric** is an optional flag that enables generation of performance information of Chimbuko. The ZMQ network is the only one presently supported; an MPI network is maintained for legacy purposes and is not recommended for use. Assuming Sonata is installed and the mochi-sonata spack module loaded, the configure script will automatically detect the installation and will build the provenance database.
 
-Once installed the unit and integration tests can be run as:  
+Once installed the unit and integration tests can be run as:
 
 .. code:: bash
-	  
+
    cd ${AD_INSTALL_DIR}/test
    ./run_all.sh
 
@@ -79,18 +68,14 @@ Summit
 Prior to building anything the user should ensure to load the required modules:
 
 .. code:: bash
-	  
+
    source ${AD_SOURCE_DIR}/env.summit.sh
 
 ..
-   We provide :download:`an installation script<files/install_adios2.sh>` for ADIOS2_, 
-   if the latest version is not availale on Summit. 
-
-A build of ADIOS2 that has been tested as compatible with Chimbuko can be found here
+   We provide :download:`an installation script<files/install_adios2.sh>` for ADIOS2_,
+   if the latest version is not availale on Summit.
 
-.. code:: bash
-	  
-   ADIOS2_INSTALL_DIR=/gpfs/alpine/world-shared/csc143/jyc/summit/sw/adios2/devel/gcc
+A build of ADIOS2 that has been tested as compatible with Chimbuko can be found `here <../appendix/appendix_install.html#ADIOS2-build>`_.
 
 The process for building Sonata_ is somewhat more complicated on Summit. The Mochi developers recommend using a Spack environment that can be setup as follows:
 
@@ -109,29 +94,20 @@ Then:
    spack env create mochi summit_spack.yaml
    spack env activate mochi
    spack install
-  
+
 The user must henceforth ensure to activate the **mochi** environment prior to building and using Chimbuko as follows:
 
 .. code:: bash
 
    spack env activate mochi
    spack load mochi-sonata
 
-GoogleTest_ is not installed by default on Summit, hence we must install it as follows:
-
-.. code:: bash
-
-   cd ${GTEST_SOURCE_DIR}
-   wget https://github.com/google/googletest/archive/release-1.10.0.tar.gz
-   tar -xvzf release-1.10.0.tar.gz
-   cd ${GTEST_BUILD_DIR}
-   CC=gcc CXX=g++ cmake -DCMAKE_INSTALL_PREFIX=${GTEST_INSTALL_DIR} ${GTEST_SOURCE_DIR}/googletest-release-1.10.0
-   make install
+GoogleTest_ is not installed by default on Summit, hence we must install it as described `here <../appendix/appendix_install.html#googletest-on-summit>`_.
 
 To build the AD:
 
 .. code:: bash
-	  
+
    cd ${AD_SOURCE_DIR}
    ./autogen.sh
    cd ${AD_BUILD_DIR}
@@ -140,10 +116,10 @@ To build the AD:
    LDFLAGS_IN="-L${GTEST_INSTALL_DIR}/lib64"
 
    CC=mpicc CXX=mpicxx LDFLAGS=${LDFLAGS_IN} CXXFLAGS=${CXXFLAGS_IN} \
-   /path/to/ad/source/configure --with-adios2=${ADIOS2_INSTALL_DIR} --with-network=ZMQ --with-perf-metric --prefix=${AD_INSTALL_DIR} 
+   /path/to/ad/source/configure --with-adios2=${ADIOS2_INSTALL_DIR} --with-network=ZMQ --with-perf-metric --prefix=${AD_INSTALL_DIR}
    make install
 
-   
+
 .. _ADIOS2: https://github.com/ornladios/ADIOS2
 .. _ZeroMQ: https://zeromq.org/
 .. _CURL: https://curl.haxx.se/
diff --git a/sphinx/source/install_usage/instrumenting.rst b/sphinx/source/install_usage/instrumenting.rst
@@ -6,13 +6,7 @@ In this section we briefly describe how to instrument an application with Tau. F
 
 The communication between Tau and Chimbuko's AD module is performed using Tau's ADIOS2 plugin. The communication is performed in batches known as *IO steps* or *IO frames*. During an IO step Tau collects data which is communicated to Chimbuko at the end of the step.
 
-In order to enable the ADIOS2 plugin Tau must be compiled with the **-adios** compile option, pointing to the ADIOS2 install directory. The user must set the following environment variables:
-
-- **TAU_MAKEFILE=${PATH_TO_TAU_MAKEFILE}** : Ensure the Makefile is one that supports ADIOS2; these contain "adios2" in the filename
-- **TAU_ADIOS2_PERIODIC=1** : This tells Tau to use stepped IO
-- **TAU_ADIOS2_PERIOD=${PERIOD}** : This is the period in microseconds(?) between IO steps. A value of **PERIOD=100000** is a common choice, but a larger value may be necessary if Chimbuko is not able to keep pace with the data flow.
-- **TAU_ADIOS2_ENGINE=SST** : The engine should be SST for an online analysis in which Chimbuko and the application are running simultaneously. Chimbuko also supports offline analysis whereby an application is run without Chimbuko and the engine is set to **BPfile**.
-- **TAU_ADIOS2_FILENAME=${STUB}** : Here ${STUB} becomes the first component of the ADIOS2 filename, eg for "${STUB}=tau-metrics" and an application "main", the filename will be "tau-metrics-main". This filename is needed to start Chimbuko; in online mode the file exists only temporarily and is used to instantiate the communication, whereas for offline mode the filename becomes the location where the trace dump is written.
+In order to enable the ADIOS2 plugin Tau must be compiled with the **-adios** compile option, pointing to the ADIOS2 install directory. The user must set a few environment variables as `described here <../appendix/appendix_instrument_with_tau.html#environment-variables-tau>`_.
 
 How Tau is used depends on the language in which the application is written. Below we describe how to instrument applications written in several common languages.
 
@@ -40,7 +34,7 @@ Calls to the GPU via CUDA are instrumented through CUDA's CUPTI performance API
 For example
 
 .. code:: bash
-	  
+
 	  tau_exec -cupti -env -um -T papi,mpi,pthread,cupti,pdt,adios2 ${APPLICATION} ${APPLICATION_OPTS}
 
 The C++ components of the application should be compiled as in the previous section. In the special case of mixed C++/CUDA code, for which the user desires to instrument also the C++ component, the CUDA compiler first separates the	CUDA and C++ code and passes the components to their corresponding compilers. We must therefore specify to the CUDA compiler that it should use the **tau_cxx.sh** compiler wrapper as its C++ compiler, thus:
@@ -51,9 +45,9 @@ For example
 
 	  nvcc -ccbin tau_cxx.sh -x cu code.cc
 
-Here the **-x cu** option ensures that the compiler treats the file as CUDA/C++ and ignores the extension. Note that options passed to the C++ compiler should be prefixed with **-Xcompiler** (for more information see the CUDA compiler documentation). 
+Here the **-x cu** option ensures that the compiler treats the file as CUDA/C++ and ignores the extension. Note that options passed to the C++ compiler should be prefixed with **-Xcompiler** (for more information see the CUDA compiler documentation).
+
 
-  
 Python
 ~~~~~~
 
@@ -62,7 +56,7 @@ As an interpreted language, Python applications must be wrapped by Tau's Python
 For example, for a python3 application:
 
 .. code:: bash
-	  
+
 	  tau_python -vv -tau-python-interpreter=python3 -adios2_trace ${APPLICATION}.py
 
 Note that the ADIOS2 filename required by Chimbuko will be set not to the application name but to the name of the python interpreter, e.g. for **TAU_ADIOS2_FILENAME=tau-metrics** and using python3.6, the filename will be "tau-metrics-python3.6".
diff --git a/sphinx/source/install_usage/run_chimbuko.rst b/sphinx/source/install_usage/run_chimbuko.rst
