Updated docs with command description and appendix

Author: sandeepmittal

sphinx/source/appendix/appendix_usage.rst

@@ -2,9 +2,17 @@
 Usage
 *********
 
+Additional ProvDB Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- **-db_write_dir** : This is used to specify a path to provenance database to write on disk.
+- **-engine** : This is the OFI libfabric provider used for the Mochi stack. Its value can be set to "ofi+tcp;ofi_rxm".
+
 Visualization Variables
 ~~~~~~~~~~~~~~~~~~~~~~~
 
+- **${provdb_writedir}** : A directory which stores provenance database
+- **${provdb_nshards}** : Number of shards used between provenance database and visualization module.
 - **${VIZ_PORT}** : The port to assign to the visualization module
 - **${VIZ_DATA_DIR}**: A directory for storing logs and temporary data (assumed to exist)
 - **${VIZ_INSTALL_DIR}**: The directory where the visualization module is installed
@@ -14,22 +22,25 @@ Parameter Server Variables
 
 - **PSERVER_NT** : The number of threads used to handle incoming communications from the AD modules
 - **PSERVER_LOGDIR** : A directory for logging output
-- **VIZ_ADDRESS** : Address of the visualization module (see above).
-- **PROVDB_ADDR**: The address of the provenance database (see above). This option enables the storing of the final globally-aggregated function profile information into the provenance database.
 - **PSERVER_ALG** : Set AD algorithm to use for online analysis: "sstd" or "hbos". Default value is "hbos".
 
 Note that all the above are optional arguments, although if the **VIZ_ADDRESS** is not provided, no information will be sent to the webserver.
 
+Additional pserver Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- **-ws_addr** : Address of the visualization module.
+- **-provdb_addr** : The address of the provenance database (see above). This option enables the storing of the final globally-aggregated function profile information into the provenance database.
+- **-prov_outputpath** : This is the path to the provenance database on disk.
+
 AD Variables
 ~~~~~~~~~~~~
 
-- **RANKS** : The number of MPI ranks that the application will be run on
-- **ADIOS2_ENGINE** : The ADIOS2 communications engine. For online analysis this should be **SST** by default (an alternative, **BP4** is discussed below)
-- **ADIOS2_FILE_DIR** : The directory in which the ADIOS2 file is written (see below)
-- **ADIOS2_FILE_PREFIX** : The ADIOS2 file prefix (see below)
-- **PSERVER_ADDR**:  The address of the parameter server from above.
-- **PROVDB_ADDR**:  The address of the provenance database from above.
-- **NSHARDS**: The number of provenance database shards
+- **${ADIOS2_ENGINE}** : The ADIOS2 communications engine. For online analysis this should be **SST** by default (an alternative, **BP4** is discussed below)
+- **${ADIOS2_PATH}** : The directory in which the ADIOS2 file is written (see below)
+- **${ADIOS2_FILE_PREFIX}** : The ADIOS2 file prefix.
+- **${EXE_NAME}** : Name of the executable of application (see examples).
+- **${ad_opts}** : This is a collection of all other `arguments <./appendix_usage.html#additional-ad-variables>`_ required by AD module for its instantiation.
 
 Additional AD Variables
 ~~~~~~~~~~~~~~~~~~~~~~~
@@ -40,5 +51,11 @@ Additional AD Variables
 - **-program_idx** : For workflows with multiple component programs, a "program index" must be supplied to the AD instances attached to those processes.
 - **-rank** : By default the data rank assigned to an AD instance is taken from its MPI rank in MPI_COMM_WORLD. This rank is used to verify the incoming trace data. This option allows the user to manually set the rank index.
 - **-override_rank** : This option disables the data rank verification and instead overwrites the data rank of the incoming trace data with the data rank stored in the AD instance. The value supplied must be the original data rank (this is used to generate the correct trace filename).
-- **-ad_algorithm** : This is an option which sets AD algorithm to use for online analysis: "sstd" or "hbos". Default value is "hbos".
-- **-hbos_threshold** : This is the threshold to control density of detected anomalies used by HBOS algorithm. Its value ranges between 0 and 1. Default value is 0.99
+- **-ad_algorithm** : This sets the AD algorithm to use for online analysis: "sstd" or "hbos". Default value is "hbos".
+- **-hbos_threshold** : This sets the threshold to control density of detected anomalies used by HBOS algorithm. Its value ranges between 0 and 1. Default value is 0.99
+
+Application Variables
+~~~~~~~~~~~~~~~~~~~~~
+
+- **${APPLICATION}** : Application executable.
+- **${APPLICATION_ARGS}** : List of arguments specific to the application.

sphinx/source/install_usage/run_chimbuko.rst

@@ -37,28 +37,30 @@ The user should also choose a port for the provenance database, to which we assi
 
 .. code:: bash
 
-	  provdb_admin ${HEAD_NODE_IP}:${PROVDB_PORT} -nshards ${NSHARDS} -nthreads ${NTHREADS} &
+	  provdb_admin ${HEAD_NODE_IP}:${PROVDB_PORT} -nshards ${NSHARDS} -nthreads ${NTHREADS} ${provdb_extra_args} &
 	  sleep 2
 
 For performance reasons the provenance database is **sharded** and the writing is threaded allowing parallel writes to different shards. By default there is only a single shard and thread, but for larger jobs the user should specify more shards and threads using the **-nshards ${NSHARDS}** and **-nthreads ${NTHREADS}** options respectively. The number of shards must also be communicated to the AD module below. The optimal number of shards and threads will depend on the system characteristics, however the number of threads should be at least as large as the number of shards. We recommend that the user run our provided benchmark application and increase the number of each to minimize the round-trip latency.
 
+**${provdb_extra_args}** is a variable that is used to provide any additional arguments to provdb_admin. A list of such additional variables can be found `here <../appendix/appendix_usage.html#additional-provdb-variables>`_.
+
 The database will be written to disk into the directory from which the **provdb_admin** application is called, under the filename **provdb.${SHARD}.unqlite** where **${SHARD}** is the index of the database shard.
 
 For use below we define the variable **PROVDB_ADDR=tcp://${HEAD_NODE_IP}:${PROVDB_PORT}**. For convenience, the **provdb_admin** application will write out a file **provider.address**, the contents of which can be used in place of manually defining this variable.
 
 ----------------------------------
 
-The second step is to instantiate the visualization module:
+The second step is to instantiate the visualization module.
 
 .. code:: bash
 
     export DATABASE_URL="sqlite:///${VIZ_DATA_DIR}/main.sqlite"
     export ANOMALY_STATS_URL="sqlite:///${VIZ_DATA_DIR}/anomaly_stats.sqlite"
     export ANOMALY_DATA_URL="sqlite:///${VIZ_DATA_DIR}/anomaly_data.sqlite"
     export FUNC_STATS_URL="sqlite:///${VIZ_DATA_DIR}/func_stats.sqlite"
-    export PROVENANCE_DB="${PWD}/"
+    export PROVENANCE_DB="${provdb_writedir}"
     export PROVDB_ADDR=$(cat provider.address)
-    export SHARDED_NUM=1
+    export SHARDED_NUM=${provdb_nshards}
     export C_FORCE_ROOT=1   #REQUIRED FOR DOCKER IMAGES ONLY
 
     cd ${VIZ_INSTALL_DIR}
@@ -89,25 +91,36 @@ For details on the installation and usage of the visualization module, please re
 
 ----------------------------------
 
-The third step is to start the parameter server:
+The third step is to start the parameter server. This can be achieved by running the following **pserver** command.
 
 .. code:: bash
 
-	  pserver -nt ${PSERVER_NT} -logdir ${PSERVER_LOGDIR} -ws_addr ${VIZ_ADDRESS} -provdb_addr ${PROVDB_ADDR} -ad ${PSERVER_ALG} &
+	  pserver -nt ${PSERVER_NT} -logdir ${PSERVER_LOGDIR} -ad ${PSERVER_ALG} &
 	  sleep 2
 
-Description of the variables can be found `here <../appendix/appendix_usage.html#parameter-server-variables>`_.
+Description of the variables can be found `here <../appendix/appendix_usage.html#parameter-server-variables>`_. **${ps_extra_args}** can be used to provide additional arguments to the pserver command, as described `here <../appendix/appendix_usage.html#additional-pserver-variables>`_.
 
 The parameter server opens communications on TCP port 5559. For use below we define the variable **PSERVER_ADDR=${HEAD_NODE_IP}:5559**.
 
 ----------------------------------
 
-The fourth step is to instantiate the AD modules:
+The provenance database, visualization module and parameter server are launched using the following **jsrun** command:
 
 .. code:: bash
 
-	  mpirun -n ${RANKS} driver ${ADIOS2_ENGINE} ${ADIOS2_FILE_DIR} ${ADIOS2_FILE_PREFIX} -pserver_addr ${PSERVER_ADDR} -provdb_addr ${PROVDB_ADDR} -nprovdb_shards ${NSHARDS} &
-	  sleep 2
+		#Run the services
+		jsrun ${SERVICES}
+
+**${SERVICES}** is the path to a script which includes commands from the previously described first, second and third step, respectively. This command should successfully launch the provenance database, the visualization module, and the parameter server.
+
+----------------------------------
+
+Next, the AD module can be instantiated using **jsrun** command as follows:
+
+.. code:: bash
+
+    jsrun -e prepended driver ${ADIOS2_ENGINE} ${ADIOS2_PATH} ${ADIOS2_FILE_PREFIX}-${EXE_NAME} ${ad_opts}
+    sleep 2
 
 Description of the variables can be found `here <../appendix/appendix_usage.html#ad-variables>`_.
 
@@ -117,7 +130,7 @@ The **ADIOS2_ENGINE** can be chosen as either **SST** or **BP4**. The former use
 
 In the above we have assumed that the provenance database is being used. However if this component is not in use, the AD will automatically output the provenance data as JSON documents "${STEP}.anomalies.json", "${STEP}.normalexecs.json" and "${STEP}.metadata.json" placed in the directory "${PROV_DIR}/${PROGRAM_IDX}/${RANK}", where STEP is the i/o step; PROGRAM_IDX is the program index; RANK is the rank of the AD instance; and PROV_DIR is set by default to the working directory but can specified manually using the optional argument -prov_outputpath (cf. below).
 
-The AD module has a number of additional options that can be used to tune its behavior. The full list can be obtained by running **driver** without any arguments. However a few useful options are described `here <../appendix/appendix_usage.html#additional-ad-variables>`_.
+The AD module has a number of additional options that can be used to tune its behavior. The full list can be obtained by running **driver** without any arguments. However a few useful options are described `here <../appendix/appendix_usage.html#additional-ad-variables>`_. These are part of the **${ad_opts}** in the above command.
 
 For debug purposes, the AD module can be made more verbose by setting the environment variable **CHIMBUKO_VERBOSE=1**.
 
@@ -131,9 +144,10 @@ The final step is to instantiate the application
 
 .. code:: bash
 
-	  mpirun -n ${RANKS} ${APPLICATION} ${APPLICATION_ARGS}
+	  jsrun -e prepended ${APPLICATION} ${APPLICATION_ARGS}
 
 Aside from interacting with the visualization module, once complete the user can also interact directly with the provenance database using the **provdb_query** tool as described below: :ref:`install_usage/run_chimbuko:Interacting with the Provenance Database`.
+Description of variables is provided `here <../appendix/appendix_usage.html#application-variables>`_.
 
 Offline Analysis
 ~~~~~~~~~~~~~~~~