Updating guide for v1.7

AnandiKarunaratne · AnandiKarunaratne · commit 4d77ca250881 · 2024-08-22T18:38:57.000+10:00
diff --git a/jbpt-pm/entropia/guide/bib.bib b/jbpt-pm/entropia/guide/bib.bib
@@ -60,3 +60,30 @@ @inproceedings{KalenkovaP2020
   year      = {2020},
   note			= {{In Press}}
 }
+
+@inproceedings{Polyvyanyy2022Bootstrapping,
+  author    = {Polyvyanyy, Artem and Moffat, Alistair and Garc{\'{\i}}a{-}Ba{\~{n}}uelos, Luciano},
+  title     = {Bootstrapping Generalization of Process Models Discovered From Event Data},
+  booktitle = caise,
+  pages     = {36--54},
+  hidepublisher = {Springer},
+  year      = {2022}
+}
+
+@inproceedings{Kabierski2023Addressing,
+  title     = {Addressing the Log Representativeness Problem using Species Discovery},
+  author    = {Kabierski, Martin and Richter, Markus and Weidlich, Matthias},
+  booktitle = icpm,
+  pages     = {65--72},
+  year      = {2023},
+  hideorganization = {IEEE},
+}
+
+@InProceedings{Karunaratne2024Role,
+  author    = {Anandi Karunaratne and Artem Polyvyanyy and Alistair Moffat},
+  booktitle = icpm,
+  title     = {The Role of Log Representativeness in Estimating Generalization in Process Mining},
+  year      = {2024},
+  publisher = {{IEEE}},
+  note      = {{to appear}},
+}
diff --git a/jbpt-pm/entropia/guide/fig/log3.pdf b/jbpt-pm/entropia/guide/fig/log3.pdf
diff --git a/jbpt-pm/entropia/guide/fig/model3.pdf b/jbpt-pm/entropia/guide/fig/model3.pdf
diff --git a/jbpt-pm/entropia/guide/fig/system3.pdf b/jbpt-pm/entropia/guide/fig/system3.pdf
diff --git a/jbpt-pm/entropia/guide/guide.tex b/jbpt-pm/entropia/guide/guide.tex
@@ -2,22 +2,23 @@
 \input{include}
 \input{style}
 
-\title{\textbf{Entropia 1.5 \\ \large User's Guide}} 
-\author[1]{Artem Polyvyanyy}
-\author[1]{Hanan Alkhammash}
-\author[2]{Claudio Di Ciccio}
-\author[3]{Luciano García-Bañuelos}
-\author[1]{Anna Kalenkova}
-\author[4]{Sander J. J. Leemans}
-\author[5]{Jan Mendling}
-\author[1]{Alistair Moffat}
-\author[6]{Matthias Weidlich}
-\affil[1]{The University of Melbourne \authorcr {\tt {\{artem.polyvyanyy,anna.kalenkova,ammoffat\}@unimelb.edu.au}}\\ {\tt halkhammash@student.unimelb.edu.au}\vspace{1.3ex}}
-\affil[2]{Sapienza University of Rome, \authorcr {\tt {diciccio@di.uniroma1.it}\vspace{1.3ex}}}
-\affil[3]{Tecnológico de Monterrey, \authorcr {\tt {luciano.garcia@tec.mx}\vspace{1.3ex}}}
-\affil[4]{Queensland University of Technology, \authorcr {\tt {s.leemans@qut.edu.au}\vspace{1.3ex}}}
-\affil[5]{Vienna University of Economics and Business, \authorcr {\tt {jan.mendling@wu.ac.at}\vspace{1.3ex}}}
-\affil[6]{Humboldt-Universität zu Berlin, \authorcr {\tt {matthias.weidlich@hu-berlin.de}}}
+\title{\textbf{Entropia 1.7 \\ \large User's Guide}}
+%\author[1]{Artem Polyvyanyy}
+\author[1]{Hanan Alkhammash\thanks{Documentation completed for version 1.5 on August 22, 2020.}}
+%\author[2]{Claudio Di Ciccio}
+%\author[3]{Luciano García-Bañuelos}
+%\author[1]{Anna Kalenkova}
+%\author[4]{Sander J. J. Leemans}
+%\author[5]{Jan Mendling}
+%\author[1]{Alistair Moffat}
+%\author[6]{Matthias Weidlich}
+\author[1]{Anandi Karunaratne\thanks{Documentation updated for version 1.7 on \today.}}
+\affil[1]{The University of Melbourne \authorcr {\tt {\{halkhammash, anandik\}@student.unimelb.edu.au}}}
+%\affil[2]{Sapienza University of Rome, \authorcr {\tt {diciccio@di.uniroma1.it}\vspace{1.3ex}}}
+%\affil[3]{Tecnológico de Monterrey, \authorcr {\tt {luciano.garcia@tec.mx}\vspace{1.3ex}}}
+%\affil[4]{Queensland University of Technology, \authorcr {\tt {s.leemans@qut.edu.au}\vspace{1.3ex}}}
+%\affil[5]{Vienna University of Economics and Business, \authorcr {\tt {jan.mendling@wu.ac.at}\vspace{1.3ex}}}
+%\affil[6]{Humboldt-Universität zu Berlin, \authorcr {\tt {matthias.weidlich@hu-berlin.de}}}
 \date{}
 
 
@@ -31,6 +32,8 @@
 \input{sec_02}
 \input{sec_03}
 \input{sec_04}
+\input{sec_05}
+\input{sec_06}
 
 \newpage
 \bibliographystyle{plain}
diff --git a/jbpt-pm/entropia/guide/intro.tex b/jbpt-pm/entropia/guide/intro.tex
@@ -15,4 +15,6 @@
 \item[1.] Stochastic Precision and Recall  \cite{Leemans2020}; and
 \item[2.] Entropic Relevance \cite{abs-2007-09310}. 
 \end{enumerate}
+\item Bootstrap Generalization: A section detailing the \textit{Entropia} commands and processes for applying the Bootstrap Generalization approach to assess the generalization of a discovered process model described as a directly-follows-graph (DFG).
+\item Log Representativeness Measures: Learn about the measures used to evaluate log representativeness and how to apply them using \textit{Entropia}, ensuring that your logs adequately reflect the underlying process.
 \end{enumerate}
diff --git a/jbpt-pm/entropia/guide/screens/screen_(-bgen1).txt b/jbpt-pm/entropia/guide/screens/screen_(-bgen1).txt
@@ -0,0 +1,30 @@
+Computing bootstrap generalization.
+The technique is described in:
+Artem Polyvyanyy, Alistair Moffat, Luciano Garcia-Bonuelos. Bootstrapping
+Generalization of Process Models Discovered from Event Data. CAiSE 2022
+
+===========================Calculating generalization===========================
+
+Sample Size = 264
+Number of Log Generations = 16
+Crossover Subtrace Length = 2
+Breeding Probability = 1.0
+Threshold for confidence interval of bootstrap samples = 0.01
+
+Model-log precision and recall calculated for bootstrap sample 1: 0.8640223021976667, 0.9481225021326511
+Model-log precision and recall calculated for bootstrap sample 2: 0.846835226895293, 0.9298562086663995
+Model-log precision and recall calculated for bootstrap sample 3: 0.8789360974064723, 0.9703714487638393
+Model-log precision and recall calculated for bootstrap sample 4: 0.8640223021901551, 0.9481512297480506
+Model-log precision and recall calculated for bootstrap sample 5: 0.8399359447535413, 0.9273141266860363
+Model-log precision and recall calculated for bootstrap sample 6: 0.8468352269011715, 0.9418416732973515
+Model-log precision and recall calculated for bootstrap sample 7: 0.8789360974064715, 0.9377155545915118
+Model-log precision and recall calculated for bootstrap sample 8: 0.8789360974064718, 0.9426623519201563
+Model-log precision and recall calculated for bootstrap sample 9: 0.8789360974064719, 0.9377155545915125
+Model-log precision and recall calculated for bootstrap sample 10: 0.8640223021081593, 0.9368237287963496
+Model-log precision and recall calculated for bootstrap sample 11: 0.8789360974064686, 0.9528753340039785
+
+===========================Calculated generalization============================
+
+Generalization calculated in 6296 ms with 11 samples.
+Model-system precision: 0.8654867083707584 +/- 0.009628352340204586
+Model-system recall: 0.9430408830179853 +/- 0.007616113385628391
diff --git a/jbpt-pm/entropia/guide/screens/screen_(-bgen2).txt b/jbpt-pm/entropia/guide/screens/screen_(-bgen2).txt
@@ -0,0 +1,2 @@
+> java -jar jbpt-pm-entropia-1.7.jar -bgen -rel=examples/log3.xes -ret=examples/model3.json -s -m=1000 -ep=0.005
+0.8646947781195624, 0.9443517823270182
diff --git a/jbpt-pm/entropia/guide/screens/screen_(-bgen3).txt b/jbpt-pm/entropia/guide/screens/screen_(-bgen3).txt
@@ -0,0 +1,39 @@
+Computing bootstrap generalization.
+The technique is described in:
+Artem Polyvyanyy, Alistair Moffat, Luciano Garcia-Bonuelos. Bootstrapping
+Generalization of Process Models Discovered from Event Data. CAiSE 2022
+
+===========================Calculating generalization===========================
+
+Sample Size = 1000
+Number of Samples = 20
+Number of Log Generations = 16
+Crossover Subtrace Length = 2
+Breeding Probability = 0.5
+
+Model-log precision and recall calculated for bootstrap sample 1: 0.8640223023960756, 0.9316078514432434
+Model-log precision and recall calculated for bootstrap sample 2: 0.8882953280705439, 0.9416055428031388
+Model-log precision and recall calculated for bootstrap sample 3: 0.8640223022539603, 0.921805445155556
+Model-log precision and recall calculated for bootstrap sample 4: 0.8640223022681677, 0.9481512298336585
+Model-log precision and recall calculated for bootstrap sample 5: 0.8640223022654226, 0.9368237289668605
+Model-log precision and recall calculated for bootstrap sample 6: 0.864022302362209, 0.9481512299368573
+Model-log precision and recall calculated for bootstrap sample 7: 0.8640223022423997, 0.9667939526581532
+Model-log precision and recall calculated for bootstrap sample 8: 0.864022302265727, 0.9481512298309965
+Model-log precision and recall calculated for bootstrap sample 9: 0.8640223022506948, 0.9316078512864893
+Model-log precision and recall calculated for bootstrap sample 10: 0.86402230226115, 0.9481512298259573
+Model-log precision and recall calculated for bootstrap sample 11: 0.8789360974064712, 0.9703714487638323
+Model-log precision and recall calculated for bootstrap sample 12: 0.8640223022574599, 0.9481512298219085
+Model-log precision and recall calculated for bootstrap sample 13: 0.8789360974064726, 0.9376442101432018
+Model-log precision and recall calculated for bootstrap sample 14: 0.8789360974064709, 0.942662351920154
+Model-log precision and recall calculated for bootstrap sample 15: 0.8640223022271509, 0.9539061777646949
+Model-log precision and recall calculated for bootstrap sample 16: 0.864022302276446, 0.9218042849623546
+Model-log precision and recall calculated for bootstrap sample 17: 0.878936097406473, 0.9528750949556504
+Model-log precision and recall calculated for bootstrap sample 18: 0.8640223022513908, 0.9481512298152467
+Model-log precision and recall calculated for bootstrap sample 19: 0.8789360974064728, 0.9377155545497414
+Model-log precision and recall calculated for bootstrap sample 20: 0.8640223022648623, 0.9539061778063306
+
+===========================Calculated generalization============================
+
+Generalization calculated in 12226 ms with 20 samples.
+Model-system precision: 0.8689644023473011 +/- 0.0036445059613284285
+Model-system recall: 0.9445018526122011 +/- 0.005750771071582731
diff --git a/jbpt-pm/entropia/guide/screens/screen_(-l1).txt b/jbpt-pm/entropia/guide/screens/screen_(-l1).txt
@@ -0,0 +1,22 @@
+Computing event log statistics
+Completeness and Coverage are described in:
+M. Kabierski, M. Richter, and M. Weidlich, "Addressing the
+log representativeness problem using species discovery," in ICPM 2023
+
+============================= Event Log Statistics =============================
+
+--------------------------- Activity-based Analysis ----------------------------
+Activities: [A, B, C]
+Number of distinct activities: 3
+Number of total activities: 121
+Completeness: 1.0
+
+------------------- Directly-follows-relation-based Analysis -------------------
+Number of distinct DF-relations: 5
+Number of total DF-relations: 88
+Completeness: 1.0
+
+----------------------------- Trace-based Analysis -----------------------------
+Number of distinct traces: 5
+Number of total traces: 33
+Completeness: 1.0
diff --git a/jbpt-pm/entropia/guide/screens/screen_(-l2).txt b/jbpt-pm/entropia/guide/screens/screen_(-l2).txt
@@ -0,0 +1,17 @@
+Computing event log statistics
+Completeness and Coverage are described in:
+M. Kabierski, M. Richter, and M. Weidlich, "Addressing the
+log representativeness problem using species discovery," in ICPM 2023
+
+============================= Event Log Statistics =============================
+
+--------------------------- Activity-based Analysis ----------------------------
+Activities: [A, B, C]
+Number of distinct activities: 3
+Number of total activities: 121
+Coverage: 1.0
+
+------------------- Directly-follows-relation-based Analysis -------------------
+Number of distinct DF-relations: 5
+Number of total DF-relations: 88
+Coverage: 1.0
diff --git a/jbpt-pm/entropia/guide/screens/screen_(-l3).txt b/jbpt-pm/entropia/guide/screens/screen_(-l3).txt
@@ -0,0 +1,2 @@
+> java -jar jbpt-pm-entropia-1.7.jar -l=examples/log3.xes -lra -tr -s
+0.8484848484848485
diff --git a/jbpt-pm/entropia/guide/sec_02.tex b/jbpt-pm/entropia/guide/sec_02.tex
@@ -1,6 +1,8 @@
 \section*{Models and Event Log}
-The tutorial will use example files that are provided with the tool, which are the event log coded in XES format (log.xes), process model modelled as a Petri Net (model.pnml) and stochastic process model as an SDFA coded in JSON format (automaton.json). Table 1, Figure 1 and Figure 2 represent the three files, respectively. 
-
+For the first two sections, which cover conformance checking measures for precision and recall,
+the tutorial will use example files that are provided with the tool, which are the event log coded in XES format (log.xes), process model modelled as a Petri Net (model.pnml) and stochastic process model as an SDFA coded in JSON format (automaton.json). Table 1, Figure 1 and Figure 2 represent the three files, respectively.
+For the last two sections where the bootstrap generalization measure and log representativeness measures are explained,
+we will use Figure~\ref{fig:sys3} (system3.json) as the system, Figure~\ref{fig:model3} (model3.json) as the model, Table~\ref{fig:log3} (log3.xes) as the event log generated from the system in Figure~\ref{fig:sys3}.
 
 \begin{figure}[h!]
 \hspace{3mm}
@@ -70,16 +72,50 @@ \section*{Models and Event Log}
 \end{center}
 \end{minipage}
 
-\begin{center}
+%\begin{center}
 \vspace{5mm}
-\begin{minipage}{0.45\textwidth}
- \centering
-\includegraphics[width=0.40\textwidth]{fig/eventLog.pdf} 
-\caption*{ Table 1: Event Log}
-\label{fig:Log}
-\end{minipage}
-\end{center}
-\vspace{-4mm}
+%\begin{minipage}{0.45\textwidth}
+% \centering
+%\includegraphics[width=0.40\textwidth]{fig/eventLog.pdf}
+%\caption*{ Table 1: Event Log}
+%\label{fig:Log}
+%\end{minipage}
+%\end{center}
+%\vspace{-4mm}
 \end{figure}
 
+\begin{minipage}{\textwidth}
+	\begin{minipage}{0.34\textwidth}
+		\begin{minipage}{\textwidth}
+			\centering
+			\includegraphics[width=\textwidth]{fig/system3}
+			\vspace{-2em}
+			\captionof{figure}{A ystem represented as a DFG.}
+			\label{fig:sys3}
+		\end{minipage}
+		\begin{minipage}{\textwidth}
+			\centering
+			\includegraphics[width=\textwidth]{fig/model3}
+			\vspace{-2em}
+			\captionof{figure}{A process model represented as a DFG.}
+			\label{fig:model3}
+		\end{minipage}
+	\end{minipage}%
+	\hfill
+	\begin{minipage}{0.3\textwidth}
+		\centering
+		\includegraphics[width=0.6\textwidth]{fig/eventLog}
+		\captionof{table}{An Event Log.}
+		\label{fig:log1}
+	\end{minipage}
+	\begin{minipage}{0.32\textwidth}
+		\centering
+		\includegraphics[width=0.8\textwidth]{fig/log3}
+		\captionof{table}{An Event Log generated by system in Figure~\ref{fig:sys3}.}
+		\label{fig:log3}
+	\end{minipage}
+\end{minipage}
+
+
+\vspace{1em}
 All files are located in the folder \textbf{jbpt-pm\slash entropia\slash examples\slash}. If the folder does not contain the files, you can download them from \url{https://github.com/jbpt/codebase/tree/master/jbpt-pm/entropia/examples}. It is recommended to use the event log and models provided to understand how to run the \textit{Entropia} commands. Then, try it with your own data. Refer to Table III in the demo paper for more details about file types and formats supported by the tool.
diff --git a/jbpt-pm/entropia/guide/sec_05.tex b/jbpt-pm/entropia/guide/sec_05.tex
@@ -0,0 +1,81 @@
+\section*{Bootstrap Generalization Measure}
+\setcounter{subsection}{0}
+
+To compute the generalization of a process model with bootstrap generalization approach~\cite{Polyvyanyy2022Bootstrapping}, use the option (\textcolor{darkcandyapplered}{\footnotesize\ttfamily-bgen}) as follows.
+\begin{lstlisting}[style=CL]
+>java -jar !jbpt-pm-entropia-1.5.jar! @-bgen@ &-rel&=log3.xes &-ret&=model3.json
+\end{lstlisting}
+
+\textbf{Output Screen:}%chnage
+\lstinputlisting[style=DOS]{screens/screen_(-bgen1).txt}
+
+Note that the model you want to analyze should be provided in JSON format.
+Each DFG node has a string label for the activity name, a numerical frequency for activity execution, and a unique identifier.
+Arcs connect nodes by their source and target numbers, along with their occurrence frequency.
+The start and end nodes should be labeled \texttt{INPUT} and \texttt{OUTPUT} to indicate process boundaries.
+Refer to Listing~\ref{lst:dfg} for a sample representation.
+%\begin{wrapfigure}{R}{0.35\textwidth}
+%  \vspace{-5mm}
+
+\begin{minipage}[t]{\textwidth}
+        \vspace{0pt}
+        \begin{lstlisting}[caption={Sample JSON representation.}, label={lst:dfg}]
+{ "nodes": [
+    { "id": 1, "label": "INPUT",  "freq": 30 },
+    { "id": 2, "label": "A",      "freq": 30 },
+    { "id": 3, "label": "B",      "freq": 30 },
+    { "id": 4, "label": "OUTPUT", "freq": 30 }
+  ],
+  "arcs": [
+    { "from": 1, "to": 2, "freq": 30 },
+    { "from": 2, "to": 3, "freq": 30 },
+    { "from": 3, "to": 4, "freq": 18 }
+  ]
+}
+        \end{lstlisting}
+    \end{minipage}
+%    \vspace{-5mm}
+%\end{wrapfigure}
+
+You can adjust the underlying bootstrapping process with several parameters for bootstrap generalization estimation, including sample size (\texttt{n}), number of samples (\texttt{m}), number of log generations (\texttt{g}), crossover subtrace length (\texttt{k}), breeding probability (\texttt{p}), and threshold for confidence interval of bootstrap samples (\texttt{ep}).
+These parameters, detailed in~\cite{Polyvyanyy2022Bootstrapping}, are optional for tool usage.
+
+\begin{lstlisting}[style=CL]
+>java -jar !jbpt-pm-entropia-1.5.jar! @-bgen@ &-rel&=log3.xes &-ret&=model3.json -s &-m&=1000
+&-ep&=0.005
+\end{lstlisting}
+
+\textbf{Output Screen:}%chnage
+\lstinputlisting[style=DOS]{screens/screen_(-bgen2).txt}
+
+Here, bootstrapping continues until 1,000 samples
+are reached, unless the confidence interval for both
+precision and recall of the bootstrap samples falls
+below 0.005, triggering early termination.
+And this calculation will  be done in silent mode,
+showing only the final result.
+
+%Model-system precision: 0.922949821721433 +/- 0.00469559925550323
+%Model-system recall: 0.923083213341255 +/- 0.004965976603082337
+
+Another configuration might be as follows:
+
+\begin{lstlisting}[style=CL]
+>java -jar !jbpt-pm-entropia-1.5.jar! @-bgen@ &-rel&=log3.xes &-ret&=model3.json &-n&=1000 &-m&=20
+&-p&=0.5
+\end{lstlisting}
+
+\textbf{Output Screen:}%chnage
+\lstinputlisting[style=DOS]{screens/screen_(-bgen3).txt}
+
+This configuration generates 20 bootstrap samples, each
+containing 1,000 traces, with new traces being generated
+50\% of the time, and existing traces used the remaining 50\%.
+
+
+
+
+
+
+
+
diff --git a/jbpt-pm/entropia/guide/sec_06.tex b/jbpt-pm/entropia/guide/sec_06.tex

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+> java -jar jbpt-pm-entropia-1.7.jar -bgen -rel=examples/log3.xes -ret=examples/model3.json -s -m=1000 -ep=0.005`
	`2`	`+0.8646947781195624, 0.9443517823270182`