Uni/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

.

Browse Source
This commit is contained in:
Matthias
2017-08-16 17:37:54 +02:00
parent 18f1c768cf
commit 7d03ad7051
42 changed files with 20274 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
workflowtransformations/workflowtransformations-evaluation.tex
+143
View File
@@ -0,0 +1,143 @@
\section{Experiments}\label{sec:experiments}
We conducted all experiments on a multi-core system with four Intel Xeon E7-4870 10-core CPUs at \SI{2.40}{\giga\hertz} with \SI{128}{\giga\byte} of RAM. We used the GNU \texttt{gcc} compiler version 4.9.1 to compile the generated workflow programs with the \texttt{-O3} optimization flag.
We used a combination of real-world workflows to show that compiler transformations are practically applicable,
supplemented by a number of synthetic workflows to better highlight their potential positive effects. We manually chose the applied the transformations, and plan to research an automatic optimizer based on the experience gained from these experiments in future work.
For each experiment, we recorded the workflow enactment time in both early and late evaluation modes, and their memory utilization. The enactment time refers to the time spent by the engine for evaluating the WEP and resolving the dependencies and does not include the time spent in executing atomic activities. We used the resident size reported by the operating system to measure the engine's memory consumption. Since there is no difference in memory consumption in early and late evaluation modes, we report a single joint result for both. We also report a single value for the number of WEP nodes for early and late evaluation modes for the same reasons.
We used the HEFT~\cite{heft02} scheduling algorithm in all experiments unless otherwise stated.
\subsection{Engine Overhead Reduction}
\begin{figure*}[t]
\centering
\subfloat[Original workflow.]{\includegraphics[height=4cm]{./figures/Raincloud-experiments1.pdf}\label{fig:raincloud-structure1}}\qquad
\subfloat[After loop unswitching.]{\includegraphics[height=4cm]{./figures/Raincloud-experiments2.pdf}\label{fig:raincloud-structure2}}\qquad
\subfloat[After loop unswitching and loop unrolling.]{\includegraphics[height=4cm]{./figures/Raincloud-experiments3.pdf}\label{fig:raincloud-structure3}}
\caption{RainCloud workflow.}
\label{fig:raincloud-structure}
\end{figure*}
We show in this section the effects of transformations on reducing engine overhead for two real-world workflows.
\subsubsection{RainCloud} is a meteorological workflow for weather simulation in mountainous regions that uses a simple numerical linear model of orographic precipitations~\cite{Schuller2013} (see Figure~\ref{fig:raincloud-structure1}). The \texttt{prepareLM} activity partitions the input data in several chunks, independently processed by a \texttt{parallel for each} loop. The number of chunks and the degree of parallelism is controlled by an input port called \texttt{NGroups}. The workflow features an optional post-processing activity of the results called \texttt{PPS}, and an optional post-processing activity of all results called \texttt{PPF}.
We first applied loop unswitching that moves the \texttt{if} activity out of the \texttt{parallel foreach} loop (see Figure~\ref{fig:raincloud-structure2}), followed by a ten-fold loop unrolling, as shown in Figure~\ref{fig:raincloud-structure3} (we only unrolled the loop twice in the figure for brevity reasons).
To enable loop unrolling, we exchanged \texttt{parallel foreach} with a \texttt{parallel for} loop to process more data chunks per iteration in the transformed workflow. We also introduced three additional helper activities: (1) \texttt{H1} that calculates the loop counter boundaries of the new \texttt{parallel for} loop, (2) \texttt{H2} that distributes the data chunks among the unrolled activities, and (3) \texttt{H3} that collects the results from the unrolled activities and passes them to the output port of the \texttt{parallel for} loop.
%We compared in our experiments the original version of the Raincloud workflow with a version that only has Loop Unswitching applied, and with a version that has Loop Unswitching and Loop Unrolling applied.
\begin{figure*}[t]
\centering
\subfloat[Early evaluation enactment.]{\includegraphics[width=.33\textwidth]{./figures/raincloud-EnactmentTimeEarly.pdf}\label{fig:raincloud-results-enactment-early}}
\subfloat[Late evaluation enactment.]{\includegraphics[width=.33\textwidth]{./figures/raincloud-EnactmentTimeLate.pdf}\label{fig:raincloud-results-enactment-late}}
\subfloat[WEP length.]{\includegraphics[width=.33\textwidth]{./figures/raincloud-NodeCountLate.pdf}\label{fig:raincloud-results-nodecount}}\\
\subfloat[Early evaluation WEP visitation.]{\includegraphics[width=.33\textwidth]{./figures/raincloud-VisitationCountEarly.pdf}\label{fig:raincloud-results-visitations-early}}
\subfloat[Late evaluation WEP visitation.]{\includegraphics[width=.33\textwidth]{./figures/raincloud-VisitationCountLate.pdf}\label{fig:raincloud-results-visitations-late}}
\subfloat[Memory usage.]{\includegraphics[width=.33\textwidth]{./figures/raincloud-MemoryLate.pdf}\label{fig:raincloud-results-memory}}
\caption{RainCloud workflow experimental results.}
\label{fig:raincloud-results-enactment}
\end{figure*}
Figure~\ref{fig:raincloud-results-enactment-early} shows that loop unswitching enables an up to three times better enactment, while loop unrolling an up to 88 times better enactment as the original version. The better performance is caused on one hand by a reduced WEP length, defined as the length of its critical path (see Figure~\ref{fig:raincloud-results-nodecount}), and a consequentially lower WEP node visitation (see Figure~\ref{fig:raincloud-results-visitations-early}). On the other hand, loop unrolling reduces the loop iterations by an order of magnitude, which massively reduces the loop handling overhead.
Figure~\ref{fig:raincloud-results-enactment-late} shows a better scalability of the enactment time with the problem size, although the performance improvements are lower than in early evaluation mode. Applying loop unswitching shows an up to 1.4 times better enactment, while applying both loop unswitching and loop unrolling increases it to a twice better enactment time. The lower overhead is again caused by a reduced WEP length, a consequentially lower WEP node visitation, and a reduced loop handling overhead. Late evaluation, however, eliminates a lager number of unnecessary WEP node visits reducing the optimization potential compared to the early evaluation mode. Figure~\ref{fig:raincloud-results-visitations-late} shows that, in contrast to the early evaluation mode, the workflow after loop unswitching and loop unrolling features more WEP node visits in late evaluation mode compared to loop unswitching only. The helper activities needed by loop unrolling cause more WEP node visits in late evaluation mode than eliminated by the reduced number of loop iterations.
% memory usage of the original and the transformed versions of the RainCloud workflow.
Loop unswitching reduces the number of WEP nodes, which improves the memory usage of the engine by up to 29\% (see Figure~\ref{fig:raincloud-results-memory}).
Loop unrolling reduces loop iteration management overhead, which additionally improves the memory usage up to 4\% for a total reduction of up to 33\%.
\begin{figure*}[t]
\centering
\subfloat[Original workflow.]{\includegraphics[height=6.1cm]{./figures/meteoag01.pdf}\label{fig:meteoag-structure1}}\qquad
\subfloat[After loop unswitching.]{\includegraphics[height=6.1cm]{./figures/meteoag02.pdf}\label{fig:meteoag-structure2}}\qquad
\subfloat[After loop unswitching and loop unrolling.]{\includegraphics[height=6.1cm]{./figures/meteoag03.pdf}\label{fig:meteoag-structure3}}
\caption{MeteoAG workflow.}
\label{fig:meteoag-structure}
\end{figure*}
\subsubsection{MeteoAG} \cite{cotton2003rams} is another workflow designed for meteorological simulations based on the RAMS numerical atmospheric model (see Figure~\ref{fig:meteoag-structure1}). The simulations produce spatial and temporal fields of heavy precipitation cases over the western part of Austria to resolve most alpine watersheds and thunderstorms. The \texttt{simulation\_init} activity splits the input data into several simulation cases, independently processed by the \texttt{parallel for each} loop. For each simulation case, another nested parallel loop is executed with different parameter values. The workflow contains two optional \texttt{rams\_hist} and \texttt{revu\_dump} activities within an \texttt{if} activity. As with RainCloud, we first applied loop unswitching to move the \texttt{if} activity out of the \texttt{parallel foreach} loop (see Figure~\ref{fig:meteoag-structure2}), followed by a ten-fold loop unrolling shown in Figure~\ref{fig:meteoag-structure3} (unrolled twice in the figure for brevity reasons). Again, the helper activity \texttt{H1} calculates the loop boundaries for the new \texttt{parallel for} loop, \texttt{H2} distributes the input data among the unrolled loop iterations, and \texttt{H3} collects the results. We fixed in our experiments the inner loop iteration number to 20 and varied the outer loop iteration count.
\begin{figure*}[t]
\centering
\subfloat[Speedup.]{\includegraphics[width=.33\textwidth]{./figures/meteoag-speedup.pdf}\label{fig:meteoag-speedup}}
\subfloat[Early evaluation enactment.]{\includegraphics[width=.33\textwidth]{./figures/meteoag-enactment-early.pdf}\label{fig:meteoag-enactment-early}}
\subfloat[Late evaluation enactment.]{\includegraphics[width=.33\textwidth]{./figures/meteoag-enactment-late.pdf}\label{fig:meteoag-enactment-late}}
\caption{MeteoAG workflow experimental results.}
\label{fig:meteoag-results}
\end{figure*}
Figure~\ref{fig:meteoag-speedup} shows the speedup of the transformed workflows of up to 1.32.
The MeteoAG workflow is much more complex than RainCloud with atomic activities having a much larger impact on enactment performance than the composite activities, as indicated by the late enactment mode having the same performance as early enactment mode (see Figures~\ref{fig:meteoag-enactment-early} and \ref{fig:meteoag-enactment-late}).
For this reason, the measured enactment times lie well within the respective error margins and, therefore, the results only hint at a better performance achieved by the transformations.
The performance improvements achieved by loop unswitching and loop unrolling are largely hidden by the remaining unoptimized workflow structure.
More specialized transformations suitable for the unoptimized workflow parts may obtain significant
performance improvements, which we plan to research
in future work.
\subsection{Atomic Activity Reduction}
We show in this section the results of applying loop-invariant code hoisting and common activity elimination. We implemented the workflow from Figure~\ref{fig:transformation-loopInvariantHoisting}, and for activity elimination the workflow from Figure~\ref{fig:transformation-CommonSubexpressionElimination}, surrounded by a \texttt{parallel for} loop. For both workflows, we implemented activities with an execution time of \SI{1}{\second} each that, in contrast to DCIs, is a common order of magnitude for manycore parallel systems.
Loop-invariant code hoisting halves the number of activities in the loop body of our example, which produces a theoretical speedup of up to 2. Figure~\ref{fig:raincloud-results-loophoisting-makespan} shows a performance speedup of up to 2.6, surpassing the theoretical estimate because the original loop loop overhead significantly contributes to the workflow makespan, as the loop body consists of two short-running activities. %Loop hoisting cuts the number of activities in half, thereby also reducing loop overhead.
Common activity elimination reduces the number of atomic activities in our example workflow from five to four, which gives us a theoretical speedup of up to 1.25. Figure~\ref{fig:raincloud-results-elimination-makespan} shows a performance improvement of up to 1.7 times, again surpassing our theoretical estimate. The reason is that the total number of activities in the transformed workflow is a multiple of the number of cores, in contrast to the original workflow where the average activity time spent in the waiting queue was twice as high.
\begin{figure}[t]
\centering
\subfloat[Loop hoisting makespan.]{\includegraphics[width=.8\columnwidth]{./figures/loophoisting-Makespan.pdf}\label{fig:raincloud-results-loophoisting-makespan}}\\
\subfloat[Common activity elimination makespan.]{\includegraphics[width=.8\columnwidth]{./figures/elimination-Makespan.pdf}\label{fig:raincloud-results-elimination-makespan}}
\caption{Loop hoisting and common activity elimination results.}
\label{fig:raincloud-results-elimination-loophoisting}
\end{figure}
\subsection{Schedule Improvement}
We show in this section the effects of transformations to improving scheduling of the Sparselu workflow, adapted from the BOTS benchmark suite~\cite{duran2009barcelona} for LU factorization of sparse matrices. The original version of the program consists of three parallel inner loops (see Figure~\ref{fig:Sparselu-experiments1}) within a sequential outer loop. However, the first and the second inner loops have the same loop range and the contained activities \texttt{fwd} and \texttt{bvid} operate on separate parts of the sparse matrix and have no dependencies on each other. Therefore, we can apply loop fusion to combine the two inner loops (see Figure~\ref{fig:Sparselu-experiments2}) in a new
inner loop containing twice as many parallel activities. This gives the scheduler more opportunities to utilize the available computing resources.
%We simulated in this experiment the execution of the activities to be able to assign them arbitrary execution times matching the predicted execution times used by HEFT.
The iteration count of the inner loops determines the number of activities to be executed in parallel. Figure~\ref{fig:Sparselu-Heft-Speedup} shows that
%the speed-up of the transformed workflow version.
in all scenarios where the number of inner loop iterations is not a multiple of the number of cores, the transformed workflow version shows a significant speedup up to 36\%. Even when the inner loop iteration count is a multiple of the number of cores, the transformed version still shows a slightly better performance when the outer loop counter is large enough. The higher number of activities to be simultaneously scheduled allows a better mapping.
\begin{figure}[t]
\centering
\subfloat[Original workflow.]{\includegraphics[height=4.2cm]{./figures/Sparselu-experiments1.pdf}\label{fig:Sparselu-experiments1}}\qquad
\subfloat[After loop fusion.]{\includegraphics[height=4.2cm]{./figures/Sparselu-experiments2.pdf}\label{fig:Sparselu-experiments2}}
\caption{Sparselu workflow.}
\label{fig:sparselu-structure}
\end{figure}
\begin{figure}[t]
\centering
\subfloat[Early evaluation mode.]{\includegraphics[width=.8\columnwidth]{./figures/Sparselu-Heft-Speedup-early.pdf}\label{fig:Sparselu-Heft-Speedup-early}}\\
\subfloat[Late evaluation mode.]{\includegraphics[width=.8\columnwidth]{./figures/Sparselu-Heft-Speedup-late.pdf}\label{fig:Sparselu-Heft-Speedup-late}}
\caption{Speedup of Sparselu caused by scheduling improvements.}
\label{fig:Sparselu-Heft-Speedup}
\end{figure}
\subsection{Parallel Throughput Improvement}
\begin{figure}[t]
\centering
\subfloat[Original workflow.]{\includegraphics[height=4.5cm]{./figures/blast-experiment01.pdf}\label{fig:blast-experiment01}}\hspace{2mm}
\subfloat[After reduction recognition.]{\includegraphics[height=4.5cm]{./figures/blast-experiment02.pdf}\label{fig:blast-experiment02}}
\caption{Blast workflow.}
\label{fig:blast-structure}
\end{figure}
We first show the results of applying reduction recognition to the Basic Local Alignment Search Tool Blast~\cite{altschul1990basic} workflow (see Figure~\ref{fig:blast-structure}). Blast is a bioinformatics application that finds regions of local similarity between sequences by comparing nucleotide or protein sequences to sequence databases, and calculating the statistical significance of matches. The workflow consists of a parallel \texttt{foreach} loop that conducts the parallel search operation, and a few subsequent activities that collect the intermediate results and produce the final result. Each iteration of the \texttt{foreach} loop produces three files (called \texttt{output}, \texttt{error} and \texttt{total}), collected by the \texttt{cat\_output}, \texttt{cat\_error} and \texttt{cat\_total} activities and merged into a single file each. The merge operations are associative and, therefore, appropriate for the reduction recognition transformation.
Figure~\ref{fig:blast-experiment02} shows the transformed workflow featuring an additional \texttt{for} loop, which distributes the results of the \texttt{foreach} loop among several parallel merge activities \texttt{cat\_*\_p}.
The helper activity \texttt{H1} splits the original input of the \texttt{foreach} loop.
The results of the parallel merge operations are then collected and merged by the final merge operations \texttt{cat\_*\_f}. To exclusively measure the time for the merge operation, we replaced in our experiments all activities not involved in the merge operation with empty implementations. The execution time of the merge operations is around \SI{1}{\milli\second} per file. We configured the outer loop to have 40 parallel iterations to fully utilize all available cores, and varied the iteration count of the inner loop.
Figure~\ref{fig:results-reduction-makespan} shows that reduction recognition can significantly improve the performance of the merge operation up to 3.8 times. However, this is still far from the theoretical estimate of 13.3 times because the parallel merge operations have a very short execution time and therefore, a comparably larger execution overhead. However, we expect a higher speedup for longer running reduction operations.
For software pipelining, we implemented the workflow from Figure~\ref{fig:transformation-SoftwarePipelining} with activities' execution time of \SI{1}{\second} each. Figure~\ref{fig:results-pipelining-makespan} shows that executing the three previously sequential activities in parallel improves the performance by up to 2.8 times, close to the theoretical estimate of 3.
\begin{figure}[t]
\centering
\subfloat[Reduction recognition makespan.]{\includegraphics[width=.8\columnwidth]{./figures/blast-reduction.pdf}\label{fig:results-reduction-makespan}}\\
\subfloat[Software pipelining makespan.]{\includegraphics[width=.8\columnwidth]{./figures/pipelining-Makespan.pdf}\label{fig:results-pipelining-makespan}}
\caption{Software pipelining and reduction recognition results.}
\label{fig:results-pipelining-reduction}
\end{figure}
workflowtransformations/workflowtransformations-main.tex
+4
View File
@@ -1,2 +1,6 @@
\chapter{Workflow Transformations}
\input{workflowtransformations/workflowtransformations-transformations}
\input{workflowtransformations/workflowtransformations-evaluation}
workflowtransformations/workflowtransformations-transformations.tex
+157
View File
@@ -0,0 +1,157 @@
\section{Workflow Transformations}\label{sec:optimization-goals}
\subsection{Workflow Transformation Properties}
Since scientific workflows are of a different nature than traditional imperative sequential and parallel applications, applied transformations have different properties.
\subsubsection{Hardware agnosticism}
One distinguishing property of workflow applications is that their structure belongs to the abstract part (see Section~\ref{sec:workflowmodel}) that is hardware-agnostic. Therefore, only hardware-agnostic transformations can be applied to workflows, in contrast to traditional compiler transformations that are both hardware-agnostic and specific.
\subsubsection{Hidden data access patterns}
Most part of the execution of sequential and parallel applications is spent in loops iterating over arrays. For this reason, there exist a large number of compiler transformations that modify their data access patterns to eliminate data dependencies and improve performance by, for example, better utilizing the CPU cache. While workflow applications also contain loops that iterate over arrays, they are usually contained within an atomic activities not visible outside and, therefore, cannot be modified by a transformation.
\subsubsection{Black-box principle}
One important constraint that needs to be preserved by program transformations is \emph{program correctness}. A transformed program is considered correct when it produces exactly the same output as the original program for all identical executions~\cite{bacon1994compiler}.
%The correctness of compiler transformations is usually verified using static program analysis by knowing the effects and the pre- and post-conditions of each program instruction.
%The smallest functional units of traditional imperative applications consist of a well-defined and \emph{finite} set of invariant machine instructions for which static program analysis becomes feasible. In contrast, the smallest functional units of workflow applications consist of a possibly \emph{infinite} set of atomic instructions considered as black-boxes for which only their input and output ports are known (but not their internals) with possible sexist several different implementations.
%This precludes a static program analysis of a workflow program because of unknown side-effects, pre- and post-conditions. To be able to conduct a static analysis, we need to relax the black-box principle and give the engine additional information about workflow activities that can be used by the static analysis to determine whether a transformed program is correct.
Scientific workflow activities are black-boxes that represent an undefined set of atomic instructions with potentially unknown side-effects, pre- and post-conditions, which precludes a static program analysis to determine program correctness. To still be able to decide whether a transformed program is correct, we need to relax the black-box principle and give additional information about workflow activities to be used for program analysis.
We solved this problem in MWRE by implementing a tagging system that allows to attach tags to atomic activity types. Using these tags, we can specify properties satisfied by an atomic activity, stored inside the activity repository accessible to the source-to-source compiler. For example, a \emph{side-effect freeness} tag indicates that an atomic activity does not have any hidden state that can influence the activity execution, but always produces the same result for the same input values. Any activity without this tag cannot have any transformations applied because it may accidentally alter a hidden state and produce wrong results.
%\textcolor{red}{For example, an activity that does not have any external data dependencies but only returns the current time is called in each iteration of a loop. From the lack of external data dependencies one could conclude that loop invariant code hoisting can be applied, but the current time represents a hidden state leading to different return values for each invocation.}
%For example, let us assume an activity called in each iteration of an enclosing loop that does not have any external data dependencies but only returns the current time. While the lack of data dependencies makes this activity a candidate for loop invariant code hoisting, the current time represents a hidden state leading to different return values for each activity invocation.
\subsubsection{Data manipulation}
Many compiler transformations require data manipulation. For example, a loop normalization transformation modifies the \texttt{for} loop counter so that it starts at 1 or 0 and is incremented by 1 in every iteration. However, most workflow environments do not allow data manipulations outside atomic activities, and we cannot directly modify the affected atomic activities implementations to correctly work with the new data values. To solve this problem, we introduce new atomic activities that implement the needed data manipulation.
%however, this approach has the disadvantage of causing relatively high overheads.
However, data manipulations required by transformations are usually computationally cheap, and executing them through the entire entire execution stack may cause significant unnecessary overheads. We solve this problem in MWRE by introducing a special atomic \emph{assign activity} (inspired from BPEL\footnote{\url{http://docs.oasis-open.org/wsbpel/2.0/wsbpel-v2.0.html}}) executed directly by the workflow engine without going through the entire execution stack.
%but is directly in a transparent way so that the scheduler does not notice its existence.
\subsection{Workflow Transformation Categories}
In this section, we discuss different categories of workflow transformations representing the optimization goals.
We give a selected list of the most important examples from each category for brevity reasons.
As our goal is to verify transformations that improve the workflow performance, we only consider well-known transformations for simplicity reasons.
We plan to investigate further workflow-specific transformations in future work.
\begin{figure*}[t]
\centering
\subfloat[Loop-invariant code hosting.]{\includegraphics[width=.3\textwidth]{./figures/transformation-loopInvariantHoisting.pdf}\label{fig:transformation-loopInvariantHoisting}}
\subfloat[Common subexpression elimination.]{\includegraphics[width=.3\textwidth]{./figures/transformation-CommonSubexpressionElimination.pdf}\label{fig:transformation-CommonSubexpressionElimination}}
\subfloat[If optimization.]{\includegraphics[width=.25\textwidth]{./figures/transformation-IfOptimization.pdf}\label{fig:transformation-IfOptimization}}\\
\subfloat[Reduction recognition.]{\includegraphics[width=.3\textwidth]{./figures/transformation-ReductionRecognition.pdf}\label{fig:transformation-ReductionRecognition}}
\subfloat[Code hoisting.]{\includegraphics[width=.3\textwidth]{./figures/transformation-CodeHoisting.pdf}\label{fig:transformation-CodeHoisting}}
\subfloat[Software pipelining.]{\includegraphics[width=.3\textwidth]{./figures/transformation-SoftwarePipelining.pdf}\label{fig:transformation-SoftwarePipelining}}
\caption{Example workflow transformations.}
\label{fig:example-transformations}
\end{figure*}
\subsubsection{Atomic activities reduction}\label{sec:reduce-executed-atomic}
This category represents transformations that eliminate redundant atomic activity invocations (the most expensive parts in the workflow enactment),
with a possible huge impact on performance. This category is similar to traditional transformations trying to reduce the number of executed instructions (replaced by activities here) such as loop-invariant code hoisting, common subexpression/activity elimination and constant propagation.
\paragraph{Loop-invariant code hoisting} moves the loop-invariant code to be executed only once before the loop instead of every loop iteration. With workflow applications, this transformation moves atomic activities with loop-invariant input data outside of the loop, reducing the number of activity invocations. Figure~\ref{fig:transformation-loopInvariantHoisting} shows an example where the activity \texttt{A2} only has a loop-invariant data dependency on activity \texttt{A1} and can be moved out of the loop. Section~\ref{sec:experiments} features an example of loop-invariant code hoisting.
%\begin{figure}[t]
% \centering
% \includegraphics[width=.5\columnwidth]{./figures/transformation-loopInvariantHoisting.pdf}
% \caption{Loop-Invariant Code Hosting.}
% \label{fig:transformation-loopInvariantHoisting}
%\end{figure}
\paragraph{Common subexpression/activity elimination} searches for several instances of identical expressions evaluated to the same value, and replaces them with a variable holding the computed value. We use this transformation in workflow applications to eliminate several invocations of the same activity type with the same input values. In case of an execution path between two invocations, we eliminate the redundant second invocation and connect its involved ports to the output ports of the first invocation. In the example from Figure~\ref{fig:transformation-loopInvariantHoisting}, the activities \texttt{B1} and \texttt{B2} share the same activity type and the same input values, and \texttt{B2} is reachable from \texttt{B1}. In this case, we eliminate \texttt{B2} and connect the input ports of activity \texttt{D} with the output ports of \texttt{B1}. Section~\ref{sec:experiments} features an example of common activity elimination.
%\begin{figure}[t]
% \centering
% \includegraphics[width=.75\columnwidth]{./figures/transformation-CommonSubexpressionElimination.pdf}
% \caption{Common subexpression elimination.}
% \label{fig:transformation-CommonSubexpressionElimination}
%\end{figure}
\paragraph{Constant propagation} evaluates a constant expression at compile-time and replaces it with the computed value. With workflows, we evaluate atomic activities that have constant input values at compile-time and replace the corresponding data dependencies in the successors with the computed value.
\subsubsection{Engine overhead reduction}
This category represents transformations that eliminate engine overhead by optimizing the WEP. Workflow engines that support full-ahead scheduling need to maintain and evaluate state information for each activity instance and for each loop iteration and therefore, exhibit limited scalability.
%This state information is stored inside the WEP, and
By reducing the number of nodes in the WEP, we reduce the state information, which improves the engine's scalability and reduces the callback function invocations.
There exists no equivalent category in traditional compiler transformations, as full-ahead scheduling has never been a topic in this area. However, there are transformations that can be re-purposed to reduce the number of WEP nodes like loop unswitching, loop unrolling and if optimization.
\paragraph{Loop unswitching} moves a conditional activity out of a loop and duplicates the loop in each branch of the condition. This way, the condition is only evaluated once instead of every loop iteration thus simplifying the WEP.
%Applying this transformation to a workflow produces a WEP that only contains a single set of nodes encompassing the loop, instead of a set of nodes within each loop iteration, thus reducing the number of nodes in the WEP.
Section~\ref{sec:experiments} features a loop unswitching example applied to a real-world workflow. The WEP of the original version of this workflow (see Figure~\ref{fig:raincloud-structure1}) features four, respectively three nodes per iteration, while loop unswitching (see Figure~\ref{fig:raincloud-structure2}) reduces it to two, respectively one node per iteration.
\paragraph{Loop unrolling} replicates the loop body several times, so that a single iteration of the modified loop does the same computation as several iterations of the original loop, reducing the number of iterations needed to get the same result.
%Applying this transformation to a workflow application allows us to reduce the number of loop iteration nodes in the WEP and the number of invocations of the associated callback function, which improves scalability.
Section~\ref{sec:experiments} features an example of a loop unrolled ten times that reduces the number of loop iteration nodes nodes in the WEP and the number of loop iteration callback invocations by a factor of ten (see Figure~\ref{fig:raincloud-structure3}).
\paragraph{If optimization} combines consecutive conditional activities with equivalent conditions into a single one, and flattens the nested conditional activities when the inner condition always evaluates to true and the outer condition is true, resulting in a simpler WEP.
%When applied to a workflow application, the WEP of the modified workflow contains fewer conditional nodes and, therefore, features less overhead when traversing the WEP and evaluating the nodes.
Figure~\ref{fig:transformation-IfOptimization} shows an example of two consecutive conditional activities, both having the same condition. By collapsing them into a single conditional activity, we can reduce the number of conditional activity nodes in the WEP and the number of invocations of the associated condition evaluation callback by a factor of two.
%\begin{figure}[ht]
% \centering
% \includegraphics[width=4cm,keepaspectratio=true]{./figures/transformation-IfOptimization.pdf}
% \caption{If Optimization.}
% \label{fig:transformation-IfOptimization}
%\end{figure}
\subsubsection{Scheduling and parallel throughput improvement}
The performance of workflow applications is highly influenced by the mapping of activities onto the parallel computing resources. %conducted by the scheduler.
This category contains transformations that rearrange activities to either increase the number of activities that can be executed in parallel, or to help generating a better schedule. While there exists traditional transformation that aims to improve the parallel throughput, most of them target loops and modify data access patterns to eliminate dependencies that hinder parallelization.
With workflow applications that hide data access patterns inside atomic activates, however, such transformations are not applicable.
Examples of transformations that do not modify data access patterns and can be applied to workflow applications are reduction recognition, code hoisting and software pipelining.
\paragraph{Reduction recognition} recognizes array operations (usually the output of a loop) reduced to a single value. Associative reduction operations can be parallelized by splitting the loop and spawning several parallel reduction operations, each reducing a part of the array, followed by a final reduction operation that reduces the partial results.
To be able to recognize activities suitable for parallel reduction, we tag them with an \texttt{associative} tag.
Figure~\ref{fig:transformation-ReductionRecognition} shows an example of a parallel loop producing an array reduced by the activity \texttt{R}. Reduction recognition splits the original loop in several loops that are executed and reduced in parallel by the activities \texttt{Rp}, followed by a final reduction \texttt{Rf}. Section~\ref{sec:experiments} features an example of reduction recognition.
%This allows us to improve parallel performance because instead of a single activity~\texttt{R} we have several parallel activities~\texttt{Rp} executing at the same time.
%\begin{figure}[ht]
% \centering
% \includegraphics[width=4cm,keepaspectratio=true]{./figures/transformation-ReductionRecognition.pdf}
% \caption{Reduction Recognition.}
% \label{fig:transformation-ReductionRecognition}
%\end{figure}
\paragraph{Code hoisting} is a generalization of the loop-invariant code hoisting (see Section~\ref{sec:reduce-executed-atomic}) that moves code not only out of loops, but also out of any hierarchical code block.
It can be used to improve the late enactment mode, when the contents of a composite activity are evaluated only after all its input values are available. By moving the activities with no input dependencies inside the composite activity out of it, the scheduler can schedule and execute them before the composite activity is ready.
%This transformation can be used to improve workflow scheduling in case of engine running in late enactment mode (see Section~\ref{sec:workflowenactment}), which has the problem that the contents of a composite activity are only evaluated after all their input values have been resolved. However, child activities that only depend on activities outside the parent activity can already be scheduled and executed before all the input values of the parent activities are available. Moving these activities outside of their parent activity gives the scheduler the opportunity to generate a better mapping.
Figure~\ref{fig:transformation-CodeHoisting} shows an example workflow containing an activity \texttt{B} that only depends on activities outside the composite activity. In late evaluation mode, the activity \texttt{B} is only executed after the activity \texttt{A} has finished and all the input values of the composite activity are available. By hoisting the activity \texttt{B} out of the composite activity, activities \texttt{A} and \texttt{B} can be executed in parallel.
%When using the late evaluation mode, the scheduler can only execute the activity \texttt{B} after the activity \texttt{A} finished and all input values of the composite activity are available. Assuming that each activity has an execution time of one time slot, the total execution time of the workflow is three time slots. Moving the activity \texttt{B} outside of the composite activity, the engine can execute activities \texttt{A} and \texttt{B} in parallel reducing the execution time to two time slots.
%\begin{figure}[ht]
% \centering
% \includegraphics[width=4cm,keepaspectratio=true]{./figures/transformation-CodeHoisting.pdf}
% \caption{Code Hoisting.}
% \label{fig:transformation-CodeHoisting}
%\end{figure}
\paragraph{Software pipelining} parallelizes a sequential loop containing a linear sequence of operations by overlapping the individual loop iterations. This transformation can also be applied to a workflow to improve its parallel throughput.
Figure~\ref{fig:transformation-SoftwarePipelining} shows a workflow consisting of a sequential loop with three activities \texttt{A}, \texttt{B} and \texttt{C} in a pipeline. The index \texttt{i} in the parenthesis refers to the loop iteration. The output of activity \texttt{A(i)} is input into the activity \texttt{A(i+1)} via a loop port, which hinders the use a \texttt{parallel for} loop. Software pipelining executes the activities \texttt{A(i+2)}, \texttt{B(i+1)} and \texttt{C(i)} in parallel, which improves parallelism and reduces the execution time
(see Section~\ref{sec:experiments}).% features an example of software pipelining.
%\begin{figure}[ht]
% \centering
% \includegraphics[width=6cm,keepaspectratio=true]{./figures/transformation-SoftwarePipelining.pdf}
% \caption{Software Pipelining}
% \label{fig:transformation-SoftwarePipelining}
%\end{figure}
\subsubsection{Further transformation support}
This category represents transformations that do not directly improve performance, but modify the workflow in a way that enables further transformations.
Most traditional transformations falling in this category primarily target loop-carried dependencies to enable loop parallelization.
These transformations can also be applied to workflows, as long as they do not modify data access patterns hidden inside atomic activities.
Examples of such transformations are loop peeling and loop fission.
\paragraph{Loop peeling} splits the first or the last loop iterations, required for example for loop unrolling where
%In the loop unrolling example in Section~\ref{sec:experiments},
the number of loop iterations needs to be a multiple of the unrolling factor. We can support arbitrary loop iterations by applying loop peeling beforehand, so that the remaining number of iterations is a multiple of the unrolling factor.
\paragraph{Loop fission} breaks up a loop into several ones over the same index range, each containing a part of the body. This allows moving activities that prevent, for example, loop unrolling or software pipelining into a separate loop.