158 lines
20 KiB
TeX
158 lines
20 KiB
TeX
\section{Workflow Transformations}\label{sec:optimization-goals}
|
|
|
|
\subsection{Workflow Transformation Properties}
|
|
|
|
Since scientific workflows are of a different nature than traditional imperative sequential and parallel applications, applied transformations have different properties.
|
|
|
|
\subsubsection{Hardware agnosticism}
|
|
One distinguishing property of workflow applications is that their structure belongs to the abstract part (see Section~\ref{sec:workflowmodel}) that is hardware-agnostic. Therefore, only hardware-agnostic transformations can be applied to workflows, in contrast to traditional compiler transformations that are both hardware-agnostic and specific.
|
|
|
|
\subsubsection{Hidden data access patterns}
|
|
Most part of the execution of sequential and parallel applications is spent in loops iterating over arrays. For this reason, there exist a large number of compiler transformations that modify their data access patterns to eliminate data dependencies and improve performance by, for example, better utilizing the CPU cache. While workflow applications also contain loops that iterate over arrays, they are usually contained within an atomic activities not visible outside and, therefore, cannot be modified by a transformation.
|
|
|
|
|
|
\subsubsection{Black-box principle}
|
|
One important constraint that needs to be preserved by program transformations is \emph{program correctness}. A transformed program is considered correct when it produces exactly the same output as the original program for all identical executions~\cite{bacon1994compiler}.
|
|
%The correctness of compiler transformations is usually verified using static program analysis by knowing the effects and the pre- and post-conditions of each program instruction.
|
|
%The smallest functional units of traditional imperative applications consist of a well-defined and \emph{finite} set of invariant machine instructions for which static program analysis becomes feasible. In contrast, the smallest functional units of workflow applications consist of a possibly \emph{infinite} set of atomic instructions considered as black-boxes for which only their input and output ports are known (but not their internals) with possible sexist several different implementations.
|
|
%This precludes a static program analysis of a workflow program because of unknown side-effects, pre- and post-conditions. To be able to conduct a static analysis, we need to relax the black-box principle and give the engine additional information about workflow activities that can be used by the static analysis to determine whether a transformed program is correct.
|
|
Scientific workflow activities are black-boxes that represent an undefined set of atomic instructions with potentially unknown side-effects, pre- and post-conditions, which precludes a static program analysis to determine program correctness. To still be able to decide whether a transformed program is correct, we need to relax the black-box principle and give additional information about workflow activities to be used for program analysis.
|
|
We solved this problem in MWRE by implementing a tagging system that allows to attach tags to atomic activity types. Using these tags, we can specify properties satisfied by an atomic activity, stored inside the activity repository accessible to the source-to-source compiler. For example, a \emph{side-effect freeness} tag indicates that an atomic activity does not have any hidden state that can influence the activity execution, but always produces the same result for the same input values. Any activity without this tag cannot have any transformations applied because it may accidentally alter a hidden state and produce wrong results.
|
|
%\textcolor{red}{For example, an activity that does not have any external data dependencies but only returns the current time is called in each iteration of a loop. From the lack of external data dependencies one could conclude that loop invariant code hoisting can be applied, but the current time represents a hidden state leading to different return values for each invocation.}
|
|
%For example, let us assume an activity called in each iteration of an enclosing loop that does not have any external data dependencies but only returns the current time. While the lack of data dependencies makes this activity a candidate for loop invariant code hoisting, the current time represents a hidden state leading to different return values for each activity invocation.
|
|
|
|
\subsubsection{Data manipulation}
|
|
Many compiler transformations require data manipulation. For example, a loop normalization transformation modifies the \texttt{for} loop counter so that it starts at 1 or 0 and is incremented by 1 in every iteration. However, most workflow environments do not allow data manipulations outside atomic activities, and we cannot directly modify the affected atomic activities implementations to correctly work with the new data values. To solve this problem, we introduce new atomic activities that implement the needed data manipulation.
|
|
%however, this approach has the disadvantage of causing relatively high overheads.
|
|
However, data manipulations required by transformations are usually computationally cheap, and executing them through the entire entire execution stack may cause significant unnecessary overheads. We solve this problem in MWRE by introducing a special atomic \emph{assign activity} (inspired from BPEL\footnote{\url{http://docs.oasis-open.org/wsbpel/2.0/wsbpel-v2.0.html}}) executed directly by the workflow engine without going through the entire execution stack.
|
|
%but is directly in a transparent way so that the scheduler does not notice its existence.
|
|
|
|
\subsection{Workflow Transformation Categories}
|
|
|
|
In this section, we discuss different categories of workflow transformations representing the optimization goals.
|
|
We give a selected list of the most important examples from each category for brevity reasons.
|
|
As our goal is to verify transformations that improve the workflow performance, we only consider well-known transformations for simplicity reasons.
|
|
We plan to investigate further workflow-specific transformations in future work.
|
|
|
|
\begin{figure*}[t]
|
|
\centering
|
|
\subfloat[Loop-invariant code hosting.]{\includegraphics[width=.3\textwidth]{./figures/transformation-loopInvariantHoisting.pdf}\label{fig:transformation-loopInvariantHoisting}}
|
|
\subfloat[Common subexpression elimination.]{\includegraphics[width=.3\textwidth]{./figures/transformation-CommonSubexpressionElimination.pdf}\label{fig:transformation-CommonSubexpressionElimination}}
|
|
\subfloat[If optimization.]{\includegraphics[width=.25\textwidth]{./figures/transformation-IfOptimization.pdf}\label{fig:transformation-IfOptimization}}\\
|
|
\subfloat[Reduction recognition.]{\includegraphics[width=.3\textwidth]{./figures/transformation-ReductionRecognition.pdf}\label{fig:transformation-ReductionRecognition}}
|
|
\subfloat[Code hoisting.]{\includegraphics[width=.3\textwidth]{./figures/transformation-CodeHoisting.pdf}\label{fig:transformation-CodeHoisting}}
|
|
\subfloat[Software pipelining.]{\includegraphics[width=.3\textwidth]{./figures/transformation-SoftwarePipelining.pdf}\label{fig:transformation-SoftwarePipelining}}
|
|
\caption{Example workflow transformations.}
|
|
\label{fig:example-transformations}
|
|
\end{figure*}
|
|
|
|
|
|
\subsubsection{Atomic activities reduction}\label{sec:reduce-executed-atomic}
|
|
This category represents transformations that eliminate redundant atomic activity invocations (the most expensive parts in the workflow enactment),
|
|
with a possible huge impact on performance. This category is similar to traditional transformations trying to reduce the number of executed instructions (replaced by activities here) such as loop-invariant code hoisting, common subexpression/activity elimination and constant propagation.
|
|
|
|
\paragraph{Loop-invariant code hoisting} moves the loop-invariant code to be executed only once before the loop instead of every loop iteration. With workflow applications, this transformation moves atomic activities with loop-invariant input data outside of the loop, reducing the number of activity invocations. Figure~\ref{fig:transformation-loopInvariantHoisting} shows an example where the activity \texttt{A2} only has a loop-invariant data dependency on activity \texttt{A1} and can be moved out of the loop. Section~\ref{sec:experiments} features an example of loop-invariant code hoisting.
|
|
|
|
%\begin{figure}[t]
|
|
% \centering
|
|
% \includegraphics[width=.5\columnwidth]{./figures/transformation-loopInvariantHoisting.pdf}
|
|
% \caption{Loop-Invariant Code Hosting.}
|
|
% \label{fig:transformation-loopInvariantHoisting}
|
|
%\end{figure}
|
|
|
|
\paragraph{Common subexpression/activity elimination} searches for several instances of identical expressions evaluated to the same value, and replaces them with a variable holding the computed value. We use this transformation in workflow applications to eliminate several invocations of the same activity type with the same input values. In case of an execution path between two invocations, we eliminate the redundant second invocation and connect its involved ports to the output ports of the first invocation. In the example from Figure~\ref{fig:transformation-loopInvariantHoisting}, the activities \texttt{B1} and \texttt{B2} share the same activity type and the same input values, and \texttt{B2} is reachable from \texttt{B1}. In this case, we eliminate \texttt{B2} and connect the input ports of activity \texttt{D} with the output ports of \texttt{B1}. Section~\ref{sec:experiments} features an example of common activity elimination.
|
|
|
|
%\begin{figure}[t]
|
|
% \centering
|
|
% \includegraphics[width=.75\columnwidth]{./figures/transformation-CommonSubexpressionElimination.pdf}
|
|
% \caption{Common subexpression elimination.}
|
|
% \label{fig:transformation-CommonSubexpressionElimination}
|
|
%\end{figure}
|
|
|
|
|
|
\paragraph{Constant propagation} evaluates a constant expression at compile-time and replaces it with the computed value. With workflows, we evaluate atomic activities that have constant input values at compile-time and replace the corresponding data dependencies in the successors with the computed value.
|
|
|
|
|
|
\subsubsection{Engine overhead reduction}
|
|
This category represents transformations that eliminate engine overhead by optimizing the WEP. Workflow engines that support full-ahead scheduling need to maintain and evaluate state information for each activity instance and for each loop iteration and therefore, exhibit limited scalability.
|
|
%This state information is stored inside the WEP, and
|
|
By reducing the number of nodes in the WEP, we reduce the state information, which improves the engine's scalability and reduces the callback function invocations.
|
|
There exists no equivalent category in traditional compiler transformations, as full-ahead scheduling has never been a topic in this area. However, there are transformations that can be re-purposed to reduce the number of WEP nodes like loop unswitching, loop unrolling and if optimization.
|
|
|
|
\paragraph{Loop unswitching} moves a conditional activity out of a loop and duplicates the loop in each branch of the condition. This way, the condition is only evaluated once instead of every loop iteration thus simplifying the WEP.
|
|
%Applying this transformation to a workflow produces a WEP that only contains a single set of nodes encompassing the loop, instead of a set of nodes within each loop iteration, thus reducing the number of nodes in the WEP.
|
|
Section~\ref{sec:experiments} features a loop unswitching example applied to a real-world workflow. The WEP of the original version of this workflow (see Figure~\ref{fig:raincloud-structure1}) features four, respectively three nodes per iteration, while loop unswitching (see Figure~\ref{fig:raincloud-structure2}) reduces it to two, respectively one node per iteration.
|
|
|
|
\paragraph{Loop unrolling} replicates the loop body several times, so that a single iteration of the modified loop does the same computation as several iterations of the original loop, reducing the number of iterations needed to get the same result.
|
|
%Applying this transformation to a workflow application allows us to reduce the number of loop iteration nodes in the WEP and the number of invocations of the associated callback function, which improves scalability.
|
|
Section~\ref{sec:experiments} features an example of a loop unrolled ten times that reduces the number of loop iteration nodes nodes in the WEP and the number of loop iteration callback invocations by a factor of ten (see Figure~\ref{fig:raincloud-structure3}).
|
|
|
|
\paragraph{If optimization} combines consecutive conditional activities with equivalent conditions into a single one, and flattens the nested conditional activities when the inner condition always evaluates to true and the outer condition is true, resulting in a simpler WEP.
|
|
%When applied to a workflow application, the WEP of the modified workflow contains fewer conditional nodes and, therefore, features less overhead when traversing the WEP and evaluating the nodes.
|
|
Figure~\ref{fig:transformation-IfOptimization} shows an example of two consecutive conditional activities, both having the same condition. By collapsing them into a single conditional activity, we can reduce the number of conditional activity nodes in the WEP and the number of invocations of the associated condition evaluation callback by a factor of two.
|
|
|
|
%\begin{figure}[ht]
|
|
% \centering
|
|
% \includegraphics[width=4cm,keepaspectratio=true]{./figures/transformation-IfOptimization.pdf}
|
|
% \caption{If Optimization.}
|
|
% \label{fig:transformation-IfOptimization}
|
|
%\end{figure}
|
|
|
|
|
|
\subsubsection{Scheduling and parallel throughput improvement}
|
|
The performance of workflow applications is highly influenced by the mapping of activities onto the parallel computing resources. %conducted by the scheduler.
|
|
This category contains transformations that rearrange activities to either increase the number of activities that can be executed in parallel, or to help generating a better schedule. While there exists traditional transformation that aims to improve the parallel throughput, most of them target loops and modify data access patterns to eliminate dependencies that hinder parallelization.
|
|
With workflow applications that hide data access patterns inside atomic activates, however, such transformations are not applicable.
|
|
Examples of transformations that do not modify data access patterns and can be applied to workflow applications are reduction recognition, code hoisting and software pipelining.
|
|
|
|
\paragraph{Reduction recognition} recognizes array operations (usually the output of a loop) reduced to a single value. Associative reduction operations can be parallelized by splitting the loop and spawning several parallel reduction operations, each reducing a part of the array, followed by a final reduction operation that reduces the partial results.
|
|
To be able to recognize activities suitable for parallel reduction, we tag them with an \texttt{associative} tag.
|
|
Figure~\ref{fig:transformation-ReductionRecognition} shows an example of a parallel loop producing an array reduced by the activity \texttt{R}. Reduction recognition splits the original loop in several loops that are executed and reduced in parallel by the activities \texttt{Rp}, followed by a final reduction \texttt{Rf}. Section~\ref{sec:experiments} features an example of reduction recognition.
|
|
%This allows us to improve parallel performance because instead of a single activity~\texttt{R} we have several parallel activities~\texttt{Rp} executing at the same time.
|
|
|
|
%\begin{figure}[ht]
|
|
% \centering
|
|
% \includegraphics[width=4cm,keepaspectratio=true]{./figures/transformation-ReductionRecognition.pdf}
|
|
% \caption{Reduction Recognition.}
|
|
% \label{fig:transformation-ReductionRecognition}
|
|
%\end{figure}
|
|
|
|
|
|
\paragraph{Code hoisting} is a generalization of the loop-invariant code hoisting (see Section~\ref{sec:reduce-executed-atomic}) that moves code not only out of loops, but also out of any hierarchical code block.
|
|
It can be used to improve the late enactment mode, when the contents of a composite activity are evaluated only after all its input values are available. By moving the activities with no input dependencies inside the composite activity out of it, the scheduler can schedule and execute them before the composite activity is ready.
|
|
%This transformation can be used to improve workflow scheduling in case of engine running in late enactment mode (see Section~\ref{sec:workflowenactment}), which has the problem that the contents of a composite activity are only evaluated after all their input values have been resolved. However, child activities that only depend on activities outside the parent activity can already be scheduled and executed before all the input values of the parent activities are available. Moving these activities outside of their parent activity gives the scheduler the opportunity to generate a better mapping.
|
|
Figure~\ref{fig:transformation-CodeHoisting} shows an example workflow containing an activity \texttt{B} that only depends on activities outside the composite activity. In late evaluation mode, the activity \texttt{B} is only executed after the activity \texttt{A} has finished and all the input values of the composite activity are available. By hoisting the activity \texttt{B} out of the composite activity, activities \texttt{A} and \texttt{B} can be executed in parallel.
|
|
%When using the late evaluation mode, the scheduler can only execute the activity \texttt{B} after the activity \texttt{A} finished and all input values of the composite activity are available. Assuming that each activity has an execution time of one time slot, the total execution time of the workflow is three time slots. Moving the activity \texttt{B} outside of the composite activity, the engine can execute activities \texttt{A} and \texttt{B} in parallel reducing the execution time to two time slots.
|
|
|
|
%\begin{figure}[ht]
|
|
% \centering
|
|
% \includegraphics[width=4cm,keepaspectratio=true]{./figures/transformation-CodeHoisting.pdf}
|
|
% \caption{Code Hoisting.}
|
|
% \label{fig:transformation-CodeHoisting}
|
|
%\end{figure}
|
|
|
|
|
|
\paragraph{Software pipelining} parallelizes a sequential loop containing a linear sequence of operations by overlapping the individual loop iterations. This transformation can also be applied to a workflow to improve its parallel throughput.
|
|
Figure~\ref{fig:transformation-SoftwarePipelining} shows a workflow consisting of a sequential loop with three activities \texttt{A}, \texttt{B} and \texttt{C} in a pipeline. The index \texttt{i} in the parenthesis refers to the loop iteration. The output of activity \texttt{A(i)} is input into the activity \texttt{A(i+1)} via a loop port, which hinders the use a \texttt{parallel for} loop. Software pipelining executes the activities \texttt{A(i+2)}, \texttt{B(i+1)} and \texttt{C(i)} in parallel, which improves parallelism and reduces the execution time
|
|
(see Section~\ref{sec:experiments}).% features an example of software pipelining.
|
|
|
|
%\begin{figure}[ht]
|
|
% \centering
|
|
% \includegraphics[width=6cm,keepaspectratio=true]{./figures/transformation-SoftwarePipelining.pdf}
|
|
% \caption{Software Pipelining}
|
|
% \label{fig:transformation-SoftwarePipelining}
|
|
%\end{figure}
|
|
|
|
|
|
\subsubsection{Further transformation support}
|
|
This category represents transformations that do not directly improve performance, but modify the workflow in a way that enables further transformations.
|
|
Most traditional transformations falling in this category primarily target loop-carried dependencies to enable loop parallelization.
|
|
These transformations can also be applied to workflows, as long as they do not modify data access patterns hidden inside atomic activities.
|
|
Examples of such transformations are loop peeling and loop fission.
|
|
|
|
\paragraph{Loop peeling} splits the first or the last loop iterations, required for example for loop unrolling where
|
|
%In the loop unrolling example in Section~\ref{sec:experiments},
|
|
the number of loop iterations needs to be a multiple of the unrolling factor. We can support arbitrary loop iterations by applying loop peeling beforehand, so that the remaining number of iterations is a multiple of the unrolling factor.
|
|
|
|
\paragraph{Loop fission} breaks up a loop into several ones over the same index range, each containing a part of the body. This allows moving activities that prevent, for example, loop unrolling or software pipelining into a separate loop.
|