Tuesday, April 28, 2015

0 What happens when you run a workflow?



When a PowerCenter workflow is ran, it initializes itself and then it

initializes the session(s) within it in the order defined. When a session

initializes, it establishes connectivity to its source and target database(s). If the

mapping is built to read from file(s), it will verify that the source file indeed

exists. If the mapping is built to write to a file, it will attempt to create the

file/overwrite it/append to it depending on the session configuration. Then it

will start reading the source data – one block at a time. Once a block of data is

read, it is transformed and then written to the target. Each block of data is

independently read, transformed and written in parallel. Look at the picture

below












While a session is physically a single component, it depends on several

processes to perform its tasks at runtime. The 3 most important processes are

shown in the picture above. Reader is a process that reads data from the

sources defined in the mapping. DTM is the Data Transformation Manager

(engine), powered by vibe the Informatica's data virtual machine, which

transforms the data as per the mapping specification. Writer is the process that writes the data to the target. In the above picture, the processes are shown

against a timeline. In the sample timeline above, it can be noticed that the

reader process starts at about 09:05 and starts buffering blocks of data. Once it

has enough blocks to transform, the DTM kicks in at about 09:20 and starts

transforming the data that is already buffered by the reader. Once the DTM has

transformed enough blocks, the writer process is triggered (at about 09:35),

which starts writing the transformed data into the target. From about 09:35, all

the three processes run in parallel. Writer is writing the transformed data

provided by DTM, while DTM is transforming the data buffered by reader and

the reader is buffering the next set of blocks. At 11:15, reader has completed

buffering the last set of blocks from the source, while DTM completes

transforming at 11:30 and the writer finally completes writing its last block at

about 11:50.


No comments:

Post a Comment