DM Jobs and Memory Usage

[vmanojrc30]

In a DM Sequence with multiple DM steps (Data Import or consolidations) does the memory that each DM step consume will be released at the completion of each DM step OR it will be on hold till the completion of the DM sequence it is part of?

[MikeG]

Hi @vmanojrc30 When you run a DM Sequence that has multiple Steps, you can be confident that Step 2 will not start until Step 1 is completed.  If Step 1 is doing a Write/Save to the database, Step 2 can use that stored data cell in it's step.  So if your Step 1 is to perform a calculation, and Step 2 is to Consolidate the Entity hierarchy, Step 2 will not start until Step 1 is completed.  Whether there are 2 or 10 Steps it works the same where the next step does not start until after the previous step completes.

Hope this helps.

[vmanojrc30]

Mike ,

Thanks for your response. My question is specific to the app server memory that each step uses. In a DM Sequence with multiple steps, when Step 1 is completed, will the memory that it uses be released after the steps completion OR will it be released after the entire sequence (ALL steps within sequence) is completed?

I have multiple data import (Workflow) DM steps part of a sequence. After few steps the Data Import performance degrades, and sequence get cancelled with an out of memory error. Performance degrades at Validate Intersections or write to cube steps. If I run each step one by one manually, the steps complete without any memory errors.