Forum Discussion

Sridhar_M's avatar
Sridhar_M
Contributor
2 years ago

Workflow Overlapped siblings taking long time to load the data to cube

Hi All, 

We have 8 sibling Import workflows under one parent(Base Input Type Parent). We have one entity that's being used to load data in all 8 import workflows. Because we have good volume of data and multiple users loading the data to each workflow, whenever someone try to load the file through any workflow it is processing Load Cube for all remaining workflows as well and this process taking long time. It seems to be expected behavior of workflow design but Is there any recommendations that I can try to reduce the processing time ? Any suggestions would be much appreciated.

Thank you. 

Workflow
  • LeeB's avatar
    LeeB
    Contributor II
    2 years ago

    The system is checking for overlapped WF Data Units and finding them, you can turn this check off but if an overlapping condition occurs you might find it doesn’t load data as expected. To improve load times you may need to use Workflow Channels here to break up the Data Unit.

  • ckattookaran's avatar
    ckattookaran
    VIP
    2 years ago

    That is per design, as you can see OS needs to load the overlapped sibling level 3 unit data so that data is not lost. Even with channels, you'll see the same behavior happening. 

    • Sridhar_M's avatar
      Sridhar_M
      Contributor
      2 years ago

      So there is no technical solution to control that behavior except creating a separate workflow just for that entity and load data through that workflow ? 

  • ckattookaran's avatar
    ckattookaran
    VIP
    2 years ago

    If you move it to a separate workflow then that will remove all the data from other workflows. Channels will help in some way. However if you are touching the level 3 data unit then you don't have much options. It could reduce the load timings. However, that is the way OS works.