Level 2 Data Unit Overlap Help - Data Loss

[t_kaplanis]

Hello Fellow OneStreamers,

Please bear with the long post.

I have the following workflow profile structure that contains Level 2 data unit overlap I am trying to change based on a financial planning requirement.

The requirement comes in when an analyst reconfigures an hourly labor position in one of the assumption forms. A clear, retransform, load, and seed process should occur to align loaded data with the new configuration that will be used throughout the planning process for a particular position.

The current workflow structure is a hinderance here because kicking off this process for a particular position retransforms and loads all entities which is much more data than needed. It also contains Level 2 data unit overlap because the same data unit is currently being loaded for the specified accounts within the current workflow profile structure.

Our Current Production State:

Base Input Workflow Structure Current State:

• Import Labor Data (Base Input Workflow Profile Step)
  • Import Actuals
    • Loads All Entities
    • Loads Hourly Positions GL Account: 5000
  • Import Actuals HC
    • Loads All Entities
    • Loads Hourly Positions Headcount Account: Headcount
  • Import Salary
    • Loads All Entities
    • Loads Salary Positions Accounts: 5000 and Headcount

As you can see, the “Import Salary” step causes the Level 2 Data Unit overlap. This occurs since account “5000” is being loaded for all data units at “Import Actuals” and account “Headcount” is being loaded for all data units at “Import Actuals HC”.

This has been passable until now because OneStream forces a reload of all steps when “Import Salary” is loaded. There is no data loss.

Development Test Environment State:

In a development application, I have changed this workflow profile structure to the following so that I can automate the clear, retransform, load, and seed process for the specific entity in which the specific HOURLY position that was reconfigured resides. This is an attempt to speed the process up and not retransform so much data unnecessarily.

Per the current requirement SALARY positions are rarely reconfigured, thus no need to retransform and load them. One load for salary positions per period should suffice.

Development Test Workflow Structure:

• Import By Entity Actuals Data (Base Input Workflow Profile Step)
  • Import Entity 1
    • Loads Entity 1
    • Loads Hourly Positions GL Account: 5000
  • Import Entity 2
    • Loads Entity 2
    • Loads Hourly Positions GL Account: 5000
  • Import Entity 3
    • Loads Entity 3
    • Loads Hourly Positions GL Account: 5000
• Import By Entity Actuals HC Data (Base Input Workflow Profile Step)
  • Import Entity 1
    • Loads Entity 1
    • Loads Hourly Positions Headcount Account: Headcount
  • Import Entity 2
    • Loads Entity 2
    • Loads Hourly Positions Headcount Account: Headcount
  • Import Entity 3
    • Loads Entity 3
    • Loads Hourly Positions Headcount Account: Headcount
• Import Salary Data (Base Input Workflow Profile Step)
  • Import Salary
    • Loads All Entities
    • Loads Salary Positions Accounts: 5000 and Headcount

This test workflow structure allows me to kick off specific retransforms and loads by entity for hourly positions.

HOWEVER, whenever “Import Salary” is loaded data loss occurs because of the Level 2 data unit overlap. The salary data load now overwrites previous loads for hourly positions at account “5000” and “Headcount”. This structure gives control, but there is data loss.

Due to where our data resides for each load and its format, combining the hourly and salary position information is a decent effort.

I am wondering if something like a Workflow Channel would be best to implement here? I have read the documentation on the Workflow Channel, but a solution using them in this example was not clear to me when reading through it.

Any help would be much appreciated. Thank you for taking the time to read this post.

[JackLacava · Accepted solution]

Channels is likely what you want. They allow you to segregate data by Account and/or one UD dimension. Assuming your loads target different UD intersections for at least one dimension (since you say Accounts overlap), you can ensure that different loads will be guaranteed not to step on each other's toes. Give another read at the page about Workflow Channels , you'll find a section called "Workflow Channel User Defined Phasing"; I'd try to implement that, then combine it with Account phasing if necessary. Make sure you also read the bit about the standard channels (NoDataLock etc), because you'll probably want them in some profile to keep things sane. This is also covered in the Foundation handbook, and there is probably material on Navigator too.

Channels can be a bit mind-bending to implement, particularly when they get added to an existing application; so expect some breakage of existing customizations (e.g. "this dashboard worked fine but now it won't allow me to enter data!" "Yes, because it's now in the wrong profile for the channel it affects..."); but they are super-powerful and will allow you to solve this kind of situation in a very robust way.