Forum Discussion

RonnieKarpinsk's avatar
RonnieKarpinsk
New Contributor III
3 years ago

Batch Automation Uploads

I am looking for tips on batch automation business rules. Currently when our scheduled trial balances are loaded the whole upload will fail if one transformation mapping error is present. As you can imagine this can slow the close process down significantly. Does anyone have any suggestions on how to allow the batch to load the validated accounts and only error out the transformation errors? 

  • You can create a mask for unmapped values across every dimension and use the Order to make sure that will be the last rule applied, but it is a VERY BAD IDEA since the whole purpose of a data quality layer is to make sure the whole dataset gets loaded. If your customer has data quality issues and you want to use OneStream to analyze that data you can use a Blend load with Garbage mask rules, allows the users to review the transactions in blend and create a second load with the updated mappings. 

    If the source GL data has additional metadata that needs to be created you can include the metadata creation in the connector rule itself so loads will find the appropriate members when loading data. 

     

  • We use the TransfromationEventHandler to either add new members under "Unmapped_Members" parent if the we are using one-to-one mapping, or map them to an "Unmapped_Members" base level member if we are using many-to-one mapping. The rule captures the name of the new members that are added and sends an e-mail to the admin to let them know that new members were added. Admin still has to move the new members or update the maps as needed, but at least the load process doesn't fail.

  • LeeB's avatar
    LeeB
    Contributor II

    There isn't a "load with errors" option in OneStream.

    I've seen solutions for overnight loads where unmapped values are caught in a mask transformation rule which maps them to a member named "Unmapped" or similar.  An Administrator would then review and remap any accounts mapped to this member.

    However, automatically importing incomplete data sets such as Trial Balances presents some obvious issues.  Where possible, I would favour implementing some form of master data governance to ensure any changes are identified and dealt with across all systems thereby preventing the mapping errors from occuring.

  • You can create a mask for unmapped values across every dimension and use the Order to make sure that will be the last rule applied, but it is a VERY BAD IDEA since the whole purpose of a data quality layer is to make sure the whole dataset gets loaded. If your customer has data quality issues and you want to use OneStream to analyze that data you can use a Blend load with Garbage mask rules, allows the users to review the transactions in blend and create a second load with the updated mappings. 

    If the source GL data has additional metadata that needs to be created you can include the metadata creation in the connector rule itself so loads will find the appropriate members when loading data. 

     

  • We use the TransfromationEventHandler to either add new members under "Unmapped_Members" parent if the we are using one-to-one mapping, or map them to an "Unmapped_Members" base level member if we are using many-to-one mapping. The rule captures the name of the new members that are added and sends an e-mail to the admin to let them know that new members were added. Admin still has to move the new members or update the maps as needed, but at least the load process doesn't fail.

  • Ando's avatar
    Ando
    New Contributor II

    Would you have an example of that rule you are using to do that?

    • Gidon_Albert's avatar
      Gidon_Albert
      Contributor II

      The rule we use to dynamically update members is highly customized and specific to our application. It will require some work to make it work for your needs.

      OneStream has a MarketPlace solution called ACM that can update new members during data load. It works well and is easy to configure. It is also free!

      Let me know if the ACM doesn't work for you and I'll be happy to share our TransformationEventHandler.