Batch Automation Uploads

RonnieKarpinsk
New Contributor III

I am looking for tips on batch automation business rules. Currently when our scheduled trial balances are loaded the whole upload will fail if one transformation mapping error is present. As you can imagine this can slow the close process down significantly. Does anyone have any suggestions on how to allow the batch to load the validated accounts and only error out the transformation errors? 

2 ACCEPTED SOLUTIONS

tsandi
Contributor

You can create a mask for unmapped values across every dimension and use the Order to make sure that will be the last rule applied, but it is a VERY BAD IDEA since the whole purpose of a data quality layer is to make sure the whole dataset gets loaded. If your customer has data quality issues and you want to use OneStream to analyze that data you can use a Blend load with Garbage mask rules, allows the users to review the transactions in blend and create a second load with the updated mappings. 

tsandi_0-1639413092489.png

If the source GL data has additional metadata that needs to be created you can include the metadata creation in the connector rule itself so loads will find the appropriate members when loading data. 

 

View solution in original post

Gidon_Albert
Contributor

We use the TransfromationEventHandler to either add new members under "Unmapped_Members" parent if the we are using one-to-one mapping, or map them to an "Unmapped_Members" base level member if we are using many-to-one mapping. The rule captures the name of the new members that are added and sends an e-mail to the admin to let them know that new members were added. Admin still has to move the new members or update the maps as needed, but at least the load process doesn't fail.

View solution in original post

5 REPLIES 5

LeeB
Contributor II

There isn't a "load with errors" option in OneStream.

I've seen solutions for overnight loads where unmapped values are caught in a mask transformation rule which maps them to a member named "Unmapped" or similar.  An Administrator would then review and remap any accounts mapped to this member.

However, automatically importing incomplete data sets such as Trial Balances presents some obvious issues.  Where possible, I would favour implementing some form of master data governance to ensure any changes are identified and dealt with across all systems thereby preventing the mapping errors from occuring.

tsandi
Contributor

You can create a mask for unmapped values across every dimension and use the Order to make sure that will be the last rule applied, but it is a VERY BAD IDEA since the whole purpose of a data quality layer is to make sure the whole dataset gets loaded. If your customer has data quality issues and you want to use OneStream to analyze that data you can use a Blend load with Garbage mask rules, allows the users to review the transactions in blend and create a second load with the updated mappings. 

tsandi_0-1639413092489.png

If the source GL data has additional metadata that needs to be created you can include the metadata creation in the connector rule itself so loads will find the appropriate members when loading data. 

 

Gidon_Albert
Contributor

We use the TransfromationEventHandler to either add new members under "Unmapped_Members" parent if the we are using one-to-one mapping, or map them to an "Unmapped_Members" base level member if we are using many-to-one mapping. The rule captures the name of the new members that are added and sends an e-mail to the admin to let them know that new members were added. Admin still has to move the new members or update the maps as needed, but at least the load process doesn't fail.

Ando
New Contributor II

Would you have an example of that rule you are using to do that?

The rule we use to dynamically update members is highly customized and specific to our application. It will require some work to make it work for your needs.

OneStream has a MarketPlace solution called ACM that can update new members during data load. It works well and is easy to configure. It is also free!

Let me know if the ACM doesn't work for you and I'll be happy to share our TransformationEventHandler.

Please sign in! RonnieKarpinsk