Exporting OneStream Data to Snowflake Database Parallel

Hello Team 

I am currently using the "FDXExportDataUnit" function to load data into a DataTable using the following code:

Dim dt As DataTable = BRApi.Import.Data.FdxExecuteDataUnit(si,"FPA_CM","E#Entity_top.base","USD",scenarioTID,Scenario,timefilter,"MTD",True,"Amount <> 0 And Flow='EB'",16,False)

 

he resulting DataTable contains 678,123 records. I plan to insert these records into a Snowflake database. To speed up the inserts, I am dividing the DataTable into multiple DataTables of 150,000 records each. However, the inserts are currently happening sequentially, and I want to run them in parallel.

I am considering the following options:

1. Pass the split DataTable as a parameter to a Data Management sequence:
   This would allow me to call the sequence using "QueueDataManagementSequence" so the process runs on different servers. I need assistance on how to pass the DataTable as a parameter to the sequence.

2. Run bulk insert SQL statements in parallel using threading tasks:
   I need help constructing the Parallel.ForEach loop. My current idea is as follows which is not working

 

parallel.Foreach(splitTable.AsEnumerable(),sf_load.PrepareSqlInsertQueryWithoutColumns(si ,"dev_fin360_core.fpa_integration.CM_CUBE", splitTable, "SFDEVW",""))

Any input or suggestions would be greatly appreciated.

Best Regards

Karun