Forum Discussion

KurtMayer's avatar
KurtMayer
Contributor
16 days ago

Is there guidance on the setting for the number of parallel executions for Harvest Batch Loads?

When using the BRAPi.Utilities.ExecuteFileHarvestBatchParallel function in an Extender BR, does anyone have any guidance and/or opinions on the proper setting of the parallelBatchCount parameter?  ...
Data Integration
Data Management
Import
Workflow
KH1's avatar
KH1
Contributor II
15 days ago

How did you get "parallelBatchCount" = "maximum of the number of CPUs - 1"?
- An earlier post suggested using a Count < "max CPUs - 1".
- We raised a Case: Support suggested using a "trial and error".

Please review our findings below:
1. Parallelism:
Raise a Case: to set Parallelism = the number of CPU Cores of Servers if not already.

Parallelism = the number of CPU Cores (32 Cores for our Servers)

2. Trial and Error:
Run jobs: parallelBatchCount = 16, 18, 20, 22, 24 (and higher as you wish).
- Our tests showed that jobs ran longer when parallelBatchCount >= 20.

Please share your findings.

Thank you and Good luck, Kurt.