Forum Discussion

ST1's avatar
ST1
New Contributor III
4 months ago

Transformation Event Handler using Batch Load

When loading data through batch file, does transformation event handler invoked the same way as running through manual workflow steps?

Also, how can I execute Workflow (import->validate->load) through batch?

Workflow
  • JoakimK's avatar
    JoakimK
    3 months ago

    If you are using the standard "BRApi.Utilities.ExecuteFileHarvestBatch" function then the answer is yes, it will invoke all the same steps.

    Secondly, you can decide which steps of the import process to be executed by setting the relevant boolean flags that are used in the same BRApi call above, the easiest way to do that is to create parameters for each flag, and then have the parameter in the function.

    Example:

    Dim valTransform As Boolean = True
    Dim valIntersect As Boolean = True
    Dim loadCube As Boolean = True
    Dim processCube As Boolean = False
    Dim confirm As Boolean = False
    Dim autoCertify As Boolean = False

    Dim batchInfo As WorkflowBatchFileCollection = BRApi.Utilities.ExecuteFileHarvestBatch(si, Scenario, Time, valTransform, valIntersect, loadCube, processCube, confirm, autoCertify, False)

     

    Joakim

  • SimonHesford's avatar
    SimonHesford
    Contributor
    4 months ago

    Yes it does. All the batch process does is automate the manual process but the execution steps are the same

  • JoakimK's avatar
    JoakimK
    Contributor
    3 months ago

    If you are using the standard "BRApi.Utilities.ExecuteFileHarvestBatch" function then the answer is yes, it will invoke all the same steps.

    Secondly, you can decide which steps of the import process to be executed by setting the relevant boolean flags that are used in the same BRApi call above, the easiest way to do that is to create parameters for each flag, and then have the parameter in the function.

    Example:

    Dim valTransform As Boolean = True
    Dim valIntersect As Boolean = True
    Dim loadCube As Boolean = True
    Dim processCube As Boolean = False
    Dim confirm As Boolean = False
    Dim autoCertify As Boolean = False

    Dim batchInfo As WorkflowBatchFileCollection = BRApi.Utilities.ExecuteFileHarvestBatch(si, Scenario, Time, valTransform, valIntersect, loadCube, processCube, confirm, autoCertify, False)

     

    Joakim

  • ssmith-nova's avatar
    ssmith-nova
    New Contributor III
    11 days ago

    How do I catch failures here?  When I load a file if fails in the validation step.   Below is what I am catching so far.  Even with the validate stage failing, the batchinfo.TaskActivityItem.HasError is set to false.  

    Dim detailString As New Text.StringBuilder
detailString .AppendLine($"Batch Name: {batchInfo.GetBatchName}")
detailString .AppendLine($"Load Group Name:{batchInfo.GetFileGroupName(fileGroupID)}")
detailString .AppendLine($"All Load Count:{batchInfo.GetFileCount(fileGroupID, False)}")
detailString .AppendLine($"Processed Load Count:{batchInfo.GetFileCount(fileGroupID, True)}")
detailString .AppendLine($"All Files Completed:{batchInfo.AllFilesCompleted(fileGroupID).ToString}")
detailString .AppendLine($"All Files Loaded:{batchInfo.AllFilesLoaded(fileGroupID).ToString}")
detailString .AppendLine($"All Files Can Process:{batchInfo.AllFilesCanProcess(fileGroupID).ToString}")
detailString .AppendLine($"All Files Processed:{batchInfo.AllFilesProcessed(fileGroupID).ToString}")
detailString .AppendLine($"Task Description:{batchInfo.TaskActivityItem.Description}")
detailString .AppendLine($"Task Activity Status:{batchinfo.TaskActivityItem.TaskActivityStatus}")
detailString .AppendLine($"Task Activity HasError?:{batchinfo.TaskActivityItem.HasError}")
detailString .AppendLine($"Task Activity ErrorLogId?:{batchinfo.TaskActivityItem.ErrorLogID}")