Forum Discussion

Ankit_Shah's avatar
Ankit_Shah
New Contributor II
2 years ago

How to speed up force calculate step?

Hi all,

We are running "force calculate" step to perform force calculation of 80% base entities  . It runs for 4-6 hours. How to improve performance , any way run this in multi thread/parallel by entity (servers are hosted by OneStream) ? Thanks!

  • JackLacava's avatar
    JackLacava
    Honored Contributor

    The system already parallelizes everything it can. There isn't a magic "go faster" button.

    Force Calculate will run on all previous periods, ignoring calculation status of every dataunit. So, in December, you're recalculating the whole year.

    That, plus (likely) inefficient rules, can result in bad performance like you're seeing. 

    You'll need to look at the performance of your rules, scouring the log for timings (you can use the With Logging option for more details, although it will slow down things further) and checking the various Application Reports for dense dataunits, bad rules using #All, excessive reliance on Business Rules rather than Member Formula, etc etc.

  • Ankit_Shah's avatar
    Ankit_Shah
    New Contributor II

    Thanks Jack, I am already ware of what you said , may I should rephrase my question. I want to know a way to trigger same calc.step in parallel for 5-6 different group of entities that way I can achieve faster execution. Currently we pass value1 and all base entities of value1 is force calc. it has appx 10000 entities underneath. Now value1 is a parent in entity dimension so I am looking for a way to launch from dashboard button in single click, if there is way to run force calc for value1.1, value1.2, value1.3 at same time?

    • ChristianW's avatar
      ChristianW
      Valued Contributor

      You can run calculations on different servers using data management jobs. There is a selection box for server in data management jobs. 

      Other tips:

      • Member formulas run in parallel.
      • You can aggregate instead of consolidate.
      • You can run custom calcs in a parallel.ForEach loop from a extender business rule.
      • You can avoid aggregating/consolidating multiple periods. Set the base entities (or a subset of it) to dirty with a business rule and then run a standard aggregate/consolidate instead of force. This can be done with a business rule and a data management job.

      And of cause you can combine some of these ideas.

      • FrankDK's avatar
        FrankDK
        Contributor

        Christian, out of curiosity, would triggering custom calculate using a parallel.ForEach run faster compared to triggering a DM Step for 100 entities? I thought the DM Step would also include some sort of parallelization

    • JackLacava's avatar
      JackLacava
      Honored Contributor

      Ah, I see what you mean. I was going to suggest using a Data Management job, but it looks like it works in the same way you mention - i.e. when you use a filter on  a Force Calculate step, the resulting members are calculated sequentially.

      The most efficient approach would probably be to just Consolidate parents instead, letting OneStream do what it does best (i.e. parallelizing dataunit calculation).

      If that's not possible, depending on what you're doing in those calculations and how long they take individually, it *might* make sense to play around with .Net parallelization yourself, as shown in the example extender below. However, in my tests this adds significant overhead - although they are nominally started at the same time, having to set up internal machinery for each additional thread actually slows down the total execution time by several seconds per thread. If each calculation can take minutes, that's a price worth paying; otherwise, we're making things worse. It may or may not be faster to launch parallel Custom Calcs with ExecuteCustomCalculateBusinessRule instead of going through DataManagement, but at that point you cannot use regular Calculate. Also, it's completely possible that some stuff will work (or not work) in unexpected ways, as it's always the case when one plays with threads; in that case, maybe just launch the sequences one after the other in a simple loop, and hope that OneStream will parallelize some of them (and likely remove some of the threading overhead).

       

      Case Is = ExtenderFunctionType.ExecuteDataMgmtBusinessRuleStep
      	' define which dimension and parent to get base members for
      	Dim myDimPk As DimPk = brapi.Finance.Dim.GetDim(si, _
      							args.NameValuePairs("Dimension")).DimPk
      	Dim parentId As Integer = brapi.Finance.Members.GetMemberId(si, 
      									DimType.Entity.Id, _
      									args.NameValuePairs("ParentEntity"))
      	' retrieve base members
      	Dim baseMembers As List(Of Member) = brapi.Finance.Members.GetBaseMembers(si, _
      																myDimPk, _
      																parentId)
      	' set up parameters for our parallel jobs, 
      	' in this case containing the name of the member we want to calc
      	Dim paramDicts As New List(Of Dictionary(Of String, String))
      	For Each M As Member In baseMembers
      		paramDicts.Add(New Dictionary(Of String, String) From {{"Child", M.name}})
      	Next
      	' launch execution in parallel. The sequence will have one step, calculating the member
      	Parallel.ForEach(paramDicts, _
      		Sub(paramObj)
      			BRApi.Utilities.StartDataMgmtSequence(si, "FCSeq", paramObj)
      		End Sub)
      

       

       

      • dipawar's avatar
        dipawar
        New Contributor II

        Will this above code work with finance business rule??