Recent Content
Data Processing and Performance - A comprehensive guide of tables, and design
Overview To maintain well performing application, one must understand how the underlying database works and more importantly its limitations. Understanding how a system works, allows designers and administrators to create reliable, stable, and optimal performing applications. This white paper is intended to guide the design of those optimal data processing strategies for the OneStream platform. First, this document will provide a detailed look at the data structures used by the stage engine as well as those used by the in-memory financial analytic engine, providing a deep understanding of how the OneStream stage engine functions in relation to the in-memory financial analytic engine. The relationship between stage engine data structures and finance engine data structures will be discussed in detail. Understanding how data is stored and manipulated by these engines will help consultants build OneStream applications that are optimized for high-volume data processing. Second, the workflow engine configuration will be examined in detail throughout the document since it acts as the controller / orchestrated of most tasks in the system. The workflow engine is the primary tool used to configure data processing sequences and performance characteristics in an OneStream application. The are many different workflow structures and settings that specifically relate to data processing and these settings will be discussed in relation to the processing engine that they impact. Finally, this document will define best practices and logical data processing limits. This will include suggestions on how to create workflow structures and settings for specific data processing workloads. With respect to data defining processing limits, this document will help define practical / logical data processing limits in relation to hard/physical data processing limits and will provide a detailed explanation of the suggested logical limits. This is an important topic because in many situations the physical data processing limit will accept/tolerate that amount of data that is being processed, but the same data may be able to be processed in a much more efficient manner by adhering to logical limits and building the appropriate workflow structures to partition data. These concepts are particularly important because they enable efficient storage, potential parallel processing and high-performance reporting/consumption when properly implemented. Conclusion Large Data Units can create problems for loading, calculating, consolidating, and reporting data. This really is a limitation of what the hardware and networks can support. Your design needs to consider this. But from this paper, I hope you can take away some options to relieve some of the pressure points that could appear.Can you give an example of the best way to sum a row from a cube view? For example, if I add two rows to a cube view, one with a selection of members, how do I make the second row a custom sum of the first? I know one way to do it, but I don’t love it, I would love to see what else you guys recommend.
Clarification – this is referring to using Cube View Row/Column math when members expansions are in use. Answer No “elegant” ways to address this currently – options are: Avoid using member filters (i.e., break the source rows down to more detail) Replicate the member filter in a GetDataCell function on the total row Create a CV Helper function to return the same thing Enhancement Status: There have been numerous enhancement requests to allow CV Row/Column math to work against member expansions, as well as suppressed rows/columns. These have all been approved by Development and should be in one of the upcoming quarterly releases in 2020. Source: Office Hours 2020-03-26 Partner Enablement
Cube Dimension Assignment
Summary The following details offer a quick snapshot of this article’s core content and primary focus to ensure that it is most relevant to your needs. What: Cube dimension assignment When: Early build Why: Enable future flexibility How: Assign dimensions to specific Scenario Types and change “(Use Default)” to the Root dimension Overview To enable future flexibility, it is foundationally critical to properly configure the cube dimension assignments prior to loading data. Once data has been loaded to a cube, the assignments for the (Default) Scenario Type are locked in. The Root dimension assignments in the above image can be updated on the (Default) Scenario Type in the future, but any Scenario Types that have data and are set to “(Use Default)” like the image below, cannot be changed. This means that if not configured properly, the entire cube must abide by the updates to the (Default) Scenario Type. If the cube dimensions are configured properly, additional dimensions can be added to specific Scenario Types in the future. The example use case illustrated in this guide is adding a customer dimension in Budget to expand the annual planning capabilities. This guide provides example configurations to illustrate the recommended approach and common misconfigurations. Recommendation When a cube is created, dimension assignments on specific Scenario Types are set to “(Use Default)” on the Cube Dimensions tab. To properly configure an application for extensibility and enable data model flexibility/expansion in the future, these settings should be updated for the active Scenario Types within each cube. (Default) Scenario Type: Assign the Data Unit dimensions of entity and scenario. Leave all non-Data Unit dimensions as Root. For all active Scenario Types: Entity and scenario will remain as “(Use Default).” All non-Data Unit dimensions should be assigned a specific dimension. Select Root for all unused dimensions. “(Use Default)” should not remain for any dimension. Leave inactive Scenario Types as-is until ready to be activated. Use Case & Examples Use Case: A client with a live OneStream application wants to enable a customer dimension in Budget to expand their annual revenue planning capabilities and include their top customers. Data has already been loaded to Actual and a prior year Budget. Configuration #1: Recommended Configuration The recommended configuration of cube dimension assignments will enable the application to take full advantage of extensibility. This configuration will allow the addition of new dimensions to specific Scenario Types in the future and eliminate the need to “stub out” unused dimensions for future use. To configure properly, the Data Unit dimensions (entity and scenario) will be assigned to the (Default) Scenario Type, and all remaining active dimensions will be assigned to their respective active Scenario Types. Any inactive dimensions should be set to "Root” instead of “(Use Default)”. Recommended initial assignment is as follows: (Default) Actual Budget After the Actual and Budget Scenario Types both have data in them, we are still able to change our UD4 Dimension assignment in the Budget Scenario Type to include our new summary customer dimension. **Be aware that once you hit save, the new UD4 assignment will be locked in, and you will be unable to change it if there is data in that cube and Scenario Type combination. Changing from a Root dimension is a one-time change that cannot be reverted if there is data in this cube and Scenario Type combination. ** After adding the new dimension to the Budget Scenario Type, one will see the history in UD4#None, and the new dimension members active for input in subsequent budget cycles. Since it was assigned to the specific Scenario Type and not (Default), you will notice that this new UD4 dimension is invalid for the Actual Scenario Type. This configuration will also allow the future addition of UD5 and UD6 dimensions by following these same steps. Configuration #2: Improper Assignment to the (Default) Scenario Type A common error is to assign all dimensions to the (Default) Scenario Type and only use the Scenario Type-specific tabs for those that differ. This configuration will work and will also allow you to add additional dimensions in the future but is much less flexible. Additional dimensions in this setup must be assigned to the (Default) Scenario Type and will therefore apply to all active scenarios. In the example below, all active dimensions have been assigned to the (Default) Scenario Type and a different Account dimension has been assigned to the Budget Scenario Type to enable the use of extensibility. The remaining non-Data Unit dimensions have been left as (Use Default) for both the Actual and Budget Scenario Types: (Default) Actual Budget In this setup, attempting to assign our new customer dimension to UD4 on the Budget Scenario Type: Will display this error: Due to the use of (Use Default) on the active Scenario Types, they are now locked into whatever the (Default) Scenario Type has set for these dimensions and they cannot be updated. To add our new customer dimension, one is forced to assign it to the (Default) Scenario Type: When assigning to the (Default) Scenario Type, you will notice that it works for Budget as required (same as the recommended configuration), but it is now active for the Actual Scenario Type as well which was not the desired result. With this configuration, existing business rules and member formulas will need to be validated throughout the application to ensure the right intersections are specified. This additional dimension contains valid intersections in all Scenario Types; therefore, rules need to be more explicit in their filtering and writing of data. If written improperly or too open, this new dimension may cause a performance impact or zeros and other bad data being calculated in these new intersections. Configuration #3: Improper Assignment of (Use Default) Another erroneous configuration is to assign all dimensions to their respective Scenario Type but leave unused dimensions as (Use Default). This configuration will also work and will allow you to add additional dimensions in the future, but also is not as flexible as the recommended setup. Additional dimensions in this setup must be assigned to the (Default) Scenario Type and will therefore apply to all active scenarios. In the example below, all active dimensions have been assigned to their respective Scenario Types. The remaining inactive dimensions have been left as (Use Default) for both the Actual and Budget Scenario Types: (Default) Actual Budget In this setup, attempting to assign our new customer dimension to UD4 on the Budget Scenario Type will result in the same error as configuration #2 above: Again, forcing the assignment to the (Default) Scenario Type which will apply to all active Scenario Types with the setting of (Use Default) for UD4. This assignment will also work for Budget, but as with configuration #2 above, you will notice that it is now active for the Actual Scenario Type as well which was not the desired result. As with configuration #2 above, existing business rules and member formulas should be validated throughout the application to ensure the right intersections are specified. Additional no input rules may be necessary to limit input to these intersections in Scenario Types that do not apply. Considerations The recommended configuration for cube dimension assignment eliminates the need to “stub out” unused dimensions for future use. If unused dimensions are assigned to “Root” on their respective Scenario Types, they can be changed in the future. One should not create a placeholder dimension for those that are unused (UD4, UD5, and UD6 in our example above) as this will only limit future flexibility. If additional dimensions are not configured properly, you can only update from Root to a specific dimension once. If you accidentally save an incorrect dimension update, you’re locked into that change. Plan and make sure these settings are properly updated before saving the changes. Despite adding flexibility for the future, configuring the cube dimensions in this way still does not allow you to change active dimensions with data. Conclusion As you can see from the examples above, non-Data Unit dimensions should be assigned to the cube by Scenario Type with (Use Default) changed to the Root dimension for those that are inactive at setup. Assigning non-Data Unit dimensions to the (Default) Scenario Type and leaving (Use Default) on the specific Scenario Types will limit the benefits and flexibility provided by extensibility. Improper setup will force the entire cube to conform to future updates to the (Default) Scenario Type. Conversely, assigning non-Data Unit cube dimensions to specific Scenario Types and utilizing the Root dimensions instead of (Use Default) will open additional growth opportunities for the application. This recommended configuration is also more flexible than “stubbing out” dimensions for future use as you do not have to consider the potential pitfalls related to that design.
Question: Is there a way to report the total of a group of members based on a text attribute with multiple dimensions in a row?
Question (cont) The example below is for an operating expense that I want to be filtered using a text property of “HP” (a specific business line). I thought the following would work, but it is still pulling in the total company business line, not just a total of the “HP” UD3 members. A#Marketing:U1#TotalBU:U2#Top:U3#CustomerGroup_0002.Base.Where(Text1 contains HP):Name(Market) I don’t need an expansion of the customers (UD3) but a total of the relevant members. Answer Using CVR Math in a cubeview row will not work in this situation. CVR Math Example A Dashboard XFBR String will be needed to aggregate the members with the Text property to produce a total. XFBR Example Source: Office Hours 2021-02-04 - Partner EnablementExtensibility Series: An Overview of Extensibility in OneStream
What is Extensibility? The concept of Extensibility in OneStream is the capability to incorporate multiple use cases and future growth with a single foundation. I like to relate this to a dinner table that can expand and add additional table leaves while maintaining the same integrity. The OneStream platform, in tandem with Workflow and Extensible Dimensionality expands on this concept by providing users with multiple ways to extend their platform footprint. When designing an application or planning for expansion to the existing footprint, these concepts are crucial to understand and apply correctly. Extensibility in OneStream is a broad topic and can mean something different to each person in the community so I would like to break our language on this topic down further into the following categories: Horizontal Extensibility Vertical Extensibility Workflow Extensibility Platform Extensibility Horizontal or Scenario extensibility relates to the ability to extend and use different levels of a hierarchy for different business purposes. It also provides the ability to target when and where dimensions need to be included in the data model. Have you ever wanted to input data at a parent level? Through horizontal extensibility, that parent can become a base for input in a different scenario by using the scenario type settings and properly applying Cube Dimension Assignments. What if you have highly detailed metadata that only applies to a specific use case? Horizontal extensibility can help limit the potential intersections that aren’t valid for all the other use cases by assigning it only where it makes sense. Vertical or Entity/Cube extensibility relates to the ability to include/exclude detail at different levels up the entity hierarchy. The Data Unit is a key concept to understand in OneStream and it is important to properly manage its size to allow for optimal performance while accounting for future growth. Vertical extensibility also relates to varying dimensionality across business units. When you report consolidated financials, do you need to see the lowest level of department detail? Each individual product? Every project? The most granular GL accounts? If the answer is no to any of these, vertical extensibility can help. Lower-level entities can still report at a detailed level, but the data can be collapsed to a summary level to facilitate the reporting and increase performance. Does your organization have Business Units with very different operations? Perhaps vertical extensibility can provide the flexibility you need to vary the dimensionality at a detailed level but consolidate to a common summary level. Workflow extensibility relates to the ability to vary the input steps & methods within each process flow. Workflow steps and settings can be adjusted on each scenario type or can be combined if multiple processes follow the same responsibility hierarchy. Workflow extensibility can be configured on each parent cube to tailor the software interface to match the process needs. Is your Actual data collection process more import driven and the Planning process more forms, calculations, and dashboard driven? Workflow extensibility can help split these processes and make them easier to manage from an administration standpoint. Are some data collections imported in a centralized fashion while others have their responsibility distributed to more end users? Entities can only be assigned once in a Workflow hierarchy so to vary the entity signoff responsibilities, Workflow extensibility should be utilized to allow for differing entity assignments. Platform extensibility relates to the ability to vary where data is stored and how it is utilized within the platform. It also includes the ability to have multiple applications within one environment that can talk to each other. OneStream has the unique ability to consume, utilize, and report on data regardless of if it is stored in cubes, relational tables, or even externally. The capabilities in this category are expanding rapidly and should be considered during all solution design activities. Do you plan at a named personnel level? By each individual capital project? It’s important to determine what is necessary in the cube for consolidated reporting versus what can live outside the cube to be reported on more at a base entity level. Through platform extensibility, we can combine cube data with relational data to achieve the optimal balance between performance and reporting needs. Is the process you are designing more operationally driven and your data dimensions more transient in nature? Perhaps none of a specific data set needs to live in a cube, or even OneStream at all. Platform extensibility allows us to utilize entirely relational data, web content, and even external data sets. How should one think about Extensibility? Extensibility is foundational to OneStream. It should be thought of as a tool as essential as the level. Without it, you can probably get the job done and, on the surface, it might look okay as well. But over time, you are likely to discover structural integrity issues. It is probable that what you built may no longer be able to do everything you need it to. We use extensibility to right-size data units. We use extensibility to input at the right level. We use extensibility to fit the business process. We use extensibility to set the foundation for the future. I’ve heard people talk about extensibility in that you are “locked in” to the choices you make now. While there is some truth in that, it should not be thought about as a box, but a key to the future. Applying extensibility opens the door to so many more options in the future. Design the process and use extensibility as the tool to bring it all together. As mentioned in the Guiding Principles article, the importance of designing the process cannot be stressed enough. Don’t look for a tool, look for a problem and use the tools provided. Be forward thinking during design and ask questions to all stakeholders to make sure future functionality is accommodated for. Be sure to understand how the business operates and what is on the roadmap so that the proper foundation can be built. Recommendations I will begin with a disclaimer, there is not a single be-all, end-all way to implement extensibility in OneStream. I have seen applications with no extensibility and ones with too much extensibility. While there is a middle ground that should be found, the applications without extensibility are those that much more commonly have issues. A lack of vertical and platform extensibility tends to lead to performance issues. A lack of horizontal and Workflow extensibility tends to lead to flexibility issues. The applications with too much extensibility less commonly run into performance or flexibility issues, but they do have a higher maintenance burden. This is why, as architects, it is our job to balance performance, usability, and maintenance when thinking about these four types of extensibility. It is our recommendation that extensibility be considered in every single design and that it should be implemented nearly every time. To not use extensibility should be an exception, not the norm. During a solution design, I like to fill out a matrix like the one below to visualize what detail needs to be included where. With this, you can start to shape the Scenario Types, cubes, dimensions, and any platform extensibility. When looking for extensibility configuration examples, look no further than our CPM Blueprint application. This application has example configurations using our leading practices. Looking at UD1 as an example, one can see our common configuration of a “MainUD1” dimension parent to summarize the BU and Cost Center details into a common dimension. This is a concept we apply to all user defined dimensions to facilitate both vertical and horizontal extensibility. To facilitate vertical extensibility, dimensional detail that is not needed in a parent cube can be collapsed by assigning MainUD1. The dimensional detail is then extended from “TotUD1” to expand into the necessary levels of detail for each data collection and reporting need. This allows both “None” and “Top” to be active at all levels in the dimensional hierarchy. Another example of extensibility on display in the CPM Blueprint application is in the cube configuration. Focusing on the financial reporting structure in this application, it follows our recommendation for a base-summary cube relationship between Business Unit and total company reporting. I commonly apply this configuration even if there is only a single child cube and a single parent cube because it opens the door to so many more options in the future: More flexibility to expand child cubes horizontally and plug in different dimensionalities Greater ability to collapse the data unit if its size becomes an issue Further future-proofing as it allows for more platform expansion with the same foundation Finally, this application also has Workflow extensibility on display. On the cube settings, you can see the connection between top level and base cubes. You can also see the Workflow suffixing applied in the CPM Blueprint application. In this example, the Actual Scenario Type has a different process flow and responsibility hierarchy from other data collections, so it has been given its own suffix of “ACT.” Budget and Forecast follow the same process flow and responsibility hierarchy so therefore share a Workflow suffix of “BUDFCST.” This allows each process to have its own configuration and entity assignment. Conclusion Extensibility in OneStream cannot be overlooked. During a solution design, each of the four types of extensibility should be weighed and discussed to see which tool is right for the job: Horizontal Extensibility Vertical Extensibility Workflow Extensibility Platform Extensibility If you conclude that extensibility is not right for you, be absolutely sure. If the choice was up to me, the benefits of future flexibility and performance reliability greatly outweigh the potential need for additional administration overhead and end user training that come with extensibility.
