FDX methods (like FdxExecuteCubeView) ... Why are they faster?
Recently I heard in these forums that FDX methods (like FdxExecuteCubeView) should be used from a BR whenever the performance of the data is critical.
Can someone help me understand why they are faster?
We've already built quite a lot of client-integration software using cubeviews. The cubeviews are being executed via the REST api. We are retrieving data with "api/DataProvider/GetAdoDataSetForCubeViewCommand".
Now that I have discovered the "FDX" methods, it puts me in the troublesome position. I realize that a ton of our integration software should probably be re-written to use those FDX methods. The re-write may take some time. Why won't onestream just extend the "normal" cubeview interface of the REST api, and add an option to run it in FDX mode? That would be similar to the "turbo" buttons they used to put on a the tower of a PC to make it run faster (for those users who had a preference between a fast computer and a slow one.)
It would be helpful to understand the differences between these two (fast and slow) mechanisms for retrieving data from cube views.
To add to Peter's points, the FDX methods were specifically created to programmatically extract data from the appropriate OneStream tables or other source systems in the most efficient way. The FDX connectors were primarily created in conjunction of the BI Blend process but can be used for any process that requires data to be extracted from OneStream
FDX, Fast Data Extract, BRAPIs allow a variety of options for connecting to DataSources for BI Blend. A key differentiator between the FDX BRAPI’s and other collection methods is support of parallel processing, in memory processing, and management of the Time dimension.
FDX BRAPI’s provide functionality to build Connectors to extract data from:
- Cube Views: Extract data through a Cube View definition. Ideal for defining data definitions through a Cube View, including Dynamic Calc results.
- Across Cube Data Units: Extract Cube data to a BI Blend target table through defined Data Unit filters.
- Stage Workflow Imports: Ability to leverage existing Stage Data. Uses may be reporting on existing “attribute” records contained in Stage, or simply enhanced dashboard reporting on Stage data.
- Source Systems / Data Warehouses: Performance oriented solution to connect to source system.
Performance is gained through the BRAPI’s ability to parallel process. For example, extracting data by Cube Data Unit will parallel process all the Data Units defined in the filter. Second, the FDX BRAPI’s do not generate a “.CSV” file as do Data Management File “Export Data” or “Export File” processes. The results of the export are managed during the BI Blend “in-memory” processing.
In cases of very large data sets, which where multiple periods are loaded, the processing time can be slow because each period is reflected as a data record. FDX BRAPI’s offer solutions to pivot the Time records to columns in order to create a matrix data layout. The Datasource can associate each of the periods with an “Attribute Value” field within the Integration settings. The design will treat each record as a collection of 12 periods when processing.
- FdxExecuteCubeView: Extract data defined through a Cube View. Any data presented in the Cube View will be extracted, such as Dynamic Calculated results.
- FdxExectuteCubeViewTimePivot: Cube View Data will generate all time as Columns which can be assigned as Attribute Value members in the Data Source.
- FdxExecuteDataUnit: Cube Data extract solution to extract data from Data Unit members.
- FdxExecuteDataUnitTimePivot: Cube Data extract solution to extract data from Data Unit members. Generate all time as Columns, which can be assigned as Attribute Value members in the Data Source.
- FdxExecuteStageTargetTimePivot: Extract existing Workflow’s Stage Data. Generate all time as Columns, which can be assigned as Attribute Value members in the Data Source.
- FdxExecuteWarehouseTimePivot: Extract data from an external source system.
- FdxGetCubeViewOrDataUnitColumnList: Connector BRAPI used to return field names.
- FdxGetStageTargetColumnList: Connector BRAPI used to return field names.
- FdxGetWarehouseColumnList: Connector BRAPI used to return field names.
