The OneStream Community is temporarily frozen until June 29th due to the ongoing maintenance. Please read the blog post here to learn more.
Forum Discussion
dbeavon
4 years agoContributor
FDX methods (like FdxExecuteCubeView) ... Why are they faster?
Recently I heard in these forums that FDX methods (like FdxExecuteCubeView) should be used from a BR whenever the performance of the data is critical.
Can someone help me understand why they are fa...
- 4 years ago
To add to Peter's points, the FDX methods were specifically created to programmatically extract data from the appropriate OneStream tables or other source systems in the most efficient way. The FDX connectors were primarily created in conjunction of the BI Blend process but can be used for any process that requires data to be extracted from OneStream
FDX, Fast Data Extract, BRAPIs allow a variety of options for connecting to DataSources for BI Blend. A key differentiator between the FDX BRAPI’s and other collection methods is support of parallel processing, in memory processing, and management of the Time dimension.
FDX BRAPI’s provide functionality to build Connectors to extract data from:
- Cube Views: Extract data through a Cube View definition. Ideal for defining data definitions through a Cube View, including Dynamic Calc results.
- Across Cube Data Units: Extract Cube data to a BI Blend target table through defined Data Unit filters.
- Stage Workflow Imports: Ability to leverage existing Stage Data. Uses may be reporting on existing “attribute” records contained in Stage, or simply enhanced dashboard reporting on Stage data.
- Source Systems / Data Warehouses: Performance oriented solution to connect to source system.
Performance is gained through the BRAPI’s ability to parallel process. For example, extracting data by Cube Data Unit will parallel process all the Data Units defined in the filter. Second, the FDX BRAPI’s do not generate a “.CSV” file as do Data Management File “Export Data” or “Export File” processes. The results of the export are managed during the BI Blend “in-memory” processing.
In cases of very large data sets, which where multiple periods are loaded, the processing time can be slow because each period is reflected as a data record. FDX BRAPI’s offer solutions to pivot the Time records to columns in order to create a matrix data layout. The Datasource can associate each of the periods with an “Attribute Value” field within the Integration settings. The design will treat each record as a collection of 12 periods when processing.
- FdxExecuteCubeView: Extract data defined through a Cube View. Any data presented in the Cube View will be extracted, such as Dynamic Calculated results.
- FdxExectuteCubeViewTimePivot: Cube View Data will generate all time as Columns which can be assigned as Attribute Value members in the Data Source.
- FdxExecuteDataUnit: Cube Data extract solution to extract data from Data Unit members.
- FdxExecuteDataUnitTimePivot: Cube Data extract solution to extract data from Data Unit members. Generate all time as Columns, which can be assigned as Attribute Value members in the Data Source.
- FdxExecuteStageTargetTimePivot: Extract existing Workflow’s Stage Data. Generate all time as Columns, which can be assigned as Attribute Value members in the Data Source.
- FdxExecuteWarehouseTimePivot: Extract data from an external source system.
- FdxGetCubeViewOrDataUnitColumnList: Connector BRAPI used to return field names.
- FdxGetStageTargetColumnList: Connector BRAPI used to return field names.
- FdxGetWarehouseColumnList: Connector BRAPI used to return field names.
PeterFu
4 years agoContributor II
Hi,
This is not my area of expertise, but I found something from the AnalyticBlend Training that I have attended. See print screen below. It looks like the FDX are BRapi's that can run in paralell, making the performce better in this way.
Peter
dbeavon
4 years agoContributor
I noticed that as well. I think that is part of it.
But I think there is more to it than just parallelization. I already am running parallel cubeviews via the REST api. But even if I introduce the parallelization, I still don't get the same overall efficiency as those FDX methods.
Additionally the data from the FDX methods seems a bit more "low-level" or "raw" in nature. It returns only member names (not descriptions), and for whatever reason it annotates the current/selected member of all dimensions (UD's) even when those dimensions do not appear in the definition of the cubeview rows or columns.
Related Content
- 2 years ago
- 3 years ago
- 4 months ago
- 2 years ago