What's the best way to pull 16M+ rows of data
Hello! I have a user group who needs a specific set of data pulled just after close every month in order to populate other reporting tools they use. Here's the formatting they require: I'm ...
Forum Discussion
Thanks so much for the direction! I've set up the BR and the data adapter and when I test, I get the same error I received when I built and attempted to export the initial CV:
What am I doing wrong? Here's the code (which I shamelessly stole from this post Solved: FDX Cube View Rule - OneStream Community (onestreamsoftware.com)):
And my data adapter set up:
Any direction would be very much appreciated!!
Some thoughts:
1. FdxExecuteCubeViewTimePivot might get you closer with periods in your columns but 16mil/12 is still more than 1mil so probably not helpful
2. Have you tried the other FDX options that don't rely on a cube view? E.g., BRapi.Import.Data.FDXExecuteDataUnit or BRapi.Import.Data.FDXExecuteDataUnitTimePivot? Not sure what their limits are.
Hi!
Thanks - I don't want periods in the columns though so that wouldn't be an option anyway.I haven't but that was my next step! Thanks 🙂
Well I wouldn't forget that you're just trying to get the data out here. You can always sort out the format/structure of it at a later stage.
I guess another option is splitting your extract by one of your dimensions and then putting it back together after (or letting the user do it).
I'm trying to solve without manual user manipulation, not just trying to get the data out. I can do a flat file DM job if I wasn't looking for a one button solution.