I am looking to implement an incremental data extraction feature for our cube data. Specifically, I need to extract all data from inception and continue to do so on a quarterly basis, including any updates to previous quarters. Any ideas or best practices on how to achieve this effectively would be greatly appreciated.
Setting up data extraction on a quarterly basis is easy and could be done with a combination of Data Management Sequences, Task Scheduler, and an optional dashboard. For example, after the quarter is closed, you would extract all data for that quarter. The tricky part is figuring out what data has changed in any months the you've already extracted. There is just really no way to do that.
The only thing I could thing you could do is force all changes after a quarter is closed to go through a specific UD and then you could extract only that UD. Another option would be to copy the data to another cube and then run cube views to compare them to see what data has changed. These aren't great option but all I got right now.
Thanks for your response. I will try these options.
