Forum Discussion

photon's avatar
photon
Contributor
1 month ago
Solved

Data retention and deletion

As shocking as it may seem, someone has decided that we actually should delete data from OneStream. In this case, we've been instructed to retain a maximum of five years of data. Everything older than that must be expunged.

Manual, painful, detailed steps aside: is there a good way to clear all old data that I just have never learned about because I have never actually worked with an org who was willing to let go of even one byte of information?

I can manually delete the cube data with enough work (clear rules, consols, the odd custom rule or update to a member formula.) I can delete stage source data with even less work, though its still a bit tedious to click through a bunch of workflows. I haven't yet looked into how to delete stage target data; I'm hoping to find an easy solution but I'm also hoping someone will just tell me so I don't have to learn the hard way. What other data have I not even thought of yet? How do I get rid of it?

Going forward, after we finish our year-end close, I suspect the next step for we admins will be to clear the oldest year. Ultimately, I'll have to write this into a guide that those who come after can follow but there's still lot I don't know that I don't know.

  • KH1's avatar
    KH1
    1 month ago
    We set up "Reset Scenario" Jobs by S# by Yr.

     

7 Replies

    • photon's avatar
      photon
      Contributor
      1 month ago

      I don't think a scenario reset will work for us as we definitely want to keep most of our Actual data, just for one example. We want to selectively purge one year of data across every scenario, not purge one scenario absolutely.

      I did consider making a "storage" scenario that would hold a backup of a scenario so I could reset the original scenario, then import all but the oldest year of data from the copy, then reset the copy itself. However, that loses a lot of other audit data, drillback info, stage data, etc. that we'd rather keep for those years that are still recent enough to retain.

      • KH1's avatar
        KH1
        Contributor III
        1 month ago
        We set up "Reset Scenario" Jobs by S# by Yr.

         

  • photon's avatar
    photon
    Contributor
    1 month ago

    Looking around a little more, I still see millions of rows in stage. A reset didn't clear those. Is there an existing way to clear old years of those?

    • KH1's avatar
      KH1
      Contributor III
      1 month ago

      I suggest setting up additional "Reset Scenario" DM Jobs using a range of Yrs by S#.
      - Stage tables are critical to the "Data Integrity" in OS apps.
      - The best option is to use new RS Jobs to safely "wipe clean" your apps' many Stage tables in OS.

      • photon's avatar
        photon
        Contributor
        1 month ago

        I'm thinking I may have just misread my own stage tables. I had manually cleared a few years of stage from the most common scenarios before I started using the Reset. As I ran the Reset for more and more scenarios, I expected to see the total rows shrinking. However, I was apparently more thorough in my manual deletes than I realized as there's no data in stage prior to the years I want to keep. (What I suppose this means is I know my data and cleared exactly what needed to be cleared. I just wouldn't have needed to do it if I'd used the Reset first.)