Forum Discussion

tmwright's avatar
tmwright
New Contributor
2 years ago

vStageSourceAndTargetDataWithAttributes 'Ri' field definition

I am trying to load data into a new cube from the vStageSourceAndTargetDataWithAttributes table.  I am seeing most lines duplicated numerous times and the only field I am seeing with different values is the 'Ri' field.  I have not been able to find a listing of the definitions of the fields in this table and so am struggling to determine why the data is duplicated in Stage.  I have checked the actual data import from Snowflake and it does not show duplicates at any stage of the import process. The SQL I am using to pull the data is: 

SELECT TOP 100 *

FROM
"vStageSourceAndTargetDataWithAttributes"
WHERE Si = 'SF_Rev_Summary'
AND Ac = 'ACCT0334_AC'
AND Et = '0034_ME'
AND Tm = '2023M1'

The results from the data adapter are below, the first 5 rows are duplicated apparently due to the 'Ri' field

I appreciate an insight or feedback folks may have. 

Thanks

Data Integration
Data Sources
Import
  • JackLacava's avatar
    JackLacava
    Honored Contributor
    2 years ago

    I think that field is related to Transformation Rules. You probably have Derivative rules duplicating records.

  • franciscoamores's avatar
    franciscoamores
    Contributor II
    2 years ago

    Your duplicated values are for the workflow unit and Fak (file archive key)

    It looks like a connector BR, are you sure you are not getting duplicated data from there?

    If you have derivative rules, I'd expect Rt (Row Type) to have different value for the generated rows.

  • tmwright's avatar
    tmwright
    New Contributor
    2 years ago

    Thanks for the feedback.  The initial load into OneStream is using a connector BR, however, I am not seeing duplicate data at the import/validate/load steps in the workflow import process.  I am only seeing the duplication when querying the stage tables.  I am using the same connector BR for other data sets (distinguished by Si) and am not seeing the duplication issues with the others - only with the particular Si in the code above.  

    • franciscoamores's avatar
      franciscoamores
      Contributor II
      2 years ago

      that view does a left join between stage source and stage attribute table.

      The duplicates you see, did you check all columns?

  • Valeria_Velez's avatar
    Valeria_Velez
    New Contributor
    2 years ago

    Hi, if you don't need the Ri field, you can probably use the SELECT DISTINCT statement in the SQL Script. This would help you to remove the duplicates. 

    I hope this works!