Extract Row count for Import only stage load
Hello all - Looking to capture the actual # of rows inserted into the StageSourceData, NOT just the # of rows that the api.Parser.ProcessDataTable(si, dt, True, api.ProcessInfo) command tries to inse...
Forum Discussion
Hi,
the api (Transformer) object hast the Parser property (Type Parser) where you can get all those metrics.
E.g.: api.Parser.LinesValid
HTH
I started down that path previously and it wasn't giving me what I was looking for. In the example I am working on here, it breaks down like this:
# of Source Rows - 42738 (when i query the external source)
# of Lines Read - 42738
#of Lines Valid - 42738
However, in the processing log there are 42000+ rows that are showing with the 5 ZP (Zero Suppression). I can confirm these are in fact being suppressed in the staging area because there are only 100 or so rows showing in the import screen.
So, I believe the import itself is working correctly by applying zero suppression, but the processing log and the API.Parser methods are not correctly capturing the # records that are being skipped. If I can get the # of rows being skipped, then I can easily subtract that from the # of lines ready to accurately report how many rows actually got loaded.
api.Parser.LinesSkippedZeroSuppress is returning zero, which I think it should be returning the 42k+ rows that are zero suppressed in the log.
Can you attach screenshot of your log?
I had to edit all the other suppressed rows out of the picture to make it fit, but assume 40k+ were suppressed
can i see the bottom of the log?
is that file loading to multiple periods?