Forum Discussion
CAIGuySeanF
Contributor
I tried multiple variations and double / rounding conversions (examples above), but no luck. The error does change to my string value: NaN in the error log though.
JackLacava
2 years agoHonored Contributor
The following approach is a last resort (string roundtrips should be avoided), but I believe in practice it should be fairly bombproof:
' get the original big number
Dim longNum As Decimal = yourCell.CellAmount ' 0.98765432109876543210987654321
' allocate a variable that will contain the rounded Decimal
Dim x As Decimal
' round the Decimal, convert it to string, then reparse it as Decimal
Decimal.TryParse(Math.Round(longNum, 8).XFToStringForFormula, x)
' do the rest
Return x ^ (1/6) - 1
This effectively ensures that the rounded Decimal is not carrying around allocated zeros that still count, by recreating it completely.
For the record, this is what I'm seeing in my instance (7.2.2):
' this is a valid decimal - signed 28 digits
Dim dc = api.Data.GetDataCell("0.9876543210987654321098765432")
' this is not a valid decimal, would error out
' Dim dc = api.Data.GetDataCell("0.98765432109876543210987654321")
' this works fine
Return dc.CellAmount ^ 99 - 1
If these snippets don't work in your environment, there might be something dodgy in the installation that you might want to discuss with Support.
Related Content
- 12 months ago
- 8 months ago
- 6 months ago
- 3 years ago