Extract Dimension Hierarchy Metadata XML Through DM and Extender BR

AdamB
New Contributor II

Hi all,

 

I came across a very helpful post that shows how to automate extracting metadata for backup. (original code at the bottom of post)

https://community.onestreamsoftware.com/t5/Accepted-Code-Samples/Extender-Automate-Application-Metad... 

 

I tried this business rule and it works perfectly, however I am looking for something a little more specific since this returns a massive dump of data. Basically, I am looking to automate a metadata xml extract of specific dimensions and their members / relationships separately into different files and combine them into a zip package. Is there any documentation that shows how to edit this xmlOptions syntax below to include specified return xml values instead of just extracting all items? 

'Set the extract options
Dim xmlOptions As New XmlExtractOptions
xmlOptions.ExtractAllItems = True

 

Ideally in the package, each dimension would have two files. I would have two files as such:

- TopU4 Members

- TopU4 Relationships

abrandm_0-1698263517970.png

abrandm_0-1698264100829.png

'Prepare the Stage Data Extract File path
Dim configSettings As AppServerConfigSettings = AppServerConfig.GetSettings(si)

'Folder path to save export
Dim folderPath As String = FileShareFolderHelper.GetDataManagementExportUsernameFolderForApp(si, _
	True, configSettings.FileShareRootFolder, si.AppToken.AppName) & _
	"\" & DateTime.UtcNow.ToString("yyyyMMdd") & "\MetadataExtracts"

'Check if folder path for export exists
If Not Directory.Exists(folderPath) Then
	Directory.CreateDirectory(folderPath)
end if

Dim filePath As String = folderPath & "\" & si.AppToken.AppName & ".zip"

'Check if file already exists.  If so then delete exsiting copy
If File.Exists(filePath) Then
	File.Delete(filePath)
end if

'Set the extract options
Dim xmlOptions As New XmlExtractOptions
xmlOptions.ExtractAllItems = True

'Execute the Metadata Extract
Using dbConnFW As DBConnInfo = BRAPi.Database.CreateFrameworkDbConnInfo(si)
	Using dbConnApp As DBConnInfo = BRAPi.Database.CreateApplicationDbConnInfo(si)
		
		Dim zipBytes As Byte() = ApplicationZipFileHelper.Extract( _
					dbConnFW, dbConnApp, Nothing, xmlOptions)

		'Push the resulting content to a new zip file we create
		Using FS As New FileStream(filePath, FileMode.Append, FileAccess.Write)

			'Create a binary writer, and write all bytes to the FileStream at once
			Using BW As New BinaryWriter(FS)
				BW.Write(zipBytes)
			End Using
		End Using
	End Using
End Using

 

 Thank you in advance!!

6 REPLIES 6

PeterFu
Contributor II

Hi Abrandm,

 

Take a look at the BR below for extract of an Entity dimension.

Imports System
Imports System.Data
Imports System.Data.Common
Imports System.IO
Imports System.Collections.Generic
Imports System.Globalization
Imports System.Linq
Imports Microsoft.VisualBasic
Imports System.Windows.Forms
Imports OneStream.Shared.Common
Imports OneStream.Shared.Wcf
Imports OneStream.Shared.Engine
Imports OneStream.Shared.Database
Imports OneStream.Stage.Engine
Imports OneStream.Stage.Database
Imports OneStream.Finance.Engine
Imports OneStream.Finance.Database

Namespace OneStream.BusinessRule.Extender.XFR_ExportAppTypes

	Public Class MainClass

		'------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
		'Reference Code: XFR_ExportAppTypes
		'
		'Description:                   Executes application extracts and produces XML file.
		'                                         This can be run for different extract types like Metadata or CubeViews
		'                                         The file is written to the application Data Mgmt Export folder in the file share directory.
		'
		'Usage:                             Can be executed from Business Rule editor or run as part of a Data Management sequence.
		'                                             
		'Created By:                    Peter Fu
		'
		'Date Created:                06-05-2021
		'------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
		Public Function Main(ByVal si As SessionInfo, ByVal globals As BRGlobals, ByVal api As Object, ByVal args As ExtenderArgs) As Object
			Try
				Select Case args.FunctionType
					Case Is = ExtenderFunctionType.ExecuteDataMgmtBusinessRuleStep
						'Get Configuration Settings
						Dim configSettings As AppServerConfigSettings = AppServerConfig.GetSettings(si)
						'Data Management extract location
						Dim folderPath As String = FileShareFolderHelper.GetDataManagementExportUsernameFolderForApp(si, True, configSettings.FileShareRootFolder,
						 si.AppToken.AppName
						 ) & "\" & DateTime.UtcNow.ToString("yyyyMMdd") & "\Extracts"  'If the directory does not exist create
If Not Directory.Exists(folderPath) Then Directory.CreateDirectory(folderPath)
						'Full path and file name for extract
						Dim filePath As String = folderPath & "\MetadataEntity " & DateTime.UtcNow.ToString("yyyyMMdd") & ".xml"
						'If file already exist
						If File.Exists(filePath) Then File.Delete(filePath)
						'Extract Options
						Dim xmlOptions As New XmlExtractOptions
						xmlOptions.ExtractAllItems = False
						'Extract dimensions
						Dim extractDict As New Dictionary(Of XmlExtractItemPk, Boolean)
						extractDict.Add(New XmlExtractItemPk(XmlExtractItemType.Dimensions), True)
						'Extract Houston Entities
						Dim dimID As Integer = BRApi.Finance.Dim.GetDimPk(si, "HoustonEntities").DimId
						extractDict.Add(New XmlExtractItemPk(XmlExtractItemType.Dimension, dimID), True)
						'Execute the Metadata Extract
						Using dbConnFW As DbConnInfo = BRApi.Database.CreateFrameworkDbConnInfo(si)
							Using dbConnApp As DbConnInfo = BRApi.Database.CreateApplicationDbConnInfo(si)
								'Extract XML metadata to target location
								File.WriteAllText(filePath, MetadataExtract.ExtractXml(dbConnFW, dbConnApp, xmlOptions, extractDict))
							End Using
						End Using

				End Select

				Return Nothing
			Catch ex As Exception
				Throw ErrorHandler.LogWrite(si, New XFException(si, ex))
			End Try
		End Function
	End Class
End Namespace

 

Regards,

Peter

AdamB
New Contributor II

Thank you Peter, this is very helpful! I was able to stack multiple dimensions into one metadata extract with this.

To take it one step further, do you know if there is a way to extract just the members out of a dimension or just the relationships?  I would think that this could be done by adding this to the extractDict. I do not see anything related to this under the XmlExtractItemType though.

Thanks,

Adam

PeterFu
Contributor II

Hi Adam,

 

Take a look at the BR below. 

 

Imports System
Imports System.Data
Imports System.Data.Common
Imports System.IO
Imports System.Collections.Generic
Imports System.Globalization
Imports System.Linq
Imports Microsoft.VisualBasic
Imports System.Windows.Forms
Imports OneStream.Shared.Common
Imports OneStream.Shared.Wcf
Imports OneStream.Shared.Engine
Imports OneStream.Shared.Database
Imports OneStream.Stage.Engine
Imports OneStream.Stage.Database
Imports OneStream.Finance.Engine
Imports OneStream.Finance.Database

Namespace OneStream.BusinessRule.Extender.XFR_ExportAppTypes
    Public Class MainClass
   
      '------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      'Reference Code:         XFR_ExportAppTypes
          '
          'Description:        Executes application extracts and produces XML file.
          '                    This can be run for different extract types like Metadata or CubeViews
          '                    The file is written to the application Data Mgmt Export folder in the file share directory.
          '
          'Usage:              Can be executed from Business Rule editor or run as part of a Data Management sequence.
          '                                              
          'Created By:         Peter Fu
          '
          'Date Created:       06-05-2021
      '------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
   
        Public Function Main(ByVal si As SessionInfo, ByVal globals As BRGlobals, ByVal api As Object, ByVal args As ExtenderArgs) As Object
            Try
                Select Case args.FunctionType
                       
                    Case Is = ExtenderFunctionType.ExecuteDataMgmtBusinessRuleStep
                       
                       
                        'Get Configuration Settings
                        Dim configSettings As AppServerConfigSettings =  AppServerConfig.GetSettings(si)
                       
                        'Data Management extract location
                        Dim folderPath As String = FileShareFolderHelper.GetDataManagementExportUsernameFolderForApp(si, True, configSettings.FileShareRootFolder, _
                        si.AppToken.AppName) & "\" & DateTime.UtcNow.ToString("yyyyMMdd") & "\Extracts"
                       
                        'If the directory does not exist create
                        If Not Directory.Exists(folderPath) Then Directory.CreateDirectory(folderPath)
                       
                        'Full path and file name for extract
                        Dim filePath As String = folderPath & "\MetadataEntity " & DateTime.UtcNow.ToString("yyyyMMdd") & ".xml"
                       
                        'If file already exist
                        If File.Exists(filePath) Then File.Delete(filePath)
                        '-------------------------------------------------------------------------------------------------------------------------
                       
                        'Extract Options
                        Dim xmlOptions As New XmlExtractOptions
                        xmlOptions.ExtractAllItems = False
                   
                        'Extract dimensions
                        Dim extractDict As New Dictionary(Of XmlExtractItemPk, Boolean)
                        extractDict.Add(New XmlExtractItemPk(XmlExtractItemType.Dimensions), True)
                       
                        'Extract Entities
                        Dim dimID As Integer = BRApi.Finance.Dim.GetDimPk(si, "CorpEntities").DimId
                        extractDict.Add(New XmlExtractItemPk(XmlExtractItemType.Dimension, dimID), True)
                       
                        'Define dimension options for extract
                        Dim xmlDimOptions As New XmlExtractMetadataDimOptions
                       
                        'Extract specified members
                        xmlDimOptions.ExtractAllMembers = False
                       
                        'Member(s) to extract
                        Dim memExtract As New Dictionary(Of String, Object)
                        memExtract.Add("All Orgs", Nothing)
                        'memExtract.Add("Corporate", Nothing)
                        xmlDimOptions.MembersToExtract = memExtract
                       
                        'Extract all member relationships
                        xmlDimOptions.ExtractAllRelationships = False
                       
                        'Extract members for each relationship
                        xmlDimOptions.ExtractMembersForEachSelectedRelationship = False
                       
                        'Define Metadata extract options
                        'Dim EntID As New Dictionary(Of DimPk, XmlExtractMetadataDimOptions)
                        'EntID = BRApi.Finance.Dim.GetDimPk(si, "Total Golfstream")
                        Dim xmlMetadataOptions As New XmlExtractMetadataOptions()
                       
                        'Entity Dimension type
                        Dim dimT As Integer = DimType.Entity.Id
                       
                        'Set the extract dimension options for specified dimension
                        xmlMetadataOptions.SetOptionsForDimension(dimT, dimID, xmlDimOptions)
                       
                        'Add dimension member options to metadata options
                        xmlOptions.MetadataOptions = xmlMetadataOptions                    
                       
                        'Execute the Metadata Extract
                        Using dbConnFW As DbConnInfo = BRApi.Database.CreateFrameworkDbConnInfo(si)
                            Using dbConnApp As DbConnInfo = BRApi.Database.CreateApplicationDbConnInfo(si)
                           
                                'Extract XML metadata to target location
                                File.WriteAllText(filePath, MetadataExtract.ExtractXml(dbConnFW, dbConnApp, xmlOptions, extractDict))
                               
                            End Using
                        End Using      
               
                End Select

                Return Nothing
            Catch ex As Exception
                Throw ErrorHandler.LogWrite(si, New XFException(si, ex))
            End Try
        End Function
    End Class
End Namespace

#End Region "Export Dimension Metadata to XML File"
 
Regards,
Peter

Peter, I know I am not the poster of the original question but I ran across your solution above as I am trying to automate the extract of simple parent/child relationships for a several of our dimensions.  I tried your code above and I am getting the following error when compiling.

denisefockler_0-1699642396999.png

 

AdamB
New Contributor II

Hi Denise, I am having this same issue. Thanks for your reply

ChrisFeller
New Contributor

Does this extract all dimensions?  When I run this rule I get a zip file and when I download the zip file I don't get the xml file showing all dimensions.  Am I doing something wrong?