Forum Discussion

AdamB's avatar
AdamB
New Contributor II
2 years ago

Extract Dimension Hierarchy Metadata XML Through DM and Extender BR

Hi all,

 

I came across a very helpful post that shows how to automate extracting metadata for backup. (original code at the bottom of post)

https://community.onestreamsoftware.com/t5/Accepted-Code-Samples/Extender-Automate-Application-Metadata-Backup/ta-p/6914 

 

I tried this business rule and it works perfectly, however I am looking for something a little more specific since this returns a massive dump of data. Basically, I am looking to automate a metadata xml extract of specific dimensions and their members / relationships separately into different files and combine them into a zip package. Is there any documentation that shows how to edit this xmlOptions syntax below to include specified return xml values instead of just extracting all items? 

'Set the extract options
Dim xmlOptions As New XmlExtractOptions
xmlOptions.ExtractAllItems = True

 

Ideally in the package, each dimension would have two files. I would have two files as such:

- TopU4 Members

- TopU4 Relationships

'Prepare the Stage Data Extract File path
Dim configSettings As AppServerConfigSettings = AppServerConfig.GetSettings(si)

'Folder path to save export
Dim folderPath As String = FileShareFolderHelper.GetDataManagementExportUsernameFolderForApp(si, _
	True, configSettings.FileShareRootFolder, si.AppToken.AppName) & _
	"\" & DateTime.UtcNow.ToString("yyyyMMdd") & "\MetadataExtracts"

'Check if folder path for export exists
If Not Directory.Exists(folderPath) Then
	Directory.CreateDirectory(folderPath)
end if

Dim filePath As String = folderPath & "\" & si.AppToken.AppName & ".zip"

'Check if file already exists.  If so then delete exsiting copy
If File.Exists(filePath) Then
	File.Delete(filePath)
end if

'Set the extract options
Dim xmlOptions As New XmlExtractOptions
xmlOptions.ExtractAllItems = True

'Execute the Metadata Extract
Using dbConnFW As DBConnInfo = BRAPi.Database.CreateFrameworkDbConnInfo(si)
	Using dbConnApp As DBConnInfo = BRAPi.Database.CreateApplicationDbConnInfo(si)
		
		Dim zipBytes As Byte() = ApplicationZipFileHelper.Extract( _
					dbConnFW, dbConnApp, Nothing, xmlOptions)

		'Push the resulting content to a new zip file we create
		Using FS As New FileStream(filePath, FileMode.Append, FileAccess.Write)

			'Create a binary writer, and write all bytes to the FileStream at once
			Using BW As New BinaryWriter(FS)
				BW.Write(zipBytes)
			End Using
		End Using
	End Using
End Using

 

 Thank you in advance!!

  • PeterFu's avatar
    PeterFu
    Contributor II

    Hi Abrandm,

     

    Take a look at the BR below for extract of an Entity dimension.

    Imports System
    Imports System.Data
    Imports System.Data.Common
    Imports System.IO
    Imports System.Collections.Generic
    Imports System.Globalization
    Imports System.Linq
    Imports Microsoft.VisualBasic
    Imports System.Windows.Forms
    Imports OneStream.Shared.Common
    Imports OneStream.Shared.Wcf
    Imports OneStream.Shared.Engine
    Imports OneStream.Shared.Database
    Imports OneStream.Stage.Engine
    Imports OneStream.Stage.Database
    Imports OneStream.Finance.Engine
    Imports OneStream.Finance.Database
    
    Namespace OneStream.BusinessRule.Extender.XFR_ExportAppTypes
    
    	Public Class MainClass
    
    		'------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    		'Reference Code: XFR_ExportAppTypes
    		'
    		'Description:                   Executes application extracts and produces XML file.
    		'                                         This can be run for different extract types like Metadata or CubeViews
    		'                                         The file is written to the application Data Mgmt Export folder in the file share directory.
    		'
    		'Usage:                             Can be executed from Business Rule editor or run as part of a Data Management sequence.
    		'                                             
    		'Created By:                    Peter Fu
    		'
    		'Date Created:                06-05-2021
    		'------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    		Public Function Main(ByVal si As SessionInfo, ByVal globals As BRGlobals, ByVal api As Object, ByVal args As ExtenderArgs) As Object
    			Try
    				Select Case args.FunctionType
    					Case Is = ExtenderFunctionType.ExecuteDataMgmtBusinessRuleStep
    						'Get Configuration Settings
    						Dim configSettings As AppServerConfigSettings = AppServerConfig.GetSettings(si)
    						'Data Management extract location
    						Dim folderPath As String = FileShareFolderHelper.GetDataManagementExportUsernameFolderForApp(si, True, configSettings.FileShareRootFolder,
    						 si.AppToken.AppName
    						 ) & "\" & DateTime.UtcNow.ToString("yyyyMMdd") & "\Extracts"  'If the directory does not exist create
    If Not Directory.Exists(folderPath) Then Directory.CreateDirectory(folderPath)
    						'Full path and file name for extract
    						Dim filePath As String = folderPath & "\MetadataEntity " & DateTime.UtcNow.ToString("yyyyMMdd") & ".xml"
    						'If file already exist
    						If File.Exists(filePath) Then File.Delete(filePath)
    						'Extract Options
    						Dim xmlOptions As New XmlExtractOptions
    						xmlOptions.ExtractAllItems = False
    						'Extract dimensions
    						Dim extractDict As New Dictionary(Of XmlExtractItemPk, Boolean)
    						extractDict.Add(New XmlExtractItemPk(XmlExtractItemType.Dimensions), True)
    						'Extract Houston Entities
    						Dim dimID As Integer = BRApi.Finance.Dim.GetDimPk(si, "HoustonEntities").DimId
    						extractDict.Add(New XmlExtractItemPk(XmlExtractItemType.Dimension, dimID), True)
    						'Execute the Metadata Extract
    						Using dbConnFW As DbConnInfo = BRApi.Database.CreateFrameworkDbConnInfo(si)
    							Using dbConnApp As DbConnInfo = BRApi.Database.CreateApplicationDbConnInfo(si)
    								'Extract XML metadata to target location
    								File.WriteAllText(filePath, MetadataExtract.ExtractXml(dbConnFW, dbConnApp, xmlOptions, extractDict))
    							End Using
    						End Using
    
    				End Select
    
    				Return Nothing
    			Catch ex As Exception
    				Throw ErrorHandler.LogWrite(si, New XFException(si, ex))
    			End Try
    		End Function
    	End Class
    End Namespace
    

     

    Regards,

    Peter

    • AdamB's avatar
      AdamB
      New Contributor II

      Thank you Peter, this is very helpful! I was able to stack multiple dimensions into one metadata extract with this.

      To take it one step further, do you know if there is a way to extract just the members out of a dimension or just the relationships?  I would think that this could be done by adding this to the extractDict. I do not see anything related to this under the XmlExtractItemType though.

      Thanks,

      Adam

      • PeterFu's avatar
        PeterFu
        Contributor II

        Hi Adam,

         

        Take a look at the BR below. 

         

        Imports System
        Imports System.Data
        Imports System.Data.Common
        Imports System.IO
        Imports System.Collections.Generic
        Imports System.Globalization
        Imports System.Linq
        Imports Microsoft.VisualBasic
        Imports System.Windows.Forms
        Imports OneStream.Shared.Common
        Imports OneStream.Shared.Wcf
        Imports OneStream.Shared.Engine
        Imports OneStream.Shared.Database
        Imports OneStream.Stage.Engine
        Imports OneStream.Stage.Database
        Imports OneStream.Finance.Engine
        Imports OneStream.Finance.Database

        Namespace OneStream.BusinessRule.Extender.XFR_ExportAppTypes
            Public Class MainClass
           
              '------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
              'Reference Code:         XFR_ExportAppTypes
                  '
                  'Description:        Executes application extracts and produces XML file.
                  '                    This can be run for different extract types like Metadata or CubeViews
                  '                    The file is written to the application Data Mgmt Export folder in the file share directory.
                  '
                  'Usage:              Can be executed from Business Rule editor or run as part of a Data Management sequence.
                  '                                              
                  'Created By:         Peter Fu
                  '
                  'Date Created:       06-05-2021
              '------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
           
                Public Function Main(ByVal si As SessionInfo, ByVal globals As BRGlobals, ByVal api As Object, ByVal args As ExtenderArgs) As Object
                    Try
                        Select Case args.FunctionType
                               
                            Case Is = ExtenderFunctionType.ExecuteDataMgmtBusinessRuleStep
                               
                               
                                'Get Configuration Settings
                                Dim configSettings As AppServerConfigSettings =  AppServerConfig.GetSettings(si)
                               
                                'Data Management extract location
                                Dim folderPath As String = FileShareFolderHelper.GetDataManagementExportUsernameFolderForApp(si, True, configSettings.FileShareRootFolder, _
                                si.AppToken.AppName) & "\" & DateTime.UtcNow.ToString("yyyyMMdd") & "\Extracts"
                               
                                'If the directory does not exist create
                                If Not Directory.Exists(folderPath) Then Directory.CreateDirectory(folderPath)
                               
                                'Full path and file name for extract
                                Dim filePath As String = folderPath & "\MetadataEntity " & DateTime.UtcNow.ToString("yyyyMMdd") & ".xml"
                               
                                'If file already exist
                                If File.Exists(filePath) Then File.Delete(filePath)
                                '-------------------------------------------------------------------------------------------------------------------------
                               
                                'Extract Options
                                Dim xmlOptions As New XmlExtractOptions
                                xmlOptions.ExtractAllItems = False
                           
                                'Extract dimensions
                                Dim extractDict As New Dictionary(Of XmlExtractItemPk, Boolean)
                                extractDict.Add(New XmlExtractItemPk(XmlExtractItemType.Dimensions), True)
                               
                                'Extract Entities
                                Dim dimID As Integer = BRApi.Finance.Dim.GetDimPk(si, "CorpEntities").DimId
                                extractDict.Add(New XmlExtractItemPk(XmlExtractItemType.Dimension, dimID), True)
                               
                                'Define dimension options for extract
                                Dim xmlDimOptions As New XmlExtractMetadataDimOptions
                               
                                'Extract specified members
                                xmlDimOptions.ExtractAllMembers = False
                               
                                'Member(s) to extract
                                Dim memExtract As New Dictionary(Of String, Object)
                                memExtract.Add("All Orgs", Nothing)
                                'memExtract.Add("Corporate", Nothing)
                                xmlDimOptions.MembersToExtract = memExtract
                               
                                'Extract all member relationships
                                xmlDimOptions.ExtractAllRelationships = False
                               
                                'Extract members for each relationship
                                xmlDimOptions.ExtractMembersForEachSelectedRelationship = False
                               
                                'Define Metadata extract options
                                'Dim EntID As New Dictionary(Of DimPk, XmlExtractMetadataDimOptions)
                                'EntID = BRApi.Finance.Dim.GetDimPk(si, "Total Golfstream")
                                Dim xmlMetadataOptions As New XmlExtractMetadataOptions()
                               
                                'Entity Dimension type
                                Dim dimT As Integer = DimType.Entity.Id
                               
                                'Set the extract dimension options for specified dimension
                                xmlMetadataOptions.SetOptionsForDimension(dimT, dimID, xmlDimOptions)
                               
                                'Add dimension member options to metadata options
                                xmlOptions.MetadataOptions = xmlMetadataOptions                    
                               
                                'Execute the Metadata Extract
                                Using dbConnFW As DbConnInfo = BRApi.Database.CreateFrameworkDbConnInfo(si)
                                    Using dbConnApp As DbConnInfo = BRApi.Database.CreateApplicationDbConnInfo(si)
                                   
                                        'Extract XML metadata to target location
                                        File.WriteAllText(filePath, MetadataExtract.ExtractXml(dbConnFW, dbConnApp, xmlOptions, extractDict))
                                       
                                    End Using
                                End Using      
                       
                        End Select

                        Return Nothing
                    Catch ex As Exception
                        Throw ErrorHandler.LogWrite(si, New XFException(si, ex))
                    End Try
                End Function
            End Class
        End Namespace

        #End Region "Export Dimension Metadata to XML File"
         
        Regards,
        Peter
  • Does this extract all dimensions?  When I run this rule I get a zip file and when I download the zip file I don't get the xml file showing all dimensions.  Am I doing something wrong?