diff --git a/d365fo.tools/bin/d365fo.tools-index.json b/d365fo.tools/bin/d365fo.tools-index.json index d26f287f..2aa79301 100644 --- a/d365fo.tools/bin/d365fo.tools-index.json +++ b/d365fo.tools/bin/d365fo.tools-index.json @@ -10346,6 +10346,75 @@ "Examples": "-------------------------- EXAMPLE 1 --------------------------\nPS C:\\\u003eRename-D365Instance -NewName \"Demo1\"\nThis will rename the D365 for Finance \u0026 Operations instance to \"Demo1\".\r\nThis IIS will be restarted while doing it.", "Syntax": "Rename-D365Instance [-NewName] \u003cString\u003e [[-AosServiceWebRootPath] \u003cString\u003e] [[-IISServerApplicationHostConfigFile] \u003cString\u003e] [[-HostsFile] \u003cString\u003e] [[-BackupExtension] \u003cString\u003e] [[-MRConfigFile] \u003cString\u003e] [\u003cCommonParameters\u003e]" }, + { + "CommandName": "Repair-D365BacpacModelFile", + "Description": "As the backend of the Azure SQL infrastructure keeps evolving, the bacpac file can contain invalid instructions while we are trying to import into a local SQL Server installation on a Tier1 environment", + "Params": [ + [ + "Path", + "Path to the bacpac model file that you want to work against", + "", + true, + "false", + "" + ], + [ + "OutputPath", + "Path to where the repaired model file should be placed\nThe default value is going to create a file next to the Path (input) file, with the \u0027-edited\u0027 name appended to it", + "", + false, + "false", + "" + ], + [ + "PathRepairSimple", + "Path to the json file, that contains all the instructions to be executed in the \"Simple\" section\nThe default json file is part of the module, and can be located with the below command:\r\nexplorer.exe $(Join-Path -Path $(Split-Path -Path (Get-Module d365fo.tools -ListAvailable)[0].Path -Parent) -ChildPath \"internal\\misc\")\r\n- Look for the \"RepairBacpac.Simple.json\" file\nOr you can see the latest version, online, inside the github repository: https://github.com/d365collaborative/d365fo.tools/tree/master/d365fo.tools/internal/misc/RepairBacpac.Simple.json\nSimple means, that we can remove complex elements, based on some basic logic. E.g.\n{\r\n\"Search\": \"*\u003cElement Type=\\\"SqlPermissionStatement\\\"*ms_db_configreader*\",\r\n\"End\": \"*\u003c/Element\u003e*\"\r\n}\n\"*\u003cElement Type=\\\"SqlPermissionStatement\\\"*ms_db_configreader*\" can identify below, and together with \"*\u003c/Element\u003e*\" - we know when to stop.\n\u003cElement Type=\"SqlPermissionStatement\" Name=\"[Grant.Delete.Object].[ms_db_configreader].[dbo].[dbo].[AutotuneBase]\"\u003e\r\n\u003cProperty Name=\"Permission\" Value=\"4\" /\u003e\r\n\u003cRelationship Name=\"Grantee\"\u003e\r\n\u003cEntry\u003e\r\n\u003cReferences Name=\"[ms_db_configreader]\" /\u003e\r\n\u003c/Entry\u003e\r\n\u003c/Relationship\u003e\r\n\u003cRelationship Name=\"Grantor\"\u003e\r\n\u003cEntry\u003e\r\n\u003cReferences ExternalSource=\"BuiltIns\" Name=\"[dbo]\" /\u003e\r\n\u003c/Entry\u003e\r\n\u003c/Relationship\u003e\r\n\u003cRelationship Name=\"SecuredObject\"\u003e\r\n\u003cEntry\u003e\r\n\u003cReferences Name=\"[dbo].[AutotuneBase]\" /\u003e\r\n\u003c/Entry\u003e\r\n\u003c/Relationship\u003e\r\n\u003c/Element\u003e", + "", + false, + "false", + "\"$script:ModuleRoot\\internal\\misc\\RepairBacpac.Simple.json\"" + ], + [ + "PathRepairQualifier", + "Path to the json file, that contains all the instructions to be executed in the \"Qualifier\" section\nThe default json file is part of the module, and can be located with the below command:\r\nexplorer.exe $(Join-Path -Path $(Split-Path -Path (Get-Module d365fo.tools -ListAvailable)[0].Path -Parent) -ChildPath \"internal\\misc\")\r\n- Look for the \"RepairBacpac.Qualifier.json\" file\nOr you can see the latest version, online, inside the github repository: https://github.com/d365collaborative/d365fo.tools/tree/master/d365fo.tools/internal/misc/RepairBacpac.Qualifier.json\nQualifier means, that we can remove complex elements, based on some basic logic. E.g.\n{\r\n\"Search\": \"*\u003cElement Type=\\\"SqlRoleMembership\\\"\u003e*\",\r\n\"Qualifier\": \"*\u003cReferences Name=*ms_db_configwriter*\",\r\n\"End\": \"*\u003c/Element\u003e*\"\r\n}\n\"*\u003cElement Type=\\\"SqlRoleMembership\\\"\u003e*\" can identify below, \"*\u003cReferences Name=*ms_db_configwriter*\" qualifies that we are locating the correct one and together with \"*\u003c/Element\u003e*\" - we know when to \r\nstop.\n\u003cElement Type=\"SqlRoleMembership\"\u003e\r\n\u003cRelationship Name=\"Member\"\u003e\r\n\u003cEntry\u003e\r\n\u003cReferences Name=\"[ms_db_configwriter]\" /\u003e\r\n\u003c/Entry\u003e\r\n\u003c/Relationship\u003e\r\n\u003cRelationship Name=\"Role\"\u003e\r\n\u003cEntry\u003e\r\n\u003cReferences ExternalSource=\"BuiltIns\" Name=\"[db_ddladmin]\" /\u003e\r\n\u003c/Entry\u003e\r\n\u003c/Relationship\u003e\r\n\u003c/Element\u003e", + "", + false, + "false", + "\"$script:ModuleRoot\\internal\\misc\\RepairBacpac.Qualifier.json\"" + ], + [ + "PathRepairReplace", + "Path to the json file, that contains all the instructions to be executed in the \"Replace\" section\nThe default json file is part of the module, and can be located with the below command:\r\nexplorer.exe $(Join-Path -Path $(Split-Path -Path (Get-Module d365fo.tools -ListAvailable)[0].Path -Parent) -ChildPath \"internal\\misc\")\r\n- Look for the \"RepairBacpac.Replace.json\" file\nOr you can see the latest version, online, inside the github repository: https://github.com/d365collaborative/d365fo.tools/tree/master/d365fo.tools/internal/misc/RepairBacpac.Replace.json\nReplace means, that we can replace/remove strings, based on some basic logic. E.g.\n{\r\n\"Search\": \"\u003cProperty Name=\\\"AutoDrop\\\" Value=\\\"True\\\" /\u003e\",\r\n\"Replace\": \"\"\r\n}\n\"\u003cProperty Name=\\\"AutoDrop\\\" Value=\\\"True\\\" /\u003e\" can identify below, and \"\" is the value we want to replace with it.\n\u003cProperty Name=\"AutoDrop\" Value=\"True\" /\u003e", + "", + false, + "false", + "\"$script:ModuleRoot\\internal\\misc\\RepairBacpac.Replace.json\"" + ], + [ + "KeepFiles", + "Instruct the cmdlet to keep the files from the repair process\nThe files are very large, so only use this as a way to analyze why your model file didn\u0027t end up in the desired state\nUse it while you evolve/develop your instructions, but remove it from ANY full automation scripts", + "", + false, + "false", + "False" + ], + [ + "Force", + "Instruct the cmdlet to overwrite the file specified in the OutputPath if it already exists", + "", + false, + "false", + "False" + ] + ], + "Alias": "", + "Author": "Mötz Jensen (@Splaxi)", + "Synopsis": "Repair a bacpac model file", + "Name": "Repair-D365BacpacModelFile", + "Links": null, + "Examples": "-------------------------- EXAMPLE 1 --------------------------\nPS C:\\\u003eRepair-D365BacpacModelFile -Path C:\\Temp\\INOX\\Bacpac\\Base.xml -PathRepairSimple \u0027\u0027 -PathRepairQualifier \u0027\u0027 -PathRepairReplace \u0027C:\\Temp\\RepairBacpac.Replace.Custom.json\u0027\nThis will only process the Replace section, as the other repair paths are empty - indicating to skip them.\r\nIt will load the instructions from the \u0027C:\\Temp\\RepairBacpac.Replace.Custom.json\u0027 file and run those in the Replace section.\n-------------------------- EXAMPLE 2 --------------------------\nPS C:\\\u003eRepair-D365BacpacModelFile -Path C:\\Temp\\INOX\\Bacpac\\Base.xml -KeepFiles -Force\nThis will process all repair sections.\r\nIt will keep the files in the temporary work directory, for the user to analyze the files further.\r\nIt will Force overwrite the output file, if it exists already.", + "Syntax": "Repair-D365BacpacModelFile [-Path] \u003cString\u003e [[-OutputPath] \u003cString\u003e] [[-PathRepairSimple] \u003cString\u003e] [[-PathRepairQualifier] \u003cString\u003e] [[-PathRepairReplace] \u003cString\u003e] [-KeepFiles] [-Force] [\u003cCommonParameters\u003e]" + }, { "CommandName": "Restart-D365Environment", "Description": "Restart the different services in a Dynamics 365 Finance \u0026 Operations environment", diff --git a/d365fo.tools/functions/repair-d365bacpacmodelfile.ps1 b/d365fo.tools/functions/repair-d365bacpacmodelfile.ps1 index 1506e62e..1df5871f 100644 --- a/d365fo.tools/functions/repair-d365bacpacmodelfile.ps1 +++ b/d365fo.tools/functions/repair-d365bacpacmodelfile.ps1 @@ -33,22 +33,22 @@ "**" - we know when to stop. - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + .PARAMETER PathRepairQualifier @@ -71,16 +71,16 @@ "**" can identify below, "**" - we know when to stop. - - - - - - - - - - + + + + + + + + + + .PARAMETER PathRepairReplace @@ -118,14 +118,14 @@ This will only process the Replace section, as the other repair paths are empty - indicating to skip them. It will load the instructions from the 'C:\Temp\RepairBacpac.Replace.Custom.json' file and run those in the Replace section. - + .EXAMPLE PS C:\> Repair-D365BacpacModelFile -Path C:\Temp\INOX\Bacpac\Base.xml -KeepFiles -Force This will process all repair sections. It will keep the files in the temporary work directory, for the user to analyze the files further. It will Force overwrite the output file, if it exists already. - + .NOTES Author: Mötz Jensen (@Splaxi) Author: Florian Hopfner (@FH-Inway) diff --git a/d365fo.tools/internal/functions/repair-bacpacmodelqualifier.ps1 b/d365fo.tools/internal/functions/repair-bacpacmodelqualifier.ps1 index 1f48e623..072f2184 100644 --- a/d365fo.tools/internal/functions/repair-bacpacmodelqualifier.ps1 +++ b/d365fo.tools/internal/functions/repair-bacpacmodelqualifier.ps1 @@ -39,16 +39,16 @@ This will remove the below section from the model file - - - - - - - - - - + + + + + + + + + + .NOTES diff --git a/d365fo.tools/internal/functions/repair-bacpacmodelsimpleandreplace.ps1 b/d365fo.tools/internal/functions/repair-bacpacmodelsimpleandreplace.ps1 index 952d26f9..52805333 100644 --- a/d365fo.tools/internal/functions/repair-bacpacmodelsimpleandreplace.ps1 +++ b/d365fo.tools/internal/functions/repair-bacpacmodelsimpleandreplace.ps1 @@ -35,9 +35,9 @@ E.g. "" Replace value that you want to substitute your search value with - + E.g. "" - + .EXAMPLE PS C:\> $removeIns1 = [pscustomobject][ordered]@{Search = '* $replace1 = [pscustomobject][ordered]@{Search = '';Replace = ''} @@ -46,22 +46,22 @@ This will remove the below section from the model file, based on the RemoveInstructions: - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + This will remove the below section from the model file, based on the ReplaceInstructions: diff --git a/docs/Repair-D365BacpacModelFile.md b/docs/Repair-D365BacpacModelFile.md new file mode 100644 index 00000000..4e7382b4 --- /dev/null +++ b/docs/Repair-D365BacpacModelFile.md @@ -0,0 +1,264 @@ +--- +external help file: d365fo.tools-help.xml +Module Name: d365fo.tools +online version: +schema: 2.0.0 +--- + +# Repair-D365BacpacModelFile + +## SYNOPSIS +Repair a bacpac model file + +## SYNTAX + +``` +Repair-D365BacpacModelFile [-Path] [[-OutputPath] ] [[-PathRepairSimple] ] + [[-PathRepairQualifier] ] [[-PathRepairReplace] ] [-KeepFiles] [-Force] [] +``` + +## DESCRIPTION +As the backend of the Azure SQL infrastructure keeps evolving, the bacpac file can contain invalid instructions while we are trying to import into a local SQL Server installation on a Tier1 environment + +## EXAMPLES + +### EXAMPLE 1 +``` +Repair-D365BacpacModelFile -Path C:\Temp\INOX\Bacpac\Base.xml -PathRepairSimple '' -PathRepairQualifier '' -PathRepairReplace 'C:\Temp\RepairBacpac.Replace.Custom.json' +``` + +This will only process the Replace section, as the other repair paths are empty - indicating to skip them. +It will load the instructions from the 'C:\Temp\RepairBacpac.Replace.Custom.json' file and run those in the Replace section. + +### EXAMPLE 2 +``` +Repair-D365BacpacModelFile -Path C:\Temp\INOX\Bacpac\Base.xml -KeepFiles -Force +``` + +This will process all repair sections. +It will keep the files in the temporary work directory, for the user to analyze the files further. +It will Force overwrite the output file, if it exists already. + +## PARAMETERS + +### -Path +Path to the bacpac model file that you want to work against + +```yaml +Type: String +Parameter Sets: (All) +Aliases: + +Required: True +Position: 1 +Default value: None +Accept pipeline input: False +Accept wildcard characters: False +``` + +### -OutputPath +Path to where the repaired model file should be placed + +The default value is going to create a file next to the Path (input) file, with the '-edited' name appended to it + +```yaml +Type: String +Parameter Sets: (All) +Aliases: + +Required: False +Position: 2 +Default value: None +Accept pipeline input: False +Accept wildcard characters: False +``` + +### -PathRepairSimple +Path to the json file, that contains all the instructions to be executed in the "Simple" section + +The default json file is part of the module, and can be located with the below command: +explorer.exe $(Join-Path -Path $(Split-Path -Path (Get-Module d365fo.tools -ListAvailable)\[0\].Path -Parent) -ChildPath "internal\misc") +- Look for the "RepairBacpac.Simple.json" file + +Or you can see the latest version, online, inside the github repository: https://github.com/d365collaborative/d365fo.tools/tree/master/d365fo.tools/internal/misc/RepairBacpac.Simple.json + +Simple means, that we can remove complex elements, based on some basic logic. +E.g. + +{ +"Search": "*\*" +} + +"*\*" - we know when to stop. + +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ + +```yaml +Type: String +Parameter Sets: (All) +Aliases: + +Required: False +Position: 3 +Default value: "$script:ModuleRoot\internal\misc\RepairBacpac.Simple.json" +Accept pipeline input: False +Accept wildcard characters: False +``` + +### -PathRepairQualifier +Path to the json file, that contains all the instructions to be executed in the "Qualifier" section + +The default json file is part of the module, and can be located with the below command: +explorer.exe $(Join-Path -Path $(Split-Path -Path (Get-Module d365fo.tools -ListAvailable)\[0\].Path -Parent) -ChildPath "internal\misc") +- Look for the "RepairBacpac.Qualifier.json" file + +Or you can see the latest version, online, inside the github repository: https://github.com/d365collaborative/d365fo.tools/tree/master/d365fo.tools/internal/misc/RepairBacpac.Qualifier.json + +Qualifier means, that we can remove complex elements, based on some basic logic. +E.g. + +{ +"Search": "*\*", +"Qualifier": "*\*" +} + +"*\*" can identify below, "*\*" - we know when to stop. + +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ +\ + +```yaml +Type: String +Parameter Sets: (All) +Aliases: + +Required: False +Position: 4 +Default value: "$script:ModuleRoot\internal\misc\RepairBacpac.Qualifier.json" +Accept pipeline input: False +Accept wildcard characters: False +``` + +### -PathRepairReplace +Path to the json file, that contains all the instructions to be executed in the "Replace" section + +The default json file is part of the module, and can be located with the below command: +explorer.exe $(Join-Path -Path $(Split-Path -Path (Get-Module d365fo.tools -ListAvailable)\[0\].Path -Parent) -ChildPath "internal\misc") +- Look for the "RepairBacpac.Replace.json" file + +Or you can see the latest version, online, inside the github repository: https://github.com/d365collaborative/d365fo.tools/tree/master/d365fo.tools/internal/misc/RepairBacpac.Replace.json + +Replace means, that we can replace/remove strings, based on some basic logic. +E.g. + +{ +"Search": "\", +"Replace": "" +} + +"\" can identify below, and "" is the value we want to replace with it. + +\ + +```yaml +Type: String +Parameter Sets: (All) +Aliases: + +Required: False +Position: 5 +Default value: "$script:ModuleRoot\internal\misc\RepairBacpac.Replace.json" +Accept pipeline input: False +Accept wildcard characters: False +``` + +### -KeepFiles +Instruct the cmdlet to keep the files from the repair process + +The files are very large, so only use this as a way to analyze why your model file didn't end up in the desired state + +Use it while you evolve/develop your instructions, but remove it from ANY full automation scripts + +```yaml +Type: SwitchParameter +Parameter Sets: (All) +Aliases: + +Required: False +Position: Named +Default value: False +Accept pipeline input: False +Accept wildcard characters: False +``` + +### -Force +Instruct the cmdlet to overwrite the file specified in the OutputPath if it already exists + +```yaml +Type: SwitchParameter +Parameter Sets: (All) +Aliases: + +Required: False +Position: Named +Default value: False +Accept pipeline input: False +Accept wildcard characters: False +``` + +### CommonParameters +This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutVariable, -OutBuffer, -PipelineVariable, -Verbose, -WarningAction, and -WarningVariable. For more information, see [about_CommonParameters](http://go.microsoft.com/fwlink/?LinkID=113216). + +## INPUTS + +## OUTPUTS + +## NOTES +Author: Mötz Jensen (@Splaxi) +Author: Florian Hopfner (@FH-Inway) + +Json files has to be an array directly in the root of the file. +All " (double quotes) has to be escaped with \" - otherwise it will not work as intended. + +This cmdlet is inspired by the work of "Brad Bateman" (github: @batetech) + +His github profile can be found here: +https://github.com/batetech + +Florian Hopfner did a gist implementation, which has been used as the foundation for this implementation + +The original gist is: https://gist.github.com/FH-Inway/f485c720b43b72bffaca5fb6c094707e + +His github profile can be found here: +https://github.com/FH-Inway + +## RELATED LINKS