Scripting SSIS Package Deployments

http://www.sqlservercentral.com/blogs/chadmiller/2012/11/06/scripting-ssis-package-deployments/#comments

Chad Miller

Chad Miller is a Senior Manager of Database Administration at Raymond James Financial. Chad has worked with Microsoft SQL Server since 1999 and has been automating administration tasks using Windows Powershell since 2007. Chad is the Project Coordinator/Developer of the Powershell-based Codeplex project SQL Server PowerShell Extensions (SQLPSX). Chad leads the Tampa Powershell User Group and is a frequent speaker at users groups, SQL Saturdays and Code Camps.

Scripting SSIS Package Deployments

• Posted on 6 November 2012
• Comments
•   Briefcase
•   Print

Before I delve into the subject of scripting SSIS package deployments, I’m going to take a slight detour and explain why the general subject of automating deployments is important.

Automating Deployments

One of the keys areas you should be looking at automation, possibly through scripting is deployments. If you’re doing deployments and you’re not using a scripted repeatable process, free of GUI and typing then you’re doing it wrong. I’ve seen many times in my career where deployments too often rely on complex instructions rather than using a tested script-based approach. It is inevitable; relying on even the most detailed step-by-step manual instructions will lead to deployments errors because of the human operator factor. When it actually comes time to deploy changes there should be zero typing or clicking. And if it’s not a fully automated deployment then any manual steps should be made as simple as possible such as “run this script with these parameters through copy and paste and report back results.” End Detour.

 SSIS Package Deployments

My SSIS package deployment requirements:

1. The solution must support 2005, 2008, 2008 R2 and 2012 because I have a mixed environment
2. The solution must support deploying to a SQL Server data storage in msdb from a dtsx file
3. The solution must include verification of package installation
4. The solution must be able to create any needed folder structures automatically
5. The solution must include error handling and detailed output on operations performed
6. The solution must support constrained parameters based on using SQL Server data store of a ServerInstance, the dtsx file and the full destination path on the SSIS server

When automating any task I’ll see if there’s already a solution either from Microsoft or third parties. I couldn’t find anything that out-of-the-box does meet all my requirements, but I did find two ways which provide partial solutions.

The first, writing Powershell code directly against Microsoft.SqlServer.ManagedDTS like I’ve done in the SSIS Powershell module I created for SQL Server Powershell Extensions. There’s is a function in the SSIS module called Copy-ISItemFileToSQL, however it provides only part of the solution and there’s a bigger problem of incompatibilities between versions to handle. The assembly for SSIS changes between 2005 and 2008/2008 R2 and 2012 which make crafting a complete solution difficult. I’ve given up on going down this path because it quickly becomes complex.

The second option and the one I went with, is to use the command-line utility dtutil.exe. The nice thing about dtutil–its included with  SQL Server 2005 and higher, well-documented and removes some of complexity of coding against the SSIS classes directly.Although dtutil.exe only meets requirements 1 through 3 above, I can fill in the rest with a bit of Powershell code. I present my Powershell script solution install-ispackage.ps1.

Using Install-ISpackage

To use install-ispackage simply download the script and from PoshCode and run by providing three parameters. Here’s an example of installing a dtsx file to my SSIS server:

1	./install-ispackage.ps1 -DtsxFullName "C:\Users\Public\bin\SSIS\sqlpsx1.dtsx" -ServerInstance "Z001\SQL1" -PackageFullName "SQLPSX\sqlpsx1"

Install-ISPackage Explanined

The install-ISPackage script provides an example of how you can approach calling native console applications (exe’s) from Powershell. You see error handling and handling output differs greatly when calling an exe vs. using cmdlets or .NET code. The former does not trigger errors and instead relies on exit codes defined by the console application developer. You have to check lastexitcode and read whatever documentation is provided with console application to determine what the exit codes mean.

I’ll step through a few things to explain:

When I’m dealing with scripts that make changes I like to set $ErrorActionPreference to Stop instead of the default of Continue. This way I can wrap some error handling and logging around any errors and be assured the script won’t proceed to the next step should an error occur.

I also like to make the exit code more user friendly. I’ll do this by reading the documentation for the command-line utility. On the msdn page for dtutil there a nice table under dtutil Exit Codes which I then create as a hashtable at the top of the script:

1 2 3 4 5 6

$exitCode = @{ 0="The utility executed successfully." 1="The utility failed." 4="The utility cannot locate the requested package." 5="The utility cannot load the requested package." 6="The utility cannot resolve the command line because it contains either syntactic or semantic errors"}

I can then return a more useful error message by using the hastable with the built-in variable $lasterrorcode:

1	throw $exitcode[$lastexitcode]

You’ll notice in the Get-SqlVersion function I’m just using the classic sqlcmd.exe console application to run a query to get the SQL Server version number:

1	$SqlVersion = sqlcmd -S "$ServerInstance" -d "master" -Q "SET NOCOUNT ON; SELECT SERVERPROPERTY('ProductVersion')" -h -1 -W

I choose to use sqlcmd.exe instead of invoke-sqlcmd Powershell cmdlet because it’s installed on every SQL 2005 machine and it’s easier to use when I just want to return a single string:

1 2	C:Users\Public\bin\>Get-SqlVersion -ServerInstance Z001\sql1 10.50.2550.0

The Set-DtutilPath function tries to find the “right” dtutil.exe based on the SQL version being deployed to. You see although parameters for dtutil.exe are identical between version the utility isn’t backwards or forward compatible. You have to use the 9.0 version for 2005,  the 10.0 version for both 2008 and 2008 R2 and the 11.0 version for 2012.

The rest of the functions follow a basic pattern:

Run dtutil.exe and save the output to $result variable

$result will be an array of strings so create a single string separated by newlines:

1	$result = $result -join "`n"

Rather than returning an error on failure or nothing on success, instead return an object with details of what was run:

1 2 3 4 5 6

new-object psobject -property @{ ExitCode = $lastexitcode ExitDescription = "$($exitcode[$lastexitcode])" Command = "$Script:dtutil /File `"$DtsxFullName`" /DestServer `"$ServerInstance`" /Copy SQL;`"$PackageFullName`" /Quiet" Result = $result Success = ($lastexitcode -eq 0)}

I really like using this technique so that if there are failures as part of troubleshooting you can just run the Command property and you get other relevant details. The key here is you can always get back to the base utility so if something doesn’t work in the script you can prove it’s not the script when you get the same error in the utility alone. Note: I have seen errors a few times, usually because a developer will create an SSIS package in later version than the server being deployed to.

Check the $lasterrorcode after calling the utility and returning an object with details:

1 2 3

if ($lastexitcode -ne 0) { throw $exitcode[$lastexitcode] }

Here I’ll use the hashtable defined at the top of script to return a more friendly error message. If errors occur between the error returned and the detailed object I can troubleshoot any issues.

The rest of the functions follow a similar pattern. I will point out a non-zero exit code doesn’t necessarily mean an error. Some console application developers will use error code of 1 or other numbers to mean something other than error as is the case when testing if a folder path exists in dtutil. If it doesn’t exist an error code of 1 is returned. Sometimes it’s hard to determine when a non-zero error code means something other than error except through using the utility. Fortunately Powershell cmdlets don’t use weird exit codes to return status, they generally return an object or error object, but if you’re going to write Powershell scripts against command-line utilities you’ll need to be aware of exit codes and specific exit code meaning for the utility you’re using.

The other thing I’ll point out is the logic to create nest folder paths in Get-FolderList  and new-folder functions. The functions are in place to satisfy my fourth requirement to automatically create folders if they don’t exist.

The main section executes the series of functions in order, wrapped in a try/catch block and since I set my $ErrorAction and check the $lasterrorcode throwing an error in each function, the script will stop should an error occur.