Implementing Team Build, part 2

(A continuation from Implementing Team Build, part 1)

Setting up

So, where to begin? Well, for one thing it's good to know how Team Build works and what you can do with - and not. MSDN is as usual a great source of information (although there's an awful lot of it, you might miss some important bits in all the "debris"). Ewald Hoffman's posts on how to Customize Team Build 2010 was also a great source of information that we followed almost word-for-word. My colleague and I also had the 'luxury' of attending a course in administering Team Build that was useful, although somewhat railroaded (as courses tend to be).

Use third-party or make your own activities?

Well, both it turned out. We set out to use Community TFS Build Extensions as much as possible, but in those cases these didn't live up to our specific requirements, we made our own. In some cases, we extended activities from TFS Extensions (we're going to submit these changes back once they're thoroughly tested). 

Project set-up

As we were certain that we were going to need custom activities, we followed mr. Hoffman's guidance on how to set up a project for this. This is crucial, since when you've started adding your custom activities to a build process, that process cannot be read without first opening the project/solution.

One caveat is that if you are going to use more than one process template for your builds, you will need to edit each of them in an XML editor in order to make the project compile. Visual Studio for some reason want's each process template to use unique tags to the common activities, so we had to difference each process template by exchanging the top line

<Activity mc:Ignorable="sads sap" x:Class="TfsBuild.Process" ..>

with

<Activity mc:Ignorable="sads sap" x:Class="TfsBuild2.Process" ...>

for each extra process template you have in the solution.

Assembly locations

A build controller can have one and only one source-controlled folder where custom/external assemblies can be found. This might seem limited, but that's how it is. We selected to have a folder outside of the project where we also stored the assemblies for TFS Community Build Extensions. This meant that deploying our own assemblies is somewhat cumbersome (checkout, build, manual copy, checkin), but it works.

Remote debugging

Being able to debug custom-made activities would have been awesome, and we followed Hoffman's post on how to set this up. But we never got it working, no matter how much we tried; in the end we gave up and did it the hard way, i.e. make changes, code review, build, checkin new assemblies and test run. This meant that using Team Build in production got somewhat delayed since we wanted just about everything to work before starting to use it for "real" builds.

Implementing Team Build, part 1

Our company is certainly not the first to jump onto the Team Foundation Build system for managing builds, but probably not the last either. We CM's inherited a CruiseControl.Net installation which had grown to include quite a few features, and naturally we wanted to keep those. These were what we needed, that weren't supported with 'basic' Team Build process templates and actions:

• Automatic update of assembly information.
• Build dependencies, i.e. building of one product automatically queues the build of another (and another, and...).
• Unit testing with NUnit, continuously and at intervals. (We have a lot of tests, roughly 8000, and one single test run could sometimes take as much as 7+ hours).
• Building installation packages.
• Automatic e-mail to users upon build or test failure.

In fact, we almost went with JetBrain's Team City because it had all these features from start. But luckily, we decided to try to implement these features by extending Team Build first. It has been quite a trip, but now we're nearly there.

Following that decision, we naturally wanted to add the 'winning' features of Team Build as well:

• Changeset and Work Item integration
• Gated check-in's
• Build retention
• Symbol server

In the following posts, I'll try to go through our implementation step by step.

Replacing Visual Studio Command Prompt

Working a lot with Visual Studio, I naturally wanted to be able to use Powershell as the preferred command line tool instead of cmd.exe that Visual Studio Command Prompt uses. Not being aware of the Invoke-Batchfile functionality available in Powershell Community Extensions, and seeing that the script logic could be made so much more efficient in Powershell, I ended up re-writing the whole script in Powershell.

One thing that simplified it for me was the fact that I only need the x64 variant, but it should be ver easy to extend this to cover all other variants. In fact, I started that but when I found that I didn't need them myself that part naturally got demoted... And it could surely be bettered some day.

Anyway, here's the script:

#vcvars64.ps1, for setting Visual Studio environment variables for 64-bit architecture
[CmdletBinding()]
param()

function Set-EnvVar
{
 param (
  [string]$Variable,
  [string]$Value,
  [switch]$Append
 )
 
 if ($Append)
 {
  if (Test-Path ('env:' + $Variable))
  {
   $Value = (Get-Item ('env:' + $Variable)).Value + ";$Value"
  }
 }
 
 [environment]::SetEnvironmentVariable($Variable, $Value)
}

function Set-EnvVarFromRegistry
{
 param (
  [string[]]$keys,
  [string]$property,
  [string]$environmentVariable,
  [string]$suffix,
  [switch]$append
 )
 
 foreach ($key in $keys)
 {
  if (Test-Path $key)
  {
   $regValue = (Get-ItemProperty $key).$property
   if (-not ($regValue -eq ""))
   {
    Set-EnvVar $environmentVariable ($regValue + $suffix)
    break
   }
  }
 }
}

Write-Host -ForegroundColor Yellow "Setting environment for using Microsoft Visual Studio 2010 x64 tools.`n"

$VSRegKeys32 = @("HKLM:\SOFTWARE\Microsoft\VisualStudio\SxS\VS7", "HKCU:\SOFTWARE\Microsoft\VisualStudio\SxS\VS7")
$VSRegKeys64 = @("HKLM:\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VS7", "HKCU:\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VS7")
$VSRegKeys = ($VSRegKeys32 + $VSRegKeys64)
$VCRegKeys32 = @("HKLM:\SOFTWARE\Microsoft\VisualStudio\SxS\VC7", "HKCU:\SOFTWARE\Microsoft\VisualStudio\SxS\VC7")
$VCRegKeys64 = @("HKLM:\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VC7", "HKCU:\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VC7")
$VCRegKeys = ($VCRegKeys32 + $VCRegKeys64)
$FSharpRegKeys = @("HKLM:\SOFTWARE\Microsoft\VisualStudio\10.0\Setup\F#", "HKCU:\SOFTWARE\Microsoft\VisualStudio\10.0\Setup\F#", "HKLM:\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\10.0\Setup\F#", , "HKCU:\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\10.0\Setup\F#")
$WindowsSDKRegKeys = @("HKLM:\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v7.0A", "HKCU:\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v7.0A")

Set-EnvVarFromRegistry $VSRegKeys "10.0" VS100COMNTOOLS "Common7\Tools\"
Set-EnvVarFromRegistry $VSRegKeys "10.0" VSINSTALLDIR
Set-EnvVarFromRegistry $VCRegKeys "10.0" VCINSTALLDIR
Set-EnvVarFromRegistry $FSharpRegKeys ProductDir FSHARPINSTALLDIR
Set-EnvVarFromRegistry $VCRegKeys64 FrameworkDir64 FrameworkDir
Set-EnvVarFromRegistry $VCRegKeys64 FrameworkVer64 FrameworkVersion
Set-EnvVar Framework35Version "v3.5"
Set-EnvVar VCbuildToolsPath ($env:VCINSTALLDIR + "VCPackages")

Set-EnvVarFromRegistry $WindowsSDKRegKeys InstallationFolder WindowsSdkDir
if ($env:WindowsSdkDir -eq "") { Set-EnvVar "WindowsSdkDir" "$env:VCINSTALLDIR\PlatformSDK\" }

if ($env:WindowsSdkDir)
{
 Set-EnvVar PATH ($env:WindowsSdkDir + "bin\NETFX 4.0 Tools\x64;" + $env:WindowsSdkDir + "bin\x64;" + $env:WindowsSdkDir + "bin") -Append
 Set-EnvVar INCLUDE ($env:WindowsSdkDir + "include") -Append
 Set-EnvVar LIB ($env:WindowsSdkDir + "lib\x64") -Append
}

# PATH
Set-EnvVar PATH ($env:VSINSTALLDIR + "Common7\Tools;" + $env:VSINSTALLDIR + "Common7\IDE;$($env:VCINSTALLDIR)VCPackages;$env:FrameworkDir\$env:FrameworkVersion;$env:FrameworkDir\$env:Framework35Version;" + $env:VCINSTALLDIR + "bin\amd64") -Append
if (Test-Path "$env:VSINSTALLDIR\Team Tools\Performance Tools\x64") { Set-EnvVar PATH ($env:VSINSTALLDIR + "Team Tools\Performance Tools\x64;" + $env:VSINSTALLDIR + "Team Tools\Performance Tools") -Append }
if (Test-Path "$env:ProgramFiles\HTML Help Workshop") { Set-EnvVar PATH "$env:ProgramFiles\HTML Help Workshop" -Append }
if (Test-Path "$env:ProgramFiles(x86)\HTML Help Workshop") { Set-EnvVar PATH "$env:ProgramFiles(x86)\HTML Help Workshop" -Append }
Write-Verbose "PATH set to $env:Path"

# INCLUDE
if (Test-Path "$env:VCINSTALLDIR\ATLMFC\INCLUDE") {Set-EnvVar INCLUDE ($env:VCINSTALLDIR + "ATLMFC\INCLUDE") -Append }
Set-EnvVar INCLUDE ($env:VCINSTALLDIR + "INCLUDE;") -Append
Write-Verbose "INCLUDE set to $env:INCLUDE"

# LIB
Set-EnvVar LIB ($env:VCINSTALLDIR + "LIB\amd64;") -Append
if (Test-Path "$env:VCINSTALLDIR\ATLMFC\LIB\amd64") {Set-EnvVar LIB ("$($env:VCINSTALLDIR)ATLMFC\LIB\amd64") -Append }
Write-Verbose "LIB set to $env:LIB"

# LIBPATH
Set-EnvVar LIBPATH ("$env:FrameworkDir\$env:FrameworkVersion;$env:FrameworkDir\$env:Framework35Version;$($env:VCINSTALLDIR)LIB\amd64") -Append
if (Test-Path "$env:VCINSTALLDIR\ATLMFC\LIB\amd64") {Set-EnvVar LIBPATH ($env:VCINSTALLDIR + "ATLMFC\LIB\amd64") -Append }
Write-Verbose "LIBPATH set to $env:LIBPATH"

Set-EnvVar Platform X64
Write-Verbose "Platform set to X64"
Set-EnvVar CommmandPromptType Native
Write-Verbose "CommmandPromptType set to Native"

Powershell: Catching an unspecified number of arguments into an array

I recently had the need for creating a command to be executed on an unspecified number of objects, and being picky regarding the command syntax, I wanted it to be called in the "usual" way, like:

My-Command namedParameter file1 file2 ... file

I.e. the user should not have to know that the list of objects (in this case, files) had to be specified as an array.

This was fairly easy in Powershell, making use of the $MyInvocation predefined variable: simply define those parameters you want as named parameters, and catch the 'rest' into an array.

function My-Command {
   param ($namedParameter)
   #Catch the items to be labelled from the unbound arguments
   if ($MyInvocation.UnboundArguments.Count -gt 0) {
      [string[]]$items = $MyInvocation.UnboundArguments
   }
   else
   {
      [string[]]$items = (($pwd).toString())
   }
   ...
}

In this case, if no unbound arguments are given, the $items array at least gets one value consisting of the current directory path, but you could easily replace that with for example a throw()-statement instead.

Enhancing ClearCase with Powershell

Using Powershell to access IBM Rational ClearCase data gives (as with just about everything that you use with PS) enormous enhancements. Here's just an example: since you can interface with COM objects from Powershell and ClearCase has a (rather useful) COM application, you gain a much more flexible and powerful access to ClearCase data and commands than with the usual cleartool commands. And it's faster.

Here's a replacement for the cleartool command, using the COM interface instead. I haven't done any real tests, but a simple "eye-measure" test with an advanced search told me it's about three times faster:

function Invoke-Cleartool
{
    $CCCleartool = New-Object -COM ClearCase.Cleartool    #Get all arguments as one command-line
    $command = ""
    $MyInvocation.UnboundArguments | Foreach-Object {
        if ($_ -eq ".") {$command += " $pwd"}
        else { $command += " $_" }
    }
   
    # Create a "real" array to store the result in
    $result = New-Object System.Collections.ArrayList
   
    #Parse the output and store as items in the array
    ($CCCleartool.CmdExec($command)).replace($pwd,".").split("`n") | Foreach-Object { $result.Add($_) >$null}
   
    #Remove the last item (which will always be an empty string)
    $result.RemoveAt($result.Count - 1)
   
    $result
}

Update: Just noted that there's a bug here when cleartool requires quotations around the arguments (for example when calling find -name). Powershell is good at stripping any unneeded quotations from arguments, but in this case that doesn't suit me very well...

Testing for parameters in Powershell functions

I recently needed to test for parameter existence in a Powershell function. The idea was that if parameters were provided, the command should run directly in shell; else a GUI should be presented where the user can enter any of the parameters in a user-friendly way.

I started out with the "standard" test, using $args:
    if ($args.Count -gt 0)
    {
        ...
    }
    else
    {
        Show-GUI
    }




Unfortunately, $Args turned out to always be zero (at least when using from within a module as in this case). I found help at this post at EggheadCafé though, switching $args for $MyInvocation.BoundParameters:

    if ($MyInvocation.BoundParameters.Count -gt 0)
    {
        ...
    }
    else
    {
        Show-GUI
    }

A small word of warning though: $MyInvocation always refers to the "current context", which means that if you debug and try getting data, it will show the data for the current shell, not the function being debugged.

Powershell: Error-handling in modules

I've recently started using modules for my Powershell commands for managing my custom commands, since I this neing a great way to encapsulate and distribute such commands. Besides, I can choose the purpose of my session by selecting what modules to import.

However, there seems to be a BIG caveat with modules: the global variables seems not to be available. In particular, the automatic $error variable cannot be used inside modules. You can try this example:

New-Module -ScriptBlock {
    function div ($number) {
       try { 1/$number }
       catch { Write-Host $Error[0] }
    }
} -name DivMod | Import-module

PS > div 1

1

PS > div 0

Note the empty line: the $Error variable is empty! I need to dig further into this, but since not even Googling seems to have any answers(!), I think I've come up with a workaround, at least: use Invoke-Expression with the -ErrorVariable parameter for 'sensitive' commands (e.g. those that you need good error-handling for).

New-Module -ScriptBlock {
   function div ($number) {
      try { Invoke-Expression '1/$number' -ErrorVariable e }
      catch { Write-Error $e }
   }
} -name DivMod | Import-module

PS > div 1

1

PS > div 0
System.Management.Automation.CmdletInvocationException: Attempted to divide by zero...

Using this, you can use try..catch in modules almost as you would in scripts.

If anyone out there has a better solution (I'd really like to be able to use the $error variable...) don't hesitate to comment/point me in the right direction!