Querying TFS Test Managment Service from C#

Query Test Plans from a C# app

Our shop uses Visual Studio 2010 and Microsoft Test Manager (MTM) for entering and running manual test cases. They wondered how to export a flat list of test cases to Excel that captured all the test case steps and the result of the last test pass. Interestingly, we couldn't find any way to do it from MTM or Visual Studio.

I did some searching and cobbled together a little Winform app in C# that has worked great for our purposes. I removed error handling for brevity but left in the core stuff.

Set References to the TFS APIs

using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.TestManagement.Client;
using Microsoft.TeamFoundation.WorkItemTracking.Client;
using Microsoft.TeamFoundation.VersionControl.Client;

First you can get a List of all your Team Projects containing Test Plans

(You need to know the URI of your TFS server)
public List<string> GetTeamProjects()
{
   var tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(TestPlanQueryTool.Properties.Settings.Default.TfsUri));
      var service = tfs.GetService<VersionControlServer>();
      return service.GetAllTeamProjects(true).Select(i => i.Name).ToList();
}

I used the List of projects to populate a DropDown ComboBox.
On selecting a team project, I would....

Get a List of Test Plans for the Selected Team Project

public List<string> GetTestPlans(string teamProject)
{
    var tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(
            new Uri(TestPlanQueryTool.Properties.Settings.Default.TfsUri));
    var service = tfs.GetService<ITestManagementService>();
    ITestManagementTeamProject teamProject = service.GetTeamProject(teamProject);
    return _teamProject.TestPlans.Query("SELECT * FROM TestPlan").Select (i => i.Name).ToList();
}

I populated a listview control with the list of Test Plans. Now I needed to query all the test suites contained in the test plan.

Automating a WCF Service Test using VSTS

I needed to veer off from testing our website UI using webtests to testing the service layer of our website. My client company has standardized on Visual Studio 2010/2012 for our test harness.

I had a couple of options: use the unit test framework or the webtest format, a declarative XML style. I’ll outline my thought process here, with credit to other sources including the VSTS Forum where this decision is discussed: Issue with VSTS webtest on WCF Servce.

I already have a bunch of test cases coded in the webtest format that I run using a custom test launcher and preferred to stick with the webtest style if possible.

The developer of the service expressed some concern about whether the webtest would be a good method of testing his new service, as he was most familiar with creating a service proxy and testing the WCF service by exercising properties and methods from that. He wanted me to evaluate calling the web methods from a VSTS webtest vs. a unit test by watching the http traffic using the Fiddler tool to make sure the service would be called in the same manner.

Fiddler Web Debugging Tool

Sure enough, my test harness proved that it didn’t matter whether I called the web service from a web test or a unit test, the http traffic recorded by Fiddler was identical. So for me, it was a no-brainer, I’d create the tests using the webtest format. If I had to summarize the pros and cons, it would go something like this:

Automating a WCF Service test

Unit Test Format
Pros

• Easier to call the service after creating a web service proxy class
• Neater, cleaner code when setting properties and calling methods of the proxy object

Cons

• The service endpoint is written into a config file and is more work to parameterize.
• The test project is compiled to a DLL unlike the declarative XML format of the webtest.

Webtest Format (pretty much the opposite of the above)
Pros

• The test format is a readable declarative XML file format.
• You can record a call to the WCF service using the wcftestclient.exe tool from VSTS, record the call in the Fiddler tool, and save the session in .webtest format very quickly.
• It is very easy to parameterize any part of the service request URI, string body, and header using the concept of VSTS context name-value parameters.

Cons

• Creating validation code may require you create a custom validation rule class to perform detailed validation of the service response packet.

It was the final point in favor of the webtest format that won me over -- we can easily point our service to a dev, test, stage, or prod environment, or change the input parameters on the method by changing some context paramter values from our custom test framework.

Running VSTS Webtests using MSTest.exe's /testcontainer option

I created a little window app that allows selecting one or more of our webtests and running the MSTest executable using .Net's ProcessStartInfo class in a separate thread created using the BackgroundWorker class.  I like using the BackgroundWorker class as it keeps multi-threading simple.

It took some trial and error plus some careful reading of MSDN's MSTest.exe Command-Line Options to figure out that I couldn't use the /testmetadata option. The reason was our team had created mulitple test lists with the same webtest appearing in several of the lists.  When I used this syntax:

     MSTest.exe /testmetada:mytests.vsmdi /test:webtest1.webtest

my tests got executed one time per test list!

That took some head scratching to figure out the workaround:  use the /testcontainer: option.  This syntax worked:

        MSTest.exe /testsettings:mysettings.testsettings /testcontainer:webtest1 /testcontainer:webtest2 /testcontainer:webtest3

I've used this syntax successfully even when starting 30 or more webtests in one command.