Automating a WCF Service Test using VSTS

I needed to veer off from testing our website UI using webtests to testing the service layer of our website. My client company has standardized on Visual Studio 2010/2012 for our test harness.

I had a couple of options: use the unit test framework or the webtest format, a declarative XML style. I’ll outline my thought process here, with credit to other sources including the VSTS Forum where this decision is discussed: Issue with VSTS webtest on WCF Servce.

I already have a bunch of test cases coded in the webtest format that I run using a custom test launcher and preferred to stick with the webtest style if possible.

The developer of the service expressed some concern about whether the webtest would be a good method of testing his new service, as he was most familiar with creating a service proxy and testing the WCF service by exercising properties and methods from that. He wanted me to evaluate calling the web methods from a VSTS webtest vs. a unit test by watching the http traffic using the Fiddler tool to make sure the service would be called in the same manner.

Fiddler Web Debugging Tool

Sure enough, my test harness proved that it didn’t matter whether I called the web service from a web test or a unit test, the http traffic recorded by Fiddler was identical. So for me, it was a no-brainer, I’d create the tests using the webtest format. If I had to summarize the pros and cons, it would go something like this:

Automating a WCF Service test

Unit Test Format
Pros

• Easier to call the service after creating a web service proxy class
• Neater, cleaner code when setting properties and calling methods of the proxy object

Cons

• The service endpoint is written into a config file and is more work to parameterize.
• The test project is compiled to a DLL unlike the declarative XML format of the webtest.

Webtest Format (pretty much the opposite of the above)
Pros

• The test format is a readable declarative XML file format.
• You can record a call to the WCF service using the wcftestclient.exe tool from VSTS, record the call in the Fiddler tool, and save the session in .webtest format very quickly.
• It is very easy to parameterize any part of the service request URI, string body, and header using the concept of VSTS context name-value parameters.

Cons

• Creating validation code may require you create a custom validation rule class to perform detailed validation of the service response packet.

It was the final point in favor of the webtest format that won me over -- we can easily point our service to a dev, test, stage, or prod environment, or change the input parameters on the method by changing some context paramter values from our custom test framework.

Using Fiddler to create VSTS Web Performance Tests

My shop uses Visual Studio 2010 Ultimate Edition for our test framework. Frankly, I spent a few months automating test cases using the coded-UI style of browser automation before I decided that the “Web Performance Test” was better suited to our testing needs.  (I’ll refer to the web performance test as webtest in the rest of this post.)

Our website makes frequent AJAX calls for server-side user entry validation, and frankly, the recorded coded-UI tests would hang in a non-deterministic way. Even though I inserted custom retry logic when entering user values, the framework would hang and cause our team no end of frustration.

VSTS Web Performance Test (*.webtest)

On a happier note, I’ve had a lot of success using the webtest, a type of test built-in to VSTS which works at the http layer rather than through interaction with the web browser. In our case we use the webtest as a functional test to test the server-side code, though we also use sets of them in load tests as well.

That’s a little background on why I use webtests, but I want to comment more about using Fiddler. If you’re not familiar with Fiddler, it’s a free web debugging tool that logs all http and https traffic between your computer and the internet.

Export Fiddler Sessions when Visual Studio Webtest Recorder Can't

We soon discovered to our dismay that when recording our shopping cart wizard using a web performance test inside VSTS, that not all http requests (read AJAX) were captured. But happily Fiddler DOES capture all this traffic, and the bonus is that Fiddler allows you to export recorded sessions in VSTS web performance test (.webtest) format!

 

Screens in Fiddler after Choosing to Export Sessions

 

 

Sometimes we’ll create a hybrid recording, creating a webtest in VSTS. If some of the requests were not captured, we’ll capture the workflow in Fiddler and then export it to webtest.  After adding the Fiddler-exported webtest to our project, we cut/paste the requests we want into our main webtest.

We’ve had much success using Fiddler to help us build out a complete functional test suite of some fairly complex workflows and would have been stymied in doing this without this great tool.