Technical Differences btw. VS Load Test and Load Test Agents and Controllers

Hi,

I was asked today if there are any significant differences btw. the load test functionality of the Tester's Role / Team Suite and the agents Does the Studio have a load test controller and agent included for doing load tests

Another question that came up was, if it is possible to use a VS Test Client as an additional agent in a rig

Last but not least I was not sure where exactly (on which server) the testee is run when using a rig for testing.

Thanks in advance, for any answers.

Tom

Visual Studio Team System16

Hi Tom,

I'll try to answer you questions.
Ok first of all the Team Test Load Agent SKU is a separate product, that you can use to execute your tests on different machines. It basically consists of an agent and a controller. The controller can have one or many agents; you will be sending you execution requests to the controller, and the controller will distribute the tests to the different agents available (and the tests are run in the machines where the agents exist). You don't need to install VSTS client in machines where you only want to have an agent.

With a load test, which is included in Visual Studio Team Suite or Visual Studio Team Test, you can simulate as many users as your machine can allow; but if you use the Team Test Load Agent SKU with your load tests, then you can simulate many more users by using as many machines as you need.

You still need VSTT or VSTS to construct and execute your tests, but the Team Test Load Agent SKU will give you that extra power of being able to run your tests remotely and adding more load to your load tests.

Hope this answers your questions
For more info please take a look here:
http://msdn2.microsoft.com/en-us/library/ms243155.aspx

Thanks,
David Gorena Elizondo
[MSFT] VSTS

To answer you second question, you cannot run a secondary copy of VSTS as an agent. They can run independantly, but you won't be able to gather any combined data. As for your third question, the controller for the rig should Identify which machines you are running on.

Last, note that the Agent is not part of VSTS, http://msdn.microsoft.com/vstudio/teamsystem/products/load/default.aspx. It's not even available via the MSDN. One will need to go to their volume liscense Vendor or partner to purshase the agent.

Hi David, Hi Johnny,

there are still some things I would like to know:

What stands SKU for I only know this abbreviation for stock keeping units.
Is the same code / technology used to run load tests on the rigs as it is used in the VSTT/VSTS My client wants to know this.
I'm not yet sure where the solution I want to test is running within a rig Let's say I want to do a load test (stress test) for a website (data tier). Where is the web site (data tier) deployed for the test Within the rig, e.g. on the controller server or out side the rig I really didn't get this point so far.

I read the installation guide, MSDN articles you referenced and the MSDN forums available to this topic couldn't find the answers so far. Pls point me to the sources I know where exactly I have to look for the answers.

Lots of thanks to David and Johnny for your answers!

Tom

Hi Tom,

1. Yep that is right. We have different SKUs around Team System.

2. The same code and technology is used to execute tests in both places.

3. When you are running a stress test you would have your target application installed on it own hardware and a seperate machine for each agent you want to use. If you have a fairly powerful desktop you could run the controller and VS on the same machine.

For #2, there are common components but also some significant differences.

When a test is run locally from Visual Studio, we launch a process VSTestHost.exe that executes the test on the local machine. VSTestHost hosts controller object, agent service object, and agent object in it's process.

When running on a rig, the controller runs as a service and hosts the controller object. The agent service runs as a service. When it gets a test or set of tests to run it will start QTAgent.exe process that actually runs the tests.

The code in VS is nearly identical for communicating with a controller or vstesthost.

Some key differences:

Agent service has restart capability that vstesthost does not. If the load generator process gets a fatal error part way through the test, the agent service will restart the test process to pick up where it left off. Similarly, if the agent is running 10 unit tests and gets a fatal error after the 5th test, it will restart the agent process and continue to run the last four tests.
Agent service supports IP Switching. This is important for a load test against a web farm so you can get appropriate load balancing on the web farm.
Controller supports agent selection. So you control which agents will run a test.
Distributed processing. You can run VS, controller and agent on the same or different machines. The controller, for example, collects all stats during the test. If the controller is running on the same machine as a load agent (e.g. in the local case), it may fall behind on data collection because it gets starved out by the load generator. And of course, agents enable you to scale-out your load farm so you can generate more load and aren't capped by what a single machine can do.

Please see our blog posts here for articles on the agent and controller. There is also some good info in the agent/controller install guide which I've posted on my blog.

Ed.

Chris, in response to your questions:

1. When I say SKU, I am refering to the mulitple flavors or editions of Visual Studio 2005. We consider each one a different SKU. For example with have Visual Studio 2005, professional or architectual editions. VSTS is built on top of the profession Visual Studio 2005 edition. In the case of Visual Studio 2005 Team System, we have five SKU's:

i. Visual Studio 2005 Team System for Software Testers

ii. Visual Studio 2005 Team System for Software Developers

iii. Visual Studio 2005 Team System for Software Architects

iv. Visual Studio 2005 Team System Suite (Comprised of the previous 3 SKUs.)

v. Visual Studio 2005 Load Test Agent (Sold on a per processor basis and Requires sku i.)

2. I believe Edglass had an excellent answer to #2.

3. The Load Test Agent is a seperate purchase, it does not come with VSTS, You would only need it if you consider VSTS cannot accomodate the amount of load you wish to generate. You should be able to test your web or applicaiton tier using VSTS web test and or unit test.

I do not intend to hijack this thread, I have posted a question in the Load Test forum but havent got an answer. This discussion seems relevant to what I am looking for, hence using this thread.

Regarding the differences between VSTestHost.exe and QTAgent.exe, when a load test is executed locally, the vusers are run as threads under VSTestHost.exe (I believe). Is that same case when the test is run on a agent Is it possible to run the load test with vusers running as separate processes rather than threads I have run into a problem where our app code is not thread safe and I am looking for alternative options.

Thanks.