Quality Indicators Report  
Author Message
Badajoz95





PostPosted: Team Foundation Server - Reporting & Warehouse, Quality Indicators Report Top

I'm feeling mad, no where can I find an help / doco / explanations for the TFS Reports. Specifically, I have questions about the "Quality Indicators" report.

It looks ****y lovely & useful; it is working superb for "Code Churn", "Code Coverage", "Tests Past", "Tests Failed" (Nighly Build).

... but...

  1. "Active Bugs" is always zero. Yet if I look at the default Active Bugs WorkItem Query, we definately have some.
  2. "Tests Inconclusive" is always zero. Yet I have no idea how to affect this.

If anyone can give me some pointers to the doco or perhaps bash me into the right direction I would be really most grateful.



Visual Studio Team System14  
 
 
Tom Patton





PostPosted: Team Foundation Server - Reporting & Warehouse, Quality Indicators Report Top

The documentation for the reports is included in the process guidance. Please try going to the Team Project and choosing "Show Project Portal" from the context menu. If you click on "Index" and then "Reports" there is a description of how to interpret the Quality Indicators report.

However, that documentation is a bit out of synch with the actual report, and does not have the level of detail to answer your questions. Future updates to the MSF Process templates will include updated descriptions of the report, and the points you bring up here are good input into that process.

As for the issues you mention, The "Active Bugs" being zero is pretty suspicous. The Active Bugs uses the "Build Start Time" of the build that is shown on the X axis and finds out how many bugs were active at that point in time. Could you please look at the build start time attribute in the data warehouse and see what they are This will help us diagnose the issue. Specifically,

- Open Excel, or another cube browser, and connect to the cube

- Drag the "Build.Build" and Build.Build Start Time onto the rows of the pivot table

- Drag "Build Count" to the data section

Then, do an active bug trend using a different pivot table:

- In the excel pivot, put date.year month day on the filter, and select a date range that overlaps with the builds you saw

- Put "Cumulative Count" measure in the data section of the pivot

- Put "Date.Date" on the row axis.

- Put "State in the filter" and select "Active"

This will show how many active bugs there wer across that timeline. Let us know what you find out and we can try and get to the bottom of the issue.

As for "Tests Inconclusive", if a tetst is published against a build that is selected in the QI report, and its outcome was not "Passed" and was not "Failed", it is counted as inconclusive. You can use a technique in excel similar to that described above to look at the outcomes of tests against builds.

There is further information on using excel to view data in the cube here:

http://msdn2.microsoft.com/en-us/library/ms244713.aspx

Hope that helps.

-tom


 
 
Badajoz95





PostPosted: Team Foundation Server - Reporting & Warehouse, Quality Indicators Report Top

Most useful, thanks very much for that (again!) I had not used the cube and so this gave me a good familiarisation with it. I ran through the pivots that you described and all the data did seem OK with the cumulative seeming OK...

... instead, it turns out that the problem is the fact that I had not updated the reports since Beta 3 - see the answer you gave me on http://forums.microsoft.com/MSDN/ShowPost.aspx PostID=507829&SiteID=1&mode=1. The only outward sign that the Quality Indicators report was broken was that the Active Bugs was zero. Uploading the new report definition fixed. it.

With specific regard to the "Tests Inconclusive" column, I think you pointed me in the right direction, basically using (for Unit Tests) the Assert.Inconclusive method (http://msdn.microsoft.com/library/default.asp url=/library/en-us/dnvs05/html/utfwvs05tmsys.asp)... is that right (and one presumes there is the same feature for Manual Tests, etc.)

Thanks very much,

Paul.