Microsoft Test tools - codedui, MTM, Test professional interview questions

Hello everyone - here are few frequent questions and answers that we have been getting asked below.

Automation

Q: What is the difference between what Microsoft Test Manager offers and Record and Playback?

A: Test Manager's record/playback is intelligent and robust. We try to figure user intention based on actual captured input to reduce dependency on specific location of controls, etc... Focus is on helping manual tester accelerate through some already recorded steps.

Q. Can testers view and modify code created by the recorder?

 A: While the action log created by the recorder cannot be modified, you can create a Coded UI test (either C# or VB.Net code) which can be viewed or modified in the Visual Studio IDE. We made the rerecording of the actions easy enough that we don’t expect you to edit the recording.

Q. How does the “assisted manual testing” react to UI whose properties are dynamically generated?

 A: This should work fine as long as the generated properties are deterministic. Here is a blog that enables you to control search properties.

  

Q. How easy would it be for a developer to re-use the rich bug to test the fix?

 A: This is fairly straight forward. If the test has been automated, the developer can open the associated test case, choose to run the test case and playback the recorded actions against the fixed application.

 Q: Would you recommend leveraging the UI automation created in Test Manager in a performance test?

 A: While UI automation can be used in a performance test, that would not be our recommendation since you won’t be able to drive significant load using a UI tool that will bottleneck at the UI layer. Instead you might be better off using Web Performance Tests to drive the server load without the UI dependency.

 Q: When you convert test actions to code, what languages do you support?

 A: We generate either C# or VB.Net code from the recorded actions.

 Q: Can you parameterize your automated tests to run for multiple sets of data?

 A: Yes, you can parameterize your tests to be run for multiple sets of data.  If you like, the data stored on a parameterized Manual Test can  be bound to and leveraged by it's automated counterpart.

  

Q: Can you drive your automated tests from data in a database?

A: You can bind your automated tests to a database to run them for multiple iterations of data.  Please refer to this article for more details.

 Code Coverage

Q: What level of support do you provide for code coverage?

 A: Users of Visual Studio are able to see the code covered by their automated tests in Visual Studio. Unfortunately Code Coverage information is not currently collected when Manually Testing.

 Competition

 Q:  How does this tool compare to Selenium and other open-source testing tools?

 A:  Our offering engages testers in the overall ALM effort through integration with lab management and TFS for work-item tracking, source control, build management, and reporting.  Many of the open-source testing tools are stand alone and tend to focus on a particular platform or type of testing (e.g., Selenium is specifically for browser-based testing of web applications). 

Q: What are the key benefit this tool provides over HP's QC/QTP?

A: We feel that Visual Studio Test Professional 2010 is optimized for the way that testing is performed today where more than 70% of testing is done manually through ad-hoc tools such as Microsoft Excel and Word. It is an integrated testing tool set that delivers a complete plan-test-track workflow while allowing testers to collaborate with developers effectively and test efficiently.

Integration

 Q: Does this tool integrate with Office Project?

 A:  Yes, via TFS.  Office Project can be used to manage data stored in TFS (such as dependent tasks) to facilitate planning projects, scheduling tasks, assigning resources, and tracking changes.  By using Office Project, you gain access to features such as a project calendar, Gantt charts, and resource views.

    

Q: Does this tool integrate with Office SharePoint?

 A: Yes, via TFS, which enables creation of a team project portal associated with a SharePoint site.  The project portal includes a customizable set of dashboards (SharePoint Web parts) that enable team members to monitor project data in the form of PivotChart reports and lists or counts of work items.  The testing tools are also integrated with SharePoint in the sense that they can be used to test SharePoint sites.

  

Q:  Does this tool have the capacity to integrate with other testing systems such as Rally, QTP, or QC?

 A:  Not directly; but if you can export test cases, bugs, and other artifacts from the other system into Office Excel, you can then import them to TFS as work items.  You can use the Test Case Migrator Tool (available at http://tcmimport.codeplex.com/) to import test case steps and parameterized data along with your test cases.

  

Intellitrace

Q: Does seeing the line of code that failed require a special type of build or deployment?

A: No, no special build or deployment is required.  A tester must however configure to capture intellitrace data on the machine which is running the code while they are testing, this is done via the Test Settings.

 Q: Will the developer be able to see the line of code for web-based development where the code sits on the server and the test is executed via client browser?

 A: Yes the developer will be able to see the line of code that failed, so long as the tester configures the webserver to capture intellitrace logs.

Internal Use of Test Professional

Q: Were these test tools used during the development of Visual Studio 2010 or any other MSFT products?

A: Absolutely.  The entire Visual Studio team has used this toolset to build Visual Studio 2010.  There are many other teams at Microsoft which are using them as well.

    

Platform Support

Q: Can you provide some information about the platform support for Microsoft Test Manager? What class of applications can I test with the tool?

A: You can use the tool to manually test almost any platform, though your experience around the creation of rich bugs and support for automation will be better if you are targeting IE7/IE8, Windows Forms, WPF, Win32, MFC or SharePoint. This blog post has an updated enumeration of the automation platform support and will be updated over time as we add additional support.

  

Requirements

 Q: Does Test Professional have requirement tracing built in? 

 A: Requirements are treated as first class citizens in Test Professional.  One can see full traceability of Requirement -> Test -> Code. One can also base a test suite off of a requirement.  This blog post speaks very extensively on the topic.

Q: Can a user produce a Requirement traceability report using Test Professional?

 A: Test Professional provides full requirement traceability.  Out of the box, users can see a requirements traceability report on their team dashboard.

  

Test Manager

 Q: Can I email a defect in Visual Studio 2010?

 A: Yes, you can email a link to a defect from Visual Studio. 

  

Q: How can I link Bugs to Test Cases? How can I link bugs to Product Backlog items? 

A: Defects, test cases, backlog items are all work items on Team Foundation Server. Test Professional automatically links the work items together when appropriate (for example, when a test creates a bug, the test and the bug are linked together).  Users can also manually link work items together in Test Manager or inside Visual Studio.

   

Q: Can a tester send a video of their testing to a developer? 

A: Yes a tester can send a video of their testing to a developer.  When a tester creates a bug, with background video recording on, the video of the testing which caused the bug is referenced on the bug form which the testers assigns to the developer.  If you wish, you can save the video which is stored on the test result to a network share as well.  Demo 2 of the webinar demonstrated the rich bug which is accessible to the developer.

  

Q: Is there any way to track testing across a variety of operating systems?

A: By setting up a configuration for each operating system, you can break down your testing for each operating system.  To read more about configurations, please refer to this blog post.

 Q: What does Analyze Test Runs do?

 A: Analyze Test Runs allows you to review the results of your automated and manual test runs.  For more details, please refer to this blog post.

  

Q: Can we access Test Manager via the web?

A: Test Manager 2010 installs as a client tool. When it launches, you are presented with the various projects on TFS which you can connect to.

  

Q: We have a large number of testers, can a test manager assign test cases to individuals

A: Test managers can assign test cases to individuals. Testers can also pull from a pool of test cases. Test case assignment is done in the Test Plan Contents, it can be done by clicking on the assign button.

  

Q: Is Test Manager available as a plugin for Visual Studio 2008?

A: No, Test Manager is only available via Visual Studio Test Professional 2010 or Visual Studio Ultimate 2010

  

Q: Does Test Professional support the branching of Test Cases?

A: Test Manager does not support branching of work items.  However you can import test cases from one plan to another so you would be able to use the same test case across code branches.

  

Q: Is it possible to work with components of tests being shared between test cases?

A: Yes, shared steps can be created once and shared between test cases.  They are really powerful. Please refer to this blog post to learn more about them.

    

Q: Are test steps and expected results which span multiple lines supported?

A: Yes, You can enter multiple lines for the test step and the expected result. The text wraps in both the test runner and while authoring the steps.

  

Q: Is Test Professional compatible with Agile Methods?

A: Yes it is compatible with Agile Methods.  We would suggest that you use 1 test plan for a sprint of work and 1 requirement based suite for each user story. This blog post describes our suggested approach.

Q: When I'm editing a test case in MTM, the steps / parameters section of my work item is changing height, or is very small. This is driving me crazy!

A: This is a known bug, and we're looking at addressing it in a future release. For the moment, there is a work around: Adjust the height of the window. This should make the issue go away.

Q: How can I find all the test cases that I have action recordings for, so I can create recordings for them?

A: Today, there is not an easy way to do this. The recording is stored on the Test Result, and it is not possibly to query between Work Items, and test results. It's important to note that with Fast Forward for Navigation, the expectation is that people will record, and update, their action recordings as they are performing their normal manual testing.

Q: How do I view the results of more than one query?

A: Just click the 'queries' link at the top of the application again. This will open a new Queries view, and allow you to have two queries open. You can find the other Queries you have open in the 'Open Items' drop down.

 Test Impact

 Q: How does Recommended Tests work?

 A: When the Test Impact Data Collector is turned on, Test Runner records the code paths taken by a Manual or Automated Test Case when it is executed.  Recommended Tests recommends test for execution when a developer makes a check in that touches the code covered by the test case. 

  

Q: Do applications need to be Debug builds to track the lines of code hit by the test?

A: No applications do not need to be debug builds to capture Test Impact Data.  The Test Impact Data Collector must just be turned on while testing.

  

Q: What is the difference between Test Impact Analysis and Code Coverage?

A: Code Coverage is a report which can be generated which speaks to the lines of your code under test which have been covered by your automated testing. Test Impact analysis collects the lines of code which your Tests cover to report which Test Cases are suggested for you to re-run based upon code churn.