Thursday, October 9, 2008

What's the Best Software Testing/QA Tool?

Frequently people will post a question like "What's the best tool?". It drives me nuts sometimes! Software testing and quality assurance are definitely challenging professions, but it's not fair to depend on others to solve your challenges for you! I've answered questions like this in the agile-testing@yahoo.com group, and on the MSDN forum, and I thought it was time I brought my answers together into one blog posting. That way, I can refer people back to the posting.

MSDN Answer

Lifted from my post on MSDN's software testing forum:

An American car manufacturer once tried to make a car for all things - it was a 5-6 passenger car in three different models, each with four wheel drive AND good fuel economy. It ended up mediocre at all things. You need to beware; resist the urge to pick one monolithic tool solution for such a wide variety of testing needs. The vendors (well, NOT Microsoft, of course ) would love to have you believe their tool can do it all. And in some aspects, their software testing tools probably can. However, will that one tool solution be effective and efficient in everything you do? No.

I'm not advocating a pantheon of tools, but you need to think like an engineer more than a manager or a customer. You find the right tool for the job at hand. Over time, you'll settle down to a group of 3-4 tools and you'll stick to them.

I've never used <some tool the asker referenced> - someone else will need to comment on that product's ability to do everything you're looking for. My take? It'd be too good to be true if it actually could. And I tell my three sons all the time 'If it's too good to be true, it's probably not true'.

Be wary.

Here's how you [should] approach this as an engineer:

  • List out the applications you'll be testing 6-12 months from now

  • Think about the test scenarios, especially those you'll want to automate

  • Ask yourself how many releases of that application/project your company will perform--will quality assurance be a repeated process, or are you testing this software just once?

  • Based on that, how much automation is 'worth the investment'? Microsoft Office 2007 *might* still be running automation I write in 1997--probably not, but I know 2003 runs it AND that automation is still being run, every time there's a patch or SP released.

  • Now that you know how much investment to make, look for best of breed solutions for each project. Don't focus on all-in-one solutions; just look for the best tool for the job. Use demos, read reviews, ask questions. Don't ask "Is this the right tool?" but rather "Which tool do you recommend" or "I'm considering this tool for this job - does anyone have experience using this tool to do this?"

  • Once you have a short list of tools, look and see if there is commonality/overlap. You'll see patterns. It's possible Rational or another all-in-one tool will appear in the list; it's equally possible none of those tools will appear.

  • If there's good overlap, ask yourself what you think the pain will be if you force a tool into a job it wasn't designed for. If you can live with the pain, go for it... If not, keep looking or open yourself up to a larger set of tools.

Hope that helps (in spite of not answering your question),

John O.

 

Commentary

OK - I searched through a number of documents and can't find my other posting on this subject, so I'll just write it once more.

  1. To all testers: look before you ask. Really! If you are wondering what tool you can use for a given test type, use Live Search and look for information! Performing a minimal amount of research shows respect to the audience you're turning to for help, and can actually prevent a question now and again. QA is all about searching for product defects; apply that searching capability to answering your questions.
  2. You're more likely to get help on a specific question rather than a broad question. For instance, asking "What's the right tool" won't get you much. However "I've evaluated Selenium, Selenium RC, and Watij--given my situation (describe it) what tool do you recommend?"
  3. Talk about the problem you're trying to solve. So "We realized we'll be running tests over and over and over again, but our UI changes frequently. What strategies can we take...?" is a question that raises the problem and let's people know what help you're looking for.
  4. Asking "what's the best tool?" is like asking "What's the best car?". If you live in Germany and drive the autobahn frequently, the best car for you is far different from the best car for someone who lives in Bombay, battles thick traffic, and drives Indian roads (notoriously rough). Be specific, give details, and look for recommendations. Software testing tools are the same way - a screaming "BMW" test tool will fall apart on a Bombay road. A right-hand drive Jaguar would be totally out of place on an American highway. A performance test tool isn't the right way to automate repeated quality assurance tests. The right tool is dependent on the project, the skill sets on the QA team, the timeline, and several other factors.
  5. Solve your own problems. The Internet is an incredible tool and offers us all a ton of opportunity to not have to reinvent the wheel. But don't ask other people to do your work for you! Ask for advice. If you want someone to solve your problem, ask one of us to consult (for pay) and bring us in. We'll be glad to get the job done for you!
  6. Give back: as you grow and learn, give back... Don't post a question, get your answer, and disappear. Remain an active participant in the community and 'pay it forward'.

Summary? Be specific, research before you ask, solve problems, and give back. That's how to get answers online--in a sustainable fashion.

11 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Hi,
    can any one tell me how can i record tst case for a browse button by using selenium.
    selenium.Type("File1","C:\\delete.gif");
    this code is not working for me.
    giving alert that plz select a file.

    thynx .. in advance

    ReplyDelete
  3. Oes Tsetnoc one of the ways in which we can learn seo besides Mengembalikan Jati Diri Bangsa. By participating in the Oes Tsetnoc or Mengembalikan Jati Diri Bangsa we can improve our seo skills. To find more information about Oest Tsetnoc please visit my Oes Tsetnoc pages. And to find more information about Mengembalikan Jati Diri Bangsa please visit my Mengembalikan Jati Diri Bangsa pages. Thank you So much.

    ReplyDelete
  4. I actually enjoyed reading through this posting.Many thanks.
    software product engineering

    ReplyDelete
  5. Great blog. You have made an amazing blog on testing. I love to read your post. I will frequently visit your blog for references.
    Software quality assessment

    ReplyDelete
  6. My cousin recommended this blog and she was totally right keep up the fantastic work!





    Automation Engineering

    ReplyDelete
    Replies
    1. It's nice to find a site you can trust. I like the privacy of shopping online, but you never know what sites are okay. Visit W21099.com this site was recommended by a friend so I tried it. They provide a 1099 Misc, 1099 Electronic Filing, 1099 Software, W2c Software, W-2 Electronic Filing, W2 Electronic Filing, 1099 Misc Software, W-2c Software, W2 Software, and W-2 Software at affordable price.Their service was great! I'm becoming a frequent shopper!

      Delete
  7. Thanks of your posting, which made me to join Testing tools online training @ www.Monstercourses.com

    ReplyDelete


  8. SWIFT Interview questions on

    http://testwithus.blogspot.in/p/swift.htm

    For selenium solution visit
    http://testwithus.blogspot.in/p/blog-page.html

    For QTP interview questions

    http://testwithus.blogspot.in/p/qtp-questions.html

    www.searchyourpolicy.com

    ReplyDelete
  9. This information which you provided is very much useful for us.It was very interesting and useful for qa online training.We also providing qa online training institute.

    ReplyDelete
  10. Thanks for Information Quality Assurance is the systematic process to check whether the product developed from the company is perfect and meeting the requirements specified for the product. This Quality Assurance was introduced in World War II when the weapons used were inspected and tested after manufacturing, but now the situation have changed and every company is following the advanced technologies for quality and most of the companies will have a separate department for the Quality testing.QA Online Training

    ReplyDelete