http://www.testingeducation.org/a/TheOngoingRevolution.pdf
A project is complete enough to release when it provides enough of the features, delivers enough of the benefits (the features have to work well enough together for the user to actually succeed in using the product to get real work done), is documented well enough for the user, validated well enough for regulators or other stakeholders (e.g. litigators of the future) who have a legitimate interest in the validation, has been sufficiently instrumented, documented, and troubleshot to be ready for field or phone support, is sufficiently ready for maintenance, localization or porting to the next environment (readiness might include having maintainability features in the code as well as effective version control and other maintainability-enhancing development processes in place), is acceptable to the key stakeholders, and has few enough bugs. This list is not exhaustive, but it illustrates the multidimensionality of the release decision. Many companies appraise status and make release decisions in the context of project team meetings, with representatives of all of the different workgroups involved in the project. They wouldn't need these team meetings if the status and release information were one-dimensional (bug counts). We describe these dimensions in the language of "good
enough" because projects differ in their fluidity. One organization might insist on coding everything agreed to in a requirements specification but do little or nothing to enable later modification. Another might emphasize high reliability and be willing to release a product with fewer than the desired number of features so long as
the ones that are included all work well. Even if we restrict our focus to bugs, the critical question is not how many bugs are in the product, nor how many bugs can be found in testing but is instead how reliable the product will be in the field (Musa, 1999), for example how many bugs will be encountered in the field, how often, by how many people, and how costly they will be. (Kaner & Bond, 2004)
Bugs in your shopping cart: A Taxonomy
Introduction
Imagine being asked to test a web site's shopping cart. If you hadn't tested one before, where would you start? What experience would you draw on? Where would you look for more information? Even very experienced testers have blind spots when they try to generate test ideas for an application that they have not tested. This paper presents a simple outline that will help you generate test ideas and limit your blind spots. The outline is the result of a year's research on classifying e-commerce-related failures and risks. The result has 60 top-level categories and examples of errors (potential issues to test for) under most categories. In many cases, we also link to examples of e-commerce defects that have been publicized in the press.
Goals of this outline
Using the list, you could pick a category of interest (such as accessibility or software upgrade), read descriptions of several types of problems that fit within that category, and so identify a few issues that would be appropriate to test for in your application. Based on feedback to the authors of Testing Computer Software, we believe that many testers will be able to use this list to identify potential problems that they would otherwise have missed.
We intend the outline to serve similar functions to Kaner / Falk / Nguyen's bug appendix in Testing Computer Software (TCS):
! Help testers generate ideas;
! Help test plan inspectors check a set of tests for thoroughness and coverage;
! Help testers and other stakeholders identify risks during discussions of
prioritizing the testing effort.
Kaner expects this to become the successor to the TCS list, and we have structured the outline (e.g. more top-level categories and fewer levels) based on feedback from many TCS readers
The nature of e-commerce testing
“E-commerce testing involves testing high value, high risk, and high performance
business critical systems”[1]. The cost of a failure in an e-commerce system is hence
formidable and generally the testing process involves considerable effort in designing and integrating an effective risk management process into the testing to avoid or minimize the cost of failures.
Imagine being asked to test a web site's shopping cart. If you hadn't tested one before, where would you start? What experience would you draw on? Where would you look for more information? Even very experienced testers have blind spots when they try to generate test ideas for an application that they have not tested. This paper presents a simple outline that will help you generate test ideas and limit your blind spots. The outline is the result of a year's research on classifying e-commerce-related failures and risks. The result has 60 top-level categories and examples of errors (potential issues to test for) under most categories. In many cases, we also link to examples of e-commerce defects that have been publicized in the press.
Goals of this outline
Using the list, you could pick a category of interest (such as accessibility or software upgrade), read descriptions of several types of problems that fit within that category, and so identify a few issues that would be appropriate to test for in your application. Based on feedback to the authors of Testing Computer Software, we believe that many testers will be able to use this list to identify potential problems that they would otherwise have missed.
We intend the outline to serve similar functions to Kaner / Falk / Nguyen's bug appendix in Testing Computer Software (TCS):
! Help testers generate ideas;
! Help test plan inspectors check a set of tests for thoroughness and coverage;
! Help testers and other stakeholders identify risks during discussions of
prioritizing the testing effort.
Kaner expects this to become the successor to the TCS list, and we have structured the outline (e.g. more top-level categories and fewer levels) based on feedback from many TCS readers
The nature of e-commerce testing
“E-commerce testing involves testing high value, high risk, and high performance
business critical systems”[1]. The cost of a failure in an e-commerce system is hence
formidable and generally the testing process involves considerable effort in designing and integrating an effective risk management process into the testing to avoid or minimize the cost of failures.
Subscribe to:
Posts (Atom)