A quality criterion is some requirement that defines what the product should be. By thinking about different kinds of criteria, you will be better able to plan tests that discover important problems fast. Each of the items on this list can be thought of as a potential risk area. For each item below, determine if it is important to your project, then think how you would recognize if the product worked well or poorly in that regard.
Capability. Can it perform the required functions?
- Sufficiency: the product possesses all the capabilities necessary to serve its purpose.
- Correctness: it is possible for the product to function as intended and produce acceptable output.
Reliability. Will it work well and resist failure in all required situations?
- Robustness: the product continues to function over time without degradation, under reasonable conditions.
- Error handling: the product resists failure in the case of bad data, is graceful when it fails, and recovers readily.
- Data Integrity: the data in the system is protected from loss or corruption.
- Safety: the product will not fail in such a way as to harm life or property.
Usability. How easy is it for a real user to use the product?
- Learnability: the operation of the product can be rapidly mastered by the intended user.
- Operability: the product can be operated with minimum effort and fuss.
- Accessibility: the product meets relevant accessibility standards and works with O/S accessibility features.
Charisma. How appealing is the product?
- Aesthetics: the product appeals to the senses.
- Uniqueness: the product is new or special in some way.
- Entrancement: users get hooked, have fun, are fully engaged when using the product.
- Image: the product projects the desired impression of quality.
Security. How well is the product protected against unauthorized use or intrusion?
- Authentication: the ways in which the system verifies that a user is who he says he is.
- Authorization: the rights that are granted to authenticated users at varying privilege levels.
- Privacy: the ways in which customer or employee data is protected from unauthorized people.
- Security holes: the ways in which the system cannot enforce security (e.g. social engineering vulnerabilities)
Scalability. How well does the deployment of the product scale up or down?
- Elasticity: the ability to dynamically adjust resource allocation based on current demand using cloud services.
- Service Decoupling: microservices allow individual services to be scaled independently based on their specific needs.
Compatibility. How well does it work with external components & configurations?
- Application Compatibility: the product works in conjunction with other software products.
- Operating System Compatibility: the product works with a particular operating system.
- Hardware Compatibility: the product works with particular hardware components and configurations.
- Backward Compatibility: the products works with earlier versions of itself.
- Product Footprint: the product doesn’t unnecessarily hog memory, storage, or other system resources.
Performance. How speedy and responsive is it?
- Latency: the product that has critical actions like financial trading, video conferencing e.g should have low latency.
- Load Time: how quickly does the product launch and become usable?
- Responsiveness: how quickly does the product react to user interactions (clicks, taps, etc.)?
- Operation Speeds: how fast are common operations performed by the product (e.g., searching, data processing, form submissions)?
Installability. How easily can it be installed onto its target platform(s)?
- System requirements: Does the product recognize if some necessary component is missing or insufficient?
- Configuration: What parts of the system are affected by installation? Where are files and resources stored?
- Uninstallation: When the product is uninstalled, is it removed cleanly?
- Upgrades/patches: Can new modules or versions be added easily? Do they respect the existing configuration?
- Administration: Is installation a process that is handled by special personnel, or on a special schedule?
Development. How well can we create, test, and modify it?
- Supportability: How economical will it be to provide support to users of the product?
- Testability: How effectively can the product be tested?
- Maintainability: How economical is it to build, fix or enhance the product?
- Portability: How economical will it be to port or reuse the technology elsewhere?
- Localizability: How economical will it be to adapt the product for other places?