iit.gif (699 bytes)

Questionnaire for the Testing Maturity Model

Maturity Goals

CURRENT PROJECT:

For the maturity goal questions, four possible options will be offered. They are described as follows:

1. Check YES when the practice is well established and consistently performed.

2. Check NO when the practice is not well established or is inconsistently performed.

3. Check DOES NOT APPLY when you have the required knowledge about the project or organization and the question asked, but you feel the question does not apply to the project.

4. Check DON'T KNOW when you are uncertain about how to answer the question.

LEVEL 2: PHASE DEFINITION

2.1 DEVELOP TESTING AND DEBUGGING GOALS

The purpose of this goal is to clearly differentiate the processes of testing and debugging. The goals, tasks, activities and tools for each must be identified. Responsibilities for each must be assigned. Plans and Policies must be established by management to accommodate and institutionalize both processes.

Maturity subgoals that support this goal are:

- A committee(s) on testing and debugging must be formed with support and funding.

- The committee(s) must develop and record testing policies and goals.

- The committee(s) must develop and record debugging policies and goals.

- The documented testing and debugging policies and goals must be distributed to all project managers and developers.

- Testing and debugging policies/goals must be reflected in test plans.

QUESTIONS

I. COMMITMENT TO PERFORM

1. Do the software developers follow a written organizational policy for testing?

Yes No Does Not Apply Don't Know

2. Do the developers follow a written organizational policy for debugging?

Yes No Does Not Apply Don't Know

II. ABILITY TO PERFORM

3. Have the policy documents on testing been distributed to project managers and developers?

(if "yes" to question #1) Yes No Does Not Apply Don't Know

4. Have the policy documents on debugging been distributed to project managers and developers?

(if "yes" to question #2) Yes No Does Not Apply Don't Know

5. Is a committee on testing and debugging established?

Yes No Does Not Apply Don't Know

III. ACTIVITIES PERFORMED

6. Is the process of testing defined?

Yes No Does Not Apply Don't Know

7. Is the process of debugging defined?

Yes No Does Not Apply Don't Know

8. Have policies, goals, activities and tools for the testing process been identified?

Yes No Does Not Apply Don't Know

9. Have policies, goals, activities and tools for the debugging process been identified?

Yes No Does Not Apply Don't Know

IV MEASUREMENT AND ANALYSIS

10. Are measurements used to determine achievement of testing goals?

Yes No Does Not Apply Don't Know

11. Are measurements used to determine achievement of debugging goals?

Yes No Does Not Apply Don't Know

V VERIFICATION OR IMPLEMENTATION

12. Have the testing policies and goals been developed with inputs from user/client groups with respect to their needs?

(if "yes" to question #1) Yes No Does Not Apply Don't Know

13. Have the debugging policies and goals been developed with inputs from user/client groups with respect to their needs?

(if "yes" to question #2) Yes No Does Not Apply Don't Know


2.2 INITIATE A TEST PLANNING PROCESS

The purpose of this goal is to establish test planning on an organization-wide basis. Test planning involves stating objectives, analyzing risks, outlining strategies, and developing test design specifications and test cases. The test plan must also address the allocation of resources, and the responsibilities for testing on the unit, integration, system and acceptance levels.

Maturity subgoals that support this goal are:

- An organization-wide test planning committee must be established with funding.

- An organization-wide test planning policy must be established and supported by management.

- A test plan template must be developed, recorded and distributed to project managers and developers.

- Project managers and developers must be trained to use the test plan template and develop the test plan.

- A procedure must be put in place to include user- generated requirements as inputs to the test plan.

- A procedure must be put into place to include risk- related information in the test plan.

- Basic planning tools must be evaluated, recommended and acquired. Usage must be supported by management.

QUESTIONS

1. Is there an organizational policy for test planning?

Yes No Does Not Apply Don't Know

2. Is there adequate support and funding for test planning for all projects?

Yes No Does Not Apply Don't Know

3. Are test objectives used on a baseline for test planning?

Yes No Does Not Apply Don't Know

4. Has a test plan template been developed and distributed to project managers?

Yes No Does Not Apply Don't Know

5. Have project managers been trained in the use of the template?

Yes No Does Not Apply Don't Know

6. Are developers properly trained to develop test specifications, test designs, and test case for the test plan?

Yes No Does Not Apply Don't Know

7. Is an organization wide test organization established?

Yes No Does Not Apply Don't Know

8. Are test-related risks identified?

Yes No Does Not Apply Don't Know

9. Are test-related risks used to set project testing goals?

Yes No Does Not Apply Don't Know

10. Are estimates (time, budget, tools) documented for use in testing planning?

Yes No Does Not Apply Don't Know

11. Is test planning done at the unit level?

Yes No Does Not Apply Don't Know

12. Is test planning done at the integration level?

Yes No Does Not Apply Don't Know

13. Is test planning done at the system level?

Yes No Does Not Apply Don't Know

14. Is test planning done at the acceptance level?

Yes No Does Not Apply Don't Know

15. Is there a procedure in place for soliciting user input for test planning?

Yes No Does Not Apply Don't Know

16. Are measurements used to determine the status of the activities for planning a test planing process?

Yes No Does Not Apply Don't Know

17. Do developers have the opportunity to give inputs (test definitions, test cases) to the test plan?

Yes No Does Not Apply Don't Know

18. Does the project manager review the activities for test planning on periodic and/or event-driven basis?

Yes No Does Not Apply Don't Know


2.3 INSTITUTIONALIZE BASIC TESTING TECHNIQUES AND METHODS

The purpose of this goal is to improve test process capability by applying basic testing techniques and methods. How and when these techniques and methods are to be applied and any basic tool support for them should be clearly specified. Various basic techniques and methods that are often used in the testing process are black-box and white-box testing strategies, use of a requirements validation matrix, the division of execution-based testing into subphases such as unit, integration,system and acceptance testing. Testing tools that support use of these techniques and methods are static and dynamic analyzers, test execution tools, capture/replay tools, comparators, emulators and simulators.

Maturity subgoals that support this goal are:

- An organization-wide test technology group must be formed to study, evaluate and recommend a set of basic testing techniques and methods, and a set of simple tools to support them.

- Management must institute a set of policies so that recommended techniques and methods are applied throughout the organization and are used repeatedly and consistently.

QUESTIONS

1. Are these testing techniques and methods applied on an organization-wide basis?

Yes No Does Not Apply Don't Know

2. Are these testing tools applied on an organization-wide basis?

Yes No Does Not Apply Don't Know

3. Are adequate resources provided by upper management to support the use of basic testing techniques and methods as well as basic testing tools?

Yes No Does Not Apply Don't Know

4. Have developers been trained to apply the basic tools and methods?

Yes No Does Not Apply Don't Know

5. Have basic testing techniques and methods been identified?

Yes No Does Not Apply Don't Know

6. Is usage of testing techniques and methods recorded?

Yes No Does Not Apply Don't Know

7. Have basic testing tools been identified?

Yes No Does Not Apply Don't Know

8. Are measurements used to determine the status of the activities for applying basic testing techniques and methods?

Yes No Does Not Apply Don't Know

9. Are the activities for applying basic testing techniques and methods reviewed with the project manager on both a periodic and event-driven basis?

Yes No Does Not Apply Don't Know

LEVEL 3: INTEGRATION

3.1 ESTABLISH A SOFTWARE TEST ORGANIZATION

The purpose of this goal is to identify a group of people that is responsible for testing. The test group should be responsible for test planning, test execution and recording, test-related standards, test metrics, the test database, test reuse, test tracking and evaluation.

Maturity subgoals that support this goal are:

- An organization-wide test group must be established. Its functionality and position in the reporting hierarchy must be defined.

- Leadership, support and funding for the test group must be provided.

- Roles and responsibilities must be defined for the test group.

- Well-trained and motivated members must be assigned to the test group.

- User/client communication links with the test group must be established to solicit their participation in testing activities and to collect, document, and incorporate user needs, concerns and requirements into the testing process.

QUESTIONS

1. Is testing recognized as a professional activity by the organization?

Yes No Does Not Apply Don't Know

2. Are client concerns a formal component of the test organization policies?

Yes No Does Not Apply Don't Know

3. Are adequate resources provided for the software test organization?

Yes No Does Not Apply Don't Know  (If "yes" skip question 6)

4. Are members of the software test organization trained in testing methods, theory, tools and techniques?

Yes No Does Not Apply Don't Know  (If "yes" skip question 6)

5. Are testers adequately compensated with respect to other software engineers?

Yes No Does Not Apply Don't Know

6. Is there an organization-wide software test organization responsible for the testing process?

Yes No Does Not Apply Don't Know  (If "no" skip questions 7 - 11)

7. Are the roles and responsibilities of the group defined?

Yes No Does Not Apply Don't Know

8. Are the group's activities documented and reported to upper management?

Yes No Does Not Apply Don't Know

9. Does the organizational software test group coordinate with the SQA group to enhance test effectiveness and improve software quality following the user's requirements?

Yes No Does Not Apply Don't Know

10. Are measurements used to determine the status of activities for establishing a software test organization?

Yes No Does Not Apply Don't Know

11. Are the activities for establishing a software test organization reviewed with upper management?

Yes No Does Not Apply Don't Know

3.2 ESTABLISH A TECHNICAL TRAINING PROGRAM

The purpose of this goal is to insure a skilled staff for the Testing Group and prepare the staff for the review process, instructing review leaders and instituting channels for user participation in the testing and review processes. Resources that are needed to implement a technical training program are: funds, training staff and appropriate training facilities.

Maturity subgoals that support this goal are:

- An organization-wide committee on technical training must be established with funding and resources.

- The technical training committee must develop and distribute an organizational training policy.

- Training goals and plans must be developed with input from project managers.

- An in-house training group must be established with tools, facilities and materials in place.

QUESTIONS

1. Does your organization follow a written organizational policy to meet its training needs?

Yes No Does Not Apply Don't Know

2. Are adequate resources provided to implement the technical training program?

Yes No Does Not Apply Don't Know  (If "yes" skip question 5)

3. Do participants in the test group receive the training for developing the skills and knowledge required to perform their technical roles?

Yes No Does Not Apply Don't Know

4. Are training goals and plans identified and documented?

Yes No Does Not Apply Don't Know

5. Has a technical training program been established to improve individuals' technical skill in testing and quality assurance?

Yes No Does Not Apply Don't Know  (If "no" skip questions 6 & 7)

6. Are measurements used to determine the quality and of effectiveness of the training program?

Yes No Does Not Apply Don't Know

7. Are training program activities reviewed with upper management and clients on a periodic basis?

Yes No Does Not Apply Don't Know

3.3 INTEGRATE TESTING INTO THE SOFTWARE LIFE CYCLE

The purpose of this goal is to carry out testing activities in parallel with all life cycle phases. Test planning is initiated early in the life cycle. A variation of the V-model is used by the managers, testers, and developers. User input to the testing process is solicited through established channels for several of the testing phases. Test requirements identify what is to be tested, how to test, who is responsible, and what resources are needed. Examples of testing activities are: test planning, reviews, test execution. Resources that support the integration of the testing effort are, for example: funds, qualified staff, supporting models, standard documents, and schedules.

Maturity subgoals that support this goal are:

- The test phase must be partitioned into subphases that can be integrated into the software life cycle following a written organizational policy and reviewed with management.

- Resources must be provided to support the integration of testing activity into the software life cycle.

- An institutionalized version of the V-model based on defined testing subphases must be developed and adopted.

- Standards definitions must be developed for test-related work products and it's quality must be measured.

- A procedure must be established for testers to work with developers to facilitate testing activity

QUESTIONS

1. Does the project follow a written organizational policy for the integration of testing effort?

Yes No Does Not Apply Don't Know

2. Is there a policy to handle non-compliance with standards?

Yes No Does Not Apply Don't Know

3. Are adequate resources provided for the integration of testing effort?

Yes No Does Not Apply Don't Know

4. Is some form of the V-model used to guide test integration into life-cycle activities?

Yes No Does Not Apply Don't Know

5. Are testing activities initiated early in the life cycle?

Yes No Does Not Apply Don't Know

6. Is there a procedure to facilitate testing activity integration?

Yes No Does Not Apply Don't Know

7. Are test requirements identified at every phase?

Yes No Does Not Apply Don't Know

8. Does test planning allow for V-model activities?

Yes No Does Not Apply Don't Know

9. Does project planning allow for V-model activities?

Yes No Does Not Apply Don't Know

10. Has the test organization and SQA group developed a standard document for all work products?

Yes No Does Not Apply Don't Know

11. Does the test group have adequate quantitative measurements of product quality?

Yes No Does Not Apply Don't Know

12. Are the activities for developing testing into the software life cycle reviewed with senior management on a periodic basis?

Yes No Does Not Apply Don't Know

3.4 INITIATE RISK MANAGEMENT FOR TESTING

The purpose of this goal is to use basic risk management techniques for identifying the major problems which could affect the production of quality software products. Risk management for testing is an essential activity. Risk-related information must be applied to test planning, controlling and monitoring. The testing process must be monitored to track the project's progress in resolving the risk-related items.

Maturity subgoals that support this goal are:

- An organization-wide risk management model for testing must be adopted, documented, and distributed to test managers.

- The organization must develop, document, and support procedures and policies for risk management of testing.

- Test managers must be trained in the use of the risk management model.

- Risk findings must be applied to test planning, controlling and monitoring.

QUESTIONS

1. Does the project follow a written organizational policy for risk management of testing?

Yes No Does Not Apply Don't Know

2. Is risk-related information applied to test planning for reducing the identified risks?

Yes No Does Not Apply Don't Know

3. Is risk-related information applied to controlling and monitoring?

Yes No Does Not Apply Don't Know

4. Are basic risk management techniques used to avoid some problems caused by low quality of software product?

Yes No Does Not Apply Don't Know

5. Are test managers trained to use risk management model?

Yes No Does Not Apply Don't Know  (If "yes" skip question 6)

6. Has an organization-wide risk management model for testing been developed and documented?

Yes No Does Not Apply Don't Know  (If "no" skip questions 12 &13)

7. Are risks which impact on software testing identified?

Yes No Does Not Apply Don't Know

8. Has the impact of risks from previous projects been considered?

Yes No Does Not Apply Don't Know

9. Does the test organization/SQA prepare contingent plans for the occurrence of risk-related events?

Yes No Does Not Apply Don't Know

10. Are testers, developers and users/clients consulted for identification of risk-related items?

Yes No Does Not Apply Don't Know

11. Does the testing process focus on resolving the risk-related items?

Yes No Does Not Apply Don't Know

12. Are measurements used to determine the status of the activities for risk management of testing?

Yes No Does Not Apply Don't Know

13. Are the activities for risk management of testing reviewed with senior management on both periodic event-driven basis?

Yes No Does Not Apply Don't Know

3.5 CONTROLLING AND MONITORING THE TESTING PROCESS

The purpose of this goal is to provide a track of the testing process to detect deviations from the test plans. Management can then take effective actions to correct the deviations in order to accomplish the goals in the test plan. Controlling and monitoring also give visibility to the testing process and support testing as a professional activity. Controlling and monitoring are supported by measurements and tools. We need to determine the status of test design work, the status of test case specification work, available test cases, what has been tested, which versions are tested with what tests, how much testing remains, and what are the outstanding problems.

Maturity subgoals that support this goal are:

- The organization must develop, document, and support procedures and policies for controlling and monitoring the testing process.

- A set of basic test process-related measurements must be defined, recorded and distributed.

- A set of corrective actions, and contingency plans must be developed, recorded and documented for use when test-related risks occur and/or testing deviates significantly from what is planned.

QUESTIONS

1. Does each project follow a written organizational policy for controlling and monitoring the testing process?

Yes No Does Not Apply Don't Know

2. Are adequate resources provided for controlling and monitoring the testing process?

Yes No Does Not Apply Don't Know

3. Do standards for test work products exist?

Yes No Does Not Apply Don't Know

4. Does the organization collect and store test metrics?

Yes No Does Not Apply Don't Know

5. Are test metrics used to evaluate quality of testing?

Yes No Does Not Apply Don't Know

6. Is test status data recorded and reported to the test manager?

Yes No Does Not Apply Don't Know

7. Does the test organization along with the project manager develop contingent plans for test progress?

Yes No Does Not Apply Don't Know

8. Are test items under control of a configuration management system?

Yes No Does Not Apply Don't Know

9. Are measurements used to determine the status of activities for controlling and monitoring the testing process?

Yes No Does Not Apply Don't Know

10. Are the activities for controlling and monitoring the testing process reviewed with senior management on a periodic basis?

Yes No Does Not Apply Don't Know

LEVEL 4: MANAGEMENT AND MEASUREMENT

4.1 ESTABLISH AN ORGANIZATION-WIDE REVIEW PROGRAM

Achieving this goal will result in a review program that will help an organization to: identify, catalog, and remove defects from software work products, and test work products early and effectively. Examples of items that are reviewed are: requirements documents, design documents, code, test plans.

Maturity subgoals that support this goal are:

- Upper management must develop review policies, support the review process, and take responsibility for integrating it into the organizational culture.

- The Test Group and the Software Quality Assurance Group must develop and document goals, plans, follow-up procedures, and recording mechanisms for reviews throughout the software life cycle.

- Personnel must be trained so that they understand and follow proper review policies, practices and procedures.

QUESTIONS

1. Does the project follow a written organizational policy for performing review?

Yes No Does Not Apply Don't Know

2. Are adequate resources (e.g. funding, review materials) provided to implement the review program?

Yes No Does Not Apply Don't Know

3. Are leaders and participants for reviews selected and trained?

Yes No Does Not Apply Don't Know

4. Is there a review of an organization-wide program developed to carry out reviews, inspections, and walkthroughs throughout every phase of the software life cycle?

Yes No Does Not Apply Don't Know (If "yes" skip questions 8 & 9)

5. Are reviews planned, and results reported and documented?

Yes No Does Not Apply Don't Know

6. Are the software/test work products developed at every phase of the software life cycle reviewed?

Yes No Does Not Apply Don't Know

7. Are actions related to defects identified in reviews tracked until they are resolved?

Yes No Does Not Apply Don't Know

8. Are measurements used to determine the status of activities for establishing an organization-wide review program?

Yes No Does Not Apply Don't Know

9. Are the activities for establishing an organization-wide review program reviewed with senior management on a periodic basis?

Yes No Does Not Apply Don't Know

4.2 ESTABLISH A TEST MEASUREMENT PROGRAM

The purposes of a test measurement program are to evaluate the quality of the testing process, to assess the productivity of the testing personnel and the effectiveness of the testing process, and to improve testing process. Example of test-related measurement data are: test costs, tester producting, error and defects, and product reliability.

Maturity subgoals that support this goal are:

- Organization-wide test measurement policies and goals must be defined.

- A test measurement plan must be developed with mechanisms for data collection, analysis and application.

- Action plans for applying measurement results to test process improvements must be developed and documented.

QUESTIONS

1. Does each project follow a written organizational policy for test measurement?

Yes No Does Not Apply Don't Know

2. Are client concerns reflected in the measurement policy?

Yes No Does Not Apply Don't Know

3. Are members in the test measurement program trained in the collection of measurement data from development and how to use related tools?

Yes No Does Not Apply Don't Know (If "yes" skip question 4)

4. Is there an organization-wide test measurement program?

Yes No Does Not Apply Don't Know (If "no" skip questions 5, 11, 12)

5. Is the test measurement program carefully planned?

Yes No Does Not Apply Don't Know

6. Are quantitative test measurement goals for the organization identified by manager?

Yes No Does Not Apply Don't Know

7. Are test measurement activities documented?

Yes No Does Not Apply Don't Know

8. Is measurement data for every test life cycle phase specified, recorded and collected?

Yes No Does Not Apply Don't Know

9. Is the data used to support appropriate action plans for testing process improvement?

Yes No Does Not Apply Don't Know

10. Are the selected test metrics distributed to project and test managers and developers?

Yes No Does Not Apply Don't Know

11. Are measurements used to determine the status of activities for establishing a test measurement program?

Yes No Does Not Apply Don't Know

12. Are the activities for establishing a test measurement program reviewed with senior management on a periodic basis?

Yes No Does Not Apply Don't Know

4.3 SOFTWARE QUALITY EVALUATION

The purposes of software quality evaluation are: to relate software quality issues to the adequacy of the testing process, to define measurable quality attributes, and to define quality goals for evaluating software work products. Quality attributes that are considered are: correctness, reliability, efficiency, integrity, usability, maintainability, flexibility, testability, portability, reusability and interoperability. Examples of software quality evaluation support items are: questionnaires, interviews, and usability tests.

Maturity subgoals that support this goal are:

- Management and the testing and software quality assurance groups must define quality-related policies, quality goals and quality attributes for software products.

- The organization must develop, document, and support procedures and policies for evaluating software quality.

- The testing process must be structured, measured, and evaluated to ensure the quality goals can be achieved.

QUESTIONS 

1. Does the project follow a written organizational policy for evaluating software quality?

Yes No Does Not Apply Don't Know

2. Are adequate resources provided for establishing quality goals and for accomplishing the goals?

Yes No Does Not Apply Don't Know

3. Are members of the software test group trained in software quality issues?

Yes No Does Not Apply Don't Know

4. Are quality goals for evaluating software work products identified?

Yes No Does Not Apply Don't Know

5. Are measurable quality attributes for software products specified?

Yes No Does Not Apply Don't Know

6. Is there a formed procedure in place for client input to software quality evaluation?

Yes No Does Not Apply Don't Know

7. Are results of testing evaluated to improve software quality?

Yes No Does Not Apply Don't Know

8. Are measurements used to determine the status of activities for software quality evaluation?

Yes No Does Not Apply Don't Know

9. Are the activities for software quality evaluation reviewed with senior management on a periodic basis?

Yes No Does Not Apply Don't Know

LEVEL 5: OPTIMIZATION/DEFECT PREVENTION AND QUALITY CONTROL

5.1 APPLICATION OF PROCESS DATA FOR DEFECT PREVENTION

The purpose of this goal is to apply defect prevention activities throughout the life cycle. Defect prevention activities include defect recording and tracking, defect causal analysis, action plan establishment, action implementation, training in defect prevention methods, and task kick-off or casual analysis meetings.

Maturity subgoals that support this goal are:

- The organization must develop, document, and support procedures and policies for defect prevention.

- A defect prevention team must be established with management support.

- Defects injected/removed must be identified and recorded for each life cycle phase.

- A causal analysis procedure must be established to identify the root causes of defects.

- Action plans must be developed through the interaction of managers, developers and testers to prevent identified defects from reoccurring. These plans must be tracked.

QUESTIONS

1. Does the project follow a written organizational policy for defect prevention?

Yes No Does Not Apply Don't Know

2. Are adequate resources (e.g. funding, statistical analysis tools) provided for defect prevention activities?

Yes No Does Not Apply Don't Know

3. Do members of the defect prevention group receive required training to perform their defect prevention activities?

Yes No Does Not Apply Don't Know (If "yes" skip question 4)

4. Has a defect prevention team been established?

Yes No Does Not Apply Don't Know (If "no" skip question 11)

5. Are defect prevention activities planned?

Yes No Does Not Apply Don't Know

6. For each life cycle phase, are injected/removed defects recorded?

Yes No Does Not Apply Don't Know

7. Once identified, are common causes of defects analyzed and systematically eliminated?

Yes No Does Not Apply Don't Know

8. Are specific actions offered to prevent defects from recurring?

Yes No Does Not Apply Don't Know

9. Are actions tracked to insure progress?

Yes No Does Not Apply Don't Know

10. Are measurements used to determine the status of defect prevention activities?

Yes No Does Not Apply Don't Know

11. Are the activities and work products for defect prevention reviewed by team's manager and presented to upper management?

Yes No Does Not Apply Don't Know

5.2 QUALITY CONTROL

The purpose of this goal is to incorporate techniques and tools to reduce defects and improve software quality. Automated tools support the running and rerunning of test cases and defect collection and analysis. Statistical techniques are used to support testing efforts and quality goals. Levels of confidence can be established for the reliability of software. Quantitative goals are used to determine when to stop testing. Quantitative criteria involves when a certain level of reliability is reached, when a certain level of trustworthiness is reached, when software products are certified with usage testing with a given degree of confidence, when the number of defects/unit time will find a certain level.

Maturity subgoals that support this goal are:

- The organization must develop, document, and support procedures for quality control.

- The software test group and the SQA group establish quality and quantitative goals for software products.

- Test managers must incorporate these quality goals into the test plans.

- The test group and related group must be trained in statistical methods and quality-related others.

- User input must be collected for usage modeling.

QUESTIONS

1. Does the project follow a written organizational policy for controlling software quality?

Yes No Does Not Apply Don't Know

2. Are techniques and tools employed to reduce defects and improve software quality?

Yes No Does Not Apply Don't Know

3. Are statistical methods used for evaluating software quality?

Yes No Does Not Apply Don't Know

4. Are members of test group trained in statistical methods?

Yes No Does Not Apply Don't Know

5. Do members of software engineering group and test group receive required training in software quality control?

Yes No Does Not Apply Don't Know

6. Are quality goals for software products identified by the software test group and the SQA group?

Yes No Does Not Apply Don't Know

7. Are quality-related attributes identified and documented?

Yes No Does Not Apply Don't Know

8. Are the goals of each quality-related attribute identified?

Yes No Does Not Apply Don't Know

9. Are the goals of each quality-related attribute considered in the testing process?

Yes No Does Not Apply Don't Know

10. Are activities for controlling product quality identified?

Yes No Does Not Apply Don't Know

11. Is user input collected for usage modeling?

Yes No Does Not Apply Don't Know

12. Are quantitative goals used to determine when to stop testing?

Yes No Does Not Apply Don't Know

13. Are the quantitative criteria used to determine when to stop testing specified by the testers/developers?

Yes No Does Not Apply Don't Know

14. Does the project use measurable and prioritized goals for managing the quality of its software products?

Yes No Does Not Apply Don't Know

15. Are measurements used to determine the status of the activities for managing software quality?

Yes No Does Not Apply Don't Know

16. Are the activities performed for software quality control reviewed with senior management on a periodic basis?

Yes No Does Not Apply Don't Know

5.3 TEST PROCESS OPTIMIZATION

The purposes of this goal are to identify testing practices that need to be improved, to implement the improvements, to track improvement progress, to continually evaluate new test-related tools and technologies for adaptation, and to support technology transfer.

Maturity subgoals that support this goal are:

- The organization must develop, document, and support procedures and policies for test process optimization.

- A test process improvement group must be established to monitor the testing process and identify areas for improvement.

- A procedure must be in place for evaluating new tools and technologies with respect to improving the capability and maturity of the testing process.

- The effectiveness of the testing process must be continually evaluated, and the decision on when to stop testing must be related to quality goals and made in a measurable and optimal manner.

QUESTIONS

1. Does the project follow a written organizational policy for test process optimization?

Yes No Does Not Apply Don't Know

2. Are adequate resources (e.g. funding, information about client's needs, tools) provided for the test process improvement program?

Yes No Does Not Apply Don't Know (If "yes" skip question 5)

3. Is there a plan supported by metrics and analytic technique for evaluating new testing tools and technologies to improve the capability and maturity of testing process?

Yes No Does Not Apply Don't Know

4. Are testing tools continually being evaluated for use by the testing group?

Yes No Does Not Apply Don't Know

5. Has a test process improvement program been established?

Yes No Does Not Apply Don't Know

6. Are testing practices that need to be improved identified?

Yes No Does Not Apply Don't Know

7. Is there a procedure for technology transfer when new tools have been identified?

Yes No Does Not Apply Don't Know

8. Is test process improvement tracked to planned goals?

Yes No Does Not Apply Don't Know

9. Is tool impact on the testing process adequately measured?

Yes No Does Not Apply Don't Know

10.Are measurements used to determine the status and results of the test process improvement?

Yes No Does Not Apply Don't Know

11.Are the activities for test process optimization reviewed with senior management on a periodic basis?

Yes No Does Not Apply Don't Know

Comments: