Author: Karl Eugene Wiegers
Editor:
ISBN: 9780735679665
Size: 10,82 MB
Format: PDF
Read: 633

Author: Dragan Z. Milosevic
Editor: John Wiley & Sons
ISBN: 1118973127
Size: 17,50 MB
Format: PDF, ePub, Docs
Read: 372

Author: Project Management Institute
Editor: Project Management Institute
ISBN: 1628251093
Size: 13,74 MB
Format: PDF, Kindle
Read: 677

Author: Project Management Institute
Editor: Project Management Institute
ISBN: 1628250798
Size: 12,52 MB
Format: PDF, ePub, Docs
Read: 953

Author: Richard H. Thayer
Editor: IEEE Computer Society
ISBN:
Size: 14,53 MB
Format: PDF, ePub, Docs
Read: 426

Author:
Editor:
ISBN:
Size: 15,80 MB
Format: PDF, Kindle
Read: 296

Author: E. M. Bennatan
Editor: John Wiley & Sons
ISBN:
Size: 15,52 MB
Format: PDF, ePub, Mobi
Read: 786

Author: Shari Lawrence Pfleeger
Editor: Prentice Hall
ISBN: 0136061699
Size: 13,82 MB
Format: PDF, ePub, Docs
Read: 662

Author: Paige Baltzan
Editor: McGraw-Hill Education Australia
ISBN: 1743762119
Size: 16,84 MB
Format: PDF, ePub, Mobi
Read: 415

Author: Miller Freeman Inc
Editor:
ISBN:
Size: 19,53 MB
Format: PDF, ePub, Docs
Read: 360

© 2009 by Taylor & Francis Group, LLC
Boca Raton London New York
CRC Press is an imprint of the Taylor & Francis Group, an informa business
AN AUERBACH BOOK © 2009 by Taylor & Francis Group, LLC
Auerbach Publications Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487‑2742 © 2009 by Taylor & Francis Group, LLC Auerbach is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed in the United States of America on acid‑free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number‑13: 978‑1‑4200‑8073‑5 (Hardcover) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher can‑ not assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copy‑ right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978‑750‑8400. CCC is a not‑for‑profit organization that pro‑ vides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging‑in‑Publication Data Lewis, William E. Software testing and continuous quality improvement / William E. Lewis. ‑‑ 3rd ed. p. cm. Includes bibliographical references and index. ISBN 978‑1‑4200‑8073‑5 (alk. paper) 1. Computer software‑‑Testing. 2. Computer software‑‑Quality control. I. Title. QA76.76.T48L495 2008 005.1’4‑‑dc22 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the Auerbach Web site at http://www.auerbach‑publications.com © 2009 by Taylor & Francis Group, LLC
2008046201
Contents Acknowledgments.............................................xxi Introduction................................................ xxiii About the Author............................................. xxv
Section 1 Software Quality in Perspective 1
A Brief History of Software Testing............................3 Historical Software Testing and Development Parallels................6 Extreme Programming.........................................8 Evolution of Automated Testing Tools.............................8 Static Capture/Replay Tools (without Scripting Language)........10 Static Capture/Replay Tools (with Scripting Language)..........10 Variable Capture/Replay Tools.............................10
2
Quality Assurance Framework...............................13 What Is Quality?............................................13 Prevention versus Detection....................................14 Verification versus Validation...................................15 Software Quality Assurance....................................16 Components of Quality Assurance...............................17 Software Testing........................................17 Quality Control.............................................18 Software Configuration Management........................19 Elements of Software Configuration Management..........20 Software Quality Assurance Plan................................23 Steps to Develop and Implement a Software Quality Assurance Plan..................................................23 Step 1: Document the Plan............................23 Step 2: Obtain Management Acceptance.................25 Step 3: Obtain Development Acceptance.................25
v © 2009 by Taylor & Francis Group, LLC
vi ◾ Contents
Step 4: Plan for Implementation of the SQA Plan..........26 Step 5: Execute the SQA Plan..........................26 Quality Standards............................................26 Sarbanes–Oxley.........................................26 ISO9000..............................................29 Capability Maturity Model (CMM).........................29 Level 1: Initial.....................................30 Level 2: Repeatable..................................31 Level 3: Defined....................................31 Level 4: Managed...................................32 Level 5: Optimized..................................32 People CMM...........................................33 CMMI................................................33 Malcolm Baldrige National Quality Award................... 34 Notes.....................................................37
3
Overview of Testing Techniques..............................39 Black-Box Testing (Functional).................................39 White-Box Testing (Structural)................................. 40 Gray-Box Testing (Functional and Structural)......................41 Manual versus Automated Testing...............................41 Static versus Dynamic Testing..................................41 Taxonomy of Software Testing Techniques........................ 42
4
Transforming Requirements to Testable Test Cases...............51 Introduction................................................51 Software Requirements as the Basis of Testing......................51 Requirement Quality Factors...................................52 Understandable.........................................52 Necessary..............................................53 Modifiable.............................................53 Nonredundant..........................................53 Terse.................................................54 Testable...............................................54 Traceable..............................................54 Within Scope...........................................54 Numerical Method for Evaluating Requirement Quality..............54 Process for Creating Test Cases from Good Requirements.............55 Step 1: Review the Requirements............................55 Step 2: Write a Test Plan..................................58 Step 3: Identify the Test Suite..............................58 Step 4: Name the Test Cases...............................59 Step 5: Write Test Case Descriptions and Objectives.............62
© 2009 by Taylor & Francis Group, LLC
Contents ◾ vii
Step 6: Create the Test Cases...............................62 Step 7: Review the Test Cases..............................63 Transforming Use Cases to Test Cases............................ 64 Step 1: Draw a Use Case Diagram.......................... 64 Step 2: Write the Detailed Use Case Text..................... 64 Step 3: Identify Use Case Scenarios......................... 66 Step 4: Generating the Test Cases........................... 66 Step 5: Generating Test Data...............................68 Summary..............................................68 What to Do When Requirements Are Nonexistent or Poor?............68 Ad Hoc Testing.........................................68 The Art of Ad Hoc Testing............................68 Advantages and Disadvantages of Ad Hoc Testing..........71 Exploratory Testing......................................72 The Art of Exploratory Testing.........................72 Exploratory Testing Process...........................72 Advantages and Disadvantages of Exploratory Testing.......73
5
Quality through Continuous Improvement Process...............75 Contribution of Edward Deming................................75 Role of Statistical Methods.....................................76 Cause-and-Effect Diagram................................76 Flowchart.............................................76 Pareto Chart...........................................76 Run Chart............................................ 77 Histogram............................................ 77 Scatter Diagram........................................ 77 Control Chart.......................................... 77 Deming’s 14 Quality Principles................................. 77 Point 1: Create Constancy of Purpose....................... 77 Point 2: Adopt the New Philosophy..........................78 Point 3: Cease Dependence on Mass Inspection................78 Point 4: End the Practice of Awarding Business on Price Tag Alone.............................................79 Point 5: Improve Constantly and Ceaselessly the System of Production and Service...................................79 Point 6: Institute Training and Retraining....................79 Point 7: Institute Leadership...............................80 Point 8: Drive Out Fear...................................80 Point 9: Break Down Barriers between Staff Areas..............81 Point 10: Eliminate Slogans, Exhortations, and Targets for the Workforce.............................................81 Point 11: Eliminate Numerical Goals........................81
© 2009 by Taylor & Francis Group, LLC
viii ◾ Contents
Point 12: Remove Barriers to Pride of Workmanship.............82 Point 13: Institute a Vigorous Program of Education and Retraining.............................................82 Point 14: Take Action to Accomplish the Transformation.........82 Continuous Improvement through the Plan, Do, Check, Act Process....83 Going around the PDCA Circle................................ 84
Section 2 Waterfall Testing Review 6
Overview................................................87 Waterfall Development Methodology.............................87 Continuous Improvement “Phased” Approach......................88 Psychology of Life-Cycle Testing................................89 Software Testing as a Continuous Improvement Process..............89 The Testing Bible: Software Test Plan.............................92 Major Steps in Developing a Test Plan............................93 Step 1: Define the Test Objectives...........................93 Step 2: Develop the Test Approach..........................93 Step 3: Define the Test Environment.........................95 Step 4: Develop the Test Specifications.......................95 Step 5: Schedule the Test..................................95 Step 6: Review and Approve the Test Plan.....................95 Components of a Test Plan.....................................95 Technical Reviews as a Continuous Improvement Process.............96 Motivation for Technical Reviews...............................101 Types of Reviews...........................................101 Structured Walkthroughs................................101 Inspections...........................................102 Participant Roles............................................103 Steps for an Effective Review..................................105 Step 1: Plan for the Review Process.........................105 Step 2: Schedule the Review..............................105 Step 3: Develop the Review Agenda........................106 Step 4: Create a Review Report............................106
7
Static Testing the Requirements.............................107 Testing the Requirements with Ambiguity Reviews.................108 Testing the Requirements with Technical Reviews..................109 Inspections and Walkthroughs.................................109 Checklists.................................................109 Methodology Checklist..................................109 Requirements Traceability Matrix.............................. 110 Building the System/Acceptance Test Plan........................ 111
© 2009 by Taylor & Francis Group, LLC
Contents ◾ ix
8
Static Testing the Logical Design............................115 Data Model, Process Model, and the Linkage...................... 115 Testing the Logical Design with Technical Reviews................. 117 Refining the System/Acceptance Test Plan........................ 118
9
Static Testing the Physical Design...........................121 Testing the Physical Design with Technical Reviews................121 Creating Integration Test Cases................................122 Methodology for Integration Testing............................123 Step 1: Identify Unit Interfaces............................123 Step 2: Reconcile Interfaces for Completeness.................124 Step 3: Create Integration Test Conditions...................124 Step 4: Evaluate the Completeness of Integration Test Conditions............................................124
10 Static Testing the Program Unit Design.......................127 Testing the Program Unit Design with Technical Reviews............127 Sequence.............................................127 Selection.............................................128 Iteration..............................................128 Creating Unit Test Cases.....................................128
11 Static Testing and Dynamic Testing the Code..................131 Testing Coding with Technical Reviews..........................131 Executing the Test Plan......................................132 Unit Testing...............................................133 Integration Testing..........................................134 System Testing.............................................134 Acceptance Testing..........................................134 Defect Recording...........................................135
Section 3 0007Spiral (Agile) Software Testing Methodology: Plan, Do, Check, Act 12 Development Methodology Overview.........................139 Limitations of Life-Cycle Development..........................139 The Client/Server Challenge...................................140 Psychology of Client/Server Spiral Testing........................ 141 The New School of Thought..............................141 Tester/Developer Perceptions..............................142 Project Goal: Integrate QA and Development.................143 Iterative/Spiral Development Methodology...................144 Role of JADs...............................................146 Role of Prototyping..........................................146
© 2009 by Taylor & Francis Group, LLC
x ◾ Contents
Methodology for Developing Prototypes.........................148 Step 1: Develop the Prototype.............................148 Step 2: Demonstrate Prototypes to Management...............149 Step 3: Demonstrate Prototype to Users.....................150 Step 4: Revise and Finalize Specifications....................150 Step 5: Develop the Production System...................... 151 Continuous Improvement “Spiral” Testing Approach................ 151
13 Information Gathering (Plan)...............................155 Step 1: Prepare for the Interview................................156 Task 1: Identify the Participants...........................156 Task 2: Define the Agenda................................156 Step 2: Conduct the Interview.................................156 Task 1: Understand the Project............................ 158 Task 2: Understand the Project Objectives................... 159 Task 3: Understand the Project Status.......................160 Task 4: Understand the Project Plans.......................160 Task 5: Understand the Project Development Methodology...... 161 Task 6: Identify the High-Level Business Requirements......... 161 Task 7: Perform Risk Analysis.............................162 Computer Risk Analysis.............................163 Method 1: Judgment and Instinct.....................163 Method 2: Dollar Estimation.........................163 Method 3: Identifying and Weighting Risk Attributes......164 Step 3: Summarize the Findings................................165 Task 1: Summarize the Interview...........................165 Task 2: Confirm the Interview Findings.....................165
14 Test Planning (Plan)......................................167 Step 1: Build a Test Plan......................................168 Task 1: Prepare an Introduction............................168 Task 2: Define the High-Level Functional Requirements (in Scope)...............................................170 Task 3: Identify Manual/Automated Test Types...............171 Task 4: Identify the Test Exit Criteria.......................171 Task 5: Establish Regression Test Strategy....................172 Task 6: Define the Test Deliverables........................ 174 Task 7: Organize the Test Team...........................175 Task 8: Establish a Test Environment.......................177 Task 9: Define the Dependencies...........................177 Task 10: Create a Test Schedule............................178 Task 11: Select the Test Tools.............................178 Task 12: Establish Defect Recording/Tracking Procedures.......182
© 2009 by Taylor & Francis Group, LLC
Contents ◾ xi
Task 13: Establish Change Request Procedures................184 Task 14: Establish Version Control Procedures................185 Task 15: Define Configuration Build Procedures...............186 Task 16: Define Project Issue Resolution Procedures............186 Task 17: Establish Reporting Procedures.....................187 Task 18: Define Approval Procedures.......................187 Step 2: Define the Metric Objectives............................188 Task 1: Define the Metrics................................188 Task 2: Define the Metric Points...........................189 Step 3: Review/Approve the Plan...............................194 Task 1: Schedule/Conduct the Review.......................194 Task 2: Obtain Approvals................................194
15 Test Case Design (Do).....................................195 Step 1: Design Function Tests..................................195 Task 1: Refine the Functional Test Requirements..............195 Task 2: Build a Function/Test Matrix....................... 200 Step 2: Design GUI Tests.................................... 200 Ten Guidelines for Good GUI Design...................... 200 Task 1: Identify the Application GUI Components.............202 Task 2: Define the GUI Tests.............................202 Step 3: Define the System/Acceptance Tests.......................203 Task 1: Identify Potential System Tests......................203 Task 2: Design System Fragment Tests......................205 Task 3: Identify Potential Acceptance Tests.................. 206 Step 4: Review/Approve Design................................ 206 Task 1: Schedule/Prepare for Review....................... 206 Task 2: Obtain Approvals................................ 206
16 Test Development (Do)....................................209
Step 1: Develop Test Scripts...................................209 Task 1: Script the Manual/Automated GUI/Function Tests......209 Task 2: Script the Manual/Automated System Fragment Tests....210 Step 2: Review/Approve Test Development........................210 Task 1: Schedule/Prepare for Review........................210 Task 2: Obtain Approvals................................212
17 Test Coverage through Traceability..........................213 Use Cases and Traceability.................................... 214 Summary.................................................216
18 Test Execution/Evaluation (Do/Check).......................217
Step 1: Setup and Testing..................................... 217 Task 1: Regression Test the Manual/Automated Spiral Fixes...... 217
© 2009 by Taylor & Francis Group, LLC
xii ◾ Contents
Task 2: Execute the Manual/Automated New Spiral Tests........ 219 Task 3: Document the Spiral Test Defects.................... 219 Step 2: Evaluation........................................... 219 Task 1: Analyze the Metrics............................... 219 Step 3: Publish Interim Report................................ 220 Task 1: Refine the Test Schedule........................... 220 Task 2: Identify Requirement Changes......................221
19 Prepare for the Next Spiral (Act).............................223 Step 1: Refine the Tests.......................................223 Task 1: Update the Function/GUI Tests.....................223 Task 2: Update the System Fragment Tests...................225 Task 3: Update the Acceptance Tests........................225 Step 2: Reassess the Team, Procedures, and Test Environment.........225 Task 1: Evaluate the Test Team............................225 Task 2: Review the Test Control Procedures.................. 226 Task 3: Update the Test Environment.......................227 Step 3: Publish Interim Test Report.............................227 Task 1: Publish the Metric Graphics........................227 Test Case Execution Status...........................227 Defect Gap Analysis............................... 228 Defect Severity Status.............................. 228 Test Burnout Tracking............................. 228
20 Conduct the System Test (Act)..............................233
Step 1: Complete System Test Plan..............................233 Task 1: Finalize the System Test Types......................233 Task 2: Finalize System Test Schedule.......................235 Task 3: Organize the System Test Team.....................235 Task 4: Establish the System Test Environment................238 Task 5: Install the System Test Tools........................239 Step 2: Complete System Test Cases.............................239 Task 1: Design/Script the Performance Tests..................239 Monitoring Approach...................................240 Probe Approach........................................241 Test Drivers...........................................241 Task 2: Design/Script the Security Tests.....................242 A Security Design Strategy...........................242 Task 3: Design/Script the Volume Tests......................243 Task 4: Design/Script the Stress Tests.......................243 Task 5: Design/Script the Compatibility Tests................ 244 Task 6: Design/Script the Conversion Tests...................245 Task 7: Design/Script the Usability Tests.................... 246
© 2009 by Taylor & Francis Group, LLC
Contents ◾ xiii
Task 8: Design/Script the Documentation Tests............... 246 Task 9: Design/Script the Backup Tests......................247 Task 10: Design/Script the Recovery Tests...................248 Task 11: Design/Script the Installation Tests..................248 Task 12: Design/Script Other System Test Types..............249 Step 3: Review/Approve System Tests............................250 Task 1: Schedule/Conduct the Review.......................250 Task 2: Obtain Approvals................................250 Step 4: Execute the System Tests................................251 Task 1: Regression Test the System Fixes.....................251 Task 2: Execute the New System Tests.......................251 Task 3: Document the System Defects......................251
21 Conduct Acceptance Testing................................253
Step 1: Complete Acceptance Test Planning.......................253 Task 1: Finalize the Acceptance Test Types...................253 Task 2: Finalize the Acceptance Test Schedule................255 Task 3: Organize the Acceptance Test Team..................255 Task 4: Establish the Acceptance Test Environment............256 Task 5: Install Acceptance Test Tools.......................256 Step 2: Complete Acceptance Test Cases.........................256 Task 1: Identify the System-Level Test Cases..................257 Task 2: Design/Script Additional Acceptance Tests.............257 Step 3: Review/Approve Acceptance Test Plan.....................257 Task 1: Schedule/Conduct the Review.......................257 Task 2: Obtain Approvals................................258 Step 4: Execute the Acceptance Tests............................258 Task 1: Regression Test the Acceptance Fixes.................258 Task 2: Execute the New Acceptance Tests...................259 Task 3: Document the Acceptance Defects...................259
22 Summarize/Report Test Results.............................261
Step 1: Perform Data Reduction................................261 Task 1: Ensure All Tests Were Executed/Resolved..............261 Task 2: Consolidate Test Defects by Test Number..............261 Task 3: Post Remaining Defects to a Matrix..................262 Step 2: Prepare Final Test Report...............................263 Task 1: Prepare the Project Overview.......................263 Task 2: Summarize the Test Activities.......................263 Task 3: Analyze/Create Metric Graphics.....................263 Defects by Function............................... 264 Defects by Tester.................................. 264 Defect Gap Analysis............................... 264
© 2009 by Taylor & Francis Group, LLC
xiv ◾ Contents
Defect Severity Status.............................. 264 Test Burnout Tracking............................. 264 Root Cause Analysis............................... 266 Defects by How Found............................. 266 Defects by Who Found.............................267 Functions Tested and Not Tested......................267 System Testing Defect Types......................... 268 Acceptance Testing Defect Types..................... 268 Task 4: Develop Findings/Recommendations.................269 Step 3: Review/Approve the Final Test Report.....................272 Task 1: Schedule/Conduct the Review.......................272 Task 2: Obtain Approvals................................273 Task 3: Publish the Final Test Report.......................273
Section 4 Project Management Methodology 23 The Project Management Framework.........................279
The Project Framework.......................................279 Product Quality and Project Quality............................279 Components of the Project Framework.......................... 280 The Project Framework and Continuous Quality Improvement....... 280 The Project Framework Phases.................................281 Initiation Phase........................................281 Planning Phase........................................282 Executing, Monitoring, and Controlling Phases...............282 Implement Phase.......................................283 Scoping the Project to Ensure Product Quality.....................283 Product Scope and Project Scope...............................283 The Project Charter......................................... 284 The Scope Statement.........................................285 The Role of the Project Manager in Quality Management............285 The Role of the Test Manager in Quality Management.............. 286 Analyze the Requirements............................... 286 Perform a Gap Analysis................................. 286 Avoid Duplication and Repetition..........................287 Define the Test Data....................................287 Validate the Test Environment............................287 Analyze the Test Results.................................288 Deliver the Quality.....................................288 Advice for the Test Manager...................................288 Request Help from Others................................288 Communicate Issues as They Arise.........................288 Always Update Your Business Knowledge....................289
© 2009 by Taylor & Francis Group, LLC
Contents ◾ xv
Learn the New Testing Technologies and Tools................289 Improve the Process.....................................289 Create a Knowledge Base.................................289 The Benefits of the Quality Project Management and the Project Framework................................................290
24 Project Quality Management...............................291 Project Quality Management Processes..........................291 Quality Planning...........................................292 Identifying the High-Level Project Activities......................292 Estimating the Test Work Effort................................292 Test Planning..............................................293 Effort Estimation: Model Project...............................294 Quality Standards...........................................296
25 The Defect Management Process.............................301 Quality Control and Defect Management........................301 Defect Discovery and Classification.............................301 Defect Priority.............................................302 Defect Category............................................303 Defect Tracking............................................303 Defect Reporting...................................... 304 Defect Summary........................................... 304 Defect Meetings............................................305 Defect Metrics.............................................305 Quality Standards.......................................... 306
26 Integrated Testing and Development.........................309 Quality Control and Integrated Testing..........................309 Integrated Testing...........................................309 Step 1: Organize the Test Team................................ 310 Step 2: Identify the Tasks to Integrate........................... 310 Step 3: Customize Test Steps and Tasks.......................... 311 Step 4: Select Integration Points................................ 311 Step 5: Modify the Development Methodology....................312 Step 6: Test Methodology Training.............................312 Step 7: Incorporate Defect Recording............................313 The Integrated Team.........................................313
27 Test Management Constraints...............................315 Organizational Architecture................................... 315 Traits of a Well-Established Quality Organization.................. 315 Division of Responsibilities.................................... 316 Organizational Relationships.................................. 317
© 2009 by Taylor & Francis Group, LLC
xvi ◾ Contents
Using the Project Framework Where No Quality Infrastructure Exists.. 317 Ad Hoc Testing and the Project Framework....................... 318 Using a Traceability/Validation Matrix........................... 319 Reporting the Progress....................................... 319
Section 5 Emerging Specialized Areas in Testing 28 Test Process and Automation Assessment......................323
Test Process Assessment......................................323 Process Evaluation Methodology...............................324 Step 1: Identify the Key Elements..........................324 Step 2: Gather and Analyze the Information..................325 Step 3: Analyze Test Maturity.............................326 The Requirements Definition Maturity.................326 Test Strategy Maturity..............................327 Test Effort Estimation Maturity.......................328 Test Design and Execution Maturity...................328 Regression Testing Maturity..........................329 Test Automation Maturity...........................329 Step 4: Document and Present Findings.....................330 Test Automation Assessment...................................330 Identify the Applications to Automate.......................332 Identify the Best Test Automation Tool......................332 Test Scripting Approach.................................333 Test Execution Approach.................................333 Test Script Maintenance.................................334 Test Automation Framework..................................334 Basic Features of an Automation Framework..................335 Define the Folder Structure..........................335 Modularize Scripts/Test Data to Increase Robustness.......336 Reuse Generic Functions and Application-Specific Function Libraries.................................336 Develop Scripting Guidelines and Review Checklists.......336 Define Error Handling and Recovery Functions..........337 Define the Maintenance Process.......................337 Standard Automation Frameworks.........................337 in=' association=' with=' the=' project='> 1.3 Application/System Overview 2. Testing Scope 2.1 Testing Objectives 2.4 Out of Scope 2.6 Baseline Documents 3. Testing Approach 3.1 Testing Methodology 3.3.1 Entry Criteria 0007 3.3.2 Test Planning Approach 0007 3.3.4 Test Execution Methodology 3.3.5 Test Execution Checklist 0007
Test Templates ◾ 483
3.3.7 Defect Management 0007<Entire defect management process. It includes defect meeting, defect resolution, and so on> 3.3.8 Defect Logging and Defect Reporting 0007 3.3.9 Defect Classification and Defect Life Cycle 0007 3.3.10 Defect Meetings 0007 3.3.11 Exit Criteria <Exit criteria for test execution> 4. Resources 4.1 Skills Required for the Project 4.2 Training Schedule 4.6 Test Infrastructure 4.6.1 Hardware 4.6.2 Software 5. Project Organization and Communication
484 ◾ Software Testing and Continuous Quality Improvement
5.2 Suspension and Resumption Criteria 5.3 R isk, Contingency, and Mitigation Plan 5.4 Schedule 5.4.1 Milestones 0007 5.4.2 Detailed Plan 0007 5.4.3 Deliverables 0007 6. Appendix <Appendix, as mentioned in any of the foregoing sections, should be mentioned here>
E22: Clarification Request The following Clarification Request matrix is used to document questions that may arise while the tester analyzes the requirements. This sample is also included in the CD at the back of the book. Project Name:______________________________________________________________ Project Code:_______________________________________________________________ Issue No.
Document Reference
Application/ Function
Date Raised
Clarification Requested
Raised By
Status
Response
Test Templates ◾ 485
Issue No.
Document Reference
Application/ Function
Date Raised
Clarification Requested
Raised By
Status
Response
E23: Screen Data Mapping The following Screen Data Mapping matrix is used to document the properties of the screen data. This sample is also included in the CD at the back of the book. Item No.
Test Case ID
Screen Reference (Optional)
1
TS-01
Account Number
aabbcc
alphabets
2
TS-01
Account Number
10099
numeric
######
3
TS-05
As-of Date
31101999
date
dd-mm-yyyy
Field Name
Data Required
Data Type
Data Format
Comments
486 ◾ Software Testing and Continuous Quality Improvement
E24: Test Condition versus Test Case The following Test Condition versus Test Case matrix is used to associate a requirement with each condition that is mapped to one or more test cases. This sample is also included in the CD at the back of the book. Item No.
Requirement Details/Source (Fun. Spec./Bus. Req./Other)
Condition No.
Test Condition
Test Case No.
E25: Project Status Report The following Project Status Report is used to report the status of the testing project for all key process areas. This sample is also included in the CD at the back of the book. Purpose: This template consolidates the QA project-related activities in all key process areas. It is published to all project stakeholders weekly.
Test Templates ◾ 487 Project Name _____________________ Project Code ____________________________ Project Start Date ______________________Project Manager ______________________ Project Phase ______________________ Week No. & Date ________________________ Distribution _______________________________________________________________
Key Activities Details
Remarks
Deliverables
Decisions
Weekly Progress for This Week Planned
Item
Key Processes
Actual
Activities/ Milestone, Start End Start End Status/ Deliverable Date Date Date Date Remarks Owner
Unplanned Activities Item
Activities
Start Date
End Date
Effort (Person-Hours)
Comments
End Date
Effort (Person-Hours)
Comments
Activities Planned for Next Week Item
Activities
Start Date
488 ◾ Software Testing and Continuous Quality Improvement
Planned but Not Completed
Change Requests (New)
Change Requests (Outstanding)
Issues (New)
Issues (Outstanding)
Test Templates ◾ 489
E26: Test Defect Details Report
Text box is not enabled
3
Severity
Priority
Open
3
1
Text box should be enabled
Joe
Closed
2
2
User is not allowed to enter values
User should be allowed to enter values
Sam
Fixed
3
3
4
Area 3 is not getting displayed on the list
Module 3 should get displayed on the list
Sally
Closed
2
4
5
Error message is displayed
Error message should not be displayed
June
Open
2
5
Closed Date
2
Bill
Reported Date
Log-in page is not displaying
Defect Status
Log-in page should be displayed
Detected By
Actual Results
Log-in page not getting displayed
Script ID
1
Item No.
Expected Result
Test Case Description
Defect ID
The following Test Defect Details Report is used to report the detailed defect status of the testing project for all key process areas. This sample is also included in the CD at the back of the book.
490 ◾ Software Testing and Continuous Quality Improvement
E27: Defect Report
Client Comment
Developer Comment
Tester Comment
Defect Type
Severity
Status
Actual Result
Expected Result
Test Case Description
Test Script ID
Date
Defect ID
The following Defect Report is used to report the details of a specific defect. This sample is also included in the CD at the back of the book.
E28: Test Execution Tracking Manager The Test Execution Tracking Manager is an Excel spreadsheet that provides a comprehensive and test cycle view of the number of test cases that passed/failed, the number of defects discovered by application area, the status of the defects, percentage completed, and the defect severities by defect type. The template is located in the CD at the back of the book.
Test Templates ◾ 491
E29: Final Test Summary Report The following Final Test Summary Report is used as a final report of the test project with key findings. The following is a sample table of contents that is also included in the CD at the back of the book.
1. Introduction 1.1 Executive Summary 1.3 Scope of Testing 2. Test Methodology 2.1 Test Documents 2.2 Test Iterations 2.3 Defect Management 3. Measurements 3.1 Traceability Matrix 3.3 Test Scripts Summary 4. Findings 4.1 Final Defect Summary <Summary of Defects at the end of test execution> 4.2 Deferred Defects