Chapter 3 The Realities of Software Testing

The Realities of Software
Testing
Chapter 3

Highlights
 No perfect process
 Trade-offs and concessions
 Why software can never be perfect
 Why software testing isn’t just a technical
problem
 The terms commonly used by software
testers
Testing Axiom

It’s impossible to Test a Program
Completely
 Four key reasons
 The number of possible inputs is very large
 The number of possible outputs is very large
 The number of paths through the software is
very large
 The software specification is subjective. You
might say that a bug is in the eye of beholder

Software Testing Is a Risk-Based
Exercise
 Not to test every possible test scenario
 Customer will eventually find it someday
 Dilemma
 Testing vs. Release deadline
 Stop testing vs. Costly bug
 Key concept
 Reduce the huge domain of possible tests
into a manageable set
 Make wise risk-based decision on what’s
important to test and what’s not

Optimal Amount of Testing
Number of
Missed Bugs
Cost of Testing
Optimal Amount
of Testing
Under Testing
Over Testing
Quantity
Amount of Testing

Testing Can’t Show That Bugs Don’t
Exist
 Bugs in the House
 Testing
 Can show that bugs exist
 Can’t show that bugs don’t exist
 What can you do?
 Test more and find more

The More Bugs You Find, the More
Bugs There Are
 The nature of real bug and software bug:
 Both types tend to come into groups
 “bugs follow bugs”
 Several reasons
 Programmers have bad days
 Programmers often make the same mistake
 Some bugs are really just the tip of the iceberg

The Pesticide Paradox
 The more you test software, the more
immune it becomes to your test
 Spiral model example
 How to overcome
 Write new and different tests to exercise
different parts of the program and find more
bugs

Not All Bugs You Find Will Be Fixed
 Reasons
 There’s not enough time
 It’s really not a bug
 Misunderstanding, test errors or spec changes
 Its’ too risky to fix
 Make a bug fix that causes other bugs to appear
 It’s just not worth it

When a Bug’s a Bug Is Difficult to say
 There’s problem in the software but no one
ever discover it
 Is It a bug????
 Obey the rules to define bugs in Chapter 1
 You can’t claim that a bug exists if you didn’t
see it
 Bugs that are undiscovered or haven’t yet
been observed are often referred to as latent
bugs

Product Specification Are never Final
 Constantly changing product specification
 Fast moving industry
 Longer development schedule
 The only invariance is the variance of the
requirement (specification)

Software Testers Aren’t the Most
Popular Members of a Project Team
 Goal of software tester
 Find bugs, find them as early as possible, and
make sure they get fixed
 Keep peace with your fellow team member
 Find bug early
 Temper your enthusiasm
 Don’t just report bad news

Software Testing Is a Disciplined
Technical Profession
 Professional software tester are mandatory
 Core, vital members of the staff
 Career choice
Software Testing Terms and
Definitions

Precision and Accuracy
 Neither accurate nor precise
 Precise, but not Accurate
 Accurate, but not Precse
 Accurate and Precise

Verification and Validation
 Verification
 Process confirming that something-softwaremeets
its specification
 Validation
 The process confirming that it meets the
user’s requirement
 Example: Hubble telescope

Quality and Reliability
 Quality
 The customer feel that the product is excellent
and superior to his other choices
 Reliability
 One aspect of quality
 How often the product crashes or trashes his
data

Testing and Quality Assurance (QA)
 The goal of software tester is to find bugs,
find them as early as possible, and make
sure they get fixed
 A software quality assurance person’s main
responsibility is to create and enforce
standards and methods to improve the
development process and to prevent bugs
from ever changing