Chapter 7 Testing the Software with X-Ray Glasses

Testing the Software
with X-Ray Glasses
Chapter 7
Highlights
 What dynamic white-box testing is
 The difference between debugging and
dynamic white-box testing
 What unit and integration testing are
 How to test low-level functions
 The data areas that need to be tested at a
low level
 How to force program to operate a certain
way
 What different methods you can use to
measure the thoroughness of your testing
Dynamic White-Box Testing
 Definition
• Using information you gain from seeing
what the code does and how it works to
determine what to test, what not to test,
and how to approach the testing
• Structural testing
 Calculator example
Dynamic White-Box Testing (2)
 Four areas of dynamic white-box testing
• Directly testing low-level functions, procedures,
subroutines, or libraries
• Testing the software at the top level, as a
completed program
• Gaining access to read variables and state
information from the software to help you
determine whether your tests are doing what
you thought
• Measuring how much of the code and
specifically what code you hit when you run
your tests
Dynamic White-Box Testing Versus
Debugging
 Both involve dealing with software
bugs and looking at the code
 Goals
• Dynamic white-box testing
 To find bugs
• Debugging
 Fix bugs
Dynamic White-Box Testing Versus
Debugging (2)
Dynamic White-Box
Testing
Isolating the Bug Debugging
Testing Programming
Testing the Pieces
 Why dynamic black-box testing is not
enough?
• It’s difficult and sometimes impossible
to figure out exactly what caused the
problem
• Some bugs hide others
 Never have it happen in the first
place
Unit and Integration Testing
 Code is built and tested in pieces and
gradually put together into larger and
larger portions
 Process --- incremental testing
• Unit testing or module testing
• Integration testing is performed against groups
of modules
• Continues
• Test the entire product --- system testing
Unit and Integration Testing (2)
Main
ABC
BC
A B C D E F
DEF
Incremental Testing
 Bottom up and top-town
 Example
Temperature
Display Module
Temperature
Display Module
Thermometer
Interface Module
Thermometer
Interface Module
An Example of Module Testing
 atoi() function
• ASCII to Integer
• A bottom module in the program
• Called by others but not call others
 How
• Write test driver to send test strings to the atoi( )
function, read back the return values and compare them
with expected results
• Analyze the specification to decide what black-box test
cases should be tried and then apply equivalence
partitioning to reduce total set
• Use white-box knowledge of the module to add or
remove test cases
 Creating black-box testing cases based on the
specification before white-box cases
Data Coverage
 Data and states
 Map the white-box information to the
black-box cases
Data Flow
 Tracking a piece of data completely
through the software
• Individual module or function
• Integrated modules
• Entire software product
 How
• Use a debugger and watch variables to
view the data as program runs
Sub-Boundaries
 Every piece of software will have its
own unique sub-boundaries
• Computes taxes
• OS running low on RAM
• Number precision problem
 Examine the code carefully to look
for sub-boundary conditions and
create test cases that will exercise
them
Formulas and Equations
 Formulas and equations are buried
deep in the code
 Compound interest example
Error Forcing
 Use the debugger to force error
 Make sure you aren’t creating a
situation that can never happen in
the real world
 A great way to use error forcing is to
cause all the error messages in your
software to appear
Code Coverage
 Code coverage testing
• Enter and exit every module
• Execute every line of code
• Follow every logic and decision path
 Simplest form
• Single-step through the program using
debugger to view lines of code
 Code coverage analyzer
• A function, a line of code, or logic decision is
executed, the analyzer records the information
Code Coverage (2)
 What parts of the software your test
cases don’t cover
 Which test cases are redundant
 What new test cases need to be
created for better coverage
 You will also have a general feel for
the quality of the software
Program Statement and Line
Coverage
 The most straightforward form of
code coverage
 But, it can’t tell you if you’ve taken
all the paths through the software
Branch Coverage
 Path testing
• Attempting to cover all the paths in the
software is called path testing
• Simplest form
 Branch coverage testing
 Most code coverage analyzers give
you report both statement coverage
and branch coverage
Condition Coverage
 Condition coverage testing
• Takes the extra conditions on the
branch statements into account
 Example