Examining the Specification Part 2: Testing Fundamentals

Examining the Specification
Part 2: Testing Fundamentals
Chapter 4

Highlights
 What is black-box and white box testing
 How static and dynamic testing differ
 What high-level techniques can be used for
reviewing a product specification
 What specific problems you should look for
when reviewing a product specification in
detail

Getting Started
 Product specification
 Written document using words and pictures to
describe the intended product
 Calculator example
 Reason
 Programmer’s idea != User’s idea
 Thoroughly describe the product in a specification
 Document is testable

Black-box and White Box Testing
 Black-box testing
 Tester only knows what the software is
supposed to do
 Doesn’t know how
 White-box testing
 Access to program’s code and can examine
the program
 Know how
 Risk: tailor tests to match code’s operation

Static and Dynamic Testing
 Static testing
 Testing something that’s not running
 Examining and reviewing
 Dynamic testing
 Normal testing
 Running and using
 Car testing example

Static Black-Box Testing:
Testing the Specification
 Static --- document review
 Know extraction process?
 No
 Varieties of specifications
 Words
 Diagrams
 Self-documenting computer language
 What if no spec?
 Walking, talking
 Test “mental” specification

Performing a High-Level Review of the
Specification
 Process of requirements elicitation is an
inexact science and is prone to having
problems
 Research from high level
 Large fundamental problem
 Oversights
 Omissions

Pretend to Be the Customer
 Research who the customer will be
 Who will be using the software?
 Understand customer’s expectations
 Gaining some familiarity with the field
 Tip
 Don’t forget software security
 It must be specified
 User will assume that the software is safe

Research Existing Standards and
Guidelines
 Corporate Terminology and Conventions
 Industry Requirement
 Government Standards
 Graphical User Interface (GUI)
 Security Standards
 Test instead of making decision

Review and Test similar Software
 Scale
 Fewer or greater features?
 Less or more code
 Matter in your testing?
 Complexity
 More or less complex
 Will this impact your testing
 Testability
 Resources, time, and expertise to test?
 Quality/Reliability
 Representative of the overall quality planned for your software?
 More or less reliable
 Security
 Secure?
 Advertised and actual
 Compare to what you will be offering

Review and Test similar Software (2)
 Tip
 Reading online
 Collect review and articles about the
competition
 Especially helpful for security issues

Low-Level Specification Test
Techniques
 Specification Attributes Checklist
 Specification Terminology Checklist

Specification Attributes Checklist
 Complete
 Accurate
 Precise, Unambiguous, and Clear
 Consistent
 Relevant
 Feasible
 Code-free
 Testable

Specification Terminology Checklist
 List of problem words
 The appearance of these words often signifies
that a feature isn’t yet completely thought out
 It likely falls under one of the preceding
attributes

Specification Terminology Checklist (2)
 Always, Every, All, None, Never
 Certainly, Therefore, Clearly, Obviously,
Evidently
 Some, Sometimes, Often, Usually, Ordinarily,
Customarily, Most, Mostly
 Etc., And So Forth, And So On, Such As
 Good, Fast, Cheap, Efficient, Small, Stable
 Handled, Processed, Rejected, Skipped,
Eliminated
 If…Then…(but missing Else)