All software needs to be tested. In fact, software testing is
a major part of the overall software development process that
involves many people and countless hours of detailed work.
Unfortunately, most testing efforts are under-planned: some
software testing professionals work in the field for years
without ever seeing a really comprehensive QA plan and test
suite. Part of the problem is that QA efforts often begin too
late in the release cycle when there is too much pressure to take
shortcuts.
This white paper shows how ReadySET Pro can be used to quickly
create a comprehensive system test suite with test cases.
ReadySET Pro's unique content-rich templates help you write
better test suites that can improve your product quality. These
templates make the writing process much faster than starting from
scratch, which means that you are more likely to be able to
complete your plan, even under time pressure.
The figure below illustrates where your QA plan and test suite
fit with other project documents. This white paper focuses on
the yellow "Quality Assurance" box. Ideally, your testing
documents are just part of an overall set of project documents.
But even if you do not have the others, you will be able to
follow the discussion of test planning documents.
→
→
Structural Design
Behavioral Design
User Interface
Build System
Architecture
Persistence
Security
Target Market Segment
Claimed Customer Benefits
→
Classes of Users
User Stories
→
→
QA Plan
Test Suite
Test Cases
Test Runs
Requests from Customers
→
Software development projects that don't have enough test planning tend
to bog down with defects that can put the entire project's success at
risk. Test planning helps in the following specific areas:
Requirements Validation
Designing a system test suite forces you to deeply understand
the requirements. As you understand the requirements more, you
will notice incompleteness, ambiguity, and inconsistency.
Correcting these problems early can speed up development and reduce
the number of late requirements changes.
Testing Coordination
Testing involves many people working together over time. For
the team to be effective, their efforts must be coordinated with a
written plan.
Test Coverage
Testing only half of a large system is sure to allow thousands
of defects into the shipping product. A QA plan is needed to set
coverage criteria and evaluate coverage. A test suite must be
carefully designed with the coverage criteria in mind.
Test Automation
Too often, QA teams hope to use automated testing, but end up
stuck with ad-hoc manual testing. This happens because they never
really formalize the requirements, so they must always rely on
human judgment to evaluate test outputs. Creating automated test
scripts without outlining the test suite is like writing code
without a design document. The following diagram illustrates the
gap between ad-hoc testing and automated testing, and how
systematic testing with a test suite bridges that gap.
Just see if you can break it
Make up test cases "on the fly"
Human interpretation of requirements
→ → →
Driven by explicit QA goals
Test suite designed for coverage
Tests specify expected output
→ → →
Driven by explicit QA goals
Test suite designed for coverage
Scripts need no human judgment
The rest of this white paper works through the steps shown in the
diagram below. (You may notice that this is very similar to the use case writing steps.)
1: Overall QA Planning
↓
2: Outline the Test Suite
↓
3: List Test Case Names
↓
4: Write Some Test Case Descriptions
↓
5: Write Selected Test Cases
↓
6: Evaluate Test Cases
Note that in steps 4 and 5, we recommend that you only specify
the most important test cases in detail. In any complex system,
there will be a large number of potential test cases. We
encourage you to take a breadth-first approach: map out your test
case suite first, then fill in details incrementally as needed.
This concept is key to getting the most value out of the limited
time that you have for test planning.
Step One: Overall QA Planning
Software quality is not one-size-fits-all: different software
products need different types of testing because they have
different QA goals. For example, a real-time system may place
much more priority of performance than would a typical desktop
business application.
The task of QA planning is discussed in detail in the "Quality Throughout the Life-Cycle" white
paper. The main parts of the overall QA plan are:
Select and prioritize quality goals for this release
Select QA activities to achieve those goals
Evaluate how well the activities support the goals
Plan the actions needed to carry out the activities
The overall QA plan addresses all quality activities. Quality
can be achieved by building in better quality from the start, and
by testing to find and remove defects. Specific QA activities
include: coding preconditions, reviewing design and code, unit
testing, integration testing, system testing, beta testing, using
analysis tools, and field failure reports, among others. The
rest of this paper will focus in on just the system testing
activity.
ReadySET Pro helps with every part of the overall QA plan.
The ReadySET Pro QA plan
template is pre-populated with detailed goals, activities,
evaluation, a plan of action, and a built-in checklist.
Adapting the QA plan to your project is largely a matter of
"chipping away." That means, selecting the sample answers that
fit best rather than writing new text. To see how fast and easy
chipping away can be, see the Step-By-Step Demo.
ReadySET Pro users have reported that the QA plan template
alone saved them an average of three hours. More importantly, a
good QA plan can save many more hours later by reducing the
number of defects. Use the ROI
Calculator to see how the savings for the QA plan, test
cases, and other documents really add up to save days or weeks
of your project schedule.