sss ssss rrrrrrrrrrr
ssss ss rrrr rrrr
sssss s rrrr rrrr
ssssss rrrr rrrr
ssssssss rrrr rrrr
ssssss rrrrrrrrr
s ssssss rrrr rrrr
ss sssss rrrr rrrr
sss sssss rrrr rrrr
s sssssss rrrrr rrrrr
+===================================================+
+======= Quality Techniques Newsletter =======+
+======= July 2001 =======+
+===================================================+
QUALITY TECHNIQUES NEWSLETTER (QTN) is E-mailed monthly to Subscribers
worldwide to support the Software Research, Inc. (SR), TestWorks,
QualityLabs, and eValid user communities and other interested parties to
provide information of general use to the worldwide internet and
software quality and testing community.
Permission to copy and/or re-distribute is granted, and secondary
circulation is encouraged by recipients of QTN provided that the entire
document/file is kept intact and this complete copyright notice appears
with it in all copies. Information on how to subscribe or unsubscribe
is at the end of this issue. (c) Copyright 2003 by Software Research,
Inc.
========================================================================
Contents of This Issue
o eValid Ver. 3.0 Released with Many New Features
o The Vulnerabilities of Developing on the Net, Robert A. Martin
(MITRE Corporation) (Part 2 of 2)
o QWE2001: 5th Internet and Software Quality Week Europe
o SERG Report: Interpreting Data Network Language on a Single
Processor, by Mr. Feng Cui
o eValid -- Power User Tips & Tricks
o A Handbook to Test Strategies, by Shivakumar Balasubramaniyan
o QTN Article Submittal, Subscription Information
========================================================================
eValid Ver. 3.0 Released with Many New Features
We're proud to announce major new features in eValid Ver. 3.0: FREE
Page Metrics PopUp, Comprehensive SiteMap , Applet Coverage, Automatic
Test Script Generation, and more!
A complete description of eValid Ver. 3.0 is at:
<http://www.soft.com/eValid/Products/Documentation.30/release.3.0.html>
* FREE Page Metrics PopUp: Any time you use the eValid browser you can
access the eValid Page Metrics PopUp. You'll get complete details
about each page you download, including total download and rendering
time -- instantly and automatically. It's FREE!
* eVsitemap Function: From any one URL eValid can map the entire set of
pages referenced from that page -- to any depth. You can analyze an
entire WebSite to see if there any pagest that are slow-loading (you
specify the time limit), too big (you specify the size), broken or
unavailable (even on dynamically generated pages), etc. You can even
search a WebSite for string matches in pages. All accomplished
entirely from the user's perspective -- from the eValid browser!
* eVgenerate Function: There's a new capability to generate test script
data from a values table and a parameterized script file. If you want
to test a WebSite by signing up 1000's of users with random names and
passwords, eVgenerate can do it!
* eVcoverage Feature: You can measure coverage your tests achieve on a
Java Applet as it runs in the eValid browser. TCAT/Java does the
instrumentation and the test coverage reporting; eValid's eVcoverage
option does the data collection.
* Application Mode Recording: eValid now records and plays activity in
any application that opens in the eValid browser, e.g. Adobe Acrobat,
Java Applications, MS-Word, etc.
* Extended Validation Mode: You can now validate any text from anywhere
on any browser page that you can copy to the clipboard.
* Download Location: Get the new version of eValid from:
<http://www.soft.com/eValid/Products/Download/down.evalid.html>
Note: Existing eValid Ver. 2.n keys will not work with eValid Ver.
3.0. Send email to for replacement keys.
* Supported Platforms: eValid is supported on Windows NT/2000. Some
eValid scripts that work perfectly on these platforms may have some
problems if you are running Windows 98. eValid scripts that work
perfectly on these platforms may have significant problems if you are
on Windows 95.
Complete information including details on how to obtain from
<http://www.soft.com/eValid> or <http://www.e-valid.com>.
========================================================================
The Vulnerabilities of Developing on the Net (Part 2 of 2)
Robert A. Martin
The MITRE Corporation
(ramartin@mitre.org)
The Tower of Babel
In 1998 if you tried to use these various tools, services, and databases
you were faced with a problem rooted in each ones heritage. Each had
developed its own naming standards and methods for defining individual
entries in their respective vulnerability data stores. Table 5 shows how
the same vulnerability was referred to by 12 different names by 12
leading organizations. With such confusion, it was very hard to
understand what vulnerabilities were faced, and what vulnerabilities
were or were not being looked for by each tool. Then the vulnerability
or exposure still had to be mapped to the software vendor's name for the
problem to get a fix.
-------------------------------------------------------------------
Organization Name used to refer to vulnerability
-------------------------------------------------------------------
AXENT phf CGI allows remote command execution
BindView #107 - cgi-phf
Bugtraq PHF Attacks - Fun and games for the whole family
CERIAS http_escshellcmd
CERT CA-96.06.cgi_example_code
Cisco Systems HTTP - cgi-phf
CyberSafe Network: HTTP "phf" Attack
DARPA 0x00000025 = HTTP PHF attack
IBM ERS ERS-SVA-E01-1996:002.1
ISS http - cgi-phf
Symantec #180 HTTP Server CGI code compromises http server
Security Focus #629 - phf Remote Command Execution Vulnerability
-------------------------------------------------------------------
Table 5. Vulnerability in the Tower of Babel
Driven by our own attempts to develop an integrated picture of what was
happening in our networks and in trying to select some new tools, The
MITRE Corporation started to design a method for working through the
confusion of vulnerability and exposure information. This method was
based on the creation of a reference list of unique names that would
then be mapped to the appropriate items in each tool and database. In
January 1999 the first public statement of our idea for a reference list
of unique names for vulnerabilities and exposures was presented at the
2nd Workshop on Research with Security Vulnerability Databases, held at
Purdue University. MITRE presented a paper [1] at this conference that
outlined the basic ideas and approach for what is today called the
Common Vulnerabilities and Exposures (CVE) initiative.
Our vision for CVE was to provide a mechanism for linking together
vulnerability-related databases or concepts -- and nothing more (see
Figure 1). Rather than viewing this narrow scope as a limitation, we
saw it as an advantage. By agreeing to limit the use of CVE to the role
of a logical bridge, we could avoid competing with existing and future
commercial efforts. This was important, since it was critical that
commercial organizations concur with the CVE concept and proceed to
incorporate the initiative into their various products and services.
-------------------------------------------------------------------
Software Vendor
Alerts
Vulnerability | IDS
Scanner DB=+ | +-Signature DB
| | |
CVE LIST
| | |
Security Advisories-+ | +-Software Vendor
& E-mails | Patches & Updates
Vulnerability
Web Sites &
Databases
-------------------------------------------------------------------
Figure 1. CVE List as a Bridge
By March 2001 the CVE effort had evolved into a, cross-industry effort
involving more than 30 occupations in creating and maintaining a
standard list of vulnerabilities and exposures. Almost half of the known
vulnerabilities and exposures are either listed or under review, and
presently 29 organizations are building nearly 50 products or services
that use CVE names as a key element of their functionality.
How CVE Works
The CVE initiative is an international community activity focused on
developing a list that provides common names for publicly known
information security vulnerabilities and exposures. The CVE list and
information about the CVE effort are available on the CVE Web site at
<http://cve.mitre.org/cve/.>
The common names in CVE result from open and collaborative discussions
of the CVE editorial board. This board, as shown in Table 6, includes
members from numerous information security-related organizations around
the world, including commercial security tool vendors, members of
academia, research institutions, government agencies, and other
prominent information security experts. The board identifies which
vulnerabilities or exposures will be included in CVE, then determines
the common name, description, and references for each entry. The CVE
name, for example CVE-1999-0067, is an encoding of the year that the
name was assigned and a unique number N for the Nth name assigned that
year.
-------------------------------------------------------------------
Area of Expertise Organizations
Academic/Educational UC Davis, SANS, CERIAS
Network Security Analysts Vista IT, Genuity
Other Security Experts IBM Research, MITRE,
Zero-Knowledge
Intrusion Detection Experts Silicon Defense, SANS
Tool Vendors The Nessus Project, ISS, PGP
Security-Network Associates,
BindView, AXENT, CyberSafe,
Symantec, NFR, Hiverworld,
Harris, Cisco
Software Vendors IBM, Sun Microsystems, Microsoft
Incident Response Teams CanCERT, CERT/CC, DOD-CERT
Information Providers NTBugtraq, Security Focus,
NIST, Ernst & Young,
eSecurityOnline.com
-------------------------------------------------------------------
Table 6. CVE Editorial Board Composition
MITRE maintains the CVE list and Web site, moderates editorial board
discussions, and provides guidance throughout the process to ensure that
CVE remains objective and continues to serve the public interest.
Archives of board meetings and discussions are available for review on
the CVE web site at <http://cve.mitre.org/board/archives/>.
Other information security experts are invited to participate on the
board on an as-needed basis, based upon recommendations from board
members. The key tenets of the CVE initiative are:
o One name for one vulnerability or exposure.
o One standardized description for each vulnerability or exposure.
o Existence as a dictionary rather than a database.
o Publicly accessible for review or download from the Internet.
o Industry-endorsed via the CVE editorial board and CVE-compatible
products.
What Does CVE-Compatible Mean?
CVE-compatible is a phrase that indicates that a tool, Web site,
database, or service uses CVE names in a way that allows for a cross-
link with other repositories that use CVE names. To be CVE-compatible,
the product, service, database, or Web site must meet the following
three requirements:
o CVE Searchable: A user can search using a CVE name to find related
information.
o CVE Output: Information is presented that includes the related CVE
name(s).
o Mapping: The repository owner has provided a mapping relative to a
specific version of CVE, and has made a good faith effort to
ensure accuracy of that mapping.
Different products and repositories address different portions of the
complete CVE list. For example, some might deal with UNIX, while others
cover Windows NT. When looking at CVE-compatible items, you will need to
evaluate them against your organization's specific needs in terms of
platforms coverage and the software products that you use.
Why Use CVE-Compatible Products?
CVE compatibility allows you to use your vulnerability databases and
tools together since they can "talk" to each other through shared CVE
names. For example, if a report from a vulnerability scanning tool
incorporates CVE names, you can quickly and accurately locate fix
information in one or more of the separate CVE-compatible databases and
Web sites to determine how to fix the problems identified by the
vulnerability scanner. Also, with CVE-compatible tools, you will know
exactly what each tool covers because the CVE list provides a baseline.
Simply determine how many of the CVE entries are applicable for your
platforms, operating systems, and commercial software packages, and use
this subset to compare against the tool's coverage. Before the use of
common names, it was extremely difficult to identify the vulnerabilities
of your systems7, or to determine whether a particular tool or set of
tools covered them.
Improving the Process
The CVE effort is changing the way organizations use security tools and
data sources to address their operational security posture. The example
organization in Figure 2 is able to detect an ongoing attack with its
CVE-compatible IDS system (A). In a CVE-compatible IDS, specific
vulnerabilities that are susceptible to the detected attack are provided
as part of the attack report. This information can then be compared
against the latest vulnerability scan by your CVE-compatible scanner (B)
to determine whether your enterprise has one of the vulnerabilities or
exposures that can be exploited by the attack. If it does, you can turn
to a CVE-compatible fix database at the software vendor or you can use
the services of a vulnerability Web Site like ICAT Metabase8, which lets
you identify (C) the location of the fix for a CVE entry (D), if one
exists.
----------------------------------------------------------------------
________________
| IDS Signature |
______________|_ DB |
| CVE compatible |-------+------ _______________
|----------------| | _|_____________ |
| Intrusion | | _|_____________ | |<------+
| Detection | | | Application | |_| |
| System | v | Servers |_| |
---------------- | --------------- |
Attack || (A) | || |
-----> || | || |
================================================================== |
|| | || ____||___ |
|| ________v______ || _|_______ | |
(B) || | Vulnerability | || _|_______ | |<-+
_______||____|__ DB | || | Web | |- |
| Vulnerability |-----+------ ____||____ | Servers |- |
| Scanner | | _|________ | --------- |
| System | | _|________ | |<------------+------+
|----------------| | | Database | |- |
| CVE compatible | | | Systems |- |
---------------- | ---------- |
________v______ _______|_____
(C) | Vulnerability | (D) | SW Patches |
_____________|_ Cross-index | _____________|__ & Updates |
| Vulnerability |-----+------ | SW Application |----------
| Web Site | | | Vendor |
|----------------| +-----------> |----------------|
| CVE compatible | | CVE compatible |
---------------- ----------------
----------------------------------------------------------------------
Figure 2. A CVE-enabled process
Identifying Your Risk
Another thing you can accomplish with CVE-compatible products, that
would be hard if not impossible to do before common names were adopted,
is improve how your organization responds to security advisories. If the
advisory is CVE-compatible it will include CVE entries. With that
information you can see if your scanners check for these
vulnerabilities, and determine whether your IDS has appropriate attack
signatures for the alert.
Additionally, for systems that you build or maintain for customers, the
CVE compatibility of advisories and announcements will help you directly
identify any fixes from commercial software vendors in those systems (if
the vendor fix site is CVE compatible). This is a much more structured
and predictable process for handling advisories than most organizations
currently possess.
Making Guidance Actionable
Earlier this year, a group of concerned security professionals put
together a "Top 10" list [2] that outlined the most common, critical
Internet security threats. The effort was orchestrated by the System
Administration, Networking, and Security (SANS) Institute and brought
together a consensus list from a wide variety of security experts. To
help bring specificity and make the recommendations actionable, each of
the top 10 suggestions had the appropriate CVE names, detailing each of
the specific issue areas for a variety of platforms and products. A
total of 68 CVE names were called out in the list of 10 threats.
Who Is CVE-Compatible?
While the list of organizations with CVE-compatible products is
expanding, at this writing the vendors in Table 7 are those working
toward compatibility. For a current list visit the CVE web site at:
<http://cve.mitre.org/compatible/>.
-------------------------------------------------------------------
Advanced Research Corporation Internet Security Systems, Inc.
Alliance Qualite Logiciel Max Vision Network Security
/Whitehats
AXENT Technologies, Inc. NIST
BindView Development The Nessus Project
CERIAS/Purdue University Network Security Systems
CERT Coordination Center Network Security Wizards
Cisco Systems TBugtraq
Computer Security Lab, UC Davis PGP Security, Network Associates
CyberSafe Qualys
CYRANO SANS
Ernst & Young Security Focus, Inc.
Harris Corporation Security Watch
Hiverworld, Inc. Symantec
Intranode Tivoli Systems Inc.
Intrusion.com World Wide Digital Security
-------------------------------------------------------------------
Table 7. Organizations Developing CVE-Compitable Products
Today, there are several members of each type of tool, service,
repository, and announcement capability that support CVE names.
Underrepresented areas are vendor announcement and vendor fix sites;
however, several vendors are actively discussing adding CVE names to
their announcements. By the time you read this article there should be
several vendors using CVE names in their announcements and alerts. In
addition, like the CVE editorial board, the list of organizations
working on or delivering CVE-compatible products has become
international in scope.
Conclusion
The application of all known security fixes and patches is the
complement of standard security protection mechanisms. Keeping current
on fixes offers a robust method for keeping the commercial software that
makes up your organization's software infrastructure healthy.
Vulnerabilities and exposures will always be a part of our systems, as
will the groups that find and share information about vulnerabilities
and exposures in commercial software. With common name integration and
cross-referencing abilities emerging in vulnerability and exposure
tools, web sites, and databases, it is becoming possible to deal with
these mistakes and improve our systems' security. Handling security
incidences is more systematic and predictable as CVE is supported within
the commercial and academic communities. As vendors respond to user
requests for CVE-compatible fix sites, the complete cycle of finding,
analyzing, and fixing vulnerabilities will be addressed.
========================================================================
QWE2001: 5th Internet and Software Quality Week Europe
Mark your calendars now: 12-16 November 2001 in Brussels, Belgium,
Europe!
Dozens of Tutorials, a complete Technical Conference devoted to Internet
and Software Quality issues, with multiple technical tracks, QuickStart
tracks, vendor exhibits and presentations, plus Belgian special events.
Complete details a <http://www.qualityweek.com>.
========================================================================
SERG Report 396
"Interpreting Data Network Language on a Single Processor"
by Mr. Feng Cui
Abstract: There is such a large amount of varied information available
on the Internet that it is hard for a human to query this information
without efficient tools. Using structured relational data can greatly
improve our ability to search for data and data compute new information
across networks. The McMaster University Software Engineering Research
Group (McSERG) proposes to use binary relations to organize data on
networks and use binary relation operations to do information query. A
new language, Data Network Language (DNL) [15], has been developed by
McSERG to apply this idea. This thesis presents the design and
development of the Data Network Language Interpreter System, which is
the implementation of DNL. By means of examples, this thesis shows how
the use of DNL can improve the consistency and completeness in
information retrieve.
[15] X. Yan, Data Network Language Design, SERG Report 390, McMaster
University, Hamilton, Ontario, September 2000.
The web address for downloading reports is:
<http://www.crl.mcmaster.ca/SERG/serg.publications.html>
Email contact is Doris Burns
========================================================================
eValid -- Power User Tips & Tricks
Become an eValid Power User! The aim of this short note is to help you
use eValid to get better results, quicker, and easier! eValid's Power
User Tips & Tricks sent periodically to all eValid users, evaluators,
and consultants. The aim is to address one issue in eValid use that we
hope will help you to increase your effectiveness with eValid. And, to
help you improve the quality of your WebSite!
> Using the SiteMapping Feature of eValid 3.0: Checking Page Loading
Times
Keep your web site error free with the new eValid SiteAnalysis mode.
You can search your site for broken links, specify what objects to
search, which pages to load, how deep to conduct the test, and lots
more!
Using the SiteAnalysis feature in eValid will surely increase the
effectiveness of your site in terms of functionality.
Lets do a 2-Deep Site Analysis of a site using all the browser
protocols in a "Full Browser Mode", searching for Slow Loading pages..
(1) Click: eValid > Site Analysis Mode > Site Analysis Preferences.
This gets you to the SiteAnalysis Preferences Menu.
(2) Click and change the search Depth to "2".
(3) Click "Full Browser Mode" in the Control section.
(4) Choose "Never Use Cache" in the Cache Control section.
(5) Click: eValid > Site Analysis Mode > Site Analysis Preferences >
Define Filters. Then Choose "Pages Loading Slower Than" and
specify a time, e.g. 5000 milliseconds (5 seconds). Make sure
the other filters are "off".
(6) Starting mapping your entire WebSite to Depth 2 by clicking on
the Site Map icon or pressing F3.
Sit back, relax, and watch as eValid maps the WebSite for you, and
advises you of slow loading pages as it finds them! Remember, the
eValid browser shows exactly what your users would see!
> Download the latest version of eValid from:
> To see other eValid Power User Tips & Tricks go to:
<http://www.soft.com/eValid/Products/Tips/index.html>
If you need a fresh eValid EVAL key, contact .
+-----------------------------------+----------------------------------+
| eValid Support Team | Phone: [+1] (415) 861-2800 |
| eValid, Inc. | Tollfree (USA): 1-800-942-SOFT |
| 1663 Mission Street, Suite 400 | FAX: [+1] (415) 861-9801 |
| San Francisco, CA 94103 USA | Email: evalid@soft.com |
| | WWW: http://www.e-valid.com |
+-----------------------------------+----------------------------------+
========================================================================
A Handbook to Test Strategies
by
Shivakumar Balasubramaniyan
Email: shiva@thinkbn.com
What A Strategy Really Is?
Strategies are simple but executing them is not! It might involve
giving up a destructive habit or addiction or it might be a matter of
recognizing a self-defeating pattern of behavior and somehow seeing how
to change it. For example the most of testing hours of a tester goes in
to testing the product again and again for different releases in the
same pattern. At a certain phase it leads to frustration of the testers
for doing things repeatedly. It happens sometimes that when someone is
frustrated he may sound something like, "I know its possible to see this
(or experience) differently". And often, this awareness or even hope
that there is another way of experiencing your dilemma, or problem opens
the door for it to occur. This is where strategies come in. Thus a
strategy in its synonym is a preparation for long-term battle plans or
making plans to achieve a goal
Introduction
Strategic Planning is a process to guide the members to envision the
future and develop the necessary procedures and operations to achieve
that future. The purpose of Test strategies helps us to:
* Provide a framework and a focus for improvement efforts
* Provide a means for assessing progress.
This strategic approach to testing will forecast the action plans which
includes the different types of testing that would be followed in the
testing life cycle, identifying risk issues etc. earlier so that
progress can be evaluated more precisely. The development of a test
strategy is a means of communication with the customer on the
organization of testing and the strategic choices that go with it. The
test strategy indicates how testing is to be carried out. This will
clearly picture where much emphasis on various aspects of the system has
to be given so that best possible use of resource and time can be made
use of. And also the test strategy forms an important basis for a
structured approach to testing which makes the testing process
manageable.
Why Do We Need a Test Strategy?
When test cases are written for testing the whole of the product, which
is intended to unearth all defects, why do we need a test strategy?
These test cases will say what will be tested and a test strategy will
say how this will be tested. Hence a test strategy aims at finding the
most important errors at any early stage involving minimum costs. A
more unusual approach but useful approach will be developing a test
strategy based on risks. Because a test strategy will be fruitful
provided if all the risks are identified at an early stage. This means
that assessing the damages due to the defects, which are left undetected
prior to operation and during operation. Consider the following,
Risk = Chance of Failure x Damage
So when the chance of failure is high and when the damage is more, risk
is definitely high. The chance of failure can be due to frequency of use
and the chance of error. Broadly classifying the areas in these
respective areas would give a good perception of all possible chances of
failure
Frequency of Use: The frequency of use determines the amount of risk
because if a function is used dozens of times each day the chance of an
failure is much bigger than with a function used once a year.
Chance of Error: The chance of error can be due to
1. Complex functions involved
2. Completely new functions involved
3. Functions for which certain tools or techniques were employed for
the first time
4. Inexperienced developers
5. Insufficient quality assurance during development
6. Function with many interfaces
These are some factors not all, which determine the chance of an error.
Naturally this amounts to the high chance of failure, which thereby
involves high risk.
Damage: Lets see what will be the damage when the error manifests
itself. The damage will be the
1. Cost of repair for these error,
2. Forgone income,
3. Loss of customer or customer confidence.
How to Avoid This?
It is impossible to asses risk in detail hence it is not the job of the
test manager alone to asses risk. Therefore a large number of people
should contribute to this like the customer, users, and development team
etc. This not only increases the quality of the strategy but also has
the advantage that the different parties are aware of the risks and the
extent to which testing would contribute to make this testing
manageable.
To summarize, the main aim of test strategy is to see that the test will
be organized in such a way that:
1. The most important problems will be found earlier.
2. The problems will be found much earlier.
3. The problem that requires much amount of rework will be identified.
4. Efficient use of resources are made
Establish Test Techniques
The final step in a test strategy involves the selection of the
appropriate test specification techniques that will be used to test the
system. Choosing the techniques should take into account various
factors, some of which are listed below:
* Quality Characteristic to be Tested
A technique is fit for testing one or more quality characteristics.
Some quality characteristics can best be tested with one (set of)
techniques, others with another one.
* Area of Application
Some techniques are specifically suitable for testing the
interaction (screens, reports) between the system and the users;
others are better in testing the performance of systems (web
portals).
* Use of Resources
The application of a technique requires a specific amount of
resources, in terms of man capacity as well as machine capacity. For
example the testing can be done on cross platforms like Windows,
UNIX etc. And there are various applications in communication
systems, which interact with certain devices like RDT (Remote
Digital Terminal), USX1000 etc. Thus these resources have to be made
available in advance prior to testing.
* Required Knowledge and Skills
Not each tester is equipped for each technique. For the useful
application of some techniques much business knowledge is needed.
For other techniques more analytic talent is required. Therefore,
the knowledge and skills of the test staff also influences the
choice for techniques. The selection of the test specification
techniques should be done in an early stage of the test process, for
then the test team can take the appropriate actions in training for
techniques and the necessary checklists can be made or adjusted for
the specific situation. The aim of this is to have the most
important tests take place as early as possible.
Using the test strategy as a basis the test manager may discuss with the
customer which tests can be dropped or where less thorough testing can
be done. By indicating which parts are to be tested less in relation to
the risks assessed (translated into importance levels in the strategy),
the test manager can report in a solid fashion on the increased risks
after the testing phase. The following are some of the key areas that
should be addressed when designing a test strategy,
Software Risk Issues
The purpose of discussing software risk is to determine what the primary
focus of testing should be. Generally speaking, most organizations find
that their resources are inadequate to test everything in a given
release. Outlining software risks helps the testers rank what to test,
and allows them to concentrate on those areas that are likely to fail or
have a large impact on the customer if they do fail. Features to Test
This is a listing of what will be tested from the user or customer point
of view. For example if you were testing an Automated Teller Machine
(ATM), feature to be tested will include Password Authentication,
Deposit/Withdraw Money etc. This will give a brief overview to the
testers of the number of major functionalities or features in a product,
which increase the depth of testing on these features. Hence pen down
all the major features or functionalities that would have major setback
on the customer when it is not properly tested. Features Not Tested
This section will record any features that will not be tested and why.
There are many reasons that a particular feature may not be tested, for
example you need to test a condition where the number of subscribers
generating a call in a network would be 22,000. This may not always be
feasible when the product is developed offshore where these
infrastructures may be lacking. But whatever the reason a feature is
listed in this section it all boils down the risk relatively.
Test Approach
The Test Strategy presents the recommended approach to testing of
software applications. This approach should contain a description of how
testing will be done and discuss any issues that have a major impact on
the success of the testing and ultimately of the project. This section
will throw light on various testing techniques and the objectives behind
the use of this technique in testing.
For example:
* A Sanity Testing or a Business Cycle Testing validates basic
activities the system needs to perform whenever each release is made
* A User Interface Testing will ensure that the User Interface
provides the users with the appropriate access and navigation
through the functions of the application. In addition this testing
should ensure that the User Interface should conform to corporate or
industry standards.
* Performance Testing will ensure that application is designed to test
the run time performance of software within the context of an
integrated system.
These are some examples and hence the appropriate testing techniques
that would be best suited for the application has to be foreseen by the
testing team so that a worth wile testing could be established.
Another strategy issue that should probably be addressed is the use of
tools and automation. Testing tools can be a boon to the development and
testing staff, but they can also spell disaster if their use is not
planned. Using some types of tools can actually require more time to
develop, implement and run a test set the first time than there would be
if the tests were run manually.
Training Needs
A testing team may involve mixture of testers who may or may not have
adequate knowledge on the domain or to domain context. Hence training
becomes essential and example of trainings might include use of test
tools, Testing methodologies, Management systems like Defect Tracking,
Configuration Management and use some third party tools like simulator,
which will assist in the testing life cycles. Responsibilities
This can be a matrix that quickly shows the major responsibilities such
as Establishment of test environment, Configuration Management etc.,
that are associated with the human resources who are nothing but the
testing team. Planning Risk Contingencies
Many organizations announce their commitment to quality but a closer
look at those will show that their only true commitment is towards
schedule. Most of us have taken part in projects where the schedule is
at best ambitious and at worst impossible. Let me describe a scenario,
Your VP has promised the next release of your product on a certain date.
The date seems aggressive with the available resources. Suppose one
Programmer Mr.X falls sick and thereby leaving a large gap in your
knowledge base and now an ambitious schedule will now be a mission
impossible. What are your choices or contingencies?
1. Alter the schedule
2. Reduce the scope
3. Reduce the quality (this means reducing testing thereby allowing
more hidden defects)
4. Add resources (but there are none since everyone is already working
around the clock)
5. Punt!.
All the above choices seem bad. Hence this section will identify the
risks and help us to focus and prioritize the testing to reduce them.
Conclusion
The testing of information systems should be based on the business
risks, which the organization will experience in using these information
systems. In practice, test managers should take the steps to come from
risks and test the coverage in an intuitive manner. Good risk assessment
is a part of these steps. In practice, the discussion of risks and
related testing strategies in this way proves to be a real eye-opener
for the concerned. It also enables negotiation of testing depth by
having the customer decide which elements should be tested how
thoroughly. The stepwise definition of the test strategy can also be
used for any test level (e.g., system test, acceptance test). The result
of such a test strategy gives a better insight to everyone on how the
testing will be done.
Establishing a Test Strategy is difficult but it should be borne in mind
that all things are difficult before they are easy. Hence:
"Do not follow where the path may lead.
Go instead where there is no path, and leave a trail"
Thus this idea of establishing a test strategy is to come out of the
conventional way of testing which is tedious and tiresome whereby a
testing which has a strategy will not only be challenging but is worth
doing.
========================================================================
------------>>> QTN ARTICLE SUBMITTAL POLICY <<<------------
========================================================================
QTN is E-mailed around the middle of each month to over 9000 subscribers
worldwide. To have your event listed in an upcoming issue E-mail a
complete description and full details of your Call for Papers or Call
for Participation to .
QTN's submittal policy is:
o Submission deadlines indicated in "Calls for Papers" should provide at
least a 1-month lead time from the QTN issue date. For example,
submission deadlines for "Calls for Papers" in the March issue of QTN
On-Line should be for April and beyond.
o Length of submitted non-calendar items should not exceed 350 lines
(about four pages). Longer articles are OK but may be serialized.
o Length of submitted calendar items should not exceed 60 lines.
o Publication of submitted items is determined by Software Research,
Inc., and may be edited for style and content as necessary.
DISCLAIMER: Articles and items appearing in QTN represent the opinions
of their authors or submitters; QTN disclaims any responsibility for
their content.
TRADEMARKS: eValid, STW, TestWorks, CAPBAK, SMARTS, EXDIFF,
STW/Regression, STW/Coverage, STW/Advisor, TCAT, and the SR logo are
trademarks or registered trademarks of Software Research, Inc. All other
systems are either trademarks or registered trademarks of their
respective companies.
========================================================================
-------->>> QTN SUBSCRIPTION INFORMATION <<<--------
========================================================================
To SUBSCRIBE to QTN, to UNSUBSCRIBE a current subscription, to CHANGE an
address (an UNSUBSCRIBE and a SUBSCRIBE combined) please use the
convenient Subscribe/Unsubscribe facility at:
<http://www.soft.com/News/QTN-Online/subscribe.html>.
As a backup you may send Email direct to as follows:
TO SUBSCRIBE: Include this phrase in the body of your message:
subscribe
TO UNSUBSCRIBE: Include this phrase in the body of your message:
unsubscribe
Please, when using either method to subscribe or unsubscribe, type the
exactly and completely. Requests to unsubscribe that do
not match an email address on the subscriber list are ignored.
QUALITY TECHNIQUES NEWSLETTER
Software Research, Inc.
1663 Mission Street, Suite 400
San Francisco, CA 94103 USA
Phone: +1 (415) 861-2800
Toll Free: +1 (800) 942-SOFT (USA Only)
Fax: +1 (415) 861-9801
Email: qtn@soft.com
Web: <http://www.soft.com/News/QTN-Online>