sss ssss rrrrrrrrrr
ssss ss rrrr rrrr
sssss s rrrr rrrr
ssssss rrrr rrrr
ssssssss rrrr rrrr
ssssss rrrrrrrr
s ssssss rrrr rrrr
ss sssss rrrr rrrr
sss sssss rrrr rrrr
s sssssss rrrrr rrrrr
+===================================================+
+======= Testing Techniques Newsletter (TTN) =======+
+======= ON-LINE EDITION =======+
+======= February 1995 =======+
+===================================================+
TESTING TECHNIQUES NEWSLETTER (TTN), On-Line Edition, is E-Mailed
monthly to support the Software Research, Inc. (SR) user community and
provide information of general use to the world software testing commun-
ity.
(c) Copyright 1995 by Software Research, Inc. Permission to copy and/or
re-distribute is granted to recipients of the TTN On-Line Edition pro-
vided that the entire document/file is kept intact and this copyright
notice appears with it.
TRADEMARKS: Software TestWorks, STW, STW/Regression, STW/Coverage,
STW/Advisor, X11 Virtual Display System, X11virtual and the SR logo are
trademarks of Software Research, Inc. All other systems are either
trademarks or registered trademarks of their respective companies.
========================================================================
INSIDE THIS ISSUE:
o STW AT TOOLS EUROPE 95
o NATIONAL SOFTWARE COUNCIL
o CALL FOR PAPERS - SOFTWARE QUALITY
PACIFIC NORTHWEST SOFTWARE QUALITY CONFERENCE
o CALL FOR PAPERS
INTERNATIONAL TEST CONFERENCE
o SOFTWARE TestWorks (STW)
FREQUENTLY ASKED QUESTIONS (FAQs)
o TRUE-TIME AND WIDGET BOTH NECESSARY!
o SPEAK OUT NOW!
o CALENDAR OF EVENTS
o TTN SUBMITTAL POLICY
o TTN SUBSCRIPTION INFORMATION
========================================================================
STW AT TOOLS EUROPE 95
Planning on being in France in March? TOOLS EUROPE 95 will be held in
Versailles March 6-10. It is the international meeting place for
managers and developers interested in understanding object-oriented
technology and its implications.
The vendor exhibits will be March 7-9, and Software Research Inc.'s
Software TestWorks (STW(tm)) will be there! Here is a perfect chance
for you to see STW in operation. For more information about TOOLS
EUROPE 95, write:
TOOLS EUROPE 95
104 rue Castagnary
75015 Paris-FRANCE
Tel: +33 (1) 45 32 58 80
Fax: +33 (1) 45 32 58 81
E-mail: tools@eiffel.fr
========================================================================
NATIONAL SOFTWARE COUNCIL
The National Software Council (NSC) is a proposed national-level organi-
zation that focuses on software from a variety of perspectives as a
critical resource and economic force. The purposes for which the Cor-
poration is organized are:
(a) to identify and articulate key software issues
(b) to provide a forum for analysis and discussion of software issues
(c) to propose policies to address software issues
The initiative originated in a workshop sponsored by the Office of the
Secretary of the Air Force for Acquisition, specifically Communications,
Computers, and Support Systems, which is headed by Mr. Lloyd Mosemann.
The workshop, held in August of 1993, was hosted by the Rome
Laboratory's Command and Control Directorate, specifically, Mr. Andrew
Chruscicki.
A bootstrap effort was led by Walter Ellis to gain support and endorse-
ment for the initiative and resulted in the "Report of The National
Software Council Planning Task Force" of 10 March 1994. As a result the
National Security Industrial Association (NSIA) was asked to further
this initiative by moving it into the private sector. In October of
1994, at the NSIA sponsored NSC workshop, an implementation committee
for the NSC termed the "National Software Council Implementation Commit-
tee" was formed.
Since that time the committee has taken the following steps:
(a) Incorporation as the National Software Council
(b) Organizing a Software Summit which would focus high level personnel
on the issues involved and serve to provide a public announcement
(c) Put into place a number of organizational initiatives such as
standing committees for membership and funding.
The committee held its seminal meeting in January of this year and John
Marciniak, Kaman Sciences Corporation, was elected Chair. (For further
information about how you can participate in the NSC effort contact Mr.
Marciniak at 703-329-7368, marcin-alx1@kaman.com.
========================================================================
CALL FOR PAPERS - SOFTWARE QUALITY
PACIFIC NORTHWEST SOFTWARE QUALITY CONFERENCE
The mission of the Pacific Northwest Software Quality Conference is to
increase awareness of the importance of software quality. As a non-
profit corporation, it seeks to promote software quality by providing
education and opportunities for information exchange within the software
community.
ORIGINAL PAPERS are solicited on all aspects of software quality.
Conference attendance will be about 400 software professionals from
industry, universities and government. Papers will be selected on the
basis of abstracts, then refereed for publication in the Proceedings and
presentation at the conference.
CONFERENCE INFORMATION
Abstract deadline 15 February 1995
Notice of selection 4 April 1995
Paper length 10-20 pages
Draft paper due 15 May 1995
Referee's comments returned 26 June 1995
Camera-ready paper due 19 July 1995
Conference presentation 27-29 September 1995
Questions? Contact: Peter Martin 408/777-5241
Internet peter_martin@taligent.com
or Margie Davis 503/294-4200
mrd@plaza.ds.adp.com
========================================================================
CALL FOR PAPERS
INTERNATIONAL TEST CONFERENCE
ITC 1995
October 23-25, 1995
Sheraton Washington Hotel
Washington, D.C. USA
DRIVING DOWN THE COST OF TEST
International Test Conference is the premier conference dedicated to the
complex issue of electronic test. ITC offers a comprehensive Test Week
of activities for design and test theoreticians and practitioners. The
conference includes a formal paper technical program, tutorials, panel
sessions, IEEE standards activities and commercial exhibits of pertinent
equipment, software and services -- all assembled together in one loca-
tion.
Authors are invited to submit original, unpublished papers describing
recent work. SUBMISSIONS MUST INCLUDE:
* Title of paper
* Name, affiliation, address, telephone and FAX number of each
author. (DESIGNATE THE PRESENTER. ITC will communicate with the
presenter.)
* Suggested topic or topics from the topic list below, or describe
your topic.
* TWELVE COPIES of a complete paper or extended summary of 4-6
pages. (Finished papers must be 10 pages or less.)
* An ABSTRACT of 35 words or less.
SUBMISSION DEADLINE: March 3, 1995
Since the ITC paper selection process is highly competitive; clearly
describe your work, explain its significance, highlight novel features,
provide supporting data, include practical results and conclusions, pro-
vide references to prior work and describe the current status of your
work. ITC does not accept material that does not meet all requirements
in this Call for Papers. Do not submit confidential material.
TOPIC AREAS:
IC Test Board Test
System Test Test Synthesis
MCM Test IDDQ-Based Test
DFT BIST
Microprocessor Test Mixed-Signal Test
Tester Software *Software Test*
Test Engineering Economics of Test
Quality Physical Defects
Test Data Management IC Diagnostics
Test Equipment Hardware Memory Test
ATPG
MAIL YOUR SUBMISSION TO:
Doris Thomas
International Test Conference
205 Tennyson Ave., Suite C
Altoona, PA 16602, USA
Tel: +1-814-941-4666, FAX: +1-814-941-4668
internet: dor.thomas@compmail.com
SOFTWARE TESTERS TAKE NOTE!
The International Test Conference, a long standing conference focused on
testing electronics from components to systems, is expanding its scope
to include Software Testing. There is great interest on the part of the
almost 3,000 attendees each year to hear about current research results
in software testing, particularly when these also include proof of con-
cept in the form of experiments or actual use. We would also like to
receive papers from industry that can point ITC attendees to successful
Technology Transfer or practical issues related to software testing.
This last year we had a very successful panel on legal issues related to
testing.
Topics of interest include:
o test methods: black box, white box, unit, integration, and system
level testing, verification and validation, regression testing,
and software reliability.
o testing process: test planning, management, metrics, test case
standards and specifications, organizations supporting testing
(Software Quality Assurance, Independent Validation & Verification
groups, testing groups), defect tracking
o test tools: test case generators, coverage tools, reliability
models, test managers, capture/playback tools
o Testing success stories: How we tested this product ...
Additional details of next year's conference are included in the Call
For Papers attached. While this call for papers emphasizes ITCs primary
mission, hardware testing, we do want your software testing papers or
panel suggestions.
Contact:
Dr. Anneliese von Mayrhauser avm@cs.colostate.edu
Robert McNitt robert_mcnitt@stortek.com
Software Testing Topic Coordinators, ITC 1995
========================================================================
SOFTWARE TestWorks (STW)
FREQUENTLY ASKED QUESTIONS (FAQs)
Software Research has recently added some new products to our automated
software testing line, and wished to let our users and potential users
know. Below is listed our current product information, to update the
recently published FAQ that appeared on recent InterNet postings.
------------------------------------------------------------------------
Name of Tool : STW/Regression
Kind of Tool : Regression test, test management, capture/playback
Company Name : Software Research, Inc.
Address : 901 Minnesota Street, San Francisco, CA 94107 USA USA
Internet Addr : info@soft.com
Phone and Fax : (415)550-3020, (800)942-SOFT, FAX (415)550-3030
Description : STW/Regression tools automate the entire test execution process
with test management, capture/playback including OCR based
character recognition, load generation and comparison
technology. With each modification you make, STW/Regression
helps determine if your program introduced unexpected results.
Platforms : Sun Sparc, SGI, RS/6000, HP 9000, DEC Alpha,
DECstation, 386/486 UNIX
----------------------------------------------------------------------
Name of Tool : STW/Regression/MSW
Kind of Tool : Regression test, test management, capture/playback
Company Name : Software Research, Inc.
Address : 901 Minnesota Street, San Francisco, CA 94107 USA USA
Internet Addr : info@soft.com
Phone and Fax : (415)550-3020, (800)942-SOFT, FAX (415)550-3030
Description : STW/Regression tools automate the entire test execution process
with test management, capture/playback including OCR based
character recognition and comparison technology. With each
modification you make, STW/Regression helps determine if your
program introduced unexpected results.
Platforms : MS/Windows, MS/DOS
------------------------------------------------------------------------
Name of Tool : STW/Coverage
Kind of Tool : Coverage
Company Name : Software Research, Inc.
Address : 901 Minnesota Street, San Francisco, CA 94107 USA USA
Internet Addr : info@soft.com
Phone and Fax : (415)550-3020, (800)942-SOFT, FAX (415)550-3030
Description : STW/Coverage analysis tools measure how test cases exercise a
program at branch, call-pair and path levels. STW/Coverage
gives you confidence in your code because the collected
coverage data will advise what parts of the program have been
tested or not tested, thus helping you determine if additional
testing is needed. Support for C, C++, Ada, F77, COBOL.
Platforms : Sun Sparc, SGI, RS/6000, HP 9000, DEC Alpha,
DECstation, 386/486 UNIX, MS/DOS
------------------------------------------------------------------------
Name of Tool : STW/Advisor
Kind of Tool : Metrics, static analysis, some test case generation
Company Name : Software Research
Address : 901 Minnesota Street, San Francisco, CA 94107 USA USA
Internet Addr : info@soft.com
Phone and Fax : (415)550-3020, (800) 942-SOFT, FAX (415)550-3030
Description : STW/Advisor tools establish and measure quality benchmarks with
metrics, and analyze source code for anomalies with static
analysis. By providing code measurements, STW/Advisor helps
focus your attention and guides appropriate resource
allocation.
Platforms : Sun Sparc, SGI, RS/6000, HP 9000, DEC Alpha,
DECstation, 386/486 UNIX
------------------------------------------------------------------------
Name of Tool : CAPBAK/X, CAPBAK/MSW, CAPBAK/NI
Kind of Tool : Capture/Playback
Company Name : Software Research, Inc.
Address : 901 Minnesota Street, San Francisco, CA 94107 USA USA
Internet Addr : info@soft.com
Phone and Fax : (415)550-3020, (800)942-SOFT, FAX (415)550-3030
Description : CAPBAK/X is a capture/playback system for X that includes OCR
support (from XIS); CAPBAK/MSW is for MS-WIN, also with OCR.
CAPBAK/NI is a non-invasive hardware-based capture/playback for
microchannel, AT-channel, and IDE-channel PC architectures.
All are compatible with other tools in Software Research's tool
suite "Software TestWorks (STW (tm))".
Platforms : SUN Sparc, SGI, RS/6000, HP-9000, DEC Alpha,
DECstation, 386/486 UNIX, MS/Windows
========================================================================
TRUE-TIME AND WIDGET BOTH NECESSARY!
Testing of the GUI part of a product can be sometimes be a very complex
task and when the GUI is part of a client-server application in which
more than one machine is involved it can be even more complicated. Some
test-tool suppliers would have you believe that you can NEVER do tests
on GUIs without object oriented (widget) level testing, and and other
organizations would have you believe that only ``true-time'' (100% real-
istic) recording and playback are needed. In practice, BOTH are
required, both are useful, and both methods have advantages and limita-
tions.
TRUE-TIME TESTS
Since the mid-1980's, when SR introduced automated capture and replay
technology, first for MS-DOS and then for UNIX serial-port type environ-
ments, the main technology has been to perform "true time" recording of
test sessions. Keyboard and mouse activity events are recorded and
played back in a way that maintains faithfulness to exactly what the
user typed. Done correctly, and with the correct kinds of playback syn-
chronization enhancements in place, true-time based testing can be
extremely effective. Generally, such tests are very precise; this means
that a true-time test will FAIL -- i.e. correctly identify a defect --
even if the smallest product change occurs.
From Xwindows there are some special considerations for this, such as
how to relocate windows to the original positions, etc. These matters
are discussed in more detail elsewhere, but it suffices to recognize
that most operational objections are easily overcome.
OBJECT-ORIENTED ("OO") TESTS
More recently, with the increase in need for tests that execute without
fail on multiple platforms (see below), we have implemented on UNIX and
MSWindows platforms a different kind of test recording based on use of
special techniques to record which visual image, or object, is used,
along with when and where it is used.
Versions of such object oriented tests (OO tests) play back more reli-
ably on multiple platforms because such tests inherently are less sensi-
tive to tiny changes in the application window features, border width,
placement, color, etc. Such tests' advantage is their conceptual sim-
plicity and the sense of cross-platform portability.
WHERE THE PROBLEMS LIE
Almost all OO tests need a semi-invasive technique that may introduce
its own kind of problems to the testing process. To run in OO mode on a
UNIX platform, a capture/playback system MUST instrument the underlying
windowing library (typically it is the Xt library for Motif applica-
tions). When you run your application tests this way there is the risk
that, because the application is not really the same one that you are
going to be shipping to customers, you can't really be sure that you've
really tested the same software the customer gets. In other words,
there is a risk that such invasivity -- done to simplify the testing
process -- will: (a) cause errors that are present in the application
to be missed by the tests (false positive tests); and/or, (b) find
errors that aren't really present (false negative tests). Neither of
these kinds of test outcome helps builds confidence in the application's
quality if they occur often.
Typically, one must link the application to the special Xt library. On
some platforms this is done at run time, but on some platforms this
"dynamic linking" is not available and a separate build is required.
Separate builds introduce another source of potential error.
Another concern is that some approaches,i in which the user "programs"
the way the application is going to be exercised, make the testing pro-
cess as vulnerable to error as the programming process itself. Here's
why: the tester who programs tests does so to exercise what he believes
is supposed to have been implemented. If the testers' program misses a
required feature, the test will succeed incorrectly, but that feature
will never have been tried. Even worse, if the testers' program
incorrectly exercises a feature -- for example, if it pushes a button
that is not actually visible on the screen -- then the test will also
succeed but it will have done so using "illegal" means.
DUAL-MODE TESTS:
It's clear that no single method will ever suffice to handle every
situation, but it appears that the mixture of true-time and OO mode give
the software tester the best possible chance to build tests that are:
(a) non-invasive if they have to be (using true-time recording); (b) are
based on what the GUI really shows (using the ability to derive tests
from hands-one recording rather than programmatically); (c) provides for
testing of graphics -- where true-time mode is critical -- as well as
testing of the GUI's that drive the graphics; and, (d) reliable and
flexible enough to provide real confidence increases in delivered pro-
duct quality.
EDITORS NOTE: This article is based on a lengthier Applications Note
that is available on request from SR. Send e-mail to "info@soft.com"
and ask for the AN entitled "Mixed Mode Testing Considered Essential".
========================================================================
SPEAK OUT NOW!
We have been doing TTN-Online for over a year now...and it's time to ask
our nearly 2000 readers worldwide:
What do you like about the TTN-Online?
What do you NOT like?
What would you like to see MORE of? See LESS of?
We would love to hear from you! Please address your comments and ques-
tions to "ttn@soft.com", or phone or write us at the address below:
Software Research, Inc.
901 Minnesota Street
San Francisco, CA 94107 USA
[+1] 415-550-3020
========================================================================
---------------------->>> CALENDAR OF EVENTS <<<----------------------
========================================================================
The following is a partial list of upcoming events of interest. ("o"
indicates Software Research will participate in these events.)
o February 22-25: Uniforum 1995
Dallas Convention Center
Dallas, TX
Contact: [+1] 800-545-EXPO
+ March 1-2: Software World USA
Chicago, IL
Contact: Loretta Taylor
tel: 508-470-3870
+ March 7-10: Achieving Quality Software
Sheraton Premiere
Tyson's Corner, VA
contact: Sherry Paquin
email: sap0215@sperry.mhs.compuserve.com
tel: 804-974-2078
fax: 804-974-2480
o March 6-10: TOOLS EUROPE 95
Palais de Congres
Versailles, FRANCE
email: tools@eiffel.fr
tel: +33 (1) 45 32 58 80
fax: +33 (1) 45 32 58 81
+ March 8-15: CeBIT '95
Contact: Hannover Fairs USA, Inc.
103 Carnegie Center
Princeton, NJ 08540
tel: 609-987-1202
fax: 609-987-0092
+ April 9-14: 7th Annual Software Technology Conference
Salt Lake City, Utah, USA
Contact: Dana Dovenbarger, Lynne Wade
tel: 801-777-7411 DSN 458-7411
fax: 801-777-8069 DSN 458-8069
email: dovenbar@oodis01.hill.af.mil
wadel@hillwpos.hill.af.mil
========================================================================
------------>>> TTN SUBMITTAL POLICY <<<------------
========================================================================
The TTN On-Line Edition is forwarded on the 15th of each month to sub-
scribers via InterNet. To have your event listed in an upcoming issue,
please e-mail a description of your event or Call for Papers or Partici-
pation to "ttn@soft.com".
The TTN On-Line submittal policy is as follows:
o Submission deadlines indicated in "Calls for Papers" should provide
at least a 1-month lead time from the TTN On-Line issue date. For
example, submission deadlines for "Calls for Papers" in the January
issue of TTN On-Line would be for February and beyond.
o Length of submitted items should not exceed 68 lines (one page).
o Publication of submitted items is determined by Software Research,
Inc., and may be edited as necessary.
========================================================================
----------------->>> TTN SUBSCRIPTION INFORMATION <<<-----------------
========================================================================
To request a FREE subscription or submit articles, please E-mail
"ttn@soft.com". For subscriptions, please use the keywords "Request-
TTN" or "subscribe" in the Subject line of your E-mail header. To have
your name added to the subscription list for the quarterly hard-copy
version of the TTN -- which contains additional information beyond the
monthly electronic version -- include your name, company, and postal
address.
To cancel your subscription, include the phrase "unsubscribe" or
"UNrequest-TTN" in the Subject line.
Note: To order back copies of the TTN On-Line (August 1993 onward),
please specify the month and year when E-mailing requests to
"ttn@soft.com".
TESTING TECHNIQUES NEWSLETTER
Software Research, Inc.
901 Minnesota Street
San Francisco, CA 94107 USA
Phone: (415) 550-3020
Toll Free: (800) 942-SOFT
FAX: (415) 550-3030
E-mail: ttn@soft.com
## End ##