sss ssss rrrrrrrrrrr ssss ss rrrr rrrr sssss s rrrr rrrr ssssss rrrr rrrr ssssssss rrrr rrrr ssssss rrrrrrrrr s ssssss rrrr rrrr ss sssss rrrr rrrr sss sssss rrrr rrrr s sssssss rrrrr rrrrr +===================================================+ +======= Testing Techniques Newsletter (TTN) =======+ +======= ON-LINE EDITION =======+ +======= October 1997 =======+ +===================================================+ TESTING TECHNIQUES NEWSLETTER (TTN), Online Edition, is E-mailed monthly to support the Software Research, Inc. (SR)/TestWorks user community and to provide information of general use to the worldwide software quality and community. Permission to copy and/or re-distribute is granted, and secondary circulation is encouraged by recipients of TTN-Online provided that the entire document/file is kept intact and this complete copyright notice appears with it in all copies. (c) Copyright 1997 by Software Research, Inc. ======================================================================== INSIDE THIS ISSUE: o A Guided Tour of Quality Week Europe '97 o Measured Merriment o A Must-Read for Trustworthy Software Engineers, by Larry Bernstein o An Ambulance Down In The Valley, forwarded by John Ivory o COCOMO II Goes Public: Forum In October o An Intesting Puzzle, forwarded by John Favaro o Conference Announcement and Call for Papers: EASE'98 o Apologies for our Typo o TTN SUBSCRIPTION INFORMATION ======================================================================== A GUIDED TOUR OF QUALITY WEEK EUROPE '97 (QWE'97)... CONFERENCE THEME: "Quality for the Millennium" Sheraton Hotel Brussels, Belgium 4-7 November 1997 Building on SR/Institute's QW's success in the past decade, the 1st International Software Quality Week/Europe (QWE'97) aims to bring the best of quality technology to a mainly European audience. Included in the QWE'97 technical program are several very-high-quality papers that were presented at QW'97 (San Francisco, May 1997). As the 90's draw to a close we see more and more that software quality issues are among the essential driving forces for selection, use, and world-wide software market expansion. Companies large and small, Government units, in short, everyone, are concerned to make sure that the software products they deliver provide good value and effective use. QWE'97 is a multi-threaded conference aimed at all levels of software quality people, from those just beginning new projects, to those with many years experience. People come to Quality Week to get started, to hone their skills, to share results with their colleagues ... a thousand reasons. This short writeup, a companion to the printed brochure and the on-line conference description on the WWW http://www.soft.com/QualWeek/QWE97, gives an informal summary of the entire QWE conference. TUTORIALS The conference opens with two days of tutorials, with 10 carefully selected tutorial speakers whose topics address critical issues of today as well as the issues of tomorrow. The basics of software testing are presented by the world-renowned test expert Dr. Boris Beizer (Independent Consultant), "An Overview of Testing (T-A)". This seminar is likely to be, as it has in the past, one of the best general introductions to the software quality and testing area you'll likely hear anywhere. Software designers need to know the intrinsic value of early test planning and the impact on quality, so we have arranged to bring you: Mr. Tom Gilb (Independent Consultant) "Requirements-Driven Test Management (T-G)". Real-world results have to be taken into account, too. "Good specifications beget good software," says Mr. Robert Poston (Aonix) "10X Testing: Automating Specification-Based Testing (T-J)" as he describes a remarkable high-performance end-to-end approach to testing. Technically oriented testing is the key to good quality software, as illustrated in two tutorials by the well known Mr. Robert V. Binder (RBSC Corporation). His full-day tutorial "Modal Testing Strategies for Object-Oriented Software (T-D)" deals with applying the OO paradigm to testing and his follow-on half-day tutorial "Test Automation for Object-Oriented Systems (T-I)" aims to explain how to automate such kinds of tests. The view taken by Mr. Erik Van Veenendaal & Jos Trienekens (KEMA/Eindhoven University of Technology) "Practical User-Oriented Software Product Quality Specification and Evaluation (T-L)" sustains the notion that the user is in the drivers' seat for true software quality improvement. Combined with the associated technology, the TMap approach offers some real advantages, as described in the half-day tutorial by Mr. Erik Van Veenendaal & Mr. Bart Broekman (KEMA/Eindhoven University of Technology) "Structured Testing According to TMap (T-H)". We all know that the Year 2000 problem may be a real watershed for software quality, and this topic is the focus of the tutorial by Nicholas Zvegintzov (Software Management Network), "Testing for Year 2000 (T-I)". Metric based processes have reached new levels of sophistication and maturity, as discussed in this tutorial by Mr. Hans-Ludwig Hausen (GMD) "Effective Metrication for Software Process Assessment and Software Product Evaluation (T-B)" The very-important ISO-9000 process is described by Dr. Giora Ben-Yaacov (Cadence Design Systems) "Effective Implementation of ISO 9000 for Software (T-F)" in a half-day tutorial. And we have also included a half-day tutorial by Dr. Michael Haug "A Survey of EC-Sponsored Software Process Improvement Efforts (T-C)" that describes European organizations' successes with software process improvements. Ultimately, the determining factor in any quality process is people, and in this half-day tutorial by Mr. Tom Drake (Booz Allen & Hamilton) "Testing Software Based Systems: People, Process & Technology (T-E)" we get a real-world practitioner's view of what really makes software quality processes run well. CONFERENCE KEYNOTE TALKS Special events like QWE'97 give us an opportunity to address critical issues, controversial issues and issues that are likely to be important now as well as in the future. In view of current interest in ISO-9000 you'll be pleased to know that not everyone believes ISO-9000 to be a panacea. This keynote talk by Mr. John Seddon (Consultant), "In Pursuit of Quality: The Case Against ISO 9000 (L2)" aims to clarify the ISO-9000 effectiveness question. And, we'll hear from one of the pioneers in CASE and OO design with a keynote by Mr. Tony Wasserman (Software Methods & Tools) "30 Years of Software Engineering -- What Have We Learned? (1P)". The present and future impact of quality, from two very different perspectives, and from two well known experts in the field, will be addressed by Dr. Boris Beizer (Independent Consultant) "The Future of Software Quality (7P)", and Dr. Edward F. Miller (Software Research, Inc.) "Quality for the Millennium (L1)". TECHNOLOGY TRACK The long term growth of software quality is founded in developing technologies that address current and even future problems. The Technology Track has papers from around the world that show off the latest thinking from labs and from the field that give you a good cross-section of what people are doing today that can lead to exciting results in the future. New tool approaches that have high potential impact are described in the papers by Mr. Fraser Macdonald & Mr. James Miller (University of Strathclyde Department of Computer Science) "ASSISTing with Software Inspection (11T)", and Mr. Huey-Der Chu & Prof. John E. Dobson (University of Newcastle upon Tyne) "An Integrated Test Environment for Distributed Applications (12T)". Testing, still a disputed method in many quarters, continues to grow in strength, as illustrated in the paper by Prof. Antonia Bertolino (IEI- CNR) "A General Path Generation Algorithm for Coverage Testing (4T)", and in the presentation by Dr. Agnes Arnould, Dr. Pascal Le Gall (Universite Paris) "Some Aspects of Test Data Selection from Formal Specifications (3T)". Formal methods, technically difficult but extremely powerful, is advancing in various ways. The paper by Ms. Barbara Quaquarelli (Cad.Lab S.p.A) "Quality Improvement through VErification PROcess (PROVE) (8T)" deals directly with quality issues, and the paper by Mr. Jean-Marie Condom & Khadir Ouriachi (Universite de Pau) "Meeting Quality Requirements in Robot-Based Manufacturing Using Z (10T)" brings us up to date on application of "Z". Static and complexity estimation has a very important role in software quality, and the papers by Dr. Guido Dedene & Geert Poels (Applied Economics, KU Leuven) "Assessing the Size and Complexity of Formally Specified Conceptual Models (9T)" and Dr. Denise Woit (Ryerson Polytechnic University) "Specifying Component Interactions for Modular Reliability Estimation (6T)" bring out recent advances in this important area. Object oriented approaches are important, and are becoming very practice-oriented. Real-world work reported by Ms. Janet P. Oberti, et. al. (BEA Systems, Inc.) "Testing an Object Request Broker (ORB) Using Automation (2T)", and by Mr. T. Ashok, et. al. (VeriFone India Pvt Ltd) "Class Evolution and Equivalence Categories (5T)" give a feel for how the best technologies can be applied with very positive results. TOOLS AND SOLUTIONS TRACK How modern methods are applied makes a big difference in how successful a software quality enhancement project is going to be. Real world experience is the best guide and much can be learned from what worked and what didn't. QWE'97 case study papers include: Mr. Erik Van Veenendaal (KEMA/Eindhoven University of Technology) "Practical Inspection Experiences at Philips (4S)", Mr. Sudipo Ghosh, et. al. (Purdue University/Bell Communications Research) "Software Fault Injection Testing on a Distributed System -- A Case Study (2S)", and Ms. Karen Thelen, et. al. (Honeywell Technology Center) "Model-Based Requirements Definition and Verification Test Generation for Cockpit Display Systems (9S)". Measurement based solutions, ranging from hi-level studies of how processes are used in practice to how to put static analysis to work, are represented by these papers: Mr. Don O'Neill (Independent Consultant) "National Software Quality Experiment: A Lesson in Measurement 1992-1996 (12S)", Mr. Bruno Peeters (Gemeentekrediet van Belgie) "Measurement of Software Maintainability (8S)", and Mr. Otto Vinter (Bruel & Kjaer) "How to Apply Static and Dynamic Analysis in Practice (5S)". Process oriented team enablement is often a key factor in achieving quality results, as the papers by Dr. Giora Ben-Yaacov, et. al. (Cadence Design Systems) "Improving Software Testing with ISO 9000 Processes (6S)", Mr. Bob Smith and Ms. Ana Andres (European Software Institute) "ISO-9000 Certification as a Business Driver: The SPICE Road (9M)", and Mr. Thierry Tacquet (Objectif Technologie) "Small Company Action Training and Enabling (SCATE) (3S)" make clear. New approaches, involved high interactivity and the use of the WWW, are described in these two solutions-track presentations: Mr. Sankar Chakrabarti & Mr. Harry Robinson (Hewlett Packard Company) "Catching Bugs in the Web: Using the World Wide Web to Detect Software Localization Defects (10S)" and Mr. Tony Templeton (IBM Canada Ltd.) "Rapid Testing Strategies (11S)". MANAGEMENT TRACK Process and management issues are sometimes as important as the product itself -- after all, if the process is good then the product OUGHT to be good too! But process involves more than just complying with a standard. Realistic application of good process is the key to achieving quality, and these two papers by Mr. Marc Morel-Chevillet & Stephane Depres (Objectif Technologie) "AVAL Objectives, Process and Results (8M)" and by Mr. Stefan Biffl, Mr. G. Thomas (Technical University Vienna) "How to Apply Quality Assurance in Every Project: Guidance Towards Three Levels of QA for Real Life (11M)" make this point very well. Again, in practical terms, "if you can't measure it you can't do it" still has a lot of validity. The papers by Mr. Howard Chorney (Process Software Corporation) "A Practical Approach to Using Software Metrics (12M)", and by Mr. Bruno Peeters (Gemeentekrediet van Belgie) "A COBOL Standards Compliance Checker (5M)" argue that numerics are the core of any process and give us the way to get the numbers we need. Sometimes it takes novel thinking to make an impact on quality, and these papers illustrate the point. Mr. Tom Gilb (Independent Consultant) "Evolutionary Project Management (6M)" and Mr. Morten Elvang-Goransson (SimCorp AS) "Devising a Specification Process (by Turning Gilb Style Inspections Inside Out) (3M)" provide a fine contract (with apologies to Tom Gilb), while Mr. Paolo Panaroni (Intecs Sistemi, S.p.A.) "Testing, But...The Other Way Round: A Management Perspective (4M)" gives a fundamentally different perspective. Formal or semi-formal programs to increase productivity are important, and the paper by Ms. Anne Mette Jonassen Hass & Mr. Jorn Johansen (DELTA, Software Engineering) "BOOTSTRAP - The Real Way To SPI (2M)" describes one of the most important. Finally, if the goal is world competitiveness then the paper by Mr. Don O'Neill (Independent Consultant) "Global Software Competitiveness Assessment Program (10M)" is extremely important because it describes how to measure [your own] processes to find out where you stand. ------------------------------------------------------------------------ REGISTRATION INFORMATION Complete registration with full information about the conference is available on the WWW at http://www.soft.com/QualWeek/QWE97 where you can register to attend on-line. We will be pleased to send you a QWE'97 registration package by E-mail, postal mail or FAX on request. Send your E-mail requests to: qw@soft.com or FAX or phone your request to SR/Institute at the numbers below. QWE'97: 4-7 November 1997, Brussels, Belgium EU +-----------------------------------+----------------------------------+ | Quality Week Europe '97 | Phone: [+1] (415) 550-3020 | | SR/Institute, Inc. | Toll Free: 1-800-942-SOFT | | 901 Minnesota Street | FAX: [+1] (415) 550-3030 | | San Francisco, CA 94107 USA USA | E-Mail: qw@soft.com | | | WWW: http://www.soft.com | +-----------------------------------+----------------------------------+ ======================================================================== Measured Merriment! This one is so good it is hard to pass it up! Thanks to Teresa Cleland (clelands@hqamc.safb.af.mil). Here are some important paraphrases..... 10**12 Microphones = 1 Megaphone 10**6 bicycles = 2 megacycles 500 millinaries = 1 seminary 2000 mockingbirds = two kilomockingbirds 10 cards = 1 decacards 1/2 lavatory = 1 demijohn 10**-6 fish = 1 microfiche 453.6 graham crackers = 1 pound cake 10**12 pins = 1 terrapin 10**21 picolos = 1 gigolo 10 rations = 1 decoration 100 rations = 1 C-ration 10 millipedes = 1 centipede 3 1/3 tridents = 1 decadent 5 holocausts = 1 Pentacost 10 monologues = 5 dialogues = 1 decalogue 2 monograms = 1 diagram 8 nickles = 2 paradigms 2 snake eyes = 1 paradise 2 wharves = 1 paradox ======================================================================== A Must-Read for Trustworthy Software Engineers Reviewed by Larry Bernstein Les Hatton's book "Safer C: Developing Software for High-integrity and Safety-critical Systems" (ISBN: 0-07-707640-0) describes the best way to use C and C++ in mission critical applications. The author says we must constrain the use of the language features to achieve reliable software performance and then goes to specify instruction by instruction how to do it. His theme is "The use of C in safety-related or high integrity systems is not recommended without severe and automatically enforceable constraints. However, if these are present using the formidable tool support (including the extensive C library), the best available evidence suggests that it is then possible to write software of at least as high intrinsic quality and consistency as with other commonly used languages." I am in violent agreement with Les and applaud his instruction by instruction analysis of the C language. After a discussion of infamous software failures, Les separates reliability from safety. He restricts safety to: "freedom from danger or risks" and reliability to: "Of sound and consistent character or quality." Chapter 2 contains a critique of C with the understanding that it is an intermediate language in the class set out by Nicholas Wirth is his pioneering PL/360 paper. Because the programmer can drop down to the machine architecture there are dangers. But with reasonable constraints and limitations on the use of register instructions to those very few key cases dictated by the need to achieve performance goals, C can be used to good effect. The alternative of using a high level language that protects the programmer form the machine often leads to a mix of assembly language and high level language code which brings with it all the headaches of managing configuration control and integrating modules from different code generators. Les Hatton finds the way to have the power of C with constraints to assure that the source code is well structured. He strongly recommends the use of function prototypes for interfaces. It shocked me to find a medium size supplier of telecommunications software violating his simple conclusion that all problems found at compilation must be corrected. This simple recommendation could have avoided several world-renowned software bugs. Chapter 4 reports on detailed analysis of source code from 54 projects. For example, he found that once in every 29 lines of executable source lines functions are not declared before they are used. I particularly liked the way he suggested using cyclomatic complexity measures and it was the first time I saw the use of fan-in /fan-out metrics. Software shop looking for a basis for their processes will find Safer C a treasure. Understanding and following Les Hatton's recommendations will improve the structure, reliability and predictability of software schedules and performance. McGraw Hill International, London publishes "Safer C." ======================================================================== An Ambulance Down In The Valley Twas a dangerous cliff as they freely confessed, Though to walk near its edge was so pleasant. But over its edge had slipped a Duke, And it fooled many a peasant. The people said something would have to be done, But their projects did not at all tally. Some said, "Put a fence around the edge of the cliff," Others, "An ambulance down in the valley." The lament of the crowd was profound and loud, As their hearts overflowed with pity; But the ambulance carried the cry of the day, As it spread to the neighboring cities. So a collection was made to accumulate aid, And dwellers in highway and alley, Gave dollars and cents not to furnish a fence, But an ambulance down in the valley. "For the cliff is all right if you're careful," they said, "And if folks ever slip and are falling; It's not the slipping and falling that hurts them so much, As the shock down below when they're stopping." And so for the years as these mishaps occurred, Quick forth would the rescuers sally, To pick up the victims who fell from the cliff, With the ambulance down in the valley. Said one in his plea, "It's marvel to me, That you'd give so much greater attention, To repairing results than to curing the cause; Why you'd much better aim at prevention. For the mischief, of course, should be stopped at its source; Come friends and neighbors, let us rally! It makes far better sense to rely on a fence, Than an ambulance down in the valley." "He's wrong in his head," the majority said, "He would end all our earnest endeavors. He's the kind of a man that would shirk his responsible work, But we will support it forever. Aren't we picking up all just as fast as they fall, And giving them care liberally? Why a superfluous fence is of no consequence, If the ambulance works in the valley." Now this story seems queer as I've given it here, But things oft occur which are stranger. More humane we assert to repair the hurt, Than the plan of removing the danger. The best possible course would be to safeguard the source, And to attend to things rationally. Yes, build up the fence and let us dispense, With this ambulance down in the valley. Forwarded by: John E. Ivory (ivory@tower.com) Tower Concepts, Makers of the RAZOR system. ======================================================================== COCOMO II Goes Public: Forum In October Barry Boehm boehm@sunset.usc.edu A public version of the COCOMO II (COnstructive COst MOdel) is now available at the USC Center for Software Engineering's web site: http://suset.usc.edu/COCOMOII/Cocmo.html It includes a Java program and associated manuals. The 1997 version of COCOMO II has been calibrated to 83 project data points contributed by the COCOMO II Affiliates, primarily large commercial and aerospace firms. Additional data points continue to be contributed; the COCOMO II project plans to issue annual upgrades as the model is calibrated to larger samples. COSTAR, a commercial version of COCOMO II, is also available from Dan Ligett (ligett@SoftstarSystems.com). Experiences with usage, calibration, and extensions of COCOMO II and other software cost models will be presented at the 12th International Forum on COCOMO and Software Cost Estimation. It will be held October 9-10, 1997 on the USC campus in Los Angeles, CA. It will be preceded by a three-day, hands-on COCOMO II tutorial by Don Reifer, October 6-8, 1997. ======================================================================== An Interesting Puzzle Note: Thanks for this go to John Favaro (favaro@pisa.intecs.it) Work this out as you read. You'll be surprized by the result. 1. First of all, pick the number of days a week that you would like to be handed a check for $1,000,000. 2. Multiply this number by 2. 3. Add 5. 4. Multiply it by 50. 5. If you have already had your birthday this year, add 1747. If you haven't, add 1746. 6. Last step: Subtract the four digit year that you were born. RESULT: You should now have a three digit number: - The first digit of this was your original number (how many checks you want to receive each week). - The second two digits are your age!!! It really works. Try it! More importnat, this is the only year it will ever work! ======================================================================== CONFERENCE ANNOUNCEMENT & CALL FOR PAPERS EASE'98 Empirical Assessment & Evaluation in Software Engineering 30th March - 1st April 1998 Keele University, Staffordshire, U.K. Sponsors: BT Laboratories, National Computing Centre British Computer Society, Institution of Electrical Engineers Keele University, University of Sunderland The empirical component of Software Engineering practice is still in a formative phase. The effect of this is that we continue to lack any widely accepted procedures for systematically assessing the effectiveness of both the technical and organizational practices that are used for software development and procurement, or for assessing the quality of the resulting products. Following on from the success of EASE'97, selected papers from which are to appear in November as a special issue of the Information and Software Technology journal, this conference will address issues concerned with both assessment and evaluation needs and practices, as well as the analysis element within these. EASE'98 will be relevant to researchers and practitioners in all branches of Software Engineering. Keynote speakers will include Robert L Glass, Editor of the Journal of Systems and Software and publisher of the Software Practitioner magazine and Dr. Marian Petre from the Open University, UK. TECHNICAL PAPERS: We are seeking papers that are related, but not limited, to the following topics: * Certification practices and their effectiveness * Measurement * Procurement needs * Product acceptance * Product assessment and evaluation * Process assessment and evaluation * Quality assessment * Testing strategies Once again we expect to consider conference papers for journal publication. EXPERIENCE REPORTS of a less formal nature are invited from Software Engineering practitioners and will be made available during the conference. Selected reports will be presented during an `industrial experiences' session. POSTERS: All participants will be invited to provide posters describing their evaluation needs and problems. Practitioners and research students are particularly encouraged to take up this opportunity to elicit comments from those who are studying evaluation techniques and practices. Four copies of full technical papers (3000 - 5000 words) and experience reports (up to 1000 words) should be submitted to: EASE, Department of Computer Science, University of Keele, Keele, Staffs, ST5 5BG, UK. Expressions of interest can be registered either by email or via the Web pages. IMPORTANT DATES 3 November 1997 - Submission deadline 16 January 1998 - Author notification 27 February 1998 - Final version FURTHER INFORMATION: Email: ease@cs.keele.ac.uk URL: http://www.keele.ac.uk/depts/cs/Announcements/conferences/ease98.html ======================================================================== APOLOGIES FOR OUR TYPO Yes, indeed, as many readers noted, because of a production error Mr. Muru"s kind permission to reprint his article was inadvertently included in the article itself. A corrected copy of the September 1997 issue is available on our WebSite archive of all TTN-Online issues. Check at http://www.soft.com/News for a corrected copy. ======================================================================== TTN-Online -- Mailing List Policy Statement Some subscribers have asked us to prepare a short statement outlining our policy on use of E-mail addresses of TTN-Online subscribers. This issue, and several other related issues about TTN-Online, are available in our "Mailing List Policy" statement. For a copy, send E-mail to ttn@soft.com and include the word "policy" in the body of the E-mail. ======================================================================== ------------>>> TTN SUBMITTAL POLICY <<<------------ ======================================================================== The TTN Online Edition is E-mailed around the 15th of each month to subscribers worldwide. To have your event listed in an upcoming issue E-mail a complete description and full details of your Call for Papers or Call for Participation to "ttn@soft.com". TTN On-Line's submittal policy is as follows: o Submission deadlines indicated in "Calls for Papers" should provide at least a 1-month lead time from the TTN On-Line issue date. For example, submission deadlines for "Calls for Papers" in the January issue of TTN On-Line would be for February and beyond. o Length of submitted non-calendar items should not exceed 350 lines (about four pages). Longer articles are OK and may be serialized. o Length of submitted calendar items should not exceed 60 lines (one page). o Publication of submitted items is determined by Software Research, Inc. and may be edited for style and content as necessary. DISCLAIMER: Articles and items are the opinions of their authors or submitters and TTN-Online disclaims any responsibility for their content. TRADEMARKS: STW, TestWorks, CAPBAK, SMARTS, EXDIFF, Xdemo, Xvirtual, Xflight, STW/Regression, STW/Coverage, STW/Advisor, TCAT, TCAT-PATH, T- SCOPE and the SR logo are trademarks or registered trademarks of Software Research, Inc. All other systems are either trademarks or registered trademarks of their respective companies. ======================================================================== ----------------->>> TTN SUBSCRIPTION INFORMATION <<<----------------- ======================================================================== To SUBSCRIBE to TTN-Online, to CANCEL a current subscription, to CHANGE an address (a CANCEL and a SUBSCRIBE combined) or to submit or propose an article, send E-mail to "ttn@soft.com". TO SUBSCRIBE: Include in the body of your letter the phrase "subscribe". TO UNSUBSCRIBE: Include in the body of your letter the phrase "unsubscribe ". TESTING TECHNIQUES NEWSLETTER Software Research, Inc. 901 Minnesota Street San Francisco, CA 94107 USA Phone: +1 (415) 550-3020 Toll Free: +1 (800) 942-SOFT (USA Only) FAX: +1 (415) 550-3030 E-mail: ttn@soft.com WWW URL: http://www.soft.com ## End ##