Rebel Science News
6/22/2008
Parallel Computing: The End of the Turing Madness. Part II

6/15/2008
Parallel Computing: The End of the Turing Madness. Part I

5/29/2008
The COSA Saga

5/23/2008
Encouraging Mediocrity at the Multicore Association

5/18/2008
Parallel Computing: Why Legacy Is Not Such a Big Problem

The Real Enemies of Software Reliability

Rebel Science Home
Why Software Is Bad
Project COSA
Operating System
Software Composition
Parallel QuickSort
The Devil's Advocate
The Enemy
Discussion Forums
Contact Me

Introduction

The List

Introduction

There are a few people in the software reliability industry that I consider to be the enemies of reliable software. They thrive on the continued existence of buggy systems. They have a vested interest in seeing that the crisis lasts as long as possible. They have invested much time and money in traditional methods of software engineering and they have a lot to lose if a new way is found which solves the problem once and for all. They will fight it teeth and nails. Their favorite battle cry is "there is no silver bullet", a phrase borrowed from Frederick P. Brooks's paper titled "No Silver Bullet--Essence and Accidents of Software Engineering". It is getting tiresome.

The fact is that Brooks' paper is not nearly as carefully reasoned as his followers would like to believe. It is full of logical holes and unsubstantiated claims, something that I have demonstrated in the Silver Bullet page. In my opinion, Brooks' essay on the causes of software unreliability has been a disaster, not only for computer science, but for the world at large.

Software quality experts love to talk about software reliability measurements the way bridge engineers talk about structural safety but the probability that a complex algorithmic program will fail is really unknown and will always remain so. The sad reality is that, no matter how reliable a safety-critical algorithmic software system is estimated to be, failure is not an option. In other words, extremely reliable software is not good enough. Unless the system is guaranteed to be 100% free of defects, it is potentially catastrophic.

Software reliability measurements based on probability are really accidents waiting to happen. Civil engineers can reliably predict that a bridge can be safely used under specific conditions, but software engineers cannot do the same for complex algorithmic systems. Yet, amazingly enough, an entire reliability industry has mushroomed within the last two decades whose main approach to software quality assurance is the use of so-called software quality metrics or assessment. In my opinion, the proper goal of software quality management is not to measure the reliability of software programs (a waste of time, really) but to guarantee their total reliability. There are no two ways about it: unless a piece of software is guaranteed bug-free, it should be considered defective and must not be deployed. The creation of truly reliable software requires a methodology of software construction based on a correct understanding of the true nature of computing. The software metrics approach is a joke: a computer program simply does not work like a bridge.

The List

What follows is a list of prominent individuals that I consider to be the real enemies of reliable software even though they all profess otherwise. These people are doing the computer industry and the world a great disservice. They are using their positions of authority to perpetuate a myth, and a very harmful myth at that. They have built a personality cult around Brooks and his fallacious doctrine just as they have done with Alan Turing over the years. They are not helping with the problem, their claims to the contrary notwithstanding. They are, in fact, a hindrance. The list contains each individual's name, professional title, affiliation and a representative quote, if available. I will add new names to the list from time to time as they come to my attention.

	Frederick P. Brooks, computer scientist and author of "No Silver Bullet--Essence and Accidents of Software Engineering".
	Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any--no inventions that will do for software productivity, reliability, and simplicity what electronics, transistors, and large-scale integration did for computer hardware.

	Jeff Voas, computer scientist and co-founder of Cigital.
	What disturbs me, however, is when I sit on a panel or in a debate with a software process zealot that truly believes, and wrongly so, that if you follow software process guideline X, then your software will be of such high quality, you will never need to apply any rigorous assessment measures to the software. Sadly enough, the rude consequences of this attitude will someday be felt by the public, because persons at the highest levels in the FAA, FDA, and Nuclear Regulatory Agency (as well DoD) have all bitten into this bait, hook, line, and sinker. Source

	Nancy G. Leveson, computer scientist and cofounder of Safeware Engineering.
	When a physicist makes an erroneous claim, such as in cold fusion, the idea may stay around for a while on the fringes of the field. However, the insistence on repeatability and careful experimentation allows such claims to be dismissed by the scientific majority within a relatively short period of time. We need to insist on the same level of evaluation and proof with regard to claims about software engineering techniques and tools. Unfortunately, this is rarely done and our belief in silver bullets persist. Even after Brooks' and Parnas' carefully reasoned and widely-acclaimed papers [8, 27], we are still seeing claims that the silver bullet has been found. Source

	Dr. Linda H. Rosenberg, chief scientist for software assurance for Goddard Space Flight Center (GSFC), NASA.
	[Software objects] are often heralded as the silver bullet for solving software problems, while in reality there is no silver bullet; object oriented development has proved its value for systems that must be maintained and modified. Source

	William Scherlis, co-director (?) of the SCC (CyLab) and principal research scientist at Carnegie Mellon's School of Computer Science.
	The SCC is not about developing a magical technology bullet, but rather to understand this issue as a market phenomenon and to try and move it along. Source

	Kenneth Jacobs, vice president of product strategy at Oracle.
	Our standards have inappropriately been lowered by our daily experience[]. We have to bring software engineering the kind of maturity we have in building bridges and buildings. We don't expect buildings to fall down every day. You can't expect instant results. It's a never-ending battle. It's not like there's one magic technology to solve all the problems []. "But having a common way to discuss reliability of systems will help. Source

	Bill Guttman, director of the SCC (CyLab) and Distinguished Service Professor of Economics and Technology.
	The problem, says consortium director Bill Guttman, is that unlike other engineers, programmers have no way of measuring the reliability of their designs. Source

	Watts Humphrey, Fellow of the Software Engineering Institute of Carnegie Mellon University; National Medal of Technology recipient.
	At the stage of basic code quality, quality is personal. [...] Quality is an individual issue. Source

If you believe that software quality assessment (probability-based reliability measurements) is the best way to improve software reliability, you are a hindrance to quality software. If one of your favorite articles on software reliability was written by Frederick Brooks, you are part of the problem, not the solution. If you think that the only way to achieve reliable software is through the enforcement of engineering discipline, you don't have a clue. If you believe that Alan Turing's computability model reveals the true nature of computing, you are lost in the wilderness. If your name is on the list and you feel that you have been unfairly treated or that you were somehow misquoted or misunderstood, do write me a note. And if you think that I have libeled you in any way, have your lawyer contact my lawyer.

Copy and distribute freely