Dead Sheepdog: Ethical Dilemmas in Software Development

The following case study comes from the field of engineering not software. It also deals with safety critical systems and dire consequences. It is included because it is a famous and well-understood example that provides a clear role model and that despite the stakes, presents dilemmas and pressures similar to those faced by developers on less critical systems.

Baura, G. Engineering Ethics: An Industrial Perspective, Elsevier Academic Press, Burlington VT, 2006

Roger Boisjoly was a Thiokol engineer who found “large arcs of blackened grease” on the solid boosters recovered from successful shuttle launches. He identified a correlation between cold temperatures and leakage of hot gases from the O-Ring seals in the solid boosters.

In January 1986, based on Boisjoly’s analysis and forecasts of cooler temperatures than ever experienced during a shuttle launch, Thiokol recommended the shuttle Challenger not launch.

NASA could not proceed over the contractor!s objections. “Appalled” by Thiokol!s recommendation, NASA held a private caucus with Thiokol management. A senior Thiokol executive was asked to, “take off his engineer hat and put on his management hat.” (Rogers Commission, 1986)

As a result, while still expressing concern, Thiokol withdrew their objection for lack of definitive proof. An age old argument for ignoring risk. By definition, no risk is certain.

Challenger exploded during launch killing all seven aboard. In the aftermath, Boisjoly testified before the shuttle commission which is why we know all this.

As a result of coming forward, Boisjoly experienced such a hostile workplace he was granted sick leave and then extended disability.

In 1988, Boisjoly was awarded the AAAS Scientific Freedom and Responsibility Award. He is considered a role model of ethical action.

The most important thing to learn from his example is that ethical behavior is not about certainty or infallibility.
Despite his expertise, insight and integrity lives were lost. At points he respected the chain of command even though he clearly disagreed with their decisions.

When it became clear he had, against his best efforts, contributed to tragedy, he stepped forward to take responsibility for his actions despite the consequences.

Stripped of these life-ending consequences, the situation Biosjoly faced was common. As an expert, hands on with the technology, he anticipated risks supervising technical staff and managers did not. When challenged to definitively prove his concerns, he could not and he deferred.

At stake on one side was his individual and uncertain concern for the lives of the astronauts. On the other were immense costs of delay, the reputation of the shuttle program, the revenue stream of his employer and differing convictions of other engineers and managers.

A person in this circumstance might weigh what they surmise as the likelihood of a bad outcome against concerns for their livelihood and reputation. They might also consider the possibility that they were wrong and that their managers and other technical staff were right. A person with experience acting with good will might be expected to make the same decisions.
Therefore, while the pressures put upon him and others were unethical in that they did not take due care for the lives of others, his actions in those circumstances were ethical despite the eventual consequences.

An ethical viewpoint doesn!t demand we have perfect insight into the future or that we immolate our careers but that we reasonably consider the implications of our actions on the broad set of stakeholders who might be affected.
However, while acting ethically may have absolved him of legal culpability, it did not resolve him of responsibility. Though he acted reasonably, a different course of action (however extreme) might have saved lives.

To his immense credit, Boisjoly acted from that sense of responsibility. Again, he might have chosen not to come forward leaving others to fix NASA. He could have quietly continued at Thiokol or he could have left for another job. Instead, he testified to the shuttle commission becoming a whistle blower.

In doing this, Boisjoly may have saved the lives of other astronauts. He certainly improved the execution justice and served the interests of the families of the deceased and the American public.

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment