The Economist’s Technology Quarterly has an excellent article on Software bugtraps: software that makes software better. This is something of a followup to an article they did a few years ago; most people quoted think that the situation is improving, although of course as Capers Jones points out it depends on your metrics. And why the improvement?
According to … the chairman of the Standish Group, most of this improvement is the result of better project management, including the use of new tools and techniques that help programmers work together. Indeed, there are those who argue that computer science is really a social science. Jonathan Pincus, an expert on software reliability who recently left Microsoft Research* to become an independent consultant, has observed that “the key issues [in programming] relate to people and the way they communicate and organise themselves.”
Indeed, I have argued that — in keynote talks Analysis is necessary but not sufficient at ISSTA 2000 and Steering the pyramids at ICSM 2002, and then more explicitly in the “BillG thinkweek paper” Computer science is really a social science (draft) from early 2005 and my 2006 Data Devolution keynote with Sarah Blankinship applying this lens to computer security.
So have many others, some of whom I cite in the above. More recently, Bruce Schneier gave 2007 keynotes on this at RSA and Computers, Freedom, and Privacy; and many of these themes cropped up in the National Academies/CSTB 2007 report Software for Dependable Systems: Sufficient Evidence?** Just this week, Addison Wesley published Adam Shostack and his co-author Andrew Stewart’s new book The New School of Information Security, which explicitly includes discussions of diversity in the social science sense as well as the epidemiology-based diversity arguments that Nicholas Weaver, Stefan Savage, ChenXi Wang, Dan Geer and others have made for a while.
Of course, that’s far from the only worthwhile thing in the article. There are also good short descriptions of static and dynamic analysis, including this:
Static analysis, being more difficult, is the younger of the two disciplines. In recent years several start-ups, including Klocwork, Fortify and Ounce Labs, have entered the field. Static analysis is best done as close as possible to the programmer, because the earlier a bug can be identified, the cheaper it is to fix. (An industry rule of thumb is that a bug which costs $1 to fix on the programmer’s desktop costs $100 to fix once it is incorporated into a build, and thousands of dollars if it is identified only after the software has been deployed in the field.)
My former startup Intrinsa’s PREfix is generally regarded as the first breakthrough tool in the static analysis market. No disrespect to Gimpel’s excellent PC-Lint and Abraxas’ CodeCheck, but Cisco’s multi-million dollar site license for PREfix followed by our acquisition by Microsoft made it clear that the business opportunities here were just as great as in the dynamic analysis field — and Bill Gates’ references to PREfix and its successor PREfast (now available in Microsoft’s Visual Studio, although the name may have changed) in his keynotes at OOPSLA and RSA confirmed that this stuff is for real.
And yeah, it is hard: at the time Reed Hastings of Purify (now CEO of Netflix) described static analysis as “cold fusion”. The current batch of startups, public domain tools such as FindBugs, and researchers and engineers at universities and corporations continue to move the field forward rapidly — including at Microsoft; a group of eight of us got the Chairman’s Award at last summer’s Engineering Excellence/Trustworthy Computing day. Major challenges still remain with user interface, integration into the overall software engineering environment, and language design: appallingly, new programming languages continue to fall far short of the classics CLU and Eiffel in their support for pre- and post-conditions and invariants, which assist both dynamic and static checking. Tony Hoare once famously characterized Algol 60 as “a significant improvement on its successors”; the same could be said here.
I was disappointed not to see any mention of formal methods in the article; companies like Praxis — which offers a money-back guarantee on its software — and Kestrel are doing some extremely interesting things, and I would have loved to hear what Jeannette Wing of NSF had to say. Agile programming was similarly excluded; why no mention of the xUnit test frameworks or perspectives from Laurie Williams? And I felt bad for my former colleagues that there was no mention of Microsoft (which continues to be a leader here in many ways) other than me leaving.
Ah well. There’s only so much you can fit in an article. The people and companies they did mention all are important contributors and have good things to say; props to Capers, Jim Johnson of Standish, David Grantges of Verizon, Daniel Sabbah and Grady Booch of IBM Rational, Chris Wysopal of Veracode, John Viega of McAfee, Brian Chess of Fortify, Gwyn Fisher of Klocwork, Seth Hallem of Coverity, Alberto Savoia of Agitar, Jack Danahy of Ounce, and Paul Black of NIST.
One thing that I hope really jumps out from this list: every single person quoted in the article is male.
While at first blush this seems like a pretty accurate reflection of the composition of the companies the article focused on, it also really highlights how thoroughly some perspectives are marginalized in the field to date. This is particularly relevant in light of Laura Beckwith’s fascinating research into gender-related differences in debugging.*** [Beckwith, by the way, was recently hired by Microsoft after getting her Ph.D. from Oregon State; and the author of the article is the same Jessica Mintz of AP who recently wrote about Microsoft and Yahoo!'s culture.] At some point, enough people at one of these startups or large companies will read Scott Page’s The Difference, or come to the same conclusion on their own, and realize that there’s an opportunity to be the next billion-dollar software engineering company.
However, as I say, that’s an observation of the field — not a criticism of the article. It’s very clearly written, and a solid survey of the state of the art. Like the Economist’s previous article on the subject , it’s a great introduction and a valuable snapshot … and perhaps more.
In the 2005 paper, I suggested that the field of computer science was in the midst of a major paradigm shift, perhaps even a scientific revolution — I love it when I get to use the word Kuhnian! If so, this week — with this as a major part of the The Economist’s framing and publication of The New School (see Andre Gironda’s review for more) — may be remembered as a major milestone for the rebels.
And my second mention in The Economist this year! w00t, w00t! Once again, Mom will be happy.
* minor clarification: I actually left Microsoft Research in early 2006; the presentation with Sarah was my last “publication”, although along with Tomasz Ostwald we are working on a paper. I spent the next 18 months at Microsoft, but not in research, as General Manager of Strategy Development on the Competitive Strategy team in the Online Services Marketing Group before leaving last November.
** I was a member of the committee; so were an economist, a sociologist, and two doctors, so I clearly wasn’t the only person seeing social sciences as important
*** Update, November 18: The Wikipedia page on Gender HCI has a good overview of research in this area by Beckwith, her advisor Margaret Burnett Beckwith, and others.