The Economist’s Technology Quarterly has an excellent article on Software bugtraps: software that makes software better. This is something of a followup to an article they did a few years ago; most people quoted think that the situation is improving, although of course as Capers Jones points out it depends on your metrics. And why the improvement?
According to … the chairman of the Standish Group, most of this improvement is the result of better project management, including the use of new tools and techniques that help programmers work together. Indeed, there are those who argue that computer science is really a social science. Jonathan Pincus, an expert on software reliability who recently left Microsoft Research* to become an independent consultant, has observed that “the key issues [in programming] relate to people and the way they communicate and organise themselves.â€
Indeed, I have argued that — in keynote talks Analysis is necessary but not sufficient at ISSTA 2000 and Steering the pyramids at ICSM 2002, and then more explicitly in the “BillG thinkweek paper” Computer science is really a social science (draft) from early 2005 and my 2006 Data Devolution keynote with Sarah Blankinship applying this lens to computer security.
So have many others, some of whom I cite in the above. More recently, Bruce Schneier gave 2007 keynotes on this at RSA and Computers, Freedom, and Privacy; and many of these themes cropped up in the National Academies/CSTB 2007 report Software for Dependable Systems: Sufficient Evidence?** Just this week, Addison Wesley published Adam Shostack and his co-author Andrew Stewart’s new book The New School of Information Security, which explicitly includes discussions of diversity in the social science sense as well as the epidemiology-based diversity arguments that Nicholas Weaver, Stefan Savage, ChenXi Wang, Dan Geer and others have made for a while.
Of course, that’s far from the only worthwhile thing in the article. There are also good short descriptions of static and dynamic analysis, including this:
Static analysis, being more difficult, is the younger of the two disciplines. In recent years several start-ups, including Klocwork, Fortify and Ounce Labs, have entered the field. Static analysis is best done as close as possible to the programmer, because the earlier a bug can be identified, the cheaper it is to fix. (An industry rule of thumb is that a bug which costs $1 to fix on the programmer’s desktop costs $100 to fix once it is incorporated into a build, and thousands of dollars if it is identified only after the software has been deployed in the field.)
My former startup Intrinsa’s PREfix is generally regarded as the first breakthrough tool in the static analysis market. No disrespect to Gimpel’s excellent PC-Lint and Abraxas’ CodeCheck, but Cisco’s multi-million dollar site license for PREfix followed by our acquisition by Microsoft made it clear that the business opportunities here were just as great as in the dynamic analysis field — and Bill Gates’ references to PREfix and its successor PREfast (now available in Microsoft’s Visual Studio, although the name may have changed) in his keynotes at OOPSLA and RSA confirmed that this stuff is for real.
And yeah, it is hard: at the time Reed Hastings of Purify (now CEO of Netflix) described static analysis as “cold fusion”. The current batch of startups, public domain tools such as FindBugs, and researchers and engineers at universities and corporations continue to move the field forward rapidly — including at Microsoft; a group of eight of us got the Chairman’s Award at last summer’s Engineering Excellence/Trustworthy Computing day. Major challenges still remain with user interface, integration into the overall software engineering environment, and language design: appallingly, new programming languages continue to fall far short of the classics CLU and Eiffel in their support for pre- and post-conditions and invariants, which assist both dynamic and static checking. Tony Hoare once famously characterized Algol 60 as “a significant improvement on its successors”; the same could be said here.
I was disappointed not to see any mention of formal methods in the article; companies like Praxis — which offers a money-back guarantee on its software — and Kestrel are doing some extremely interesting things, and I would have loved to hear what Jeannette Wing of NSF had to say. Agile programming was similarly excluded; why no mention of the xUnit test frameworks or perspectives from Laurie Williams? And I felt bad for my former colleagues that there was no mention of Microsoft (which continues to be a leader here in many ways) other than me leaving.
Ah well. There’s only so much you can fit in an article. The people and companies they did mention all are important contributors and have good things to say; props to Capers, Jim Johnson of Standish, David Grantges of Verizon, Daniel Sabbah and Grady Booch of IBM Rational, Chris Wysopal of Veracode, John Viega of McAfee, Brian Chess of Fortify, Gwyn Fisher of Klocwork, Seth Hallem of Coverity, Alberto Savoia of Agitar, Jack Danahy of Ounce, and Paul Black of NIST.
One thing that I hope really jumps out from this list: every single person quoted in the article is male.
While at first blush this seems like a pretty accurate reflection of the composition of the companies the article focused on, it also really highlights how thoroughly some perspectives are marginalized in the field to date. This is particularly relevant in light of Laura Beckwith’s fascinating research into gender-related differences in debugging.*** [Beckwith, by the way, was recently hired by Microsoft after getting her Ph.D. from Oregon State; and the author of the article is the same Jessica Mintz of AP who recently wrote about Microsoft and Yahoo!’s culture.] At some point, enough people at one of these startups or large companies will read Scott Page’s The Difference, or come to the same conclusion on their own, and realize that there’s an opportunity to be the next billion-dollar software engineering company.
However, as I say, that’s an observation of the field — not a criticism of the article. It’s very clearly written, and a solid survey of the state of the art. Like the Economist’s previous article on the subject , it’s a great introduction and a valuable snapshot … and perhaps more.
In the 2005 paper, I suggested that the field of computer science was in the midst of a major paradigm shift, perhaps even a scientific revolution — I love it when I get to use the word Kuhnian! If so, this week — with this as a major part of the The Economist’s framing and publication of The New School (see Andre Gironda’s review for more) — may be remembered as a major milestone for the rebels.
And my second mention in The Economist this year! w00t, w00t! Once again, Mom will be happy.
jon
* minor clarification: I actually left Microsoft Research in early 2006; the presentation with Sarah was my last “publication”, although along with Tomasz Ostwald we are working on a paper. I spent the next 18 months at Microsoft, but not in research, as General Manager of Strategy Development on the Competitive Strategy team in the Online Services Marketing Group before leaving last November.
** I was a member of the committee; so were an economist, a sociologist, and two doctors, so I clearly wasn’t the only person seeing social sciences as important 🙂
*** Update, November 18: The Wikipedia page on Gender HCI has a good overview of research in this area by Beckwith, her advisor Margaret Burnett Beckwith, and others.
Gregory K. | 12-Mar-08 at 12:53 pm | Permalink
Well worthy of *woot*ing!
I’d also note that “an economist, a sociologist, and two doctors” sounds like the set up for a great joke. Maybe for your next project?
jon | 12-Mar-08 at 12:59 pm | Permalink
Mmmmmaybe 🙂
PS: that’s w00t.
jon | 16-Mar-08 at 5:53 pm | Permalink
My comment on the Economist’s story, where they don’t allow such newfangled contraptions as HTML:
Amusingly enough, this fits in remarkably well with the rest of my posting history there.
jj | 17-Mar-08 at 12:03 am | Permalink
nice to get quoted by economist! they didnt mention your baby though 🙁
jon | 17-Mar-08 at 6:59 am | Permalink
The article a few years ago discussed PREfix and other Microsoft tools, and the author told me they didn’t want to rehash old ground. It makes a lot of sense, really: PREfix is still in use at Microsoft and Cisco (Intrinsa sold a site license to them back in 1999 a few months before we were acquired by Microsoft) but it’s not where the action is these days.
jon | 17-Mar-08 at 7:23 am | Permalink
Mark Davidson of Sun raves about the article on his java.net blog, including
Alas, his page doesn’t have any way to contact him or leave an anonymous comment, and a quick Google didn’t reveal his email address, so communication and teamwork are somewhat difficult in this case. Oh well. I’ll try a trackback. And if anybody sees this who knows Mark, could you please mention to him that I linked to him?
Jeff | 17-Mar-08 at 12:05 pm | Permalink
It’s an interesting perspective, but I’m not convinced that Computer Science has become (or is becoming) a social science as much as that there are many interesting problems forming on the social side as the field matures. I’m reminded of the distinction between architecture, civil engineering and city planning where there are separate fields that closely complement one another. Engineers deal with the details of getting the bridge built, architects design the bridge given the goals of the project and planners figure out whether or not and where to build the bridge as part of a larger transportation strategy. That they’re all involved in different parts of the same endeavor does not make all of them engineers or architects or planners.
Archimedes’ Hot Tub » Blog Archive » New software tools | 19-Mar-08 at 12:40 pm | Permalink
[…] a post on the story, Indeed! The Economist on Computer Science as a Social Science by Jonathan Pincus, who is cited in the piece, with some interesting comments, including a link to […]
Andy Chou | 24-Mar-08 at 9:30 pm | Permalink
There’s more evidence that computer science is a social science from the way that real-world static analyzers work. Modern analyzers (like ours, Coverity Prevent) don’t just look at source code from a purely grammatical point of view, they assume that code is written first for other programmers to read, and only second as instructions for machines to execute. Prose written by humans has many characteristics that you can’t derive from schoolbook grammar alone, such as a tendency to not be overly redundant or self-contradictory at a semantic level. If programs are just vast encyclopedic works of human expression, then static analyzers are the equivalent of robotic editors tirelessly combing the text for paragraphs that read like nonsense. They wouldn’t be able to do that nearly as well if programs weren’t written by, and for, people.
What excites me about this is that this information is just sitting out there waiting to be tapped for new applications.
Liminal states » Blog Archive » PWN2OWN: the stakes just got higher | 26-Mar-08 at 12:39 pm | Permalink
[…] PS: for those of you keeping score at home: disciplines this essay touches on include economics (vulnerability markets), pedagogy (participative learning), sociology, geopolitics, and ludology — as well as graph theory. When it comes to security, computer science is indeed a social science. […]
jon | 26-Mar-08 at 1:41 pm | Permalink
Agreed, Andy. Also, the code isn’t the only expression of semantics and intent; specifications, test cases, formal and informal models, user documentation, configuration management and build information … as Andreas Zeller’s archive mining project illustrates, any of these are extremely valuable.
Which just illustrates how much room there is for progress….
Liminal states » Make desire more important than fear: “Change the Way You See Yourself (Through Asset-Based Thinking)” | 29-May-08 at 4:29 pm | Permalink
[…] of the threads on Liminal States are experiments with this, such as the Indeed/pwn2own/RSA sequence on security and static analysis, and the Gender, race, age, and […]
Liminal states » Rant: I hate software | 23-Sep-08 at 9:17 am | Permalink
[…] a “grand old man” of the software engineering field of defect detection, I sometimes take it personally when I run into bugs or usability problems. My IM friends are […]
Liminal states » Voter suppression wiki: what to discuss on Meet the Bloggers? | 09-Oct-08 at 11:22 am | Permalink
[…] of our campaign. This ties in both with my recent experiences and my multi-year research agenda of computer science as a social science — and I think will be interesting in general for the Meet the Bloggers audience, because […]
Liminal states » danah boyd joins Microsoft Research’s New England Lab — computer science *is* a social science | 11-Oct-08 at 10:26 am | Permalink
[…] Jon Pincus, Indeed: The Economist on “computer science as a social scienceâ€, March […]
jon | 12-May-09 at 7:05 am | Permalink
Jer’s Just Landed: Processing, Twitter, MetaCarta & Hidden Data is a fascinating blog post about a mashup using tweets to track air travel. As @jennifergardy summarizes on Twitter
Jer mentioneds that the work was done in Processing, so i checked out processing.org.
Here’s the list of people mentioned in the processing.org’s “updates” list: Aaron, Josh, Yunsil, Hyunwoo, Andrew, Stephen, Hansi Raber, ewjordan, Michae, Lukas, Stephane, Douglas, Yonas Sandbaek, Bruno Nadeau, Andreas, Jan, Guillaume, Angus, Nikolau, Severin, Andreas, Jonathan, Dan, Florian, Carl, Emil, Daniel, Mark, Memo, Owen, Jordan.
Hmm. Assume that gender HCI findings also apply to language design and there are some differences in how men and women would prefer to interact with Processing as well. Whose needs are more likely to be met?
I don’t mean to pick on Processing, which seems like a great visualization tool. Other languages are no doubt just as skewed in terms of who’s designed and contributed to them.
Still, the end result is a system where languages and tools wind up institutionalizing male dominance, and women are systematically disadvantaged.
Order Ambien | 17-Jan-11 at 1:54 pm | Permalink
Good article. Thank you.
http://www.box.net/shared/8guoosaf43