My profession is much in the news at the moment, so I thought I would pass along such insights as I have from my career, mostly from a multibillion-dollar debacle which I and several thousand others worked on for a few years around the turn of the millennium. I will not name my employer, not that anyone with a passing familiarity with me doesn’t know who it is; nor will I name the project, although knowing the employer and the general timeframe will give you that pretty quickly too.
We spent, I believe, $4 billion, and garnered a total of 4,000 customers over the lifetime of the product, which was not aimed at large organizations which would be likely to spend millions on it, but at consumers and small businesses which would spend thousands on it, and that amount spread out over a period of several years. From an economic transparency standpoint, therefore, it would have been better to select 4,000 people at random around the country and cut them checks for $1 million apiece. Also much faster. But that wouldn’t have kept me and lots of others employed, learning whatever it is we learn from a colossally failed project.
So, a few things to keep in mind about a certain spectacularly problematic and topical IT effort:
- Large numbers of reasonably bright and very hard-working people, who have up until that point been creating significant wealth, can unite in a complete flop. Past performance is no guarantee, and all that. Because even reasonably bright, hard-working people can suffer from failures of imagination, tendencies to wishful thinking, and cultural failure in general.
- Morale has got to be rock-bottom for anybody with any degree of self-awareness working on this thing. My relevant moment was around the end of ’99 when it was announced, with great fanfare, at a large (200+ in attendance) meeting to review progress and next steps, that we had gotten a single order through the system. It had taken various people eight hours to finish the order. As of that date, we were projecting that we would be doing 1,600 orders a day in eight months. To get an idea of our actual peak rate, note the abovementioned cumulative figure of 4,000 over the multi-year lifespan of the project.
- Root cause analysis is all very well, but there are probably at least three or four fundamental problems, any one of which would have crippled the effort. As you may infer from the previous bullet point, back-office systems was one of them on that project. Others which were equally problematic included exposure to the software upgrade schedule of an irreplaceable vendor who was not at all beholden to us to produce anything by any particular date, and physical access to certain of our competitors’ facilities, which they were legally required to allow us into exactly two (2) days per year. See also “cultural failure,” above; most of us were residing and working in what is one of the most livable cities in the world in many ways, but Silicon Valley it ain’t.
- Not to overlook the obvious, there is a significant danger that the well-advertised difficulties of the website in question will become a smokescreen for the fundamental contradictions of the legislation itself. The overall program cannot work unless large numbers of people act in a counter-incentived (possibly not a word, but I’m groping for something analogous to “counterintuitive”) fashion which might politely be termed “selfless” – and do so in the near future. What we seem likely to hear, however, is that it would have worked if only certain IT architectural decisions had been better made.
This thing would be a case study for the next couple of decades if it weren’t going to be overshadowed by physically calamitous events, which I frankly expect. In another decade, Gen-X managers and Millennial line workers, inspired by Boomers, all of them much better at things than they are now, “will be in a position to guide the nation, and perhaps the world, across several painful thresholds,” to quote a relevant passage from Strauss and Howe. But getting there is going to be a matter of selection pressures, with plenty of casualties. The day will come when we long for a challenge as easy as reorganizing health care with a deadline a few weeks away.