Systems Analysis

On Being an IT Project Manager

by

My profession is much in the news at the moment, so I thought I would pass along such insights as I have from my career, mostly from a multibillion-dollar debacle which I and several thousand others worked on for a few years around the turn of the millennium. I will not name my employer, not that anyone with a passing familiarity with me doesn’t know who it is; nor will I name the project, although knowing the employer and the general timeframe will give you that pretty quickly too.
We spent, I believe, $4 billion, and garnered a total of 4,000 customers over the lifetime of the product, which was not aimed at large organizations which would be likely to spend millions on it, but at consumers and small businesses which would spend thousands on it, and that amount spread out over a period of several years. From an economic transparency standpoint, therefore, it would have been better to select 4,000 people at random around the country and cut them checks for $1 million apiece. Also much faster. But that wouldn’t have kept me and lots of others employed, learning whatever it is we learn from a colossally failed project.
So, a few things to keep in mind about a certain spectacularly problematic and topical IT effort:

  • Large numbers of reasonably bright and very hard-working people, who have up until that point been creating significant wealth, can unite in a complete flop. Past performance is no guarantee, and all that. Because even reasonably bright, hard-working people can suffer from failures of imagination, tendencies to wishful thinking, and cultural failure in general.
  • Morale has got to be rock-bottom for anybody with any degree of self-awareness working on this thing. My relevant moment was around the end of ’99 when it was announced, with great fanfare, at a large (200+ in attendance) meeting to review progress and next steps, that we had gotten a single order through the system. It had taken various people eight hours to finish the order. As of that date, we were projecting that we would be doing 1,600 orders a day in eight months. To get an idea of our actual peak rate, note the abovementioned cumulative figure of 4,000 over the multi-year lifespan of the project.
  • Root cause analysis is all very well, but there are probably at least three or four fundamental problems, any one of which would have crippled the effort. As you may infer from the previous bullet point, back-office systems was one of them on that project. Others which were equally problematic included exposure to the software upgrade schedule of an irreplaceable vendor who was not at all beholden to us to produce anything by any particular date, and physical access to certain of our competitors’ facilities, which they were legally required to allow us into exactly two (2) days per year. See also “cultural failure,” above; most of us were residing and working in what is one of the most livable cities in the world in many ways, but Silicon Valley it ain’t.
  • Not to overlook the obvious, there is a significant danger that the well-advertised difficulties of the website in question will become a smokescreen for the fundamental contradictions of the legislation itself. The overall program cannot work unless large numbers of people act in a counter-incentived (possibly not a word, but I’m groping for something analogous to “counterintuitive”) fashion which might politely be termed “selfless” and do so in the near future. What we seem likely to hear, however, is that it would have worked if only certain IT architectural decisions had been better made.

This thing would be a case study for the next couple of decades if it weren’t going to be overshadowed by physically calamitous events, which I frankly expect. In another decade, Gen-X managers and Millennial line workers, inspired by Boomers, all of them much better at things than they are now, “will be in a position to guide the nation, and perhaps the world, across several painful thresholds,” to quote a relevant passage from Strauss and Howe. But getting there is going to be a matter of selection pressures, with plenty of casualties. The day will come when we long for a challenge as easy as reorganizing health care with a deadline a few weeks away.

Upcoming talk at the Chicago Council on Global Affairs: STRATEGY: FROM THE WAR ROOM TO THE BOARD ROOM Sir Lawrence Freedman

by

STRATEGY: FROM THE WAR ROOM TO
THE BOARD ROOM
Sir Lawrence Freedman, Professor of War Studies, and Vice-Principal, King’s College London

What do modern military and corporate strategy have in common with Achilles, Sun Tzu, and primates? The answer is fluidity, flexibility, and pure unpredictability. Every day we make decisions that are built on our theory of what will give us the outcome we want. Sir Lawrence Freedman proposes that throughout history strategy has very rarely gone as planned, and that constant evaluation is necessary to achieve success—even today. Join The Chicago Council for a centuries-spanning discussion explaining how the world’s greatest minds navigate toward success.

For interested parties. Sir Lawrence Freedman has quite a few talks posted on YouTube too. Worth checking out.

Social-Media Corruption

by

What proportion of all social-media communication is by bots, spammers, people with agendas who misrepresent themselves, or severely dysfunctional people who pass as normal online? I suspect it’s a large proportion.

There’s not much hard evidence, but every once in a while something like this turns up. I’m guessing it’s the tip of an iceberg. See also this. And who can overlook the partisan trolls who show up on this and other right-of-center blogs before elections. Where do they come from?

None of this apparently widespread Internet corruption should come as a surprise. Given the low costs and lack of barriers to entry it would be surprising if attempts to game the system were less frequent than they appear to be. Nonetheless it’s prudent to keep in mind that a lot of what appears online is probably fake and certainly misleading.

(Via.)

“Studies Show” – Widespread Errors in Medical Research

by

Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice?

The arguments presented in this article seem like a good if somewhat long presentation of the general problem, and could be applied in many fields besides medicine. (Note that the comments on the article rapidly become an argument about global warming.) The same problems are also seen in the work of bloggers, journalists and “experts” who specialize in popular health, finance, relationship and other topics and have created entire advice industries out of appeals to the authority of often poorly designed studies. The world would be a better place if students of medicine, law and journalism were forced to study basic statistics and experimental design. Anecdote is not necessarily invalid; study results are not necessarily correct and are often wrong or misleading.

None of this is news, and good researchers understand the problems. However, not all researchers are competent, a few are dishonest and the research funding system and academic careerism unintentionally create incentives that make the problem worse.

(Thanks to Madhu Dahiya for her thoughtful comments.)

Institutions, Instruments, and the Innovator’s Dilemma

by

I have written several posts that use Carroll Quigley’s “institutional imperative” as a lens for understanding contemporary events. [1] Mr. Quigley suggests that all human organizations fit into one of two types: instruments and institutions. Instruments are those organizations whose role is limited to the function they were designed to perform. (Think NASA in the 1960s, defined by its mission to put a man on the moon, or the NAACP during the same timeframe, instrumental to the civil rights movement.) Institutions, in contrast, are organizations that exist for their own state; their prime function is their own survival.

Most institutions start out as instruments, but as with NASA after the end of the Cold War or the NAACP after the victories of the civil rights movement, their instrumental uses are eventually eclipsed. They are then left adrift, in search of a mission that will give new direction to their efforts, or as happens more often, these organizations begin to shift their purpose away from what they do and towards what they are. Organizations often betray their nature when called to defend themselves from outside scrutiny: ‘instruments’ tend to emphasize what their employees or volunteers aim to accomplish; ‘institutions’ tend to emphasize the importance of the heritage they embody or even the number of employees they have.

Mr. Quigley’s institutional imperative has profound implications for any democratic society especially a society host to so many publicly funded organizations as ours. Jonathan Rauch’s essay, “Demosclerosis” is the best introduction to the unsettling consequences that come when public organizations transform from instruments into institutions. [2] While Mr. Rauch does not use the terminology of the Institutional Imperative, his conclusions mesh neatly with it. Describing the history and growth of America’s bureaucratic class, Mr. Rauch suggests its greatest failing: a bureaucracy, once created, is hard to get rid of. To accomplish whatever mission it was originally tasked with a bureaucracy must hire people. It must have friends in high places. The number of people who have a professional or economic stake in the organization’s survival grows. No matter what else it may do, it inevitably becomes a publicly sponsored interest group. Any attempt to reduce its influence, power, or budget will be fought against with ferocity by the multitude of interests who now depend on it. Even when it becomes clear that this institution is no longer an instrument, the political capital needed to dismantle it is just too high to make the attempt worth a politician’s time or effort. So the size and scope of bureaucracies grow, encumbering the country with an increasing number of regulations it cannot change, employees it does not need, and organizations that it cannot get rid of.

I used to think that the naked self-interest described by Mr. Rauch was the driving force behind the Institutional Imperative. It undoubtedly plays a large role (particularly when public funds are involved), but there are other factors at play. One of the most important of these is what business strategists call Marginal Thinking.

Read more