“Statistical Quality Control Meets the NYPD”

An excellent post by Mark Draughn that reminds how we get the behavior we incentivize. In this case the NYC govt incentivized its police to ignore violent crimes and to make bogus arrests to boost their cleared-case stats:

This is a standard recipe for disaster in quality control — and CompStat is at heart a statistical quality control program. Take a bunch of people doing a job, make them report quality control data, and put pressure on them to produce good numbers. If there is little oversight and lots of pressure, then good numbers is exactly what they’ll give you. Even if they’re not true.

Worth reading in full.

15 thoughts on ““Statistical Quality Control Meets the NYPD””

  1. An old programmer’s saying from the 60s – “ GIGO – or Garbage In, Garbage Out

    The brilliance of Deming’s program – ignored by us for so long but adopted by the Japanese, was to have quality improve as an internal (to the factory) process – improving quality by feedback from the workers.

    In reading about the NYPD it seems that the bureaucracy and its desires took precedence over their primary mission – catching criminals and preventing crime.

    I am sure that they aren’t alone.

    In Ford’s case the manager responsible should have been rewarded too – one’s natural impulse would have been to fire the worker – which of course would show the others that silence was the best policy.

    This is because one of the early lessons learned by statistical quality control experts is that you can’t punish people for giving you bad news if you want accurate news.

    That’s the take home quote.

  2. I saw this over and over in the military. The incident I remember best happened when a certain broadcast squadron commander (he was new and establishing his authoriteh! in the same way that dogs do, by going around and pissing on certain spots) passed the word that he wanted only high-quality troops in the broadcasting squadron. Any of his enlisted troops who got an overall 7 or less on their yearly APR (airman performance report – ratings between 1 and 9 in certain key areas)could not expect to remain long in the squadron … or the career field. I think every det chief took it in turn to explain to him, very calmly and gently, that this would guarantee nothing but ratings of 8 or 9 for every single broadcaster in the squadron … even for the troops who were truely marginal. They might just as well give everyone a firewall 9 rating and be done with it.
    I think the personnel rating system was pretty well inflated to the point of uselessness by that time anyway, but that put a few more nails in the coffin.

  3. Sgt – it was/is the same in academia.

    Years ago at UVA I had shared some classes with graduate students.

    Either you had to have a “B”to get to the graduate level or you needed at least a “B” to stay in that level (can’t remember which) but I remember more than 1 student talking the Professor into upgrading his paper.

    BTW loved your analogy of the General “establishing his authority”.

  4. Since incentives operate on systems made of humans, for every incentive offered someone will try to game that incentive. Even the most thorough systems analysis with the most intensive consideration of second, third, and even fourth order effects will be gamed. An eternal tug-of-war between incentive provider and incentive gamer is inescapable. It’s when the system loses the ability of each player to one up each other that things go wrong.

  5. My wife, who is an OT supervisor, notices this technique with each new Director. The hospital has had four in four years. The best way to describe it is the old standard, “The floggings will continue until morale improves.” It’s a good this that Edwards Demming has died. He would have a heart attack every day watching this nonsense.

  6. One would think a businessman as Mayor would be wise to this, particularly after the way the New York City public school employees faked school performance numbers. Businessmen as politicians just don’t seem to be much of an improvement.

    Bloomberg thought he would be the education mayor. Jackass.

  7. L.C.-

    Except where employees’ incentives are aligned with those of mgmt. This is possible in some cases, for example, in financial firms that allow proprietary traders to engage in personal trading as long as it’s disclosed and there are no obvious conflicts. The same might be true in a research institution that gave its employees a percentage stake in their profitable inventions (though I don’t know if any such institutions actually do this). But these are cases of organizations that actually produce something, and are exceptional. I suspect that your point is valid in most organizations.

  8. Thus was the management Kabuki dance, as I observed it – poor me, as a front-line supervisor! Central authority (in this case, Squadron HQ and higher) insists on a bloody stupid course of action. Secondary field managers, of which I was one, try our best through the channels available to us as field managers, to dissuade HQ from this stupidity, a stupidity which is so flamingly obvious that it is even apparent to our lowly subordinates … who also objected to the course of action, loud and long, and with many pertinent details. To which, we could only say – ‘this is what squadron says’ and stifle in our communications with them, all mention of those many and detailed objections that we had made, only to be overruled by higher authority.

    And then, two months, or six months later, the manifest stupidity of what we – the front-line supervisors had said – was made obvious, in that Squadron HQ had reversed that policy, we were left to look like total idiots, in front of our subordinates. We had to back a stupid, ill-conceived policy, which everyone on the ground-level was seen as a stupid, ill-conceived policy, and one what we had advised against in the first place … and then, our superiors reversed that policy, without a backward look. Every time they did this, they undermined the authority of their front-line supervisors. They never saw this connection, and they did this repeatedly. They pissed away the credibility and the professional morale of their front-line NCOs … and then blamed us for the subsequent lack of morale.

    This is why I became a retiree more likely to work for small companies. Because then, my chances of being able to tell the boss that they’re full of it is much more likely to be heard. Also, my ability of being able to walk away from institutional stupidity is practically limitless.

  9. Sgt – I remember reading a book years ago of the best run big companies – HP and IBM were among them – and they all had one thing in common – a “bottom-up” approach to problem solving.

    They actually listened to the people on the front lines doing the work – interacting with the customers, making the things in the factory….

    It was exactly what you described that gave me a love-hate feeling about the Army – the camaraderie is something I miss even 40 years later but the mindless bureaucracy…..

    I once was so disgusted over their handling of a friend’s problem – not being paid for 3 months – that I wrote the (then) Chairman of the Join Chief of Staff – Gen Creighton Abrams.

    I told him exactly what I thought of all this mindless bureaucracy.

    Well, the weeks went by and I heard nothing until one day I get a letter with 5 gold stars – addressed to “Pfc Brandt”.

    I’m thinking “oh !@#$%^ (the decorum of this august blog prevents me from recreating the exact words), But having jumped the chain of command “just a bit” I knew there could be a few people angry at me on up the chain.

    A few.

    Anyway with some hesitation I open the letter and it only had a few sentences, starting with

    Dear Pfc Brandt:

    “I just want you to know the bureaucracy gets to me too sometimes”. …..

    Anyway my friend’s pay problem seemed to be immediately solved and I never heard anything about |feedback”.

  10. “Except where employees’ incentives are aligned with those of mgmt…”


    Even there incentive gaming can occur. The father of a friend for many years was a prominent steel detailer. Since his skills either as a direct producer or supervisor of other producers was the primary product he sold, he entered into 50/50 general partnerships with “business guys” twice over the last 30 years. The business guys would concentrate on the business side and my friend’s dad would handle the product management side. They were equally management, employees, as well as owners. Both times the business guys gamed the numbers to increase their personal take despite the specific contractual 50/50 split. Both times the partnerships were dissolved and my friend’s dad went back to the perfect alignment of incentives where he was owner, manager, and employee.

    The trick with scaling up a going concern is giving employees incentives that approximate ownership (proportionate profit, disproportionate loss) while spreading possible outcomes that accompany ownership (proportionate profit, disproportionate loss) around to insure against bad things. Too far in one direction, you get too much caution since your playing solely with your own money. Too far in the other direction, you get too much gambling since you’re playing with other people’s money. Some organizations I’ve worked for were so transactional and focused on material incentives that everyone became a mercenary free riding on the corporate treasury despite a nominal profit sharing plan. Others had that intangible esprit de corps like Apple’s Cult of Steve complete with Reality Distortion Field that drove employees to extraordinary outcomes despite the lack of a clear incentive mechanism aligning employee interests with management interest or ownership interest. Even then some are still gaming the system in spite of their cultic devotion.

    It’s hard to catch lightening in a bottle, especially when management is made up of freshly minted MBAs whose minds have been destroyed by modern business and financial theories. The only general principle I’ve found is retaining capability to fail fast and adapt and aligning incentives for all with that principle.

  11. My wife tells the story of her father being promoted earlier in his career to be the general manager of a computer production facility that had been having serious problems in quality control and employee morale.

    When he got there, he found a nicely appointed office section attached to a factory which was badly deterriorated. He asked for suggestions at an employee “get to know the new boss” meeting, and went right after those items which were clearly the most popular.

    Remodeled bathrooms and locker rooms, upgrade the cafeteria, clean and rearrange some of the layout in the production area.

    He then let it be known that he would be having lunch in the cafeteria every day, and that anyone who wanted to talk improvements was welcome to sit at the table with him. The rule was simple—talk about how to do things better in the future, not just endless bitching about the past.

    The plant went from losing money and being near closure to solid performance and profitability within 2 years.

    More promotions, and more problem assignments, were to follow.

    I attended his retirement party several years ago. Quite a few of his former peers and subordinates spoke of working with him.

    Two themes were mentioned repeatedly—his patient coaching, and his integrity.

    It was a fine evening.

    Amongst the endless horror stories, it is good to remember that 15 trillion dollar economies are not developed by fools and brigands.

  12. Very Retired

    For years the head of Intel – Andy Grove – used to work in a cubicle just like everyone else.

    I think they still are that way –

    They encourage “spirited” debate at meetings – don’t want people to “hold back”.

    A good running factory has more to do with good people than robots…

Comments are closed.