One program I was on was full of very smart people doing very innovative things–great systems engineering team, cutting edge development stuff. Another was nothing but paperwork and fingerpointing; I literally spent a month checking that changes to a document had been made in accordance with the contract. On pen and paper. With a highlighter. Eight hours a day. (But they paid me like a software engineer . . .) But then, that sort of stuff happens in the commercial world, too, doesn’t it?

Sometimes it seems like the interesting work is happening all around me; someone *else* is solving some cool SIGINT or crypto problem I’d love to be in on. Sometimes I really love working on . . . well, jets and missiles and radar and stuff. I mean, dang. Who’d trade that for consumer electronics? Sometimes I feel a great deal of pride in what I work on; I do believe the stuff I produce saves lives and advances the cause of freedom.

Then again, I read about internet startups doing radical things with languages that are barely years old, tight groups of true hackers solving hard problems in innovative ways. And I look at where I work–software built layer upon shaky layer, beyond hope of badly needed rearchitecture. Something that ought to be a ten-second job, like changing a display color or a builtin delay–instead costs $60000 and takes three weeks. I can’t directly communicate with the customer to build a product that’ll work for him; instead I have to build it to a spec that was originally a compromise written to solve a poorly-understood problem. Oh yeah, and we’re working in Ada. On a VAX.

And I think, dang. I gotta get out of this industry.

I don’t mean it, though. I can learn Lisp or Ruby or whatever on my own time, and where else do you get to work on such cool stuff?

But, he tells us, it’s now sold by Boeing. Of course; it has to be. The government’s procurement labyrinth locks out everyone but full-bore, full-line, full-time defense prime contractors.

The military ultimately gets a product this way, but it’s invariably years late, some multiple of the original budget, and often short vital capabilities.

There are two reasons for this: reactive regulation which means DOD happily squanders billions to keep contractors from stealing hundreds, and the bureaucratization of procurement under the worst SecDef, Robert S. Macnamara. Mac’s big project, the TFX, has been the model for defense boondoggles ever since. The USAF finally had a pretty good subset of the capability they were supposed to buy by the time the F-111 was an F model; the Navy version never achieved anything but killing test pilots and the Navy was able to wriggle out of the contract after Macnamara split (to the World Bank, where he brought his arrogant quantitative “expertise” and produced the Third World debt crisis singlehanded).

The Air Force version of the plane never performed on par with the aircraft it replaced, the 1950s vintage F-105. Not in speed, payload, range, maintainability, or survivability. (Macnamara feared this, and ordered the 105 tooling destroyed. Then the Air Force ran out of the things when he kept sending them against defended targets piecemeal “to send a message” to the DRV. Apparently he didn’t trust Dean Rusk (who would? But that’s another story).

Anyway, Mac was the guy who began screwing up procurement. The C-5A was another procurement boondoggle that ultimately resulted in a usable airplane, but only a handful of the things because of the magnitude of the overruns. The overruns didn’t happen because of Lockheed (which at the time was building classified systems and space systems on time and on budget with minimum oversight). It happened because of incompetent DOD micromanagement, which is now hard-coded into the department’s DNA.

Ask computer geeks of a certain age about another DOD brainstorm — standardizing on the ADA computer language.

I think the problem with gov’t contracts that was being alluded to was not so much the human oversight, but all the contractual and regulatory stuff. You can’t just plow ahead and build System A. You have to have an IMP, a SEMP, an IMS, an SDP, you have to hold an SRR, PDR, CDR, TRRs, calculate BCWP and BCWS, and deal with many dozen other acronyms. That’s all enormously expensive and may add little to the final product

As to education, I’m a Ph.D. EE and it’s too much education for most employers in and out of the government. I’ve had managers directly challenge me on that point. Live with the fact that Ph.D.s tend to be stubborn, you wouldn’t have gotten the Ph.D. otherwise, and stubborn folks can be troublesome to integrate into teams, so without a proven record of teamwork a Ph.D. may have more trouble than an MS finding a job, at least initially.

As to being a systems engineer, been there, done that, was told I was good at it and still got out of it. Even in the commercial end of things it really tends to be very conservative and, frankly, boring. The degree of freedom to innovate is inversely proportional to the level of the component in the system (think of the level of acceptable risk allowed in an IC as compared to the level of risk allowed to a communications system in a satellite), so if you’re one of those really creative folks systems engineering can be a frustrating specialty since getting buy in for anything really innovative can be an excruciating process.

If you’re unhappy with government oversight, the two most likely reasons are (1) the overseers don’t know what they’re doing but you do, or (2) they do know what they’re doing, but you don’t.

Now that the Feds have mandated drug tests for their contractors, my choices are to work in the private sector or do my nuclear physics for a less invasive and more appreciative government.

Contractors have to deal with the same issues of government slowness, and it is harder on young people who want to go in, get their hands dirty and get the job done. You have to go through an elaborate “requirements kabuki” that takes months in order to do a month or two of software development in many cases! It also doesn’t help that many times the government doesn’t plan ahead, and contracts end up with no support funding, leaving them at the mercy of any bugs that exist in them. All too often, software development work that is done for the federal government is done to replace functional systems that just had no support contract, and have to be rewritten from scratch because no one can maintain them now.

I alluded to it in my own post, about people in their late 30s/early 40s being pushed farther along their curve than historically typical. That’s typical for defense.

I remember shopping around for a while in the late 90s myself, after having worked defense for a few years already. I interviewed with a cell phone outfit, and I was asked by the hiring manager what sort of design tasks I was intreested in. Well, I was a sythesizer and PLL designer at the time, and I was looking for more perspective so I said either those, or maybe receiver chains.

He was looking for answers a lot lower down on the food chain.

I had the direct impression that the reason those phones are so advanced and come out on tight schedules is that they’ll have fifty or a hundred engineers working on a flip phone the size of a short stack of credit cards. By contrast, at my next job, my first task was a full, specialty frequency converter card. When I asked the phone house about system engineering, they (that group, at least) didn’t even have dedicated people doing it, it was just kinda informal by the older guys.

A few other scattered notes:

1) Salary increases can be gotten by job hopping between defense gigs. Anyone not bright enough to figure this out is not qualified to design weapons.

2) It’s not so much why do the systems guys have to retire at age 65, as it is, how are you going to stop them?

And it’s not just a business perspective where they decide they’ll be systems integrators and buy innovation. They simply don’t know how to innovate, or even buy innovative technologies, so the innovators are repeatedly hamstrung when they come up with new ideas. It’s painful to be innovating and turned down, time and time again.

My second reason for leaving the defense sector was the lack of seriousness in the mission. GD, BAE, Boeing, etc. didn’t have a mission statement or corporate culture that said “we deliver solutions to our armed forces so they can beat the bad guys.” It wasn’t even clear that the corporate cultures BELIEVED in bad guys anymore. Nothing like watching Loral give away the store to China to make you feel that. But the culture of protecting the USA wasn’t there at the top of the organization, and wasn’t there in the middle and low parts, either. Most engineers didn’t feel any national security loyalty anymore–didn’t understand the point of a security clearance, didn’t understand why a chinese-or pakistani-born-national-now-naturalized shouldn’t have a clearance. Engineers aren’t any more politically conservative than any other group of folks, and with one entire political party that’s unserious about defense, it’s not clear that they could create such a corporate culture now anyway.

But the NYT misses the mark in other ways. It really isn’t the money. Engineers and scientists with clearances receive VERY competitive packages all of the time, and job hopping will increase your salary quite rapidly. NYT’s comments about doctoral candidates also misses the mark. The defense sector doesn’t want them, largely they are viewed as expensive elitist who can’t be trained, can’t be managed, and can’t become part of the team. PhDs don’t support their mission. A tiny number may be a necessary evil, but the defense sector would rather hire an engineer with a master’s and leave the phds to academia and think tanks that they partner with for NSF funds.

But the last reason I left was because I could see the writing on the wall. FCS was getting plundered of innovation, and so were all the other projects. When Obama is elected, the defense sector will fall on even harder times.

And Mr. Simon is correct – the good engineers are ones who love engineering and/or really believe in the mission.

Here in the nuclear power industry, we too recognize the need for engineers starting their careers to join the team. There seems to be lots of interest in nuclear power now that the political winds have changed so dramatically the last year or two.

Interesting. I worked on the FAA’s original route control system in ‘67. TTL and wire wrapped boards. SUHL was better logic. TI had a better price. Which is why you never heard of SUHL.

As to systems engineering. I don’t understand why people have to retire at 65. Just when they are starting to get good.

The only engineers who are any damn good are those who do it for love. I can’t think of any school in the world that can teach that. None. Zero. If you have that you don’t need school. (I’m non-degreed aerospace). If you don’t have that there is no school that will help. The wash out rate should be high. That way mostly lovers will be left.

BTW the military world can bypass the bureaucracy when the need is critical. I never found them to be a handicap. The biggest handicap in mil/aerospace engineering is rigid mind sets. There is not enough zero budget thinking (what would we do if we had to start from scratch).

There’s one other big plus that no one has noted here – in federal civil service you will have a much higher degree of responsibility much earlier than in industry. I remember a contractor telling me he spent his first 5 years working on landing gear components at a drafting table (pre-CAD). Within 2 years of hiring, I was running wind tunnel research programs. By the time I had been in civil service for 5 years, I was managing a million-dollar flight test effort, with test pilots, mechanics, instrumentation technicians and engineers working for me. With 15 years experience I was flight test manager for a major Army helicopter program. My contractor counterpart was 20 years older. That’s a selling point you don’t usually hear mentioned by Government recruiters.

There are a few pockets of interesting defense work, but for the most part, daily existence for defense industry engineers is pretty much sit-in-the-cube-stare-at-the-tube. It’s not often that you see direct results of your work, and often boring enough that you wouldn’t want to (you checked off all the things on this list that appear in that document! Yeehaw!). You end up living in and dealing with large bureaucracies on slow-moving programs (reforming the defense acquisition system is horribly overdue, but that’s another subject). Gov’t pay, especially at starting levels, is so poor that it’s a wonder that they get any good engineers. You can make more money at the contractors (still, anybody much over $100K is making a _very_ good salary for this industry), but the tradeoff is shaky job security. Nothing like telling your wife and kids that your program has been cancelled and that you have to move them across the country (again) to stay employed.

So I would say a decision by young engineers to go into another industry is pretty rational. I’m a little skeptical of the conclusions drawn by the article, however, that this lack of engineers is driving cost overruns and technical problems on defense programs. I believe that’s a systemic problem (see several of the posts above) that that any number of superb systems engineers couldn’t change. Actually, this article is just a variant on the common articles decrying the lack of young people studying science and engineering, and how we are soon to have a major shortage of engineers with dire consequences. I’ve been hearing that since I was an undergraduate 30 years ago and it’s never come to pass. When defense engineers are too valuable to waste doing routine paperwork, start seeing pay raises that well exceed inflation, and maybe even have recruiters calling us with lucrative offers, then maybe I’ll believe aren’t enough people going into the field. Not much evidence of that so far…

Now I do HVAC work for an architecture and engineering firm. I mostly like the job, but wow, it was a huge paycut from being a submarine officer. I am planning to switch careers from the engineering world to the business world later this year. I expect to be making a lot more in two years in a new career than I would if I were to remain an engineer. Considering that my wife’s salary is even more modest than my own, my engineering salary isn’t really enough for us to afford daycare for a few kids (or for us to go to a single income). We do okay now without any kids, but unless I get a new job, it will be really tight when we do.

To their credit, they’re getting one subtle part of the story correct, namely, the growing lack of trained systems engineers. Their anecdotes are extremely good ones, and I could compare more than one of them to problems I’ve seen on my own projects. To a degree, individual pieces of technology can be farmed out to smaller defense contractors, or to commercial outfits with the right types of design expertise. I don’t like doing it, because it usually results in headaches, but it does not necessarily result in disaster.

What you simply cannot farm out, though, are the systems engineers. For the Big Four, that is our claim to fame. Want to procure a whole aircraft? Your supply base of prime contractors is extremely limited, and it’s limited to those companies with the expertise and pockets to keep a large stable of systems engineers employed. Want to procure a major avionics package, or ground RADAR systems, etc? Similarly. Those are vastly complex systems stretching across every major discipline of engineering.

What they miss, though, is that this is a demographic problem. The problem is not attracting young engineers, per se– now that salary adjustments put defense contracting at least in the same ballpark as commercial engineering, we have young talent coming in the doors again, at least at my site. We have more youngsters, lately, than I can easily remember the names of. (And someone, get me my walker!)

The problem is the substantial demographic staffing trough of engineers in the age 40 to, roughly, 55 bracket. People 55 and over are retiring, right now. But people 40 to 55 have melted back into the civilian engineering fields, many as a result of the dot com boom back in the late 90s. This is the pool from which good, solid systems engineers are drawn– engineers who have mastered (and not just in an academic sense, sorry my young PhDs…) at least one and often two fields of engineering expertise, even if they’re just different flavors of electrical or different flavors of mechanical engineering. These are the guys who have the perspective to notice weird and undesireable interactions between different sub-systems designed at different sites by different specialities, and have enough weight to slow people down and fix the problems in advance.

We’re not completely absent those people, but in an historical sense, we have less now than in the past. They cite responses by, e.g., Georgia Tech, to develop systems engineering curricula, but this just makes exerienced defense contracting engineers giggle that slightly manic giggle we get when we know we have a real problem. They’re nice kids, bright, eager, motivated, but they’re not systems engineers in a practical sense. Again, sorry my young systems engineering students and recent graduates, but… do yourselves a favor and double major in a traditional engineering field, too. Systems engineering can’t be taught. Systems engineering is almost defined as, “The stuff no one can figure out how to write a text book for.”

That’s the real problem, as guys in their thirties are pushed farther along that curve than they should be, so that we have replacements for the next major wave of retirements– those guys have to learn from the ones we still have, who are in turn scarce and over-worked as a result.