John Walker Lindh, the pampered son of the affluent American middle class, a new age spiritual seeker, did something the hardened professionals of the Central Intelligence Agency were not able to do. He penetrated the al Qaeda terrorist group.

In all the words written about Lindh’s being found among Osama bin Ladin’s fighters in Afghanistan, this fact has received little or no notice. This is surprising. From the moment terrorists first attacked Americans more than 30 years ago, a constant complaint has been that we do not know enough about what terrorists are planning. Our technical means of eavesdropping are unmatched but the details of terrorist plots—the who, what, when of their attacks—are typically not discussed in ways that our technical means can discover. The terrorists know what we can hear or soon learn from leaks to the press by those given access to intelligence reports. Our government’s warnings about impending terrorist attacks since September 11 have been vague, in part, because the conversations we have monitored have been deliberately vague or misleading.

The only way to get the intelligence we need on terrorists, experts agree, is to get people working for us inside the terrorist organizations. This is what people mean when they say we must improve our intelligence on terrorism, as they do after every terrorist attack. President Bush reiterated this point in a recent speech. This point is so often repeated because we have not been able to do it. We are sometimes lucky enough to have members of terrorist organizations volunteer to work with us but we have not been good at putting people inside such organizations. Yet, if Lindh was able to penetrate bin Ladin’s organization, why can’t the CIA?

The CIA would no doubt point out that Lindh did not really penetrate a terrorist organization. He, along with many other foreigners, was part of an overt military force that supported bin Ladin. But in time might he not have been brought into the more clandestine terrorist organization? Would he not have been a good candidate to return to the United States to conduct operations?

The CIA might also point out that many of the terrorist organizations we face are based on ethnic, tribal or even family connections. The narrower the base of the organization, the harder it is to find people who can fit in. This is often true but is not true of al Qaeda, bin Ladin’s organization, which for some time now has posed the greatest threat to us. This organization rests not on a shared language or common ancestor but on an idea, a certain understanding of Islam. That is why Lindh and other foreigners could become part of it.

We also often hear that the CIA can’t direct people to penetrate terrorist organizations because passage to the inner sanctums of such organizations, where the real information lies, is paid only with a murder or bombing. While in some instances this might be the case, in none of the information available on the nineteen men who conducted the September 11 attacks is there any indication that they had to prove their bona fides with such actions.

Finally, the CIA might argue that the time-consuming work of finding, training, and minding someone who worked his way slowly, perhaps over years, into a terrorist organization would not be worth the effort because the odds of such a person getting to the inner decision making circle would be very small. This is a good argument. It appears that no intelligence service has had much luck with such long-term agents. But this argument holds for organizations like states and some terrorist groups that are hierarchical and centrally controlled. In such groups, the critical information is indeed found only in the innermost leadership circles that we would have to be very lucky to reach with an agent who entered the organization at a low-level. This would be particularly true if, as noted above, access to the innermost circle depended on ethnicity or the right family connections.

Hierarchical organizations such as states are the targets at which the CIA has traditionally aimed. In addition, many of the terrorist organizations it first worked against had the same structure. But al Qaeda does not. It and the radical Muslim world of which it is part, are decentralized and fluid. Many people are plotting. Small cells form and then break up, their members joining other cells. There is no one central decision making circle that an agent must reach before he can learn something useful.

Lindh made himself part of this world. If he had been working for us, even if not part of a cell planning an operation, he still might have reported on those providing support to the terrorists, those giving them money, fake passports, explosives or training. This information would have been invaluable. Over time, because of the decentralized character of the radical Muslim world, he might have come across information of truly critical importance or been helpful to our cause in a number of other ways.

Penetrating al Qaeda and the radical Muslim world is critical to the well-being of the United States. If, as Lindh’s example suggests, al Qaeda and the radical Muslim world can be penetrated, even by young middle class Americans, why has the CIA not been more successful at this important work?

An important part of the explanation, perhaps the most important part, is rather simple. Directing an agent who is trying to penetrate an organization like al Qaeda is time-consuming work, requiring careful handling of the agent. CIA case officers are rewarded for recruiting agents. They are not rewarded for running them carefully. Thus the incentives of the CIA discourage case officers from undertaking the time-consuming work of running long-term penetrations.

Can the CIA do better? Assuming that it can overcome old habits and realize that al Qaeda is a different kind of enemy, the CIA will still face the problem of incentives. These will be difficult to change. Some reform has apparently taken place over the last decade. But those in charge of the CIA’s spying operations built their careers working under the current incentives. They are not likely to see a need for change. But if outside pressure can be brought to bear on the CIA’s spying operations, those applying it can take hope from the experience of the FBI.

In the 1970s, FBI agents were rewarded according to the number of arrests they made. They therefore made lots of arrests, whether or not the arrests were of those engaged in significant criminal activity. So insignificant were many of the arrests that Federal prosecutors refused to prosecute the cases. A new FBI Director changed the incentives, so that mere numbers were no longer the most important thing. This change was not easily accomplished but it improved the performance of the FBI, helping it focus on important criminal activity.

Such a reformation at the CIA, which by altering the incentives that motivate case officers could lead to other necessary reforms, would be as difficult if not more so than the one at the FBI. The CIA’s inner workings after all are more shielded from public view than those of the FBI. But such a reformation is necessary if we are to have the intelligence we need to succeed in the war on terrorism.

David Tucker is a Member of the Board of Advisors at the Ashbrook Center for Public Affairs at Ashland University and an Associate Professor of Defense Analysis at the Naval Postgraduate School. He is the author of Skirmishes at the Edge of Empire: The United States and International Terrorism. The views expressed here are his own and do not reflect the position of the Naval Postgraduate School, Navy Department, or Department of Defense.