In 1986, just two years after The Hunt for Red October took him from insurance salesman to best-selling author, Tom Clancy did what counts in the usually trite world of book promotion as a unique event: he spoke to staff at America’s National Security Administration.

Watching today, Clancy’s talk, full of one-liners and an uncomfortable number of quips about women, often feels like an impression of a late-night monologue. But bad jokes aside, it's clear that Clancy had US intelligence in his thrall. This was his second talk at the NSA, and not his last with the American state apparatus. (Clancy would go on to speak at the CIA later that year.) Among intelligence officials, there was real curiosity about how Clancy, a civilian, had come to fill The Hunt for Red October and its thematic sequel, Red Storm Rising, with so many technical details about, say, Russian submarines. The answer was, perhaps, a let-down: a few reference books and a publicly available war game manual. But judging by what else Clancy said in that talk, there may have been something more at play than mere library work.

On stage, wearing a suit and his signature aviator sunglasses, Clancy discusses the book he's currently working on, Patriot Games, and how the tactics of its central antagonist, the Irish Republican Army, are becoming increasingly prevalent around the globe. Instead of sending in troops, the IRA was deploying guerrilla tactics in their campaigns against the British, using fear and surprise (and, of course, bombs) as their weapons. “Wars are just too expensive in monetary terms [and] in terms of the damage that they do to the country that starts the war,” Clancy explains of the insurgents’ motives. But with terrorism, he says, “you can deny you had any part of it. [And] if it works, you can still get your message across.”

Watching that speech today, it’s hard not to feel like Clancy is telling a story about the world that US intelligence and defence would become embroiled in fifteen years later—a world where the traditional rules about military might no longer apply. As Clancy says, the US might have the superior firepower. But only with good intelligence could it know where to send it—which, of course, America would later fail to do in the years running up to 9/11.

There are a million different reasons why 9/11 did happen that don’t have to do with what Tom Clancy said in an NSA lecture hall more than 30 years ago. (The funding of the mujahideen by those same American intelligence officials is one of them.) But Clancy’s talk, full of postulations about the conflict scenarios of tomorrow, is a prime example of something that has played out countless times since at least 1897, when H.G. Wells wrote War of the Worlds and, in the process, came up with the concept of a laser gun: a civilian writer, armed with little more than public information and an active imagination, describing the future of conflict in a work of fiction.

In Ghost Fleet, a lobster-shaped robot comes to the aid of American counterinsurgents in Hawaii.

For most of the 20th century, the world’s governments were content to let writers make their predictions about state affairs at their own pace: novelists would imagine far-out scenarios involving spaceships and robots, occasionally landing on an idea that, like H.G. Wells’ laser, scientists would go on to invent. (To that list we can add holograms, bionic limbs, AI and autonomous vehicles.) But in the 1980s, the US decided to bring the sci-fi writing in-house, inviting novelists Larry Niven and Jerry Pournelle into the halls of power to lay out the framework for Ronald Reagan’s Strategic Defense Initiative Organization—the infamous Star Wars program—in a document they wrote called “Space: The Crucial Frontier.” Later, in the 1990s, that job would be passed on to another group of writers—the SIGMA group—who would produce a series of breezily written white papers for the US government under the guise of what their website describes as “futurism consulting”.

Flash forward to today and the practice of futurism consulting is arguably more popular than ever. However, the government’s taste in literature has shifted away from the more out-there sci-fi of the ’60s and towards what’s now known as ‘fictional intelligence’—made-up stories based on real technology and trends in global politics that try to imagine what could go right, or very wrong, for state actors in the near future. The formats have changed, as well as the content. Unlike the government-style document written by Niven and Pournelle, writers of fictional intelligence deliver short stories to their military clients—brief narratives that explore the potential outcomes of a given military strategy or new technology.

Such writing has been embraced as a way to offset the endless white papers, dossiers and briefs that dominate intelligence and defence work. Unlike, say, a white paper on cyber warfare, a research-based work of fiction can convey real-life details within a plot that employs the same page-turning tactics used in popular novels. The result? Instead of putting your average military strategist to sleep, they (best-case scenario) stay up late into the night reading.

While some might baulk at the idea of matters of national security being conveyed by the Dan Browns of the world, reducing life and death matters to narrative arcs in a potboiler, others argue in favour of anything that gets our top brass reading—especially if it prompts the reader to think more critically about the implications of their decisions.

This is precisely what occurred with Ghost Fleet, a techno-thriller depicting a near-future conflict between the US and China, published in 2015 by coauthors August Cole and Peter Singer. A deeply researched book (Cole was a long-time defence industry reporter for the Wall Street Journal), Ghost Fleet is full of existing technology and realistic conflict scenarios, even if the plot—post-communist China attacks an unsuspecting America and occupies Hawaii—is pure fiction.

Ambiguity, a common trait in literary fiction but one with little applicability to someone trying to deploy a new fleet of submarines, is largely absent in Useful Fiction.

Like The Hunt for Red October before it, Ghost Fleet found an audience among readers with far higher security clearances than Cole and Singer. “I can’t tell you how many military officers told me that they read it in one sitting,” Cole tells me over the phone, “which is extraordinary, because that’s not what people normally do with, say, a white paper.”

The book had also demonstrable impact on the Pentagon; according to a report by Foreign Policy, when a US Army general based there decided he wanted office memos written with more flair, he instructed his staff to ‘ghost fleet’ their reports. For the first time since the invention of the marching band, the defence industry was encouraging its employees to tap into their creative sides.

Cole and Singer could have parlayed the government's interest in their book into a few speaking gigs, like Clancy before them. Instead, they turned Ghost Fleet’s success into a consulting business. The company they founded, Useful Fiction, helps militaries around the world ‘ghost fleet’ their reports. The company employs a mix of hardened military officials and Emmy and Nebula Award-winning writers, and combines their skills to craft narratives that make intelligence information accessible and entertaining for clients like NATO and the Australian Defence Force.

State interest in narrative fiction extends beyond the US. In France, Emmanuel Macron’s government has the Red Team, a group of science fiction writers and screenwriters tasked with generative potential future scenarios for the French military. Not to be outdone, the Canadian military once hired Karl Schroeder to write Crisis in Zefra, a novel set in a mythical African city-state where a group of Canadian peacekeepers attempt to fight an insurgency using a range of technology—some of it banal, some of it fantastic, but all of it in development at the time of writing.

The French are a little more intellectual about their work than their North American counterparts. Speaking with France24 earlier this year, Commandant Jean-Baptiste Colas of the French Armed Forces described the Red Team in the kind of language more befitting a social studies major. “France took a chance on having a group of authors who tackle issues relating to philosophy, sociology and psychology,” he said, adding in the most French way possible that “this is not necessarily what our Anglo-Saxon allies are doing”.

We may laugh at the Chronicle's plot points, but they highlight the fact that different countries will find existential threats in different places.

And they certainly aren’t. In Chronicle of a Cultural Death Foretold, written by the Red Team in 2021, the French military has to respond to a COVID-like virus by evacuating whole towns. As sci-fi plots go, it’s hardly a massive leap. But then comes the surrealist twist: in addition to the virus, the military also has to contend with a society that has fractured into small communities based on ideological beliefs so strong that their residents are literally blinded by them. (I’m not exaggerating here: in one passage, a vegan is unable to see the butcher shop in front of them.)

What this has to do with preparing France for future conflicts is anyone’s guess. (Is the text a metaphor for a siloed society? A lament for our post-truth world? A cheap shot at vegans? Unclear.) But while we may laugh at the Chronicle's plot points, they nevertheless highlight the fact that different countries will find existential threats in different places. Contrast the Red Team’s philosophical concerns with Useful Fiction’s focus on technology: all of Useful Fiction's books rely on tech that either already exists or is currently in development. Their narratives don’t jump more than one or two generations ahead of the present either, both in terms of society or technology. In Ghost Fleet, for example, a lobster-shaped robot piloted by Navy SEALS comes to the aid of American counterinsurgents in Hawaii. Its real-life counterpart is a biomimetic robot, also shaped like a lobster, developed by Northeastern University’s Marine Science Center with support from DARPA.

While working for the Australian Defence Force, Useful Fiction drafted An Eye For a Storm, a series of fictional vignettes that imagine what military education—and conflict—might look like in the year 2035. There are obligatory nods to Australian cultural touchstones like flat whites and surfing, and young recruits are brought in through e-sports streams and trained to pilot autonomous armoured vehicles.

The narratives are sometimes cringe-y, and the writing can be flat-footed or shallow. Writing in Dirt, a newsletter about digital culture, critic Peter Matthews asks of fictional intelligence, “Who needs psychological depth when we are learning what the future will look like?” It’s a snide question, but an apt one: when your goal is to relay information, you can get away with less-than-literary narratives. (It also shouldn’t be surprising that ambiguity, a common trait in literary fiction but one with little applicability to someone trying to work out how best to deploy a new fleet of submarines, is largely absent from Useful Fiction’s oeuvre.)

But if fictional intelligence largely fails as fiction, how well does it work as intelligence? We may have to wait a few decades to see whether the world’s generals are any wiser for having read Ghost Fleet. But the genre’s champions argue that anything that gets people to think outside the square has its use. “I don’t know that I’d suggest creating a national security strategy based on a novel,” Useful Fiction’s August Cole tells me, but adds that there are particular circumstances where a novel may be the best option. “In trying to communicate a national security strategy, narrative—particularly fiction—can actually help people understand what the world would look like if [the strategy you're considering deploying] is successful.” Whether that world looks more Star Trek or Blade Runner remains to be seen. All we can do for now is keep on reading.