IDF Unit 9900
“Imagine a tourist arriving in a foreign city,” the Israeli intel officer tells me, “the first thing they do is open Google Maps and look for a restaurant. Google helps them find a place. Helps them navigate. Helps them get there on time. We do the same.” Well, not exactly. The augmented reality mapping application Lieutenant-Colonel “N” is describing is designed to find hidden terrorists, not restaurants. “Mistakes can be fatal,” he tells me, “we need to get the right house on the right street.”
Welcome to the battlefield of the future—artificial intelligence, multi-source data fusion, augmented reality. Everything edge-based and real-time. Except this isn’t really a battlefield, as such. “What happened to us,” the officer tells me, “is that our enemies have adopted a technique to merge into urban areas populated with civilians, we need to unveil the enemy, precisely, and stop the threat.”
So, now you start to get the picture. Think Google Street View—except it’s not Google. And an augmented reality overlay that comes from the fusion of multiple sources of highly classified intelligence not big tech’s cloud servers And if that isn’t enough, there’s also AI running pattern analytics on prior enemy tactics, techniques and procedures to infer what a hidden enemy is likely to do next, in real time.
This is military augmented reality and it’s not unique—such systems are under development, gaming-style headsets overlaying friendlies and likely combatants, helping targeting and the avoidance of blue on blue. Israel’s new system is different, though. The augmented reality comes from the fusion of multiple intel sources, the intent is not to present ground troops with an advanced gaming-style view of the battlefield, but to use live data to infer where actual targets are hiding.
Picture this Street View lookalike again—no screenshots, I’m afraid, it’s classified. Arrows and graphics explain to a soldier on the ground why the third-floor apartment with the wrought iron balcony is deemed a hostile environment, why anyone exiting the building can be considered a combatant. The intent is to root out threats, but also to keep others safe, to avoid collateral damage. “We need to make sure we only target the aggressor and not any civilians,” Lt-Col “N” tells me.
Israel’s idea for this “intelligence saturated combat” has been a decade in the making. The new program sits within Unit 9900, the visual intelligence operation (think maps, satellite imagery, image analysis) within Aman, the country’s military intelligence directorate, and sister unit to the better known 8200 signals intelligence unit. Unit 9900 generated headlines a few years ago when it was reported that it was recruiting autistic teenagers for their unique analytical skillset.
As Lt-Col “N” describes the work of his team, “the development of 3D mapping that is as realistic as possible,” he continually refers back to the modern-day explorer’s Google Maps view of the world, that feeling of familiarity. Yes, the location might be strange, but the viewpoint is well known, understandable in absolute real-time. “We have to build something with that user experience,” he says, “our soldiers crossing the border for the first time must be familiar with the environment.”
This “intelligence saturated” viewpoint can be presented to the solider on a smartphone or tablet, all off the shelf and “mostly Android,” or streamed directly into their binoculars or weapons sights. “They don’t know where the intel comes from,” Lt-Col “N” tells me, “but it reaches their sights, their C2 systems in real time.” The officer stresses that all targeting decisions are taken by the soldier on the ground, not by the system itself, this is an aid, not a automated targeting system.
The challenges the new unit has overcome, I’m told, include distilling down this intel, “terabytes every day,” into what is useful and relevant, that’s the role of the AI, the pattern analytics. The window is short—soldiers are given five to ten seconds to decide on any action they take. They are trained in the field with the technology, their feedback hones the program itself, “what to develop further and what to ditch.”
Essentially, putting the complexity to one side, this is a 3D, photo-realistic map, “the backbone onto which we build our intel—preliminary and real time—to understand the area and what the enemy is doing in real time.” By mining data from previous combat experiences, the AI “recognizes patterns of enemy behaviour—and can understand where the enemy is and what they’re planning.” This is overlaid with real-time intel, including open-source data on the terrain and the environment.
There has been a lot of talk about the fusion of the cyber and physical domain in the last year, not least from Israel, which became the first country to mount a physical military response to a cyber attack. A few weeks later, the U.S. did the opposite. This new concept of an “intelligence saturated” battlefield can take the cyber domain and feed it directly to troops on the ground. Those same soldiers are heaped in sensors, everything feeding back to the central intelligence system.
Lt-Col “N” often refers to the “disappearing enemy.” He means the urban shadows where combatants and civilians blend together, disasters waiting to happen. Yes, this new style of AR combat is intended to sharpen responses, but also to avoid mistakes. The officer explains that the AR display provides enough information to let the soldier understand why a location has been deemed hostile—the final targeting decision is theirs, and if they don’t understand they won’t act the right way.
I’m told that this new program within Unit 9900 has become a development hotbed, learning its approach from industry. Inside the “joint lab” you’ll find intel, combat troops, cyber and comms, Israel’s Darpa equivalent, defense contractors, even start-ups. When something new is envisaged, it’s prototyped and given to ground troops to field test. Their feedback hones to capability or consigns it to the bin.
The military world has changed, Lt-Col “N” tells me, “we needed to imagine new methods of fighting—as much as possible we use tools created outside the defense industry. We take civilian and open-source as much as possible, we access research from all over the world to help us deliver state of the art products.”
I’m told that Israel has accepted that “mil-spec” is not always best—why not plug into the billions of investment dollars piling into mapping and AR and AI, repurposing those capabilities for this. “We keep the user experience as straightforward as possible… Google Maps is a good model—how you see the world as a tourist, when you know what you see and understand where you’re going.”
This new program is now ripe for international collaboration. “Our discussions with various countries fighting terror around the world show they’re facing the same threat, enemies hiding in urban environments. This concept brings together quick intel, enhanced by AI and connected to accurate mapping. That’s its innovation.”
No details on any other countries using the tech, of course, no specifics on intel sources—all highly classified. “I can tell you this is a real-time bridge between intel and soldiers—intel wants to keep their secrets, combat operators want that intel in real time.” Testing of the new capabilities started this year.
You can add this IDF program to the multitude of new AI, IoT and AR systems being procured and developed by military customers world-wide. The concept of real-time dissemination of live intel from multiple sources, right to a soldier’s C2 or sights is novel. The challenge here is that the soldier must remain the decision maker. If there’s ever any implication that targeting has been automated, then the military world will have changed and there will be no going back.