October 1, 2012  

Small devices and big data

How to bring intelligence to the tactical edge

In Iraq, U.S. forces who recovered computers used by al-Qaida consistently found Google Maps information on them. Insurgents were using the same databases as U.S. forces to view streets, consider get-away routes and plan ambushes.

The discoveries showed how one kind of U.S. tactical advantage was eroding — and they underscore the vital need to improve the information flow to U.S. troops, all the way down to individual soldiers on patrol. We will derive our future advantage from the ability to store, access and analyze unique data, and deliver those resources through our networks to the point of need, better and faster than our enemies.

Two of the most important axes for development are moving information along the last tactical mile and automating the analysis and handling of vast flows of data.

The Last Tactical Mile

The primary driver for battlefield network evolution is the hard requirement to move data to and from the last tactical mile. Until about three years ago, the battalion level was the absolute end of the chain for intelligence. Today, we are moving information to and from company intelligence cells — critical, particularly for highly decentralized, semiautonomous operations such as Afghanistan. Platoon-and squad-level personnel make daily contact with local village elders, gleaning a vast amount of familial, tribal, religious and economic information. They gain a unique understanding of what is happening in their region because they know what the population is doing, what contact they have with the enemy, and how we can assist them.

Previously, there was no way to warehouse and share this information. Now, with a new network strategy, the military is capturing this important tactical data from company-level intelligence cells, moving it across battlefield echelons and providing universal access to it for analysts working at a variety of higher echelons.

One key component is mobile devices — smartphones, tablets, laptops and the like. New software and devices promise to allow soldiers to see the entire battlefield without being tied to local processing and analysis shops.

Relevant ISR to the Edge, or RITE, is a 3-year-old Army program meant to revolutionize network capabilities and provide intelligence, surveillance and reconnaissance data to soldiers via technology-agnostic mobile devices. Soldiers can access military applications to download data and obtain ISR information relevant to their missions from aircraft, manned and unmanned platforms and satellites in space. Unlike other efforts, RITE describes a complete LANDISRNET architecture that integrates space, aerial, terrestrial and foundation layers. [Editor’s note: EMC, the author’s employer, holds a subcontract to provide some data storage to the RITE program.]

Another program, Army Tactical Networks, should be a model for the future. It provides networked systems and communications capabilities to operational units in new ways.

Moreover, the network itself is improving. Instead of providing access to perhaps hundreds on a battlefield, the military is working to enable concurrent access for millions, on and off the battlefield, around the world. This work is urgent. Today, people on the battlefield are routinely appalled at the gap between what they can do at home with their smartphones and what they can do with military-supplied comms gear in the field. A force that cannot keep up with the commercial world cannot keep up with the technologies used by its adversaries.

Rapid adoption of commercial wireless networks is called for, and one way to accelerate the process is to allow soldiers to test new technologies early and often. That’s why the Army has begun conducting Network Integration Evaluations, semi-annual testing events designed to improve tactical communications networks. Soldiers work with various new systems in an operational environment and then report which ones were useful, which ones need improvements and which ones should be dropped. The process is helping the Army more quickly evaluate new technologies from government and industry, improve or integrate them before rolling them out to combat units, and avoid investments that lack merit.

Taming Big Data

A second driver of network evolution is the advent of Big Data. In the past, DoD has been platform- and sensor-centric, with little but lip service given to the networks that connected them. We have happily invested in satellites, planes, helicopters and unmanned platforms and pursued an incredible range of high-technology sensors. But in virtually every instance, the network required to move these vast amounts of Big Data — or Large Data, as those in the full motion video world would say — has been a lesser concern. Today, programs such as Gorgon Stare, Blue Devil 2 and ARGUS are beginning to produce hyper-scale data on the order of one petabyte a day. No network can move all of the data we collect, and no analytic tool can ingest it. Currently, the best we can do is about 10 terabytes an hour, leaving about two-thirds of the collected data unreviewed. This is no embarrassment of riches; it is an inundation that renders us unable to make predictions or even offer useful analysis.

There will never be enough bandwidth to carry all the terabytes and petabytes that our sensors can scoop up; we cannot change the laws of physics. But new technologies are available to more effectively decide which information is needed where and deliver it in the most efficient way. Several approaches include:

• Increase on-board processing of data. Each sensor, or at least the platform it sits on, needs to analyze and triage.

• Integrate analytics into data storage. This technique harvests the value from raw data where it is initially stored, and thus burdens the network only with brief reports rather than hundreds of terabytes of information. Even more unique attempts at Big Data analytics incorporate massively parallel, shared-nothing approaches that do analytics in a totally decentralized architecture. We are starting to see this capability emerge from a number of storage and network vendors.

• Move to automated tiering of data storage. This works on the principle that all data is not equal, and that we can store data intelligently, based upon use. Data viewed in the past 10 days can be stored for instant access, but data that has not been viewed in 10 days can move to a lower tier. Data that has not been viewed in 100 days can be moved to an even lower tier, etc.

Toward Smart Networks

As the momentum toward a “bring your own device” world continues to build, we are beginning to see the power of every device becoming a node on the network. The ability to enrich the data within the network and capability to share concurrently at all levels will become a driving force. The smart military networks of the future will note when analysts access specific data, then notify others working on similar projects.

The words “cloud computing” have all but lost their meaning amid a blizzard of marketing pitches, but the proper and most revolutionary understanding is “everything as a service.” In an EAAS world, you begin to abandon thoughts like “where is my data?” or “how is my data protected?” and begin to imagine how you can use and gain value from your data.

When you make a purchase online and additional recommendations pop up, you are seeing real-time Big Data analytics in the cloud. The vendor is harnessing information about what millions of other buyers are doing at the same time. The military needs to do the same. Why shouldn’t analysts around the world working on the same subject or train of thought have access to the results from not only their local team members but also analysts involved in similar research globally? If one low-level analyst is doing a search query and identifies a series of resources, other analysts executing the same or similar searches should have access to this information. In the smart networks of the new future, such information will automatically be added to a query’s search results. This is the power of the cloud and Big Data: to make connections and provide access to information, all the information.

Distributing analytics and moving only as much data as necessary will require research and development, but are essential to solving our data and network challenges in the military and distributed IT environments of the future.