Searching for an Unfriendly Face

As a soldier scans the crowded streets of Baghdad, so do another set of eyes. The second set, located on the soldier’s rifle, belongs to a camera system that instantly recognizes the faces of potential threats.

The camera’s brain, a two-pound computer system worn on a belt or in a pack, is the first portable video system capable of matching faces with known threats in real time. It’s being developed by Frederick-based ACAGI Inc., along with researchers at the University of Maryland’s A. James Clark School of Engineering, through Maryland Industrial Partnerships Program funding.

“Soldiers have cameras on their rifles or helmets, but that’s only half the equation. They don’t recognize anything,” said ACAGI CEO Peter Spatharis. “If our system sees somebody it knows, it tells you, and it tells others so quick decisions can be made.”

ACAGI’s Image Acquisition and Exploitation Camera System (IAECS) is loaded with previously taken photos of potential threats. As a soldier’s camera pans over faces, the system converts them into digital images and compares them with the stored photos. Rama Chellappa, a professor in the Clark School’s Department of Electrical and Computer Engineering, developed one of the face recognition technologies ACAGI will use. Electrical and computer engineering graduate student Raghuram Gopalan will put Chellappa’s technology into the company’s system. ACAGI will license this technology from the university.

Chellappa’s face recognition occurs in three steps. First, the system searches for oval shapes and detects that a face exists in a video stream, and saves it as a digital image made up of tiny dots, or pixels. The system then creates a simplified map of the enclosed face based upon the light values of each pixel. Lastly, it compares the newly created pixel map with existing photos, and selects a similar match, if one exists.

“Identifying a face in a crowd is harder than trying to find a needle in haystack,” said Chellappa. “We’re trying to find a piece of hay in a haystack. Our job is to make that piece of hay as easy to find as a needle.”

While the IAECS will store tens of thousands of known face records, it also categorizes new faces it sees and saves them for later, an operation the Department of Defense calls–marking and tagging–its second-highest priority next to finding improvised explosive devices, according to Spatharis.

“A person who appears to be a soldier’s friend may be his enemy tomorrow,” said Spatharis. “We’re giving users the ability to recognize somebody and say ‘where have I seen him before?’ If soldiers encounter trouble at two different locations where the same face is seen, it’s obvious there is a connection.”

Ankur Srivastava, an assistant professor in electrical and computer engineering and an expert in embedded computing systems, will optimize the IAECS’s software and hardware to perform on-the-fly, increasing the system’s power performance by as much as 40 percent. ACAGI plans to release the first IAECS prototype by the end of December.

Additional product features will include three-dimensional sound, enabling soldiers to tell where a sound, or a round of fire, came from, as well as Global Positioning System location, giving users the ability to collect local image data with the camera and match it with GPS information to figure out where they are.

Spatharis also plans to target broadcast media with his product. The IAECS could help video broadcasters index and file footage as they take it, marking when people speak (through face recognition), and using speech-to-text technology to catalog what’s being said and when. Frederick-based Three Roads Communications Inc. is testing ACAGI’s system. The company is slated to train both military and broadcast IAECS users.

ACAGI’s funding includes two Maryland Technology Development Corporation grants–$152,000 from the Maryland Technology Transfer Fund , and $83,000 from the Aberdeen Technology Transfer Initiative (ATTI)–while the MIPS project is for a total of $109,000.

The Maryland Industrial Partnerships Program, an initiative of the A. James Clark School of Engineering’s Maryland Technology Enterprise Institute, brings university innovation to the commercial sector by supporting university-based research projects to help Maryland companies develop technology-based products.

Founded in 2004, ACAGI’s goal is to change the way visual information is processed, enabling images to immediately contain data meaningful to interpretation and exploitation at the collection point, in real time, prior to transmission, editing, and/or storage — allowing the rapid availability of information to make complex, immediate and far-reaching decisions. To that effect, ACAGI developed technology and products for the commercial media and defense markets.