Big Data in Government

In 2012, the U.S. Government invested over $250 million in Big Data that traversed throughout all federal agencies. Labeled the Big Data Initiative, the intent was to eventually incorporate Big Data processes to help in areas such as border control, health care, and scientific research.

In my last article, I talked about Big Data more in relation to consumer services, and added only a touch of how Big Data applies to government agencies. However, because of the Big Data Initiative, the U.S. Government is vested to produce viable solutions and technologies to bring the massive amount of gathered data into some sort of organized and readable consistency.

The U.S. Government collects data from every area that it touches. And, while the accumulated data may not reach consumer levels, the amount of is still enormous. Legislation, rules, compliance standards, state and local rulings, state and local agencies – the list goes on and on.

The amount of data the government faces is at its highest ever and has been reported that less than 1/3 of federal IT executives believe they have sufficient Big Data strategy. So, it’s increasingly becoming more important that mandates and rules are applied as quickly as possible, but also that technology is applied that can transform large data sets into visual, actionable intelligence.

Government agencies seems to be struggling with the mandate about as much as they are still struggling with the Cloud mandates initiated in 2010 that I talked about in my last article series on American City & County. Government IT is no different than any other business IT organization and are tasked with jobs that are highly demanding. Adding mandates here and there adds to the complexity and expanse of the job. Imagine constantly being behind on normal, daily work, and then being immediately tasked with projects that even some of the largest businesses are still struggling to make happen.

Specifically, the Big Data Initiative seeks to provide the following:

• Advancing data mining processes
• Increasing server storage capacity
• Big Data security
• Cloud-like infrastructure to support Big Data

Many Big Data and Cloud projects have failed in the Private sector, primarily due the “idea” of technology being the forerunner of the actual enabling technology itself. Still, getting Big Data under control can help in various areas.

There have been many new Big Data developments just this past year from various U.S. defense and government agencies, so there’s a glimmer of hope. Now is the time to understand the trends, their subsequent effects and what the future holds.

In the next article, I’ll dig deeper into the Big Data Initiative for the U.S. Government, and give you more comprehensive insight into how things are progressing and which areas make the most sense to focus on for now.

Rod Trent is the Engagement Director for the Penton Technology Group and the Managing Editor for American City & County sister brands, Windows IT Pro, SQL Server Pro, SharePoint Pro, Dev Pro, Power IT Pro, WinSuperSite and myITforum.com. He is a leading expert on Microsoft System Center and Cloud technologies and has more than 25 years of IT experience. Rod has written many books, thousands of articles, and speaks at various conferences and user groups. His professional focus is evangelizing technical community on the web and in person. Rod was a Microsoft MVP for 10 years and is a charter member of the Dell TechCenter Rockstar program.