Beyond Bad Grades
In February, the House of Representatives Government Reform Committee announced that the federal government had achieved an overall grade of D+ on its 2004 Federal Computer Security Report Card. However, there is reason to believe that agencies are making greater progress than the simple letter grade might suggest.
Given the current emphasis being placed on Federal Information Security Management Act (FISMA) compliance, it could be that computer security progress is being masked by a faulty grading system.
It might be enlightening to reassess how the security posture for each organization is communicated. Does each individual grade really tell the whole story? Showcasing how the letter grade was derived might suggest that a lot of progress was made in an area not weighted heavily in the Office of Management and Budget (OMB) grading scale.
Government agencies are currently issued Report Card grades based on seven criteria:
annual testing (20 points);
plan of action and milestones (15 points);
certification and accreditation (20 points);
configuration management (20 points);
Incident detection and response (15 points);
Training (10 points); and
Inventory (no points — only deductions in increments of 10 points)
Also, there are criteria subcategories with differing point allocations. Training, for example, is split into four increments, with a 10 point cumulative allotment:
percentage of agency employees (including contractors) and those with significant IT security responsibilities that received security training and awareness (4 points);
percentage of employees with significant security responsibilities that received specialized security training (4 points);
whether or not the agency provided the total training costs for fiscal year 2004 (yes = 1 point); and
whether or not the agency explains policies regarding peer-to-peer file sharing in IT security awareness training, ethics training, or any other agency-wide training (yes = 1 point).
Here’s the issue: If an agency invested heavily in addressing the training requirement, arguably an important element of a comprehensive information security program, the letter grade might not reflect the progress made.
Hypothetically, if an agency fully meets six of the seven Report Card requirements but does not obtain any of the 10 points associated with training, the agency still receives a score of 90 percent or A-. Is this truly an accurate assessment of the agency’s computer security? Is it OK to have untrained people? FISMA grades do not delineate between sub-par performances in one area vs. the overall Report Card score.
This example helps demonstrate that the letter grade by itself may not be the most accurate way to define an agency’s security posture and certainly should not be used to gauge overall effort and resulting progress.
Perhaps a structured dialogue between chief information security officers (CISO) and OMB regarding the grading elements would help. It would be interesting to understand what CISOs value the most in terms of securing their agencies’ IT infrastructure.
CISOs possess a great wealth of knowledge that could be leveraged and might be used to refine grading element weights.
This communication could help other agencies to understand how the Agency for International Development managed to jump from a C- on its Report Card in 2003 to an A+ in 2004. The individual best equipped to answer this — and to help to improve the state of the government’s IT security — is the agency’s CISO, who can provide feedback to OMB as well as advice and guidance to other CISOs.
There should be more to FISMA than a simple letter grade. Real progress can be determined by looking at the underlying elements that make up the grade. Regular dialogue between CISOs and OMB will help foster rapid and continued improvement. Without such unity, it appears that there are 24 boats rowing in 24 different directions. In fact what is needed are 24 people rowing together to achieve a common goal — strong information security.
FISMA, in its entirety, is a powerful piece of legislation with the potential to transform the security of the federal government; however, there is room for improvement. A reassessment of Federal Computer Security Report Card point allocations, combined with insight and regular dialog among and between agency CISOs and OMB will help chart the course for success among federal agencies in managing their information security environments.
Richard P. Tracy is chief security officer at Telos Corp. He has managed numerous network security programs and software development for government clients.