Using Scorecards to Rank Security Needs

By Craig Harper

Last week we briefly discussed the fact that security means different things to different people depending on their job function, point-of-view, or context. Often a purchase decision needs cross-functional teams comprised of the different stakeholders affected when evaluating Enterprise or Cloud Software security solutions. What good does it do for IT to mandate a system because “it has the best built-in IDS” only to discover that it doesn’t meet the Principle Investigator’s needs for compliance? In these cases we recommend a scorecard approach that will allow everyone’s voice carry the appropriate weight.

Creating a security scorecard can follow many different methods. One way I have found that works well for many different subjects and tends to be reusable is based on scoring needs based on the following 4 categories: Implementation, Differentiation, Criticality, and Complexity.

First, create a list of security Features and sub-features that you will apply to the four categories, such as:

  • Monitoring and alerting
  • Access Control, Authorization, and Authentication
  • Data management, Export, Longevity formats
  • Authorized personnel activity accounting
  • Version control, Auditing
  • Reporting

Next, assign a numerical value to each security feature. Each feature is then rated against the four categories.

Implementation

  1. In concept / design
  2. Partially implemented/integrated
  3. Implementation complete, not tested
  4. Tested/White Hat/ Black Hat
  5. Implementation/Integration/Feature deployed & in production

Differentiation

  1. Low Differentiation
  2. Medium Differentiation
  3. High Differentiation

Criticality

  1. Nice-to-have feature
  2. Important feature
  3. Must-have feature

Complexity

  1. Simple effort to implement
  2. Moderate effort to implement
  3. Medium effort to implement
  4. High effort to implement

Feel free to change the descriptions in the lists to match the criteria that you have designed for your scorecard. Avoid over granularity, or over analysis.

My scorecards often look like this:

Feature Scorecard Example
Feature Scorecard Example

In the end you are looking to get a list of features and sub features from all of the stakeholders that your cross-functional team will then score. This scorecard can then be used to evaluate security software solutions, and will help identify the best option to meet your company’s unique needs.