InfoSec Book Excerpt: Security Metrics - Chapter 17

We like to read the latest and greatest security books, andsometimes the author and/or publisher is generous enough to share an extended with us - and you. We've selected one of our favorite topics as the sample chapter, but the entire book is well worth reading.

Step-by-step instructions to developing an effective security metrics program for the self-guided IT professional
Written by the developer of eBay's security metrics program, Security Metrics: A Beginner's Guide explains,step by step, how to develop and implement a successful security metrics program. It is a must-have tool for any networking or security practitioner looking to optimize an existing security program and demonstrate measurable results. The author illustrates how to communicate the value of an information security program, enable investment planning and decision making, and drive necessary change to improve the security of any organization.

Designed specifically for the needs of IT professionals looking to boost their skills in the fast-paced, ever-changing world of computer security, this practical resource covers project management, communication, analytics tools, identifying targets, defining objectives, obtaining stakeholder buy-in, metrics automation, data quality, and resourcing. It also provides details on cloud-based security metrics and process improvement. Templates, checklists, and examples give the hands-on help needed to get started right away.

Caroline Wong, CISSP, is the Director of Regional Product Management for Symantec's IT Governance, Risk, and Compliance products. Prior to this role, Caroline led security teams at Zynga and eBay. She is well known as a thought leader on the topics of security strategy, operations, and metrics, and has been a featured speaker at industry conferences including RSA (USA and Europe), ITWeb Summit (South Africa), Metricon, the Executive Women's Forum, ISC2, and the Information Security Forum. Caroline contributed as a technical reviewer to the Center for Information Security Consensus Metrics Definitions and is a founding member of the Cloud Security Alliance Metrics Working Group. She was awarded the 2010 Women of Influence "One to Watch" Award by the Executive Women's Forum.

Excerpted from Security Metrics: A Beginner's Guide by Caroline Wong (McGraw-Hill; 2012) with permission from McGraw-Hill.

CHAPTER 17

Security Metrics for Cloud Computing

We'll Cover

• What is cloud computing?
• Common business drivers that motivate organizations to move to the cloud
• The "new normal" for IT service delivery
• Common security metrics in the context of cloud computing
• Major cloud industry groups, especially the Cloud Security Alliance, and their efforts to describe cloud controls and related metrics

In the second decade of the 21st century, the world of computing is experiencing a revolution of sorts called cloud computing. After about a year of media hype and subsequent "cloud fatigue," IT professionals and other businesspeople are now contemplating what cloud computing means at a technical, business, and even societal level. This chapter focuses on cloud security metrics, drawing on knowledge you've already gained from previous chapters. In addition, this chapter examines how security metrics relate to cloud computing and discusses what resources are available today. Let's begin by quickly defining cloud computing and comparing what is new and not so new about the technology.

Cloud Computing Defined

In the United States, the National Institute of Standards and Technology (NIST, an agency of the U.S. Department of Commerce) provides the following broadly accepted definition of cloud computing:

Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models.¹

The following sections identify the characteristics, service models, and deployment models referenced in the preceding NIST definition.

Characteristics

The NIST definition specifies that cloud computing has the following five essential characteristics:¹

• On-demand self-service A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service's provider.
• Broad network access Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).
• Resource pooling The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Examples of resources include storage, processing, memory, network bandwidth, and virtual machines.
• Rapid elasticity Capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.
• Measured service Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the utilized service.

Service Models

NIST defines the three service models used in cloud computing as follows:¹

• Cloud Software as a Service (SaaS) The capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.
• Cloud Platform as a Service (PaaS) The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.
• Cloud Infrastructure as a Service (IaaS) The capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

LINGO

SPI is an acronym for the three general delivery models for cloud computing: SaaS, PaaS, and IaaS.

Deployment Models

NIST defines the four deployment models used in cloud computing as follows:¹

• Private cloud The cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may exist on premise[s] or off premise[s].
• Community cloud The cloud infrastructure is shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be managed by the organizations or a third party and may exist on premise[s] or off premise[s].
• Public cloud The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.
• Hybrid cloud The cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load balancing between clouds).

Cloud Computing, as Defined by ENISA

Given that IT transcends geographical boundaries, it is fitting to provide a non-U.S.-centric working definition of cloud computing from the European Network and Information Security Agency (ENISA):

Cloud computing is an on-demand service model for IT provision, often based on virtualization and distributed computing technologies. Cloud computing architectures have:

highly abstracted resources

near instant scalability and flexibility

near instantaneous provisioning

shared resources (hardware, database, memory, etc.)

"service on demand," usually with a "pay as you go" billing system

programmatic management (e.g., through WS API)²

In Actual Practice

The hype around cloud computing might lead you to believe that this is brand new technology, but enterprises have leveraged on-demand utility computing, SaaS, web applications, and distributed or grid computing since 2002. Cloud computing is a natural progression from service-oriented architectures, the growth and trust in the Internet, utility computing, and virtualization. Through widespread adoption and familiarity with these technologies, enterprises are outsourcing computing capabilities to the cloud, expecting great cost savings.

Cloud Business Drivers

This brings us to the economics of the cloud. A simple but critical question to evaluate at this point, adapted to the context of a book about security metrics, is "Will the economic benefits of moving to the cloud outweigh the security risks?" Let's begin with some economic reasons why enterprises are moving to cloud computing:

• Enterprise data centers are large investments, with choices for streamlining and cost cutting mainly coming from consolidation efforts. A lot of time and money are spent on upgrades, patching, license management, troubleshooting, incident management, and so forth.
  
• Data center utilization is often poor because of the requirement to invest in hardware and software to handle peak loads instead of having a resilient architecture that allows enterprises to expand and contract commensurately with business processing needs (elasticity).
• Cloud vendors are demonstrating they can deliver infrastructure, platform, and software cheaper, better, and faster.
• Larger cloud vendors have better economies of scale than a single enterprise.

This list is not exhaustive, but it represents some of the business drivers that enterprises (and the vendors they depend on) employ to move to cloud computing. To come back to the question, "Will the economic benefits of moving to the cloud outweigh the security risks?" you will want to be able to answer that question with facts and data, and to do that you will need to employ good security metrics.

Understanding the business and economic reasons that underlie why your company is or may be moving to cloud computing is very important to you as an IT professional. Many of the benefits of the cloud have little to do with technology and everything to do with perceived cost savings. As an IT pro, you want to be on the same page as your business decision-makers and be ready to demonstrate the relationship between security metrics and the cost savings. For example, you may have data about applying critical security fixes in your environment. It might be very beneficial to be prepared to talk about the costs associated with deploying a critical security patch so that your company can evaluate whether leveraging cloud services is a way to drive down overall operational costs. Another example, which may seem very squishy, concerns business agility. Suppose your company develops new products or services in-house but wants to leverage cloud computing to deploy them far faster than it could before. From a security metrics viewpoint, your application development security metrics may really resonate with executive management if they demonstrate that secure in-house products and services can be deployed more quickly and at a cost savings by leveraging the cloud. Not to oversimplify, but the point of these examples is to be sure you consider how your security metrics relate to your company's budget considerations.

Note

Numerous cloud vendors offer one or more of the three SPI delivery models for cloud computing.³ When your company engages one of these vendors, it is referred to as a tenant and usually is bound to terms and conditions outlined in a contract. Your company made a business decision about the capabilities of the cloud vendor and released a certain amount of control in exchange for those capabilities. For example, a cloud vendor may be able to do remote storage and backup faster, more reliably, and at a much lower cost that your company can. Perhaps the vendor uses best-of-breed technology that your company desires but cannot afford, and the vendor's personnel are experts at storage and backup, whereas your company would need to hire and train the right personnel.

In Actual Practice

The definition of a "tenant" in the context of cloud computing does not equal "customer." As an example, suppose your company leverages an SaaS offering, which in turn is hosted on a third party's cloud infrastructure or platform. In this case, both your company and the SaaS provider are tenants in someone else's environment. To your company, the data you store in the applications of the SaaS provider is most important. To your SaaS provider, if it is leveraging PaaS, then its application is what is most important; if it is leveraging IaaS, then its virtual machines, networks, and lower-layer services take precedence. In this example, it would be best to work with your legal department and contract experts not only to ensure that your contract with the SaaS provider clearly defines "tenant," but also to determine whether further due diligence is required if the provider is in turn leveraging other cloud services, to make sure you are not exposed to downstream risks. Regardless, the best practice would be to define "tenant" in your contract with the cloud vendor and not leave the interpretation open.

The New Normal

While cloud computing isn't technologically revolutionary, it is a new delivery model for IT services that presents a "new normal" we must adapt to. In this section, we'll look specifically at how metrics can be used to measure this new delivery model and thereby help us manage what is new or unknown. The following table provides some examples of what the "new normal" looks like compared to traditional enterprise computing:

Enterprise Computing

Cloud Computing

I have command and control over the data center.

Some or most of my computing processing needs are fulfilled by cloud vendor(s).

I have command and control over my data.

I'm still responsible and accountable for my data, but it's being processed outside of my control and I may have little visibility of where and how it is being processed.

I do background checks on all of my data center employees.

I may have no visibility of the hiring practices of my cloud vendor or its subcontractors.

I have control over data retention and disposal.

I don't know if my data is securely removed in the cloud, especially given the elasticity of the cloud and the ability to rapidly provision and de-provision resources.

My company's data and applications alone are running on my servers.

I am a single tenant in a multitenant environment, where my company's data and applications could be on the same physical server or the same virtual network as my competitors' data and applications.

I have established trust boundaries and my perimeter.

The highly leveraged nature of cloud computing means the trust boundaries have changed and some of the trusts are transitive to parties I am not aware of.

Don't interpret these examples to mean the new normal requires you to totally abdicate control. Remember, this book is designed to help you measure, analyze, and then manage. At a high level, the business (tenant) that is leveraging cloud services needs to think differently about who measures and reports, because the roles and responsibilities have changed and, specifically, the business doesn't have total command and control over its computing end to end. In the examples in the preceding table, the tenant is still responsible for its data, customers, and business requirements. But the tenant now needs information from its cloud vendor to complete the picture because it has leveraged some business process to a capability of a cloud vendor. If you are a tenant, you need to understand all the layers and know exactly which assets you own, which assets you touch that may not be on premises, and which assets you interface with both locally and remotely. This is the beginning to the process of answering the question of whether the economic benefits you might gain from cloud computing are worth the associated risks.

From a security metrics point of view, while cloud computing may become the new normal, with shared security responsibilities as the new model, some things haven't changed:

Both you and your vendor will measure security. You now need to define who is doing what.

• Both you and your vendor will manage functional components of an information security program (see Chapter 1) and may agree to use an internationally recognized information security management system (ISMS) standard such as ISO/IEC 27001:2005, which mandates specific requirements so that the ISMS can be explicitly managed, audited, and certified to be compliant with the standard.
• Security work is never finished (see Chapter 2). Cloud computing should motivate both you and your cloud vendor to assess what the threat landscape looks like and what new or different security threats exist in the cloud.
• Let's look at who's responsible for what in another way. The three SPI service models for cloud computing can be viewed as a stack, with platform building on infrastructure, and software building on both infrastructure and platform:⁴
• Software as a Service (SaaS)
• Platform as a Service (PaaS)
• Infrastructure as a Service (IaaS)

Just as each layer builds on the previous one, similar to the layers in the ISO Open Systems Interconnection (OSI) model, security issues are inherited up the stack.⁵ Generally, when thinking about roles and responsibilities, SaaS provides the most integrated functionality, with the cloud provider bearing most of the responsibility for security. PaaS offers a platform and assumes your company, the tenant, will build your own bespoke applications, which means application security is your responsibility. With IaaS, you have great flexibility but most of the responsibility for security, with the cloud vendor assuming responsibility only for infrastructure security.

Let's use an example to make this clearer. If your sales organization uses a cloud-based customer relationship manager (CRM) such as Salesforce.com or Microsoft Dynamics CRM, you are probably concerned about your customer data. You worry about how it is stored, how good the CRM's physical security is, whether or not the vendor uses encryption, and whether or not the vendor has good application and systems security. You will likely ask the vendor for information and assurances that its information security program is robust. You might even ask if it is ISO 27001 certified, meaning it has a standard set of ISMS components that you can compare to the components a competing vendor offers. In the end, you expect the cloud provider to demonstrate its level of security to you. If you were creating custom applications on Google's App Engine or Microsoft's Azure platform, you would want assurances about the security and integrity of that platform, but you would be responsible for the security of the applications you created.

In Actual Practice

There is a lot of buzz around getting cloud vendors to demonstrate they have the requisite security controls in place. This might come in the form of ISO 27001 certification, American Institute of CPAs (AIC) Service Organization Control (SOC) 2 report (formerly called SAS 70 reports),⁶ or BITS Shared Assessments.⁷ There are two important points to make:

All of these organizations are in the process of amending their certifications to account for cloud computing.

While it is important to evaluate the security controls of your cloud vendor, it is not sustainable for you or the vendor to create custom questionnaires. For the vendor, this would mean filling out a questionnaire for each customer. And you, the customer, cannot audit and keep track of every vendor.

Just as industry bodies have arrived at agreed standards and auditing protocols, it's important for you to look for industry-accepted and standardized ways to evaluate a cloud vendor's security controls.

Security Metrics vs. Cloud Security Metrics

Up to this point, this chapter has discussed cloud security at a high level—the definition of cloud computing, the economic drivers that support cloud computing, and the cloud delivery models. I would like to turn our attention to a discussion of common security metrics compared to cloud security metrics. This section offers a way to think about common security metrics in a cloud context.

Let's begin by slightly restating the three main benefits of an information security metrics program outlined in Chapter 1:

• Cloud metrics provides visibility—visibility for the tenant both to the cloud provider and to itself.
• Cloud metrics educates and provides a common language for understanding the information security program as applicable to the cloud vendor and to the tenant.
• Cloud metrics motivates both the cloud provider and the tenant to improve.The following table provides a structured way to think about common security metrics applied to cloud computing. Each column begins with a description of the column's content, after which two common security metrics are presented as examples, with corresponding suggestions about their relevance to the cloud.

Security Metric Cloud Addendum Vendor or Tenant? Delivery Model Target

Describes a common security metric. What is cloud specific about this security metric in a cloud context? In cloud computing, there are multiple parties involved and a business process can be outsourced to one or more cloud vendors. In addition, the tenant is still accountable for the data and the information security and privacy risks associated with its data. Each SPI delivery model, SaaS, PaaS, and IaaS, has different nuances and security responsibilities for the vendor and the tenant. What is the acceptable range or value you want to achieve?

Percentage of hosts that are up-to-date with critical security patches. Here it might be important to know whether these hosts are at the cloud vendor site, on premises at the tenant site but interface with hosts at the cloud vendor, or both. If you are a tenant, you need to understand all the layers and know exactly which assets you own, which assets you touch that may not be on premises, and which assets you interface with both locally and remotely. A vendor may be asked to report to customers on a metric similar to this on a quarterly basis in an effort to be transparent about its information security practices. Depending on the cloud delivery model, responsibility for security generally lies mostly with the vendor for SaaS and mostly with the tenant for IaaS. 100%

Application security incident: mean time to fix. No tenant wants to be exposed to application security vulnerabilities from its cloud vendor, especially if the tenant has outsourced important business functions to the cloud. Further, this could impact the tenant's customers who have dependencies on services the tenant has outsourced to the cloud vendor. On the other hand, cloud vendors who provide infrastructure and platform services need to protect themselves from tenant application security vulnerabilities that could affect the infrastructure or platform. Potentially both, depending on the deployment model. This metric applies to the cloud vendor if it is responsible for the application. It applies to the tenant in a PaaS or IaaS delivery model if it has an application with a security vulnerability.

The Cloud Security Alliance (CSA), introduced in Chapter 2, is spearheading an effort to create cloud-specific security metrics that uses a framework similar to the preceding table. The following section expands on the work that this nonprofit industry organization is doing because it has a direct bearing on security metrics.

Cloud Security Alliance

In April 2009, CSA issued version 1 of Security
Guidance for Critical Areas of Focus in Cloud Computing, and today version 2.1 is available for download.⁴ If you have not reviewed this document, it is worth reading. In addition to the guidance document, CSA has a number of working groups. One group in particular is related to our discussion of cloud security metrics—the CSA Cloud Controls Matrix (CCM) working group.⁸ The vision of this group is to be the recognized global authority for information security controls for cloud computing. CSA formed the CCM working group after realizing that to fully leverage the benefits of cloud computing, trust must be built into the cloud ecosystem by solving key problems such as the following:

• Lack of visibility into cloud providers
• Risks that are not fully understood and managed, for both the cloud providers and tenants
• Difficulty of compliance attestation

The CCM working group was also formed under the assumption that cloud providers and tenants have a shared responsibility to optimize the governance, risk management, and compliance of cloud computing.

In September 2009, the CSA Cloud Metrics working group was formed and began creating cloud-specific metrics aligned to the domains described in Security
Guidance for Critical Areas of Focus in Cloud Computing. As the Cloud Metrics working group evolved and the Cloud Controls Matrix working group published the first version of the CCM, an alignment between the two efforts became obvious.

The Cloud Controls Matrix also integrates with other CSA initiatives in addition to the Cloud Metrics working group, and with non-CSA initiatives. Here are summaries of other related activities:

• CSA Consensus Assessments Initiative (CAI) Performs research, creates tools, and creates industry partnerships to enable cloud computing assessments. This group is focused on providing industry-accepted ways to document which security controls exist in SIP offerings, to provide security control transparency. On October 12, 2010, the CAI delivered the Consensus Assessments Initiative Questionnaire. This questionnaire, available in spreadsheet format, provides a set of questions a cloud consumer and cloud auditor may wish to ask a cloud provider. It provides a series of "yes" or "no" control assertion questions that can be tailored to suit each unique cloud customer's evidentiary requirements. This question set is meant to be a companion to the CSA Security Guidance for Critical Areas of Focus in Cloud Computing and the CSA Cloud Controls Matrix, and these documents should be used together.
• CloudAudit An open standard and interface to allow cloud providers to automate audit assertions. The CCM provides CloudAudit with its Cloud Controls namespace, where CloudAudit answers the "how?" of audit assertions and CCM answers the "what?"
• Trusted Cloud Initiative (TCI) Will help cloud providers develop industry-recommended, secure and interoperable identity, access, and compliance management configurations and practices. TCI will develop reference models, education, certification criteria, and a cloud provider self-certification toolset. This will be developed in a vendor-neutral manner, inclusive of all CSA members and affiliates who wish to participate. A TCI assessor will use a combination of CCM–TCI mappings, Consensus Assessments questionnaires, and other tools as needed to cover the requirements specified in the TCI reference model. If supported by the provider, the TCI assessor will use CloudAudit-enabled tools to collect assertions.
• Common Assurance Maturity Model (CAMM) An industry initiative started by the contributors of the ENISA Cloud Computing Security Risk Assessment experts group, which includes ENISA, UK National Health Service, Amazon, Google, and Microsoft. The group has broad industry support and participation from key industry and standards stakeholders. CAMM is a methodology and solution for creating an independent maturity model-based measurement of the maturity of a cloud provider's security program. The Cloud Controls Matrix will map to CAMM's internal assessment controls, and providers can use CCM provider-specific controls to optimize their CAMM assessment scoring.

The CCM identifies security controls for the cloud that are applicable to all SPI delivery models, describe relevance to the cloud provider or tenant (or both), and are mapped to standards, regulations, and frameworks such as ISO, COBIT, HIPPA, and PCI-DSS. These controls help IT security and IT auditors leverage existing tools, processes, strategies, and frameworks to enable holistic cloud security. The link between these mapped control areas and the Cloud Metrics working group was a desire to describe a set of security metrics that could help cloud vendors and tenants alike demonstrate that these controls existed and were functioning properly. Indeed, any security management system, in the cloud or otherwise, needs metrics to report on the overall health and progress of the management system.

Even though the Cloud Metrics working group is aligned with the CCM, the group still needed to look at the control areas described and ask themselves, "What's important?" The group also knew that regulatory compliance would be a priority to most organizations. As you learned in Chapters 6 and 8, defining a target, understanding your context, and prioritizing are important steps to a successful security metrics project. The group also had a lot of different stakeholders with varying points of view creating metrics. To help them, they developed a metrics template and defined a simple lifecycle for a metric within the context of the Cloud Security Alliance and their direct link to the CCM.

CSA Cloud Metrics Working Group Template

The template that the CSA Cloud Metrics working group uses to create its metrics is presented here to show a real example of security metrics development in action. This template is a work in progress and will grow and evolve. The template addresses what the working group considers to be one of the problematic areas with security metrics—the lack of rich guidance and implementation information about a metric. To address this problem, the metrics template is detailed and allows for lengthy text in the Attribute Definition column.

Cloud Security Alliance Metric Definition Template

Author

Your name

Date

Date

Attribute

Attribute Definition

Name

The name of the metric.

Title

Short name or abbreviation.

Status

Possible states: Draft, Reviewed, Published, Under Review.

Audience

Choice of Operations, Management, or Executive.

Units of Measure

Units of measure such as incidents per month, vulnerabilities per server, or time in hours.

Targets

Values that enlighten interpretation of measurement results such as goal values, critical, normal, caution values, etc.

Description

A general description of the metric.

Objective

Succinct statement of the objective of the metric. Are there specific decisions for which this metric is useful?

Usage

Discussion regarding implementation experience such as use cases, scope, scale, or unintended consequences.

Implementation

Automation

Amenability of this metric to automation. Choice of high, medium, or low followed by rationale.

Frequency

Suggested measurement interval for the metric, e.g., daily, weekly, monthly, etc.

Sources

A general description of data required to calculate the metric and where one might obtain it.

How to Calculate

A general description of how to calculate the metric plus a formula, if possible.

Methodology

A description of any foundational models or assumptions associated with this metric. For example, if one were using linear regression to compute this metric, there is the assumption that errors are normally distributed.

Limitations

A general description of potential limitations in the implementation of this metric. What considerations are not taken into account with this metric?

Survey Interface

Question

For metrics that are calculated based upon survey responses, this is the question that would appear on the survey for this metric.

Answer

A description of the expected format of the answer, e.g., Yes/No, an integer between 0 and 100.

License

Usage restrictions.

Associations

References

Names of individuals or companies that contributed all or part of this metric definition. Also include either citations or URLs to other material that is relevant.

Tags

A comma delimited list of Contexts that are associated with this metric. For example, this field will hold a list of Control IDs from the CSA Controls Matrix, e.g., CO-2, DG-3, etc. Note that the CSA Controls Matrix will be used to automatically associate the metric with other standards such as ISO/IEC, PCI, HIPAA, NIST, and COBIT.

Cloud Relevance

Metric Collector

Who collects this metric? A cloud provider or tenant?

IaaS Relevance

Rating value between 1 and 10, 10 being most relevant.

IaaS Rationale

Rationale for rating assignment.

IaaS Sharing

Discuss the likelihood, motivation, and risks/benefits/costs of sharing this metric between provider and tenant.

PaaS Relevance

Rating value between 1 and 10, 10 being most relevant.

PaaS Rationale

Rationale for rating assignment.

PaaS Sharing

Discuss the likelihood, motivation, and risks/benefits/costs of sharing this metric between provider and tenant.

SaaS Relevance

Rating value between 1 and 10, 10 being most relevant.

SaaS Rationale

Rationale for rating assignment.

SaaS Sharing

Discuss the likelihood, motivation, and risks/benefits/costs of sharing this metric between provider and tenant.

Let's take a look at two important sections—Implementation and Cloud Relevance. These sections are important because they give you a place to discuss potential limitations of the metric, identify the source of the data needed, and indicate whether the metric is appropriate to share between tenant and cloud vendor. A metric may appear great on paper, but implementing it in real life requires some reality checks. For example, some data is too hard to gather or is in an unstructured format, and to implement the metric might require more effort than the business intelligence it provides. Further, in the Cloud Relevance section, the template asks you to describe who is collecting this information—the tenant, the cloud vendor, or both? Then the template asks you to rate on a scale of 1–10 how relevant the metric is to the three cloud service models (IaaS, PaaS, SaaS). The working group added this section to the template because it wanted to consider the main cloud delivery models and what would be the potential benefits, costs, and risks associated with sharing either the metric itself or the computed results.

Into Action

The following is a sample metric created using the Cloud Metrics working group template. You can use the blank column to create your own metric.

Cloud Security Alliance Metric Definition Template

Author

Dan Crisp and Elizabeth Nichols

Date

8/25/2010 0:00

Attribute

Proposed Definition

Name

CloudServiceDataElementCount

Title

CSDEC

Status

Draft

Audience

Management

Units of Measure

Count of data elements used by a target cloud service.

Targets

None. This is an informational metric.

Description

The CloudServiceDataElementCount metric provides counts of the number of data items that have been outsourced to a cloud provider for a specific target service. The CloudServiceDataElementCount metric can report simply the count of all data items or the count of data items that hold PII or no PII. From this metric, one can derive the percentage of data items that have PII for a given service. Additionally, along with the CloudServiceUserCount metric, per-user, per-service metrics can be derived. If data elements are tagged with additional metadata that characterizes relevant regulations and compliance requirements, then metrics can be derived on a per-compliance/regulation basis.

Attribute

Proposed Definition

Objective

The CloudServiceDataElementCount metric is designed to provide management with a simple tally of the number of attributes that are managed via a cloud provider for a target service. Additionally, it is designed to give insight into data sensitivity on a per-user and per-compliance/regulation basis.

Usage

Using CloudServiceDataElementCount conditioned by whether or not it holds PII, one can create a metric that characterizes the level of PII content for a cloud service. This metric, in turn, can be used to rank all cloud services based on their PII content. Similarly, by computing this metric on a per-compliance requirement basis, one can prioritize services based upon such criteria as local relevance of a regulation, penalties, review requirements, audit schedules and resources, etc.

Implementation

Automation

This metric can be computed automatically based upon records kept about reviews. Since it does not change frequently, manual implementation is feasible.

Frequency

Update this metric across all services once per month or once per quarter, depending upon volatility of user base.

Sources

Generically, the required data for computing this metric will come from an Application Directory and the Data Dictionary for each service, or similar. Ideally the Data Dictionary will carry annotations for each data element regarding 1) optional vs. mandatory, 2) is it PII or not, 3) a list of associated regulations or compliance requirements.

How to Calculate

Compute this metric as a simple count of attributes for a given target service, sometimes aggregated on filters derived from metadata about each data item. Example metadata includes serviceName, isPII, piiType, and others.

Methodology

Simple count. For the cases in which the data dictionary is a SQL database, simple queries can generate values for this metric across many dimensions, e.g., isPII, piiType, etc.

Limitations

This metric does not necessarily reflect any information regarding the need, justification, or time to live regarding the PII data items associated with a service.

Survey Interface

Question

Answer

License

CSA Use License

Associations

References

Tags

OP-02: Documentation

SA-03: Data Security/Integrity

DG-02: Classification

DG-03: Handling/Labeling Security Policy

Cloud Relevance

Metric Collector

Typically the tenant collects this metric.

IaaS Relevance

Provider: 10

Tenant: 10

IaaS Rationale

To both providers and tenants, this information is critical. For providers, higher values of this metric imply greater resources consumption—storage, compute power, and network bandwidth. For tenants, this metric is a proxy for service popularity, visibility, and criticality.

IaaS Sharing

Sharing of this value is typically at the discretion of the consumer. Indirectly, the provider can infer estimates of this metric via traffic analysis of the services.

PaaS Relevance

See IaaS relevance.

PaaS Rationale

See IaaS relevance.

PaaS Sharing

See IaaS relevance.

SaaS Relevance

Provider: 10

Consumer: 10

SaaS Rationale

In this case, both the tenant and provider are completely aware of the service provided and all users and data involved.

SaaS Sharing

In this case, both tenant and provider are completely aware of the service provided and all users and data involved.

CSA Cloud Metrics Working Group Lifecycle

As stated earlier in this book, metrics are subject to change and need to be reviewed and revised on a regular basis. Figure 17-1 shows the lifecycle used by the Cloud Metrics working group for this purpose. The stages of the lifecycle are as follows:

• The Cloud Metrics working group creates and reviews the cloud metrics within the working group.
• Because the working group is aligned with the Cloud Controls Matrix, it requests the CCM stakeholders to do a review. When they are finished, a broad CSA review occurs. Finally, a general public review occurs.
• After public announcement, the working group makes the metrics available in an open and vendor-neutral manner.
• In an effort to ensure its metrics are usable, the working group asks either a cloud vendor or a tenant to implement the metrics and provide feedback.
• The working group reviews the metrics along with the changes in cloud computing and decides if each metric needs to begin the cycle again with a major revision or needs to be demised.

Final Thoughts

The need for security metrics in the cloud is not much different from the need for security metrics in general. Everyone in the cloud, vendor and tenant alike, will need to measure the effectiveness of security controls and show their accountability to each other and to regulatory bodies. In the past, there was little benefit for companies to share security metrics; indeed, there were risks in doing so. With cloud computing and a world of shared accountabilities across virtual, physical, and geographic boundaries, we need to find ways to share information between vendor and tenant and across industry in responsible ways. This implies we need to remove some of the roadblocks to success and work on areas such as common definitions for terms, common metrics deployed in a consistent manner, and a consistent reporting framework. Industry bodies such as the Cloud Security Alliance are helping to achieve these goals, and many security practitioners are volunteering their time and talent.

For information security practitioners, an important first step is to measure and get a baseline that is appropriate for your business and that describes what is the new normal for security in cloud computing. Businesses will need to make decisions based on facts and data, and your security metrics program can support important planning and decision making and drive beneficial changes in your organization.

We've Covered

What is cloud computing?

• NIST definition of cloud computing
• Essential characteristics for cloud computing
• Cloud service and deployment models

Common business drivers that motivate organizations to move to the cloud

• Core question, "Will the economic benefits of moving to the cloud outweigh the security risks?"
• Budget pressures, streamlining, and consolidation efforts
• Agility and the draw to cheaper, better, faster

The "new normal" for IT service delivery

• New delivery model for IT services
• Some comparisons between aspects of traditional enterprise computing and cloud computing
• Security metrics as a way to understand the new normal

Common security metrics in the context of cloud computing

• The value of developing metrics
• Structured way to think about common security metrics in the context of cloud computing

Major cloud industry groups, especially the Cloud Security Alliance, and their efforts to describe cloud controls and related metrics

• Cloud Security Alliance (CSA) and the Security
  Guidance for Critical Areas of Focus in Cloud Computing
• Cloud Security Alliance working groups, in particular the Cloud Controls Matrix and Cloud Metrics working groups
• ENISA's Common Assurance Maturity Model (CAMM)

Endnotes

1. P. Mell and T. Grance, "The NIST Definition of Cloud Computing," NIST Special Publication 800-145 (Draft), Jan. 2011, http://csrc.nist.gov/publications/drafts/800-145/Draft-SP-800-145_cloud-definition.pdf.
2. ENISA, Cloud Computing: Benefits, Risks and Recommendations for Information Security, Nov. 2009, available at www.enisa.europa.eu/act/rm/files/deliverables/cloud-computing-risk-assessment.
3. The OpenCrowd Cloud Taxonomy (http://cloudtaxonomy.opencrowd.com/) is a good resource to consult to see the diversity of cloud offerings available today.
4. A more complete discussion of the stack concept appears in the Cloud Security Alliance's Security Guidance for Critical Areas of Focus in Cloud Computing, V2.1, www.cloudsecurityalliance.org/csaguide.pdf, which is highly recommend reading for those interested in cloud computing security.
5. Both the OSI model and the cloud stack are reference models, which means that not everything fits neatly into one layer or another. This is especially apparent when you look at the large quantity of cloud vendors who provide services throughout the stack. It may not be easy to confine a vendor or offering in a single layer.
6. AICPA press release, "AICPA Publishes Guidance on Next Generation of SAS 70," Feb. 1, 2011, www.aicpa.org/Press/PressReleases/2011/Pages/NextGenerationofSAS70.aspx.
7. BITS Shared Assessments, www.sharedassessments.org and http://www.bitsinfo.org/.
8. CSA, "CSA Cloud Controls Matrix V1.1 is Released," https://cloudsecurityalliance.org/research/projects/cloud-controls-matrix-ccm/.Figure 17-1 The proposed lifecycle of a metric, used as a guide by the CSA Cloud Metrics working group