Building an Incident Response Team and IR Process

In our world today, we have an abundance of many things, among which are -unexpected events. Falling meteorites, terrorist attacks, hacktivist demonstrations, blackouts, tsunamis…. well, you get the point.Now, although the majority of events I just mentioned probably fall into a Disaster Recovery category, they are nonetheless events that greatly impact our personal lives and disrupt the normal ebb and flow of the daily routine.On the professional side of life, there are also incidents that,although classified on a lower scale than a disaster, still create much disruption and depending on how they are handled, can have a long-lasting impact to the flow of business. The purpose of this article is to discuss some suggested methods of how to go about building an incident response team and related procedures that will enable this group to respond to these events expeditiously.

TERMINOLOGY DEFINED

Learn Incident Response

Get hands-on experience with incident response tools and techniques as you progress through nine courses.

Get Started

Before we start to discuss the mechanics behind building this elite group of technical emergency responders, let's understand what we're up against. First of all, let's get our terminology straight. What exactly am I referring to when I use the term – "event" and "incident"? To give this article some context, consider the following definitions courtesy of Merriam Webster…

• An incident is defined as "an occurrence […] that is a separate unit of experience".
• An event can be defined as "something that happens: an occurrence" or "a noteworthy happening".

Let's break this down;if we use the example of a small electrical fire in the basement of a building, this can be categorized as an individual "incident" or as a "separate unit of experience". Now, if this incident is not handled properly, it can escalate and possibly grow to become a fire so large that it consumes the entire building. The incineration of the building can be categorized as an "event", which is sort of an umbrella term that groups causes and effects for the entire disaster or "noteworthy happening" into one category.

Applying this understanding to the enterprise, items such as a data breaches, hacking attempts, critical server crashes, website defacement or social engineering attempts can be classified as individual "incidents". This is because they may affect business or the corporate reputation but may not completely halt the business flow of the company. If not addressed properly, these incidents,although small,could escalate and succeed in completely halting the business,resulting in a disaster or large scale "event". Hopefully, this explanation clarifies the difference between events and incidents as this understanding will determine how each occurrence is handled. This now brings us to our next section…

PLANNING AN INCIDENT RESPONSE PROCESS

This step can seem daunting if you've never been involved with Incident Response or you're trying to decide where a process like this might fit in to your particular environment. How can we go about organizing all the related business groups, technical areas and how can we find out if we're missing anything?The good news is that in the majority of cases, there is already some type of set process that is followed whenever incidents occur. Some problems that come up, however, could be that the process may not be documented and since it's an informal process, there is a great chance that core response components are missing or have been overlooked. The benefit to identifying any existing process that your organization may have is that it is much easier to train employees using a foundation with which they are already accustomed to. It may also be much easier to gain upper management's support and buy-in for a process that is actively being followed albeit – informally.This support is necessary because management's support will be needed for any funding that is required and for the allocation of time for the individuals that will be forming part of the official team. Without this support, it's possible that your project will never get off its feet or after all the hard work,the process could be scrapped or drastically changed and then it's back to the proverbial drawing board. This can be extremely frustrating so be sure to do your homework, identify any area that may already be built and if appropriate, incorporate this into your draft IR process.This way you'll have a deep understanding of how the process should flow when having discussions with upper management and be able to defend any modifications, enhancements or complete overhauls.

Keep in mind that when speaking with management, your initial draft is just that – a draft. Be prepared to have a detailed conversation so you can understand what their expectations are and that you properly define what your incident process is providing. It's possible that in these initial conversations you will identify areas that need to be modified or added.If this step is not accomplished correctly,it's possible that the functions of your future IR team will not be understood or properly recognized.This could result in your process not being properly advertised to the enterprise, in which case it simply becomes just another "informal process". Be sure to gain managements approval, communicate and advertise your new structure so that when an incident does occur, your new framework will be used.This will eliminate any overlap and ensure that the authority of the members of your future IR team remains fully recognized.

Some other questions that you may ponder along the way:

How far will IR processes be able to reach?

Who will make up the IR Team's client base?

The first question relating to the reach of the IR process speaks to cases where critical services and applications are provided by external third parties. In these cases, you will have to decide on how far the IR process will flow and if a "hand-off" needs to occur. This needs to be explored at length since this will make your resolution process dependent on the efforts of an outside entity.

Questions like these are highly important because in the case of many enterprise environments, there are multiple areas that are critical to business operations. This brings us to the second question regarding the IR client base. This refers to subsidiaries or operating companies that, although separate, may fall under the auspices of the parent organization. You need to understand the relationship to these companies and if they provide critical applications, services and other related business functions. More than likely, these entities will also have to fall under the scope of your IR process and it will be necessary to identify key stakeholders at those locations to support your IR. This begs the question… who should form part of the Incident Response team?

INCIDENT RESPONSE ROLES AND RESPONSIBILITIES

Depending on what you read, you may find different titles and roles for Incident Response. The following listing is an outline of some roles and responsibilities that I used when building an IR plan at a past employer. Each environment is unique, so you will need to research your own requirements and then tailor a plan that meets your needs. Generally, the types of roles that should exist within an IR function are:

Incident Response Officer – This individual is the Incident Response champion that has ultimate accountability for the actions of the IR team and IR function. This person should be an executive level employee such as a CISO or other such corporate representatives. It would be very beneficial if this individual has direct reporting access to the CEO and is a peer of other C-level executives.

Incident Response Manager – This person is the individual that leads the efforts of the IR team and coordinates activities between all of its respective groups. Normally, this person would receive initial IR alerts and be responsible for activating the IR team and managing all parts of the IR process, from discovery, assessment, remediation and finally resolution. This individual reports to the Incident Response Officer.

Incident Response Assessment Team – This group of individuals is composed of the different areas serviced by the IR team. This allows expertise from every critical discipline to weigh in on classifications and severity decisions once an incident has been identified. It is very beneficial to have representatives from IT, Security, Application Support and other business areas. In the event of an incident, the IR Manager would gather details of the incident from the affected site, begin tracking and documentation (possibly through an internal ticket management system) and then activate the Assessment Team. This group would then discuss the details of the incident and based on their expertise and knowledge of the business, would then be able to assign an initial severity. This team reports to the IR Manager.

Remote Incident Response Coordinator – This role should be assigned to qualified and capable individuals that are located in other geographic areas. These individuals ultimately report to the Incident Response Manager but in their geographic region, they are recognized as IR leaders. This will allow these assistants to manage the efforts of local custodians during an incident. This configuration is very useful, especially for organizations that have offices in multiple time zones. If an IR Manager is located in the United States but an incident occurs in a Malaysian branch, it will be helpful to have a local security leader that is able to direct efforts and provide status updates to the Incident Manager. This way, regardless of the time zone the correct actions will be invoked promptly.

Incident Response Custodians – These individuals are the technical experts and application support representatives that would be called upon to assist in the remediation and resolution of a given incident. They report to the Incident Response Manager or to the Remote IR Coordinator(s) depending on their location(s).

Once you've been able to identify the proper stakeholders that will form your team, you will have to provide an action framework they'll be able to use when carrying out their responsibilities. Think of this "action framework" as a set of training wheels that will guide your IR team. What does this mean? Let's move on to the next section to discuss this…

INCIDENT RESPONSE PROCESS FLOW

A part of outlining this framework involves the identification of IR Severity Levels. These levels will help your team understand the severity of an event and will govern the team's response. Some suggestions for these levels are the following:

Earlier in this article, I mentioned the benefit of identifying any existing informal process that your company may already be following. If so, it will now be necessary for you to step through that process mentally, keeping in mind your identified severity levels so that you can start to document each step of the process. You will undoubtedly start to remove irrelevant portions of the informal process but may opt to keep certain items in place. (For example, certain notification procedures may still be useful and you may continue to use these in your new IR process to alert members of your team). If you don't have a starting point like this and you're starting from scratch, then perhaps the following suggestions can provide some direction.

Start to create a documented action script that will outline your response steps so your IR Manager can follow them consistently. Your script should show steps similar to the following:

STEP #

ACTION

1

Incident announced

2

IR Manager alerted

3

IR Manager begins information gathering from affected site

4

IR Manager begins tracking and documentation of incident

5

IR Manager invokes Assessment Team

(Details of call bridge or other communication mechanism)

6

Assessment Team reviews details and decides on Severity Level of incident.

7

IF SEV 1 = PROCEED TO STEP #11.0

8

IF SEV 2 = PROCEED TO STEP #12.0

9

IF SEV 3 = PROCEED TO STEP #13.0

10

IF SEV 4 = PROCEED TO STEP #14.0

FOR SEVERITY LEVEL 1 – Proceed with following sequence

11.0

Determine attack vectors being used by threat

11.1

Determine network locations that are impacted

11.2

Identify areas that fall under "Parent Organization"

11.3

Identify systems or applications that are impacted

FOR SEVERITY LEVEL 2 – Proceed with following sequence

12.0

Determine attack vectors being used by threat

12.1

Alert Incident Officer to Severity 2 threat

This of course is an extremely high level example, but as you can see, it is possible to flesh out the majority of the process with specific action items for each severity level. Be sure to thoroughly research your unique environment to develop a process that fits your needs. You may have to add custom steps to cover incidents that span multiple countries and subsidiaries. Once you've created your process.you may want to consider developing small wallet size scripts for the members of your Assessment Team and other key players on which you will need to depend to make this run efficiently. In this way, each member will have necessary information on hand that will allow them to respond as expected.

This article just scratches the surface of the work that is required to build a full IR process but hopefully this has given you some direction and additional areas to explore when planning your next IR project!

Learn Incident Response

Get hands-on experience with incident response tools and techniques as you progress through nine courses.

Get Started

References:

• http://www.cert.org/csirts/Creating-A-CSIRT.html
• http://www.cert.org/csirts/Creating-A-CSIRT.html#practices
• O'Reilly Incident Response – Kenneth R. vanWyk and Richard Forno
• http://csrc.nist.gov/publications/nistpubs/800-61rev2/SP800-61rev2.pdf