SIEM Overview

Introduction

Security information and event management (SIEM) is a software solution to take event logs collected from all supported information technology (IT) infrastructure and applications and provide actionable security intelligence. These enterprise solutions provide real-time analysis of security alerts generated by applications and network hardware, providing an interface for research and analysis of provided data and alarms, while also providing an interface for deeper investigations and tracking the full scope of an event.

SIEM solutions have been around for many years, with different degrees of functionality and features depending on the product or vendor you choose.

The SIEM also provides a collection point for all logs, since a common attack profile includes an intruder attempting to cover their tracks by deleting the event logs from a compromised system. A SIEM collects the event logs in real-time, so even if the logs are deleted from the compromised system later, the events are still available for review from the SIEM copy of the logs. This helps preserve evidence and allows for detailed analysis of events even during a successful attack.

At its core, a SIEM is a data aggregator, search tool, and reporting system. SIEM gathers immense amounts of data from your entire networked environment, consolidates all that data and makes it human accessible. With the data categorized and laid out at your fingertips, an analyst can research possible data security breaches with as much detail as needed.

Summary of Capabilities

In practice many products in this area will have a mix of log-related functions, so there will often be some overlap – and many commercial vendors also promote their own terminology. A full solution will include simple collection and storage of log messages and audit trails, long-term storage as well as analysis and reporting of collected log data, real-time monitoring of new log events, correlation of events, notifications as suspicious events are collected, and console views for graphical analysis and research.

A key focus of a solution is to monitor and help manage user and service privileges, directory services and other application or system-configuration changes; as well as providing log auditing and review, analysis of related events, and incident response.

• Data aggregation: Log management aggregates data and events from many sources that include network devices, security events, servers, databases, and most applications to provide the ability to consolidate monitored data and help avoid missing crucial events. Next-generation SIEM solutions include support for cloud applications and infrastructure, enterprise applications, identity and HR data, and even non-technical data feeds as well.
• Correlation: System looks for common attributes to link two or more events together into meaningful bundles. This critical technology provides the ability to perform a variety of correlation techniques to integrate different sources in order to turn data into useful information. Correlation is typically a function of the Security Event Management portion of a full SIEM solution. Data enrichment adds context to an event. SIEM solutions sometimes enrich incoming data with identity, asset, geolocation, and threat intelligence to aid in investigations. Data enrichment fills in critical information that a SIEM needs in order to correlate related events together and aid in threat detection.
• Alerting: The automated analysis of correlated events. At a basic level, a SIEM could have the ability to integrate with a third-party security orchestration, automation, and response (SOAR) solution to assist analysts as they investigate and mitigate potential threats with some form of automation. A SOAR solution gives analysts a workbench to collect information, track steps taken during the investigation, and remember how the threat was mitigated or whether it wasn’t a real threat.
• Dashboards: Tools can take event data and turn it into informational charts to assist users in seeing patterns, identifying trends, or identifying activity that is not forming a standard pattern.
• Compliance: Applications can be employed to automate the gathering of compliance data, producing reports that adapt to existing security, governance, and auditing processes.
• User and entity behavioral analytics (UEBA): A next-generation SIEM uses advanced analytic techniques, beyond the basic signature-based approach, to catch known and unknown threats. They can use sophisticated machine learning algorithms to detect threats more accurately. UEBA is one type of advanced analytics that is integrated into most next-generation SIEMs in order to provide better detection. Another technique used are threat chain models. These models help stitch together connected alerts to consolidate separate but related alerts into a threat sequence, increasing the risk score of the overall threat.
• Retention: Employing long-term storage of historical data to facilitate correlation of data over time and to provide the retention necessary for specific compliance requirements. Long term log data retention is critical in forensic investigations as it is unlikely that discovery of a network breach will be at the exact time of the breach occurring. A report by IBM found that the average time to detect and contain a data breach is 280 days. A sophisticated attack can take several months to identify and remediate.
• Forensic analysis: The ability to search across logs on different nodes and time periods based on specific criteria is critical for security analysis. This mitigates having to aggregate log information in your head or having to manually search through thousands or even millions of log events.

Computer security researchers have identified the following SIEM use cases:

• Security Event Prioritization: It is impossible to stay ahead of the curve if your security team is buried investigating meaningless security events. You need to determine which events are most critical and which are a lower priority. Look for a solution that makes the prioritization process easy with out-of-the-box controls that can be adjusted to meet changing business requirements.
• Normalization of Disparate Data Sources: Organizations rely on multiple technologies to run their business. This makes it difficult for security teams to understand the data coming in from these disparate sources. A SIEM turns this data into actionable intelligence by normalizing it into a common format and giving it meaning and context. With a robust solution, analysts won’t need to understand the nuances of different operating systems, applications, databases, firewalls, or network appliances to know what the data means and what to do with it.
• Advanced Visibility: By aggregating all of the system logs from your on-premises and cloud-based systems, you are able to gain deeper insights into your users, endpoints, infrastructure, network traffic, system activity, and configuration settings.
• Data Enrichment: Look for the ability to get additional context behind security events for quick and thorough response. Data enrichment puts all the necessary event details and forensic analysis at your fingertips. For example, if a new user is created and then that user immediately connects to a critical system, SIEM can recognize that this is not normal behavior and escalate the incident for investigation.
• Real-Time Threat Detection: In order to minimize the impact of a breach, you have to detect threats quickly. This means having the ability to log, correlate, and prioritize events in real time to give your team a head-start on resolving and mitigating threats before they result in a devastating data breach. Most systems come pre-configured with default alert rules, but you’ll need to find the correct balance of alerts that meet your needs without too many false positives.
• Insider Threat Detection: Insider threats are concerning because these attackers don’t need to penetrate your perimeter, they’re already inside it. They can be your employees, contractors, or business associates. They might be a malicious threat actor themselves, or an employee’s account might have been compromised by external attackers, giving the attackers inside access to your infrastructure and data. In either case, the perpetrator wants to remain unnoticed, stealthily poking around, and collecting sensitive data to exfiltrate. SIEMs provide innovative, behavior-based analytics techniques that, in conjunction with peer group analysis techniques, detect variations in normal patterns when it comes to the access and usage of internal data sources. By comparing not only historical usage, but the usage of colleagues and team members, next-generation SIEMs are able to remove the noise associated with incremental changes in user behavior and highlight concerning activity.
• Streamlined Incident Response: Automatically escalate events to the right person and manage any cases that require further investigation to make your team more efficient. A SIEM might include an artificial intelligence-based recommendation engine that can suggest remediation actions based on previous behavior patterns to decrease your mean time to respond.
• Out-of-the-Box Security: As you connect to new data sources, like Windows servers, SQL Server databases, or even cloud solutions you want to make sure you can automatically apply the appropriate security controls and escalation rules. Out-of-the-box security templates make it easy to get started quickly and connections can be quickly configured as needed.
• Security and Compliance Reporting: IT operations and security teams alike are often required to provide reports to both auditors and executives on a regular basis. Most organizations also need to comply with multiple regulations, which adds to the complexity and reporting requirements and the effort required to share reports on a reliable schedule. Having a robust reporting engine in your SIEM provides for easy reporting of log data, events, and incident response activity that can greatly simplify reporting requirements. Compliance reports generated by a SIEM can even help show you how your security posture is improving over time.

All tools and systems have the possibility to produce a false-positive result. For example, too many failed login attempts can just be an employee forgetting their password and not someone trying to break into the system. It’s important that for any triggered events the steps taken are justifiable and of an appropriate measure as you wouldn’t want employees getting locked out for hours in such scenarios.

Security Challenge

Security threats are always increasing, and they can come from both internal and external sources. In addition to ongoing threats from hackers looking to breach the security protocols guarding our sensitive information, another rapidly rising concern is that of employees who accidentally misconfigure security settings in a way that essentially opens the door to attack. To address these issues, IT organizations have put various systems in place to protect against intrusion and a host of different risks.

The downside of these safeguards is they generate so much monitoring data that IT teams are then faced with the problem of interpreting millions of security events to pinpoint actual problems. In fact, the volume of security data flowing to understaffed IT security groups is largely useless unless it can be quickly analyzed and filtered into actionable alerts. Given the reams of data in question, it is no longer possible for organizations to use manual analysis to handle this critical task.

A SIEM brings together the concepts of security event management (SEM) with security information management (SIM) to achieve the best of both worlds. SEM covers the monitoring and correlating of events in real time as well as alerting the configuration and console views related to these activities. SIM takes this data to the next phase, which includes storage, analysis, and reporting of the findings.

SIEM software relays actionable intelligence that enables you to manage potential vulnerabilities proactively based on real-time information, protecting your business and your customers from devastating data breaches. With the ever-increasing incidence of these attacks, this technology is more important than ever.

Compliance Requirements

There are some required compliance considerations because many common compliance frameworks require an organization to have a properly configured SIEM as part of the minimum-security framework.

• Collect and retain all event logs for a minimum of one year, with previous years available for recovery, if possible.
• System contains at least the last three months of logs that are immediately available for analysis without any recovery.
• Logs from all in-scope systems and applications are collected by the SIEM for analysis by trained and qualified personnel. The more log sources ingested into the system also means better intelligence. Better intelligence means an opportunity for better decisions.

Some other regulations or laws may require additional efforts (i.e., longer retention periods) to demonstrate minimum legal compliance, but the product installation, proper configuration, and analysis of collected logs are the common elements to any requirement. Retaining the logs for a longer period will increase the overall cost of the solution, but disk space is one of the less expensive aspects of a SIEM solution.

Implementation Options

Aside from functionality considerations, there are other elements of your SIEM solution that will determine its long-term success and usability for your organization.

• Enterprise Solutions vs. Open Source: Some SIEM solutions fall into the enterprise category, meaning they have a dedicated development team focused on product enhancements as well as customer support. Other options are built on the open-source model and rely on a large community of developers for support and bug fixes. Open source SIEM solutions provide basic functionality that can be great for smaller organizations that are just beginning to log and analyze their security event data. But over time, many IT professionals find that open source SIEM software is too labor-intensive to be a viable option as the organization grows. In addition, some companies have policies that discourage the implementation of open-source solutions because of the high-level of expertise and time required to provide internal system support. If you have the team to support an open-source solution, it can save you money.
• Automation: Some parts of the SIEM process can be automated to save time and speed information sharing among your team. Notifications can also be routed to the right person depending on the event or data source. For example, a virus detection event coming from your Linux environment can be directly routed to your Linux admin, who will know best how to quickly isolate the system and remediate the infection before it spreads across the environment.
• Implementation and Training: Every software vendor has a different process when it comes to how their solutions are implemented and the ways your team can participate in training. Understanding your options for these services is key to getting a handle on how long it will take to get up and running on the software and when you can realistically start seeing the benefits. Intuitive solutions require a minimal amount of upfront training to start seeing results that will benefit your organization. More complicated solutions will require our team to invest a substantial amount of time in training and regular system tuning activities. Professional services usually provide integration, development, and consulting. If you do not have the resources needed to implement the solution, or you’d like to have the vendor help with the migration to the new solution, you can be sure to ask whether these services are available.
• Support: Evaluate the vendor’s customer support options to determine if support is available 24/7 or based on their normal business hours. You need to determine how you want to communicate with the vendor (web, phone, or chat) and determine if that communication option is supported. You may also want to know if support personnel are outsourced or if calls are handled locally. All of these considerations are important to think through to protect the long-term health and relevance of your SIEM application.
• Integrations: Having a complex composition of disparate security solutions can make it challenging to effectively ensure the safety of your specific environment. Integration between products like antivirus or security auditing software enhances efficiency by allowing for seamless transitions between products to create a more streamlined and centralized security profile. This helps you assess potential impacts to the security of information stored on-premises, in the cloud, or in a hybrid configuration. If you have specific integrations that are required for your business, you must make sure the selected solution supports those integrations before you make a purchasing decision.

Cost Considerations

There are some decisions that could have an impact on the overall cost of the selected solution.

• Licensing and Deployment Methods: Some SIEM software vendors charge by the amount of data generated or by how many systems the solution is managing, and others simply have a flat-rate approach. You must also remember that deployment models can be on-premises or in the cloud, and some solutions use agents while others don’t. An agent is something that has to be placed on systems to enable the sending of information from the system being monitored to the SIEM solution for normalization and evaluation. Agentless applications connect automatically to the systems they monitor to simplify administration. You must find out how basic differences in the offerings you evaluate may affect your overall complexity and your total cost of ownership.
• Impact on Headcount: Some solutions require dedicated staff to run the software and manage the interface showing security events. It can be cheaper and easier, depending on solution options, to deploy a solution managed by a third-party security partner. An efficient SIEM deployment will require 24×7 monitoring for various security alerts, and that means you will require at least four trained personnel for effective 24×7 monitoring. You can determine whether the new SIEM application will require additional headcount for day-to-day management, how much it will cost for third-party monitoring, and whether this cost is supported in your existing budget.
• Regulatory Compliance: SIEM gained popularity with large businesses working to comply with the Payment Card Industry Data Security Standard (PCI DSS). In addition, it has highly useful applications in helping you meet regulations for the EU’s General Data Protection Regulation (GDPR), Sarbanes-Oxley (SOX), and others. These laws require organizations to have mechanisms in place to detect threats and resolve them quickly. This means you must know what’s happening in a wide-reaching IT infrastructure that could span on-premises, multiple cloud environments, and hybrid environments. When used for evidence of compliance there are specific evidence collection requirements that can increase cost considerations around scope, reporting, training, and response efforts.

Recommendations

The are some general recommendations that might make the selection, purchasing, installation, configuration, and use of an effective SIEM a little easier in your environment.

• Determine Log Sources: A comprehensive listing of devices and systems that are to have their logs collected into the SIEM will help determine required capabilities. The location of these devices and systems will help determine the location of the SIEM appliance. If most everything is (or will soon be) in the public cloud, then the SIEM should probably also be in the public cloud. The number of devices and the approximate volume of events in each log that will be collected can also help determine license cost, bandwidth required to send logs to the SIEM device, drive size requirements to store the required logs for more than one year, CPU requirements to quickly process the collected logs, etc.
• Compliance Requirements: Identifying the compliance requirements you must support now and in the near future will help drive selection and configuration of a solution. If, for example, you determine at a later date that the retention of logs is twice as long as expected due to changing compliance requirements, the cost of drive storage might now double and implementation might mean potential downtime or additional other costs to support the unexpected change.
• Personnel Requirements: For a SIEM to be effective at generating actionable intelligence, it must be properly configured and maintained by trained personnel that are skilled in the selected SIEM configuration requirements. The device must also be monitored so alerts or alarms can be immediately investigated. If there is a suspected data breach reported at 2 am on Saturday, do you want the investigation and remediation to start at 2 am on Saturday or can it wait until the next scheduled shift starts at 8 am on Monday? Effective outsourcing of any configuration and monitoring can provide immediate cost saving benefits that extend past general labor savings and extend into annual savings in the area of training, recruiting, retention, etc. A quality Managed Security Services Provider (MSSP) can abbreviate deployment, save on configuration spending, perform basic product support, reduce training, and provide custom implementation for the selected SIEM solution.
• Enterprise Solutions vs. Open Source: While larger organizations are sometimes able to benefit from a robust open-source environment, a smaller organization will often find it difficult to provide internal support for mission critical open-source solutions while also providing support around normal business operations. An enterprise solution with applicable product support seems like the best solution in most environments.

Top SIEM Tools

Splunk

Splunk is a full on-premises SIEM solution that Gartner rates as a leader in this space. Splunk supports security monitoring and can provide advanced threat detection capabilities. It requires a skilled technician to adequately configure and support during the life-cycle of the product.

IBM QRadar

QRadar is another popular SIEM that you can deploy as a hardware appliance, a virtual appliance, or a software appliance, depending on your organization’s needs and capacity.

LogRhythm

LogRhythm is a good SIEM for small and mid-size organizations. It can be supported with less skilled technicians and analysts than come of the other tools.

Other Resources

Additional Information

Sample Vendors

Short list of some MSSP companies