FireEye iSIGHT Intelligence compiled extensive data from dozens of
ICS security health assessment engagements (ICS Healthcheck) performed
by Mandiant, FireEye's consulting team, to identify the most pervasive
and highest priority security risks in industrial facilities. The
information was acquired from hands-on assessments carried out over
the last few years across a broad range of industries, including
manufacturing, mining, automotive, energy, chemical, natural gas, and
utilities. In this post, we provide details of these risks, and
indicate best practices and recommendations to mitigate the identified risks.
Mandiant ICS Healthchecks
Mandiant
ICS Healthchecks and penetration
testing engagements include on-site assessments of customers' IT
and ICS systems. The ICS Healthcheck consists of workshops and
technical reviews. It captures the results in a final report that
ranks discovered findings and vulnerabilities by risk using Mandiant’s
Risk Rating method. During an onsite workshop with site technical
experts, Mandiant develops a technical understanding of the subject
control system(s), builds a network diagram of the control system,
analyzes for potential vulnerabilities and threats, and assists with
prioritizing recommended countermeasures to defend the environment.
Mandiant also collects and reviews packet captures of network
traffic from the ICS environment to validate the network diagram
constructed in the workshop and to identify any unexpected or
undesirable deviations from the intended design. This traffic is also
analyzed for evidence of compromise or misconfiguration of the ICS
network/system. Mandiant inspects the deployed security technology for
vulnerabilities and other architectural risks, such as inappropriately
configured firewalls, dual-homed control system devices, and
unnecessary connectivity to the business network or the Internet.
NOTE: Findings are discussed at a generalized level to preserve the
anonymity of our customers. This post presents a high-level overview
and is meant to be an informative first stop for customers
interested in common cyber security issues. For more information or
to request Mandiant services, please visit our website.
Methodology: Mandiant Risk Rating System
This blog post leverages information from Mandiant ICS Healthchecks,
which evaluate cyber security risk in organizations from multiple
industries. The rating of critical and high security risk is based on
the Mandiant Risk Rating System, which is determined by identifying
the exploitability and the impact of a given issue, and
cross-referencing the results (Figure 1).
Figure 1: Impact/exploitability graphic
One Third of Security Risks in ICS Environments Ranked High or Critical
We reviewed findings from all of our risk assessments and then
categorized and ranked the reported risks as critical or high, medium,
low, or informational (Figure 2).At least 33 percent of the
security issues we found in ICS organizations were rated of high or
critical risk. This means they were most likely to allow adversaries
to readily gain control of target systems and potentially compromise
other systems or networks, cause disruption of services, disclose
unauthorized information, or result in other significant negative
consequences. We suggest immediate remediation for critical risks, and
quick action to remediate high security risks.
Figure 2: Risk assessment distribution
Most Common High and Critical Security Risks in ICS Environments
FireEye iSIGHT Intelligence organized the critical and high security
risks identified during Mandiant ICS Healthchecks into nine unique
categories (Table 1). The three most common were:
Vulnerabilities, Patches,
and Updates (32 percent)
Identity and Access Management (25
percent)
Architecture and Network Segmentation (11
percent)
In most of these cases, basic security best practices would be
enough to stop (or at least make it more difficult for) threat actors
to target an organization's systems. The implications are vast because
specialized malware or actors targeting infrastructure would likely
look for these flaws first to exploit throughout the targeted attack lifecycle.
Table 1: Distribution of high and
critical security risks in ICS environments
Top Three High and Critical Risks and Recommended Mitigations
Vulnerabilities, Patches, and Updates
Vulnerability, patch, and update management procedures enable
organizations to secure off-the-shelf software, hardware, and firmware
from known security threats. Known vulnerabilities in ICS environments
can be leveraged by threat actors to access the network and move
laterally to execute targeted attacks. The following common risks were
observed during our engagements:
Infrequent procedures for
patching and updating control systems:
We encountered
organizations with no formal vulnerability and patch management
programs.
Out-of-date firmware, hardware, and
operating systems (OS), including:
Network devices and
systems such as switches, firewalls, and routers.
Hardware equipment, including desktop computers, cameras,
and programmable logic controllers (PLCs).
Unsupported
legacy operating systems such as Windows Server 2003, XP, 2000,
and NT 4.
Unaddressed known vulnerabilities
in software applications and equipment where patches are
available:
We observed outdated firewalls with up to 53
unaddressed vulnerabilities and switches with more than 200
vulnerabilities.
System management software that can be
exploited using known open source tools.
Lack of test environments to analyze patches and updates before
implementation.
Mitigations
Develop a comprehensive
ICS Vulnerability Management Strategy and include procedures to
implement patches and updates on key assets. More information is
provided by the National Institute for Standards and Technology's
(NIST) Guide for ICS Security NIST
SP800-82.
When patches and updates are no longer provided
for key infrastructure, choose one of the two following options:
Implement a security perimeter around affected assets,
protected by, at minimum, a firewall (industrial protocol
inspection/blocking if appropriate) for access control and
traffic filtering.
Decommission legacy devices that
might be exploited to gain access to the network, such as
switches.
Set up development systems or labs
that are representative of the running IT and ICS devices. These
systems can often be built from existing spares along with the
purchase or loan of additional licenses for human-machine interfaces
(HMIs) and configuration software from the system vendor. A
development system is an excellent platform to test changes and
patches, and on which to perform vulnerability scans without risk to
active systems.
Identity and Access Management
The second most common category of security issues identified was
related to the flaws in or absence of best practices for handling
passwords and credentials. Common weaknesses identified by Mandiant include:
Lack of multi-factor
authentication for remote access and critical accounts:
Users were able to remotely access ICS environments from the
corporate network without requiring multi-factor
authentication.
Lack of a comprehensive and
enforced password policy:
Weak passwords with insufficient
length or complexity used for privileged accounts, ICS user
accounts, and service accounts.
Passwords were not
changed frequently.
Passwords were reused for multiple
accounts.
Prominently displayed
passwords:
Passwords were written on the chassis of
devices.
Hard-coded and default credentials
in applications and equipment:
Mandiant discovered Remote
Terminal Units (RTUs) containing default credentials, which are
commonly available on the Internet and in the device
manuals.
A modem contained a backdoor account
incorporated by the manufacturer.
Commonly
used “administrator” accounts.
Use of shared
credentials.
Mitigations
Implement two-factor
authentication for all possible users, especially administrative
accounts.
Avoid keeping written copies of passwords and, if
necessary, secure them out of sight with limited access for only
authorized users.
Enforce password policies that require
strong passwords that are regularly modified and cannot be reused.
More information is available from SANS.
Avoid common, easily guessed user account names such as
"operator," "administrator," or
"admin." Instead, use uniquely named user accounts for all
access.
Require administrative users to log in with uniquely
named user accounts with strong passwords, tied back to an
individual person.
Avoid shared accounts when feasible.
However, if present, they should be hardened using strong passwords
that are stored in an encrypted password manager.
Network Segregation and Segmentation
Of the top three risks identified in this post, weaknesses in
network segregation and segmentation are the most important. Lack of
segregation from the corporate IT network and within the ICS network
allows threat actors opportunities to launch remote attacks against
key infrastructure by moving laterally from IT services to ICS
environments. Furthermore, it increases the risk of commodity malware
spreading to ICS networks where the malware could interact with
operational assets. The main risks identified by Mandiant included:
Plant systems accessible
from the corporate network, either directly or through bridge
devices (connected to both networks), such as unused servers, HMIs,
historians, or loosely configured shared firewalls. We also
found:
Unfiltered access to plant servers from corporate
networks through, for example, a historian communicating with
the distributed control system (DCS).
Missing
segmentation between ICS and corporate networks.
Vulnerabilities in bridge devices (e.g., outdated appliances
running vulnerable OS) that can enable lateral movement between
networks.
Business functions (e.g., data backups and
anti-virus updates) running on shared control system
networks.
Dual-homed systems, both servers
and desktop computers.
Industrial networks connected
directly to the internet.
Mitigations
Segment all access to ICS
with a network Demilitarized Zone (DMZ), as recommended by both NIST
SP 800-82 and IEC (Figure 3):
Restrict the number of
ports, services, and protocols used to establish communications
between the ICS and corporate networks to the least possible to
reduce the attack surface.
Terminate incoming access
for both regular and administrative users first in the DMZ, and
then establish another session with connectivity into the ICS
network.
Place servers (or mirrored servers) that provide
ICS data to the corporate network in the DMZ.
Use
firewalls to filter all network traffic entering or leaving the
ICS.
Firewall rules should filter both incoming traffic
from the corporate network and outgoing traffic from the ICS,
and they should only allow the minimum required amount of
traffic to pass.
Isolate the control
networks from the internet. A separate network should be used for
internet access through a DMZ, and at no time should a bridged
connection be allowed between the two networks.
Ensure that
independent, regularly patched firewalls are used to separate the
corporate network from the DMZ and ICS network, and review firewall
rulesets on a regular basis.
Identify and redirect any
non-control system traffic traversing the industrial network.
Eliminate all dual-homed servers and hosts.
Figure 3: Reference architecture for
segmentation of enterprise and control system networks
Additional Highlights
Additional common risks were identified from other categories, but
with less frequency.
Network Management and Monitoring
We identified the lack of
Network Security Monitoring, Intrusion Detection, and Intrusion
Prevention in organizations, including missing endpoint malware
protection, leaving unused ports active, and having limited
visibility into ICS networks. We recommend the following best
practices:
A comprehensive network security monitoring
strategy should be defined and implemented at the ICS level as
part of an overarching ICS security program. Special attention
should be placed on monitoring network segments where external
connectivity occurs:
Implement or increase
centralized system and network logging to provide visibility across
the entire enterprise (IT and ICS). Monitor logs for anomalous
behavior. Consider implementing additional host or network-based
security controls that generate alerts or reject traffic based on
anomalous or suspicious behavior.
Install a centrally
managed anti-malware solution on all ICS and ICS DMZ hosts. Ensure
that signature and application updates are deployed in a timely
manner.
Explore alternatives for the deployment of an
advanced endpoint protection solution that provides
detection/prevention for malware and malicious activities that do
not rely on signature-based detection methods.
Develop
procedures to identify and shut down network ports when not in
use.
Misconfigurations in Firewall Rules
We identified weak firewall rules including "ANY-ANY"
configurations, conflicting or overlapping rules, overly permissive
conditions allowing access to administrative services, and lack of
console connection timeouts. We recommend the following best practices
for secure firewall configuration:
Filtering rules should
only allow access from/to specific source/destination IP addresses
and ports.
Filter rules should specify a specific network
protocol.
ICMP filter rules should specify a specific message
type.
Filter rules should drop network packets instead of
rejecting them.
Filter rules should perform a specific
action and not rely on a default action.
Administrative
session timeout parameters should be set to terminate those sessions
after a predetermined amount of time.
Cyber Security Governance Best Practices
We identified some organizations with limited or absent formal and
comprehensive ICS security programs. We highly suggest organizations
implement ICS security programs to prioritize the following recommendations:
Establish a formal ICS
security program with a clearly defined owner, accountability, and
governance structure. It should include:
Business
expectations, policies, and technical standards for ICS
security.
Guidance on proactive security controls (e.g.,
implementation of patches and updates, change management, or
secure configurations).
Incident Response, Disaster
Recovery, and Business Continuity plans.
ICS security
awareness training plans.
Develop a
Vulnerability Management Strategy following NIST
SP800-82, including asset identification and inventory, risk
assessment and analysis methodology (with prioritization of critical
assets), remediation testing, and deployment guidelines.
Conclusion
This blog post presents a broad picture of the current risks facing
industrial organizations as observed during Mandiant ICS Healthchecks.
While the trends observed in this research align with risk areas
commonly discussed in security conference talks and media reports,
this blog draws from dozens of on-site assessments that hold real-life validity.
Our findings indicate that at least one third of the critical and
high security risks in ICS are related to vulnerabilities, patches,
and updates. Known vulnerabilities continue to represent significant
challenges for ICS owners that must oversee the daily operation of
thousands of assets in complex industrial environments. It is also
relevant to highlight that some of the most common risks we identified
could be mitigated with security best practices, such as enforcing a
comprehensive password management policy or establishing detailed
firewall rules. If you are interested in more information or to
request Mandiant services, please visit our website.