Read Countdown to Zero Day: Stuxnet and the Launch of the World's First Digital Weapon Online
Authors: Kim Zetter
Assante visited the INL lab for himself and was impressed with the group of experts the lab had recruited for its program. In addition to control-system engineers, the lab had hired a group of code warriors fresh out of high school and college who knew how to hack them. They cut through the control-system networks with little resistance, exploiting weaknesses that were invisible to the engineers who had worked on them for years. The lab also had its own substations and mini-grid—a sevenmile section of redundant grid that researchers could isolate from the public grid—to run live tests. Assante was so intrigued by the possibilities for conducting real security research on the grid—not just simulated tests—that he quit his job at AEP in 2005 and took a position with the lab.
Once there, Assante and his colleagues began to consider scenarios for how equipment might be destroyed. Until then, most of the cyber concern around the security of the grid had been focused on someone getting into a power network to open breakers and create an outage. A power outage, however, could be resolved fairly quickly by resetting the breakers. But what about an attack that defeated or bypassed security and safety systems to physically destroy a generator that couldn’t be easily fixed?
They decided to get at the generator by focusing the attack on protective relays—safety devices that monitor changes in the grid and are
responsible for tripping breakers if conditions enter a danger zone that could harm transmission lines. Disabled protective relays played a role in a large outage in February 2008, when nearly 600,000 people in Florida lost power after a field engineer with Florida Power and Light turned off the protective relays at a substation while investigating a malfunctioning switch.
58
When a fault occurred on the line that he was examining, there was nothing to keep it from radiating out. The result was a cascading outage that spread to thirty-eight substations, including one that fed electricity to a nuclear plant, causing the plant to go into automatic shutdown.
But protective relays don’t just trigger breakers on transmission lines, they also disconnect generators and other equipment from the grid if conditions grow dangerous. The power grid operates at 60 Hz—or sixty cycles a second—and devices connected to it have to be in sync or they can be damaged. Plug something into the grid when it’s out of sync and it creates torque that can destroy the equipment. When a generator connects to the grid, the load from the grid pushes back, like the gravitational force that pushes against a car climbing a hill. But when a breaker opens up and disconnects the generator from the grid, the still-running generator speeds up in the absence of any load pushing against it. Within just 10 milliseconds, the generator will be out of sync with the grid. If the breaker then closes, bringing the generator back onto the grid while the two are out of sync, the effect is similar to a car hitting a brick wall. The generator expends too much energy that has nowhere to go, and once it hits the slower grid, the force of that energy slams back against it. It’s a well-known phenomenon that has been the cause of accidents in the past.
So the question Assante’s team posed for their test was simple: If protective relays were supposed to prevent equipment from being damaged, what if they could be subverted to aid the equipment’s destruction? Designing such an attack turned out to be only slightly more complicated
than the question. The hack involved writing malicious code to change the settings of the digital relays so that the breaker for a generator opened and closed in rapid succession, causing the equipment to disconnect from the grid quickly and repeatedly and then reconnect when it was out of sync. Without the protection of the relays, there was nothing to prevent the generator from destroying itself. “This is what made it so damn insidious,” says Joe Weiss. “The thing that was supposed to stop an attack like this from happening was the thing they used to conduct the attack.” By abruptly opening and closing the protective circuit, the relay went from “providing maximum protection to inflicting maximum damage,” the DHS later wrote in a report about the test.
59
For their victim, they chose a Wärtsilä generator that had been retired from the oil fields in Alaska and was purchased through a broker for one-third of its $1 million price tag brand-new.
60
The attack lasted three minutes but could have achieved its aim in just fifteen seconds. The researchers had built in pauses to the attack to give engineers time to assess the damage and check safety systems at each stage. Each time the circuit breaker closed on the out-of-sync generator, connecting it back to the grid, the machine jumped and vibrated from the force of its own energy hitting back, until eventually the coupling between the diesel engine and the generator broke.
61
Workers in the operations center who monitored the grid for anomalies
and weren’t told of the attack before it occurred never noticed anything amiss on their monitors. The safety system that was designed to ride out little spikes and valleys that normally occurred on the grid also never registered the destructive interruption. “We could do the attack, essentially open and close a breaker so quickly that the safety systems didn’t see it,” said Perry Pederson, who headed DHS’s control-system security program at the time and oversaw the test.
62
Replacing a twenty-seven-ton generator that was destroyed in this way wouldn’t be trivial, but it was doable. But there were 800-megawatt generators at large power plants and other facilities that would take months or a year to replace, since generators that size are often built to order overseas. Not all generators powering the grid would be susceptible to this attack in the same way—it would depend on how the power on that part of the grid is balanced. But the same thing could happen with critical equipment, powering things other than the grid, that isn’t easily replaced. A substation powering a bank of pumps responsible for delivering drinking water to a major metropolitan area, for example, would cause major disruptions if taken out of commission. “I don’t know what would happen to a big 50,000 horsepower pump, but I could imagine it would be just as bad as a generator,” Pederson said.
63
Since the Aurora test took place in 2007, there have been other demonstrations of destructive cyberattacks. In a 2009 report on
60 Minutes
, researchers at Sandia National Lab showed how they could cause components at an oil refinery to overheat by simply changing the settings of a heating element and disabling the recirculation pumps that helped regulate the temperature.
64
STUXNET AND THE
Maroochy Shire incident aside, there have been no really destructive digital attacks recorded in the world to date. Experts
have offered a number of possible reasons for why this is the case—such attacks are more difficult to pull off than the evidence presented here seems to indicate, and those who possess the skills and resources to conduct them have simply lacked the motivation to take action thus far, while others who have the will to launch such an attack don’t yet have the way.
One thing, however, seems certain: given the varied and extensive possibilities for conducting such attacks, and the proof of concept provided by Stuxnet, it is only a matter of time until the lure of the digital assault becomes too irresistible for someone to pass up.
1
Author interview with Assante, September 2011.
2
SCADA systems are generally used where the systems being managed are geographically dispersed over large areas—such as in pipelines, railway systems, and water and electrical distribution. Distributed control systems, on the other hand, are best for when operators need extensive and complex control in confined facilities like refineries and water-treatment and power-generation plants, although power plants also use SCADA systems to monitor remote substations in the field. SCADA systems consist of an operator’s station, a communications network, and a remote terminal unit, or RTU, in the field. The RTUs, which are similar to PLCs, send data back through the network to the operator’s monitoring station. The operator’s station generally runs on Windows—with all of its inherent vulnerabilities—and the field devices use specialized operating systems, which generally have little security built into them.
3
Industrial control system incidents are tracked in the RISI database (Repository of Industrial Security Incidents), which began recording incidents in 2001 but fell dormant between 2006 and 2009. The subscription database is maintained by the Security Incidents Organization and can be found at
securityincidents.org
.
4
Marty Niland, “Computer Virus Brings Down Train Signals,” Associated Press, August 20, 2003, available at
informationweek.com/news/13100807
.
5
The worm arrived via the corporate network of the utility company that operated the plant, and spread from the plant’s business network to the control network. Luckily, the plant had been offline for nearly a year due to other issues, so no harm was done. Plant operators also said they had manual controls that would have served as backup with the automated ones down. See Kevin Poulsen, “Slammer Worm Crashed Ohio Nuke Plant Network,” SecurityFocus, August 19, 2003, available at
securityfocus.com/news/6767/
.
6
“Cybersecurity: Preparing for and Responding to the Enduring Threat,” Speech to the Senate Committee on Appropriations, June 12, 2013, available at
hsdl.org/?view&did=739096
.
7
Lewis was speaking on the Diane Rehm radio show, broadcast by WAMU in Southern California, on June 4, 2012. Interview available at
thedianerehmshow.org/shows/2012-06-04/growing-threat-cyberwarfare
.
8
Gregory Benford, a physicist, is credited with one of the first mentions of a computer virus in a story he wrote in 1969 called “The Scarred Man,” which was published in the May 1970 issue of
Venture
magazine. The notion of digital worms originated in John Brunner’s 1975 science-fiction book
The Shockwave Rider
, which featured a digital tapeworm that slithered from machine to machine.
9
“Teen Hacker Pleads Guilty to Crippling Mass. Airport,”
Boston Globe
, March 19, 1998.
10
“Teen Hacker Faces Federal Charges,” CNN, March 18, 1998, available at
edition.cnn.com/TECH/computing/9803/18/juvenile.hacker/index.html
.
11
“Critical Foundations: Protecting America’s Infrastructures,” President’s Commission on Critical Infrastructure Protection, October 1997. The report is available at
https://www.fas.org/sgp/library/pccip.pdf
.
12
“Electric Power Risk Assessment,” National Security Telecommunications Advisory Committee, Information Assurance Task Force, available at
solarstorms.org/ElectricAssessment.html
.
13
Information about the Maroochy Shire case comes from an author interview conducted August 2012 with Robert Stringfellow, a water district engineer who helped investigate the case, as well as from redacted court documents and a police report written by forensic examiner Peter Kingsley. Some details from the court documents were first published by Joe Weiss in his book
Protecting Industrial Control Systems from Electronic Threats
(New York: Momentum Press, 2010).
14
Between March 14 and April 23 about ninety incidents occurred. A worker traced some of the activity to the RTU at pump station 14, where the malicious radio signals seemed to originate. He knew it was easy to alter the address of an RTU simply by flipping certain switches on the device, so he concluded the intruder must be using a rogue RTU that had its switches set to 14 to send out malicious commands as if they were coming from the real pump station 14. To set a trap, he changed the address for pump station 14 to 3. If an intruder
was
sending spoofed messages, he wouldn’t know the address had changed and would continue sending his messages under the old address. That’s exactly what occurred one night when a flood of malicious traffic sailed across the network from pump station 14 aimed at crashing the central computer. Investigators concluded, then, that the attacker must be an insider with knowledge of the Maroochy system and access to Hunter WaterTech software and equipment.
15
Boden was convicted and sentenced in October 2001 to two years in prison. He later appealed, at which point his conviction on two of the charges was set aside, but his conviction on other charges remained, as well as his sentence.
16
A survey of utilities conducted by the Electronic Power Research Institute in 1996 found that only 25 percent of respondents reported using any intrusion detection methods. The survey, the EPRI Summer 1996 Electronic Information Security Survey, and the statistic are referenced at
solarstorms.org/ElectricAssessment.html
.
17
Maroochy Water Services had little choice but to involve law enforcement in the case, because the spillages were so public and threatened public safety. The incidents also brought heavy scrutiny from Australia’s environmental protection agency and from regional government officials who demanded an explanation for why they occurred.
18
Kingsley was speaking at the AusCERT2002 conference in Australia. In a report examining the Maroochy case in 2008, eight years after it occurred, the authors concluded that some of the issues raised by the incident were just beginning to be addressed by companies, while “some are unresolved with no solution in sight.” See Marshall Abrams and Joe Weiss, “Malicious Control System Cyber Security Attack Case Study–Maroochy Water Services, Australia,” February 23, 2008, available at
csrc.nist.gov/groups/SMA/fisma/ics/documents/Maroochy-Water-Services-Case-Study_report.pdf
.
19
This and other quotes from Weiss in this chapter come from an author interview, June 2012.
20
Barton Gellman, “Cyber-Attacks by Al Qaeda Feared,”
Washington Post
, June 27, 2002.
21
Ibid.
22
“Critical infrastructure” in the United States is broadly defined by the government as any facility or system that falls into one of sixteen categories that include: agriculture and food, banking and finance, chemical, commercial facilities, critical manufacturing, dams, defense industrial base, drinking-water and water-treatment systems, emergency services, energy, government facilities, information technology, nuclear reactors and waste, public health and health care, telecommunications, and transportation. See
dhs.gov/critical-infrastructure-sectors
.
23
Dan Morain, “Hackers Victimize Cal-ISO,”
Los Angeles Times
, June 9, 2001.
24
A previous SCADA testing program had launched at Sandia National Laboratory in 1998, but didn’t involve vendors. INL is the Department of Energy’s lead lab for nuclear energy research and runs the largest test reactor in the world. The Atomic Energy Commission took over the land in Idaho after World War II to build a nuclear research lab. Over the years, the lab’s work expanded to include research on the electric grid and, after the Bush administration released its National Strategy to Secure Cyberspace in February 2003, the security of industrial control systems. That strategy called for the Department of Energy and the Department of Homeland Security (DHS) to partner with private industry to address the security of control systems.
25
Department of Homeland Security, “The National Strategy for the Physical Protection of Critical Infrastructures and Key Assets” (report, The White House, February 2003), 9. Available at
dhs.gov/xlibrary/assets/Physical_Strategy.pdf
.
26
Ibid., 39.
27
One problematic loophole in the test program, however, is that the reports vendors receive describing the vulnerabilities found in their systems are covered by a nondisclosure agreement and the vendors are not required to tell customers about the vulnerabilities found in their systems.
28
Kim Zetter, “Hard-Coded Password and Other Security Holes Found in Siemens Control Systems,”
Wired.com
, August 3, 2011, available at
wired.com/2011/08/siemens-hardcoded-password
.
29
Joe Weiss estimates that more than half of all control systems have a back door embedded in them by the vendor.
30
Beresford also found one more surprise—an “Easter egg” that a Siemens programmer had hidden in the firmware. Easter eggs are inside jokes that coders bury in their programs for users to find. Often they can be seen only if a user types a specific sequence of keys or accesses an obscure part of the program. In Siemens’s case, the Easter egg consisted of an animated image of dancing chimpanzees that appeared on-screen with a German proverb. Translated loosely the proverb said, “All work and no play makes Jack a dull boy.” Though the Easter egg wasn’t malicious, it raised serious concerns about Siemens’s security. If a programmer had slipped the joke past the company’s internal code reviewers, what else might have slipped by them?
31
In 2013, two researchers found problems with a popular protocol used by control centers to communicate with PLCs and RTUs installed at substations. An intruder who couldn’t gain direct access to the control-center machine via the internet could compromise the communication device at a remote substation—either by accessing it physically or hacking into the wireless radio network it uses to communicate with the control center—and exploit a vulnerability in the protocol to send malicious commands to the control center. In this way an attacker could either crash the control-center machine or use it to distribute malicious commands to all of the substations with which that machine communicates, potentially taking out dozens or even hundreds of substations at a time, depending on the size of the utility. See Kim Zetter, “Researchers Uncover Holes That Open Power Stations to Hacking,”
Wired.com
, October 16, 2013, available at
wired.com/2013/10/ics
.
32
Jordan Robertson, “Science Fiction–Style Sabotage a Fear in New Hacks,” Associated Press, October 23, 2011, available at
news-yahoo.com/science-fiction-style-sabotage-fear-hacks-120704517.html
.
33
In 2003, according to Joe Weiss, when the SQL Slammer worm hit the internet, one controlsystem supplier warned its customers not to install a patch released by Microsoft to combat the worm, because the patch would shut down their system.
34
Safety systems at nuclear plants, luckily, are still controlled by analog means, according to Joe Weiss, and the horror of a core meltdown being caused by a cyber incident is very low for existing nuclear plants. But that could change, since designs for next-generation plants include digital, networked systems, he says, that could make it easier to attack such plants.
35
Kim Zetter, “10K Reasons to Worry About Critical Infrastructure,”
Wired.com
, January 24, 2012, available at
wired.com/2012/01/10000-control-systems-online
. Researcher Eireann Leverett was unable to determine how many of the control systems were working systems as opposed to demo systems, and couldn’t tell how many of them were critical systems as opposed to simply an office heating system at a plant. But he did identify control systems for water facilities in Ireland and sewage facilities in California among them, and even controls for a heating system can sometimes be leveraged by attackers to access other parts of a network. And only 17 percent of the 10,000 systems he found required authorization to connect to them. In some cases, the owners weren’t even aware their systems were accessible online.
36
Paul F. Roberts, “Hacker Says Texas Town Used Three Character Password to Secure Internet Facing SCADA System,” Threatpost blog, November 20, 2011, available at
threatpost.com/blogs/hacker-says-texas-town-used-three-character-password-secure-internet-facing-scada-system-11201/75914
.
37
His statements appeared on the Pastebin site on November 18, 2011. See
Pastebin.com/Wx90LLum
.
38
Ken Dilanian, “Virtual War a Real Threat,”
Los Angeles Times
, March 28, 2011.
39
Kim Zetter, “Chinese Military Linked to Hacks of More Than 100 Companies,”
Wired.com
, February 19, 2013, available at
wired.com/2013/02/chinese-army-linked-to-hacks
. For more information on the specifics of the Telvent hack, see also Kim Zetter, “Maker of Smart-Grid Control Software Hacked,”
Wired.com
, September 26, 2012, available at
wired.com/2012/09/scada-vendor-telvent-hacked
.
40
“Report of the Commission to Assess the Threat to the United States from Electromagnetic Pulse (EMP) Attack,” April 2008, available at
empcommission.org/docs/A2473-EMP_Commission-7MB.pdf
. See also
footnote 25
for a description of an intentional electromagnetic pulse attack plan.
41
A 1996 RAND study titled “The Day After … in Cyberspace” was one of the first to imagine the consequences of a multipronged attack that targeted planes, trains, phone systems, and ATMs on a number of continents. See Robert H. Anderson and Anthony C. Hearn, “An Exploration of Cyberspace Security R&D Investment Strategies for DARPA: The Day After … in Cyberspace II,” RAND, 1996, available at
rand.org/pubs/monograph_reports/MR797.html
.