Game-Changer: Ukraine's Solution to Ensuring Uninterrupted Power Supply this Winter with Revolutionary Pizza Box-Sized Equipment
Ukraine, determined to keep the lights on this winter, has discreetly imported specialized equipment, the size of a pizza box, capable of withstanding Russian electronic warfare attacks With the assistance of US officials, this innovative solution aims to safeguard their power grid
Ukraine's power grid operator, facing another harsh winter and in dire need of maintaining power supply, has covertly procured specially-designed equipment capable of withstanding Russian electronic warfare attacks, with assistance from US officials, CNN has found out.
Tech engineers at Cisco, a prominent US company, dedicated weeks to constructing and subjecting the newly-developed gear to rigorous testing in a research facility based in Austin, Texas. Subsequently, Cisco transported a prototype to Ukraine in the spring, aided by a US Air Force aircraft carrying humanitarian aid.
Ukrenergo, Ukraine's state-owned grid operator, has confirmed that new equipment has successfully functioned despite Russian attacks on its GPS systems. Cisco has delivered numerous compact hardware kits, with an estimated value of $1 million, to Ukraine. These hardware kits have been installed throughout the country, offering a potential lifeline to Ukraine's electricity grid. The grid remains a significant target of Russian attacks, as the conflict enters its second consecutive winter. Over the past two years, Russian missile and drone strikes have demolished approximately 40% of Ukrenergo's power substations and associated equipment.
Russian military intelligence agency, the GRU, was behind a recent cyberattack that caused a power outage in Ukraine in October 2022, as revealed by US experts. Illia Vitiuk, the head of cybersecurity at the Ukrainian security service SBU, expressed concerns about possible future Russian hacking attempts on power plants, especially during the upcoming winter period.
The
A steak dinner near Stanford
that Cisco intended to address stems from the interference caused by Russian radio-jammers in Ukraine. These jammers disrupt the GPS systems that Ukrenergo depends on to regulate the power flow.
According to sources from Cisco, Ukraine, and the US government, a covert operation involving US corporations has been utilized by the Biden administration to assist Ukraine in its defense, while avoiding direct conflict with Russia. As part of this effort, SpaceX has supplied satellite coverage for the Ukrainian military, Microsoft has assisted in relocating Ukrainian government data centers away from the country prior to the invasion, and the CEO of data analytics firm Palantir has claimed that their software played a significant role in target selection for the Ukrainian military.
Multiple US agencies were involved in quietly facilitating the entry of Cisco equipment into Ukraine, according to sources. The Pentagon managed the flights, the Department of Energy assisted in coordinating the delivery of the equipment, and the Department of Commerce arranged important meetings earlier this year between a select group of US technology executives and Ukrenergo managers. These managers were seeking new strategies to protect their grid from Russian attacks. In February, Ukrenergo executives engaged in discussions and exchanged experiences with their counterparts at Cisco during a dinner held at a high-end steakhouse near Stanford University.
Ukraine's grid operators revealed to their dinner companions that they were grappling with a grave yet overlooked issue. Both the Russian and Ukrainian militaries employ consistent GPS jamming to disrupt guided missiles, which inadvertently hampers the visibility of Ukraine's power grid operators. These operators heavily rely on GPS-based clocks to convey vital information about the flow of power between different locations.
In this undated photo, an employee at Ukraines state-owned grid operator, Ukrenergo, works on the power grid.
Courtesy Ukrenergo
Joe Marshall, a seasoned researcher at Talos, Cisco's cyber-intelligence unit, sat at the table that night. He listened attentively as the Ukrainians laid out their problem while enjoying their meal of steaks and drinks. Marshall's expertise lies in safeguarding electric power systems from sabotage in Ukraine and various other locations. However, Ukrenergos presented a unique challenge that he had never encountered before. Once dinner ended, Marshall retreated to his hotel room, determined to brainstorm a possible solution.
"Time played a crucial role," he emphasized. "We were deliberating on individuals' livelihoods."
Marshall dedicated extensive hours to studying YouTube videos shared by an electronic warfare specialist, seeking advice from both US authorities and industrial cybersecurity experts at Cisco and other reputable sources.
Being the largest manufacturer of computer networking equipment globally, Cisco had ample resources at their disposal. Under the leadership of Marshall, a team comprising of over twelve engineers undertook the task of customizing an industrial ethernet switch to cater to the unique requirements of the Ukrainian grid.
Although Cisco initially calculated the cost of materials and shipping for the switches to be $1 million, the company generously decided to donate the equipment to Ukrenergo without any charge.
Taras Vasyliv, the supervisor of power dispatching at Ukrenergo, compared the specially designed switches to a surgeon's flashlight in a dark operating room.
These switches enable an electric substation, responsible for converting power from high to low voltage, to establish communication with other components of the power grid. It was crucial for these switches to possess their own internal clocks capable of accurately measuring time, ensuring redundancy and granting grid operators visibility even during GPS system failures.
Otherwise, "youre blind," Vasyliv said in a phone interview from Kyiv.
The national grid relies on Ukrenergo, which operates a high-voltage electricity substation supplying power to more than 6 million consumers across multiple cities. The substation has suffered damage from a missile strike.
Vasyliv conveyed to CNN that a number of his co-workers lost their lives in the war as Russian forces relentlessly targeted Ukrenergo's infrastructure. However, his determination to ensure an uninterrupted power supply and evade impending airstrikes fuels his drive to carry on.
He asserts that he constantly reminds himself to simply fulfill his responsibilities with utmost proficiency.
Sneaking a switch into Ukraine
After their dinner in Silicon Valley, Marshall and his team were able to develop a prototype within just a few weeks. A crucial step now for Cisco was to determine the logistics of getting the prototype into Ukraine and ensuring that it functioned properly.
Marshall, an ex-Pentagon IT contractor hailing from Alabama, sought assistance from a US official in locating a military base on the East Coast that had a departing flight in April. This particular flight's route included a stop in Germany before ultimately reaching Rzeszów, Poland, a significant center for humanitarian and military aid located around 60 miles from the Ukrainian border.
Subsequently, the prototypes were carefully loaded onto a train at this location to be transported into Ukraine. Upon discreet arrival, the Ukrenergo engineers, led by Vasyliv, received the prototypes.
Vasyliv mentioned that his engineers tested the switch in a modest office located in western Ukraine as their offices in Kyiv were partially destroyed due to shelling. He remarked that the office resembled the startups in California during the 1970s rather than a sophisticated laboratory.
The functioning switches prompted Cisco to increase production in order to deliver more to Ukraine. US officials who were knowledgeable about the Cisco project refrained from sharing details about specific shipments to avoid the risk of revealing Russia's capability to disrupt them. Microsoft reported that the same GRU cyber-sabotage team, responsible for causing power outages in Ukraine, had previously attacked computers at logistics companies in Poland that were providing services to Ukraine.
Years of Russian attacks on Ukraines grid
In February, the Department of Energy stated that it had successfully identified, procured, and shipped close to 20 tons of electrical equipment to Ukraine using US Air Force cargo planes during the three-month period last winter.
US officials are frequently working behind the scenes to coordinate the provision of essential technology to Ukraine. The US Department of Defense recently announced that it is engaging SpaceX to extend its Starlink satellite service to Ukraine, as stated in July, while the price of the contract remains undisclosed.
This photo provided by Cisco shows damage to a substation in Ukraine.
Courtesy Cisco
US officials responsible for protecting the US electric sector have been studying Russia's digital sabotage of Ukraine's grid for almost ten years. This effort aims to support Ukraine in defending against such attacks and ensure US power companies are prepared to counter hacking techniques.
Following the GRU's initial use of hacking tools to disrupt power for approximately 225,000 Ukrainians in 2015, as stated in a US indictment and confirmed by private experts, the Department of Homeland Security sent a team to Ukraine to analyze the attack's forensics. A subsequent cyberattack in Ukraine in 2016 demonstrated Russia's continued development of their techniques.
On October 10, 2022, an unnamed Ukrainian electric facility became the target of the GRU. This resulted in an unplanned power outage, occurring simultaneously with air strikes by the Russian military on electric infrastructure across Ukraine, as reported by US cybersecurity firm Mandiant, who promptly responded to the hack. The extent of the power outage caused by the hacking remains uncertain. Ukrainian officials have informed CNN that distinguishing whether an outage is caused by air strikes or hacking can be challenging. However, this incident has raised concerns that the Russian hacking unit is advancing its ability to disrupt power in Ukraine more rapidly, driven by the intensity and demands of the war.
The North American grid regulator, NERC, stated to CNN that the cyberattack in Ukraine last year highlights the advancement of more powerful and faster threat capabilities in operational technology that could potentially be utilized in North America. CNN sources revealed that one of the Department of Energy's leading research laboratories, dedicated to anticipating future hacking threats to the US grid, will extensively analyze the techniques employed by the GRU during the October 2022 Ukraine hack.
