CyRC Vulnerability Advisory: Denial-of-service vulnerabilities in Zephyr Bluetooth LE stack

 

Overview

Zephyr OS is an Apache licensed real-time operating system project backed by the Linux Foundation and many big industry vendors. Zephyr is mainly used in embedded and resource-constrained systems. It supports a very wide variety of boards with different CPU architectures including ARM Cortex-M, Intel x86, ARC, NIOS II, Tensilica Xtensa, SPARC V8, and RISC-V 32. Zephyr’s native network stack supports multiple protocols including LwM2M, BSD sockets, OpenThread, full Bluetooth LE stack including Link Layer (LL), and Bluetooth Mesh.

The Bluetooth stack is split into two main components: host and controller. Zephyr’s Bluetooth LE controller is used as a part of the Synopsys Defensics Bluetooth LE fuzzing solution. To ensure the highest quality and security of our product, Zephyr’s Bluetooth LE stack’s lowest layers were fuzz tested using Defensics Bluetooth LE test suites.

Eight different vulnerabilities were discovered in the Bluetooth LE Link Layer and L2CAP implementation. The vulnerabilities can be divided into three high-level categories:

• Freeze: This vulnerability makes it possible for an attacker to remotely cause freeze or assertion failure on a target device by sending malformed input. In the case of a freeze, the behavior of a target device depends on whether assertions are enabled and if the error handler for fatal errors exists. It is common for the device to restart itself in case of hard faults. An attacker might use this to restart the device over the air with single packet when exploiting some other vulnerabilities. In some circumstances, freezing may lead to remote code execution.
• Deadlock: Some of the vulnerabilities can lead to a situation in which the target device misbehaves in a way that prevents other devices from connecting to it. The target must be rebooted to recover to normal state.
• Information leak: This vulnerability makes it possible for an attacker to gain access to potentially confidential information like encryption keys or information about memory layout. This type of vulnerability may also be used when the attacker is trying to bypass mitigation techniques like address space layout randomization, which is not present in Zephyr.

CVE ID	Vulnerability type	Host / Controller	Description
CVE-2021-3430	Freeze	Controller	Assertion failure on repeated LL_CONNECTION_PARAM_REQ
CVE-2021-3431	Freeze	Controller	Assertion failure on certain repeated LL packets
CVE-2021-3432	Freeze	Controller	Invalid interval in CONNECT_IND leads to Division by Zero
CVE-2021-3433	Deadlock	Controller	Invalid channel map in CONNECT_IND results to Deadlock
CVE-2021-3434	Freeze	Host	L2CAP: Stack based buffer overflow in le_ecred_conn_req()
CVE-2021-3435	Information leak	Host	L2CAP: Information leakage in le_ecred_conn_req()
CVE-2021-3454	Freeze	Host	L2CAP: Truncated L2CAP K-frame causes assertion failure
CVE-2021-3455	Freeze	Host	Disconnecting L2CAP channel right after invalid ATT request leads to use-after-free

Impact

All the reported vulnerabilities can be triggered from within the range of Bluetooth LE. Triggering the vulnerability does not require authentication or encryption. The only requirement is that the device is in advertising mode and accepting connections.

CVE-2021-3430: Assertion failure on repeated LL_CONNECTION_PARAM_REQ

The attacker is able to disrupt all ongoing communication and create a denial-of-service situation caused by reachable assertion by sending repeated LL_CONNECTION_PARAM_REQ packets to any Zephyr-based Bluetooth LE device that is advertising and connectable.

• Affected Zephyr versions: 2.5.0, 2.4.0, 1.14
• CVSSv3.1 overall score: 5.9 (Medium)
• CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H/E:P/RL:O/RC:C
• CWE-617

CVE-2021-3431: Assertion failure on certain repeated LL packets

The attacker is able to disrupt all ongoing communication and create a denial-of-service situation caused by reachable assertion by sending repeated LL_FEATURE_REQ, LL_PING_REQ, LL_LENGTH_REQ, or LL_PHY_REQ packets to any Zephyr-based Bluetooth LE device that is advertising and connectable.

• Affected Zephyr versions: 2.5.0, 2.4.0
• CVSSv3.1 overall score: 5.9 (Medium)
• CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H/E:P/RL:O/RC:C
• CWE-617

CVE-2021-3432: Invalid interval in CONNECT_IND leads to division by zero

The attacker is able to disrupt all ongoing communication and create a denial-of-service situation caused by division by zero by sending an invalid interval value in a CONNECT_IND packet to any Zephyr-based Bluetooth LE device that is advertising and connectable.

• Affected Zephyr versions: 2.5.0, 2.4.0, 1.14
• CVSSv3.1 overall score: 5.9 (Medium)
• CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H/E:P/RL:O/RC:C
• CWE-369

CVE-2021-3433: Invalid channel map in CONNECT_IND results to deadlock

The attacker is able to create a denial-of-service situation caused by deadlock by sending an invalid channel map value in a CONNECT_IND packet to any Zephyr-based Bluetooth LE device that is advertising and connectable.

• Affected Zephyr versions: 2.5.0, 2.4.0, 1.14
• CVSSv3.1 overall score: 3.9 (Low)
• CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L/E:P/RL:O/RC:C
• CWE-703

CVE-2021-3434: L2CAP: Stack based buffer overflow in le_ecred_conn_req()

The attacker is able to create a denial-of-service situation at minimum, but could also potentially achieve remote code execution by exploiting the stack-based buffer overflow in Zephyr’s L2CAP implementation.

• Affected Zephyr versions: 2.5.0, 2.4.0
• CVSSv3.1 overall score: 7.7 (High)
• CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H/E:U/RL:O/RC:C
• CWE-121

CVE-2021-3435: L2CAP: Information leakage in le_ecred_conn_req()

The attacker is able to read potentially confidential information, up to 6 bytes of uninitialized memory content, by sending one malformed L2CAP_CREDIT_BASED_CONNECTION_REQ packet. The request can be repeated to gain more data.

• Affected Zephyr versions: 2.5.0, 2.4.0
• CVSSv3.1 overall score: 5.9 (Medium)
• CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N/E:P/RL:O/RC:C
• CWE-908

CVE-2021-3454: L2CAP: Truncated L2CAP K-frame causes assertion failure

The attacker is able to disrupt all ongoing communication and create a denial-of-service situation caused by reachable assertion by sending a truncated L2CAP K-frame packet to any Zephyr-based Bluetooth LE device that is advertising and connectable.

• Affected Zephyr versions: 2.5.0, 2.4.0
• CVSSv3.1 overall score: 5.9 (Medium)
• CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H/E:P/RL:O/RC:C
• CWE-617

CVE-2021-3455: Disconnecting L2CAP channel right after invalid ATT request leads to use-after-free

The attacker is able to create a denial-of-service situation at minimum, but also potentially achieve remote code execution by exploiting the use-after-free in Zephyr’s L2CAP implementation.

• Affected Zephyr versions: 2.5.0, 2.4.0
• CVSSv3.1 overall score: 7.7 (High)
• CVSS:3.1/AV:A/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H/E:U/RL:O/RC:C
• CWE-416

Technical details

CVE-2021-3430: Assertion failure on repeated LL_CONNECTION_PARAM_REQ

Zephyr security advisory: https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-46h3-hjcq-2jjr

The assertion failure happened in ull_conn.c inside a ctlr_rx() function because the implementation did not clearly allow two connection parameters request procedures to be initiated by the central device at the same time. This is the correct behavior from specification point of view, but there should be a less destructive way to handle that. If the assertions were disabled, this test case would not cause problems.

This issue was resolved by replacing the assert with a comment and marking the buffer ready to be released. The fix was introduced in this pull request: https://github.com/zephyrproject-rtos/zephyr/pull/33272

CVE-2021-3431: Assertion failure on certain repeated LL packets

Zephyr security advisory: https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-7548-5m6f-mqv9

The assertion failure happened in the Nordic’s implementation of the lower Link Layer in lll_conn.c, isr_done() because of the Link Layer control procedure transaction collision. If the assertions were disabled, we saw a denial-of-service situation because the stack went into state, and it did not restart advertising after disconnection. Reboot is needed to recover.

Fixing this required adding new a locking mechanism for Link Layer control procedures. The fix was introduced in this pull request: https://github.com/zephyrproject-rtos/zephyr/pull/33340

CVE-2021-3432: Invalid interval in CONNECT_IND leads to division by zero

Zephyr security advisory: https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-7364-p4wc-8mj4
Function ull_slave_setup() in ull-slave.c did not check if the interval value was in valid range. CONNECT_IND with an interval value set to 0x0000 causes division by zero, leading to freeze.

This vulnerability was fixed in this pull request: https://github.com/zephyrproject-rtos/zephyr/pull/33278

CVE-2021-3433: Invalid channel map in CONNECT_IND results to Deadlock

Zephyr security advisory: https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-3c2f-w4v6-qxrp

Error handling of the ull_slave_setup() function in ull-slave.c was not working properly. There were two places where the function did an early return if it detected an invalid channel map value in CONNEC_IND PDU to prevent misbehavior or freeze. The problem was that the early return did not seem to be enough, as the device did not resume advertising after rejecting the connection when it received a channel map value with all zeros.

Freeze on channel map value 0x000000 was initially found in the Sweyntooth bundle of vulnerabilities and it was partially fixed with these commits:

https://github.com/zephyrproject-rtos/zephyr/commit/4a5f263e5a658bb35c1dd9215fe62939f175c859
https://github.com/zephyrproject-rtos/zephyr/commit/94d5f0854e491e594c2040fe0f061850848081fc

This vulnerability was fixed in this pull request: https://github.com/zephyrproject-rtos/zephyr/pull/33278

CVE-2021-3434: L2CAP: Stack based buffer overflow in le_ecred_conn_req()

Zephyr security advisory: https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-8w87-6rfp-cfrm

Function le_ecred_conn_req() did not check if the size of the incoming L2CAP_CREDIT_BASED_CONNECTION_REQ was too big and interpreted the overflowing data as a huge list of source CIDs. At the beginning of the function, variable chan is allocated memory from stack for L2CAP_ECRED_CHAN_MAX number of pointers to bt_l2cap_chan structs. Variable dcid is allocated memory from stack for L2CAP_ECRED_CHAN_MAX number of uint16_t values. L2CAP_ECRED_CHAN_MAX has been defined to be 5.

Later, the buffer overflow happened when l2cap_chan_accept() was called for all the source CIDs one by one. This didn’t yet freeze the execution. Another overflow happened on line 1152 where dcid is overflown.

Finally, there was another buffer overflow when constructing the L2CAP_CREDIT_BASED_CONNECTION_RSP, which then caused the freeze.

Enabling assertions led to assertion failure when trying to copy too much data into an outgoing L2CAP_CREDIT_BASED_CONNECTION_RSP packet.

This vulnerability was fixed in this pull request: https://github.com/zephyrproject-rtos/zephyr/pull/33305

CVE-2021-3435: L2CAP: Information leakage in le_ecred_conn_req()

Zephyr security advisory: https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-xhg3-gvj6-4rqh

Bug laid on the line where the value for i is calculated. This line did not take into account that if the connection request packet was too small, the buf still contains the whole packet, not just the source CIDs.

Later, uninitialized memory content was copied into the buffer of the outgoing packet.

Variable dcid is allocated from the stack. An uninitialized array could contain anything that happens to be in this memory location, for example, encryption keys.

This vulnerability was fixed in this pull request: https://github.com/zephyrproject-rtos/zephyr/pull/33305

CVE-2021-3454: L2CAP: Truncated L2CAP K-frame causes assertion failure

Zephyr security advisory: https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-fx88-6c29-vrp3

This happened because in the l2cap_chan_le_recv() function, when getting the SDU length from the buffer, the length of the buffer is not checked, and a later call to net_buf_pull_le16(buf) leads to assertion. If assertions were disabled, the SDU length would be calculated wrong and the packet would soon be dropped due to invalid SDU length.

This vulnerability was fixed in this pull request: https://github.com/zephyrproject-rtos/zephyr/pull/32588

CVE-2021-3455: Disconnecting L2CAP channel right after invalid ATT request leads to use-after-free

Zephyr security advisory: https://github.com/zephyrproject-rtos/zephyr/security/advisories/GHSA-7g38-3x9v-v7vp

When the central device connected to the peripheral and created an L2CAP connection for enhanced ATT, it sent some invalid ATT requests and disconnected, which immediately caused a freeze. This happened because the L2CAP channel was already in a disconnected state when the Zephyr stack called sent callback l2cap_chan_sdu_sent() and further bt_att_sent(). In bt_att_sent() freeze happens because the already freed memory block is being accessed.

This vulnerability was fixed in this pull request: https://github.com/zephyrproject-rtos/zephyr/pull/35597

Remediation

Product manufacturers using the Zephyr OS in their product are encouraged to update their Zephyr version to include latest security fixes. Zephyr’s security policy guarantees that security patches are backported to the two most recent releases and to active LTS release. For non-LTS Zephyr versions, manufacturers may need to take care of backporting the security patches themselves.

End users with products that include the firmware based on vulnerable Zephyr OS versions are strongly encouraged to upgrade to latest firmware version available from the vendor.

Discovery credit

Matias Karhumaa from the Synopsys Cybersecurity Research Center (CyRC) in Oulu, Finland, discovered these weaknesses with Defensics Bluetooth LE LL peripheral test suite and Bluetooth LE L2CAP server test suite.

Synopsys would like to thank the Zephyr security team and the maintainers of Zephyr Bluetooth subsystem for their responsiveness and for great cooperation.

Timeline

• February 2, 2021: Vulnerabilities discovered and analyzed.
• March 11, 2021: Vulnerabilities reported to Zephyr security team.
• March 11, 2021: Zephyr acknowledges the findings.
• March 15, 2021: Zephyr developers provide fixes to most of the Link Layer issues.
• March 17, 2021: Zephyr developers provide fixes to L2CAP related issues. However, while verifying the fixes two new issues are found in L2CAP.
• March 18, 2021: Two new L2CAP vulnerabilities reported to Zephyr security team.
• March 19, 2021: Zephyr security team acknowledges the issues.
• May 25, 2021: Last pending L2CAP vulnerability fixed and verified.
• June 5, 2021: Zephyr 2.6.0 released. The release includes fixes to all the reported vulnerabilities.
• June 22, 2021: Advisory published by Synopsys.

ML Platform Meetup: Infra for Contextual Bandits and Reinforcement Learning

 

Infrastructure for Contextual Bandits and Reinforcement Learning — theme of the ML Platform meetup hosted at Netflix, Los Gatos on Sep 12, 2019.

Contextual and Multi-armed Bandits enable faster and adaptive alternatives to traditional A/B Testing. They enable rapid learning and better decision-making for product rollouts. Broadly speaking, these approaches can be seen as a stepping stone to full-on Reinforcement Learning (RL) with closed-loop, on-policy evaluation and model objectives tied to reward functions. At Netflix, we are running several such experiments. For example, one set of experiments is focussed on personalizing our artwork assets to quickly select and leverage the “winning” images for a title we recommend to our members.

As with other traditional machine learning and deep learning paths, a lot of what the core algorithms can do depends upon the support they get from the surrounding infrastructure and the tooling that the ML platform provides. Given the infrastructure space for RL approaches is still relatively nascent, we wanted to understand what others in the community are doing in this space.

This was the motivation for the meetup’s theme. It featured three relevant talks from LinkedIn, Netflix and Facebook, and a platform architecture overview talk from first time participant Dropbox.

LinkedIn

Slides

After a brief introduction on the theme and motivation of its choice, the talks were kicked off by Kinjal Basu from LinkedIn who talked about Online Parameter Selection for Web-Based Ranking via Bayesian Optimization. In this talk, Kinjal used the example of the LinkedIn Feed, to demonstrate how they use bandit algorithms to solve for the optimal parameter selection problem efficiently.

He started by laying out some of the challenges around inefficiencies of engineering time when manually optimizing for weights/parameters in their business objective functions. The key insight was that by assuming a latent Gaussian Process (GP) prior on the key business metric actions like viral engagement, job applications, etc., they were able to reframe the problem as a straight-forward black-box optimization problem. This allowed them to use BayesOpt techniques to solve this problem.

The algorithm used to solve this reformulated optimization problem is a popular E/E technique known as Thompson Sampling. He talked about the infrastructure used to implement this. They have built an offline BayesOpt library, a parameter store to retrieve the right set of parameters, and an online serving layer to score the objective at serving time given the parameter distribution for a particular member.

He also described some practical considerations, like member-parameter stickiness, to avoid per session variance in a member’s experience. Their offline parameter distribution is recomputed hourly, so the member experience remains consistent within the hour. Some simulation results and some online A/B test results were shared, demonstrating substantial lifts in the primary business metrics, while keeping the secondary metrics above preset guardrails.

He concluded by stressing the efficiency their teams had achieved by doing online parameter exploration instead of the much slower human-in-the-loop manual explorations. In the future, they plan to explore adding new algorithms like UCB, considering formulating the problem as a grey-box optimization problem, and switching between the various business metrics to identify which is the optimal metric to optimize.

Netflix

Slides

The second talk was by Netflix on our Bandit Infrastructure built for personalization use cases. Fernando Amat and Elliot Chow jointly gave this talk.

Fernando started the first part of the talk and described the core recommendation problem of identifying the top few titles in a large catalog that will maximize the probability of play. Using the example of evidence personalization — images, text, trailers, synopsis, all assets that come together to add meaning to a title — he described how the problem is essentially a slate recommendation task and is well suited to be solved using a Bandit framework.

If such a framework is to be generic, it must support different contexts, attributions and reward functions. He described a simple Policy API that models the Slate tasks. This API supports the selection of a state given a list of options using the appropriate algorithm and a way to quantify the propensities, so the data can be de-biased. Fernando ended his part by highlighting some of the Bandit Metrics they implemented for offline policy evaluation, like Inverse Propensity Scoring (IPS), Doubly Robust (DR), and Direct Method (DM).

For Bandits, where attribution is a critical part of the equation, it’s imperative to have a flexible and robust data infrastructure. Elliot started the second part of the talk by describing the real-time framework they have built to bring together all signals in one place making them accessible through a queryable API. These signals include member activity data (login, search, playback), intent-to-treat (what title/assets the system wants to impress to the member) and the treatment (impressions of images, trailers) that actually made it to the member’s device.

Elliot talked about what is involved in “Closing the loop”. First, the intent-to-treat needs to be joined with the treatment logging along the way, the policies in effect, the features used and the various propensities. Next, the reward function needs to be updated, in near real time, on every logged action (like a playback) for both short-term and long-term rewards. And finally each new observation needs to update the policy, compute offline policy evaluation metrics and then push the policy back to production so it can generate new intents to treat.

To be able to support this, the team had to standardize on several infrastructure components. Elliot talked about the three key components — a) Standardized Logging from the treatment services, b) Real-time stream processing over Apache Flink for member activity joins, and c) an Apache Spark client for attribution and reward computation. The team has also developed a few common attribution datasets as “out-of-the-box” entities to be used by the consuming teams.

Finally, Elliot ended by talking about some of the challenges in building this Bandit framework. In particular, he talked about the misattribution potential in a complex microservice architecture where often intermediary results are cached. He also talked about common pitfalls of stream-processed data like out of order processing.

This framework has been in production for almost a year now and has been used to support several A/B tests across different recommendation use cases at Netflix.

Facebook

Slides

After a short break, the second session started with a talk from Facebook focussed on practical solutions to exploration problems. Sam Daulton described how the infrastructure and product use cases came along. He described how the adaptive experimentation efforts are aimed at enabling fast experimentation with a goal of adding varying degrees of automation for experts using the platform in an ad hoc fashion all the way to no-human-in-the-loop efforts.

He dived into a policy search problem they tried to solve: How many posts to load for a user depending upon their device’s connection quality. They modeled the problem as an infinite-arm bandit problem and used Gaussian Process (GP) regression. They used Bayesian Optimization to perform multi-metric optimization — e.g., jointly optimizing decrease in CPU utilization along with increase in user engagement. One of the challenges he described was how to efficiently choose a decision point, when the joint optimization search presented a Pareto frontier in the possible solution space. They used constraints on individual metrics in the face of noisy experiments to allow business decision makers to arrive at an optimal decision point.

Not all spaces can be efficiently explored online, so several research teams at Facebook use Simulations offline. For example, a ranking team would ingest live user traffic and subject it to a number of ranking configurations and simulate the event outcomes using predictive models running on canary rankers. The simulations were often biased and needed de-biasing (using multi-task GP regression) for them to be used alongside online results. They observed that by combining their online results with de-biased simulation results they were able to substantially improve their model fit.

To support these efforts, they developed and open sourced some tools along the way. Sam described Ax and BoTorch — Ax is a library for managing adaptive experiments and BoTorch is a library for Bayesian Optimization research. There are many applications already in production for these tools from both basic hyperparameter exploration to more involved AutoML use cases.

The final section of Sam’s talk focussed on Constrained Bayesian Contextual Bandits. They described the problem of video uploads to Facebook where the goal is to maximize the quality of the video without a decrease in reliability of the upload. They modeled it as a Thompson Sampling optimization problem using a Bayesian Linear model. To enforce the constraints, they used a modified algorithm, Constrained Thompson Sampling, to ensure a non-negative change in reliability. The reward function also similarly needed some shaping to align with the constrained objective. With this reward shaping optimization, Sam shared some results that showed how the Constrained Thompson Sampling algorithm surfaced many actions that satisfied the reliability constraints, where vanilla Thompson Sampling had failed.

Dropbox

Slides

The last talk of the event was a system architecture introduction by Dropbox’s Tsahi Glik. As a first time participant, their talk was more of an architecture overview of the ML Infra in place at Dropbox.

Tsahi started off by giving some ML usage examples at Dropbox like Smart Sync which predicts which file you will use on a particular device, so it’s preloaded. Some of the challenges he called out were the diversity and size of the disparate data sources that Dropbox has to manage. Data privacy is increasingly important and presents its own set of challenges. From an ML practice perspective, they also have to deal with a wide variety of development processes and ML frameworks, custom work for new use cases and challenges with reproducibility of training.

He shared a high level overview of their ML platform showing the various common stages of developing and deploying a model categorized by the online and offline components. He then dived into some individual components of the platform.

The first component he talked about was a user activity service to collect the input signals for the models. This service, Antenna, provides a way to query user activity events and summarizes the activity with various aggregations. The next component he dived deeper into was a content ingestion pipeline for OCR (optical character recognition). As an example, he explained how the image of a receipt is converted into contextual text. The pipeline takes the image through multiple models for various subtasks. The first classifies whether the image has some detectable text, the second does corner detection, the third does word box detection followed by deep LSTM neural net that does the core sequence based OCR. The final stage performs some lexicographical post processing.

He talked about the practical considerations of ingesting user content — they need to prevent malicious content from impacting the service. To enable this they have adopted a plugin based architecture and each task plugin runs in a sandbox jail environment.

Their offline data preparation ETLs run on Spark and they use Airflow as the orchestration layer. Their training infrastructure relies on a hybrid cloud approach. They have built a layer and command line tool called dxblearn that abstracts the training paths, allowing the researchers to train either locally or leverage AWS. dxblearn also allows them to fire off training jobs for hyperparameter tuning.

Published models are sent to a model store in S3 which are then picked up by their central model prediction service that does online inferencing for all use cases. Using a central inferencing service allows them to partition compute resources appropriately and having a standard API makes it easy to share and also run inferencing in the cloud.

They have also built a common “suggest backend” that is a generic predictive application that can be used by the various edge and production facing services that regularizes the data fetching, prediction and experiment configuration needed for a product prediction use case. This allows them to do live experimentation more easily.

The last part of Tsahi’s talk described a product use case leveraging their ML Platform. He used the example of a promotion campaign ranker, (eg “Try Dropbox business”) for up-selling. This is modeled as a multi-armed bandit problem, an example well in line with the meetup theme.

The biggest value of such meetups lies in the high bandwidth exchange of ideas from like-minded practitioners. In addition to some great questions after the talks, the 150+ attendees stayed well past 2 hours in the reception exchanging stories and lessons learnt solving similar problems at scale.

In the Personalization org at Netflix, we are always interested in exchanging ideas about this rapidly evolving ML space in general and the bandits and reinforcement learning space in particular. We are committed to sharing our learnings with the community and hope to discuss progress here, especially our work on Policy Evaluation and Bandit Metrics in future meetups. If you are interested in working on this exciting space, there are many open opportunities on both engineering and research endeavors.

Spot the Robot Dog Trots Into the Big, Bad World

Boston Dynamics' creation is starting to sniff out its role in the workforce: as a helpful canine that still sometimes needs you to hold its paw.

THIS AUTUMN, AFTER years of dropping view-amassing videos of Spot the robot dog fending off stick-wielding humans and opening doors for its pals, Boston Dynamics finally announced that the machine was hitting the market—for a select few early adopters, at least. BD’s people would be the first to tell you that they don’t fully know what the hypnotically agile robot will be best at. Things like patrolling job sites, sure. But Spot is so different than robots that have come before it that company execs are, in part, relying on customers to demonstrate how the machine might actually be useful.

After a few months on the job, Spot is beginning to show how it’ll fit in the workforce. BD’s researchers have kept close tabs on the 75 or so Spots now working at places like construction companies and mining outfits. (Oh, and one’s with MythBuster Adam Savage for the next year.) They’re seeing hints of a new kind of cooperation between humans and machines, and even machines and other machines. Starting today, you can even customize Spot to your liking—the software development kit is now publicly available on GitHub. The robot is not included, though.

As an example of how Spot can help, says Michael Perry, VP of business development at BD, the mining industry now employs self-driving subterranean vehicles. But if something goes awry, like a sensor malfunctions or a truck gets hung up on a rock, the operation has to shut down so a human worker can safely troubleshoot the problem. But with Spot, early adopters found, the human operator can stay at a safe distance, seeing through Spot’s eyes. “It's kind of an interesting cognitive leap to start thinking about robots mending and minding other robots,” says Perry. “It's a little far-fetched and it'll be interesting to see how successful these customers are with that application, but it was certainly something that I was really surprised by.” It’s the old robotics mantra dirty, dangerous, and dull in action: Advanced robots like Spot can tackle jobs humans can’t. (Or shouldn’t, really, unless you enjoy venturing into mines to get autonomous vehicles out of subterranean trouble.)

But there remains much that Spot can’t do. BD, for instance, hasn’t yet deployed the arm that allows the robot to open doors. That’ll come later this year, so for the time being Spot can’t fix a problem it might find with an autonomous mining truck. And the company has to confront the very magic that made it famous. A running criticism is that by viralizing slick videos of their robots pulling off amazing feats (a humanoid robot doing backflips, anyone?), they’re setting the public’s expectations too high. It takes a lot of work to get those tricks right, and what you’re not seeing are the many times the robots fail.

So BD’s researchers and execs have had to sit down with each prospective early adopter and talk through what their needs are, and what the robot can and can’t do for them—or whether they even need such an advanced platform in the first place. “We really try to work with customers and our own internal expectations to make sure that we're not tackling a sensing task that, if you just installed a bunch of Nest cameras, you'd have the same result,” says Perry.

At the same time, BD is trying to make Spot a flexible platform, so adopters can tailor the robot to fit their needs—think of it more like Android than iOS. That’s where the newly downloadable SDK comes in, allowing operators to program new behaviors. For example, if Spot is working a construction site, a project manager might want it to recognize and photograph certain objects. The SDK allows them to connect Spot’s cameras to computer vision models running either onboard the robot or in the cloud. Once they set Spot’s path by joysticking it around so it can remember the route, then they can let Spot autonomously roam the site, doing the work of a human who’d have to wander around doing the same.

What's tricky is figuring out how customizable to make such an advanced machine. Higher-level customization—getting Spot to recognize certain objects or walk certain routes—is one thing. But BD isn’t particularly interested in letting clients toy with how the robot’s joints work in concert to produce that famous agility. “We're assuming that our customers believe that we've got that problem solved,” says Perry. “It's not how it gets from point A to B. It's that it gets from point A to B while doing something that I care about.”

By making the SDK public, BD is opening up a platform to coders and roboticists of diverse specialties. “Developers will still need to become part of the early adopter program to lease the robot to execute their code,” adds Perry, “but all interested parties will now be able to view the SDK and existing early adopters can open source their own code.” BD is also announcing today that it’ll be putting on a developer conference in May in Boston.

And as those machines grow more capable, BD itself is transitioning: Longtime boss Marc Raibert is moving from CEO to chairman, with Robert Playter, the company’s COO, taking his place. “This is partly a transition away from us being a research-only shop to a company that's fielding commercial products,” says Perry. “So [Raibert] is still setting higher-level vision for pushing forward the envelope on robotics development at Boston Dynamics.”

But with increasingly advanced robots like Spot come increasingly sticky PR problems. From politicians and economists we're getting dire warnings about how the machines will replace humans in the workforce. It’s true that as robots have gotten better at sensing the world, they’ve been freed from factory floors, where they worked in isolation from humans. As they get better at navigating our world—be it as self-driving trucks or delivery robots—the concern is they’ll muscle people out of jobs.

Nevertheless, the outside world is awful for robots. Humanoid robots have nowhere near our stability to stay upright, never mind getting back up. Wheels may give them some traction, but then are confounded by stairs. The world outside an orderly factory is unpredictable and treacherous, even for as nimble a machine as Spot. Humans still need to hold Spot’s paw—it can only autonomously navigate an environment after you show it around. It still can’t manipulate objects. And you have to swap out its battery if you want continuous operation.

It’s worth noting that as robots escape the factory floor and enter the real world, they’re typically taking over certain parts of jobs—Knightscope’s security robots can patrol the perimeter of a building, but a human security guard still has to supervise it (especially if someone gets angry and assaults it). And that’s when its operators aren’t busy fishing it out of water fountains.

Which is all to say that Spot is delicately exploring a new kind of automation, in which sensitive machines work closely with humans, instead of replacing them outright. Where exactly Spot trots from here is anyone’s guess—including BD’s. But with these first few baby steps, the internet’s most famous robot dog finally gets to work.

The Right(s) Question: Can and Should Robots have Rights?

Why did you write “Robot Rights”?

We are it seems in the middle of a robot apocalypse. The machines are now everywhere and doing virtually everything. We chat with them online, we collaborate with them at work, and we rely on their capabilities to manage many aspects of our increasingly complex data-driven lives. Consequently, the “robot invasion” is not some future catastrophe that will transpire as we have imagined it in our science fiction, with a marauding army of rogue androids taking up arms against the human population. It is an already occurring event with machines of various configurations and capabilities coming to take up positions in our world through a slow but steady incursion. It looks less like Westworld and more like the fall of Rome.

As these various mechanisms take up increasingly influential positions in contemporary culture—positions where they are not necessarily just tools but interactive social entities in their own right—we will need to ask ourselves interesting and difficult questions. At what point might a robot, an algorithm or another autonomous system be held accountable for the decisions it makes or the actions it initiates? When, if ever, would it make sense to say, “It’s the robot’s fault”? Conversely, when might a robot, an intelligent artifact or other socially interactive mechanism be due some level of social standing or respect? When, in other words, would it no longer be considered nonsense to inquire about the rights of robots? I wrote the book to begin a conversation about this question and to get some traction on providing a workable solution.

Who is the intended audience?

The book is intended for those individuals and communities involved with emerging technology. This includes not only the people and organizations involved in designing, building and marketing AI applications and robots—things like self-driving cars, deep-learning algorithms, social robots, etc.—but also those concerned with the social impact and consequences of releasing these increasingly autonomous machines into the world. We are in the process of re-engineering social reality as we know it. The book speaks to the opportunities, challenges and even fears that we now face and are going to be facing in the next several decades.

What are you hoping the book will accomplish?

Simply put, a reality check. When it comes to robots and the question of their social status, there is a spectrum of responses bounded by two extreme positions. On one end of the spectrum, there are those individuals who find the very question of robot rights to be simply preposterous. On the other end, there are those who think robots and other forms of AI—especially human-like androids—will need to have rights on par with or substantially similar to what is extended to human beings. Both sides get it wrong. The one side is too conservative, while the other is too liberal. The book simply tries to formulate a more realistic and practical way to address the opportunities and challenges of robots and AI in our world.

How do you respond to people who think the notion of robot rights is silly?

This is a reasonable and intuitive response, mainly because it is rooted in some rather old and well-established assumptions. Currently, in both law and ethics, we divide up the world into two kinds of entities—persons and things. We have moral and legal obligations to other persons; they can be benefitted or harmed by our decisions and actions. But there are no such obligations to things; they are property that can be used, misused and even abused as we see fit. A robot, like any other manufactured artifact or piece of technology, would appear to be just another thing. Case closed, period.

But not so fast. “Person” is a socially constructed moral and legal category that applies to a wide range of different kinds of entities and not just human individuals. In fact, we already live in a world overrun by artificial entities that have the rights (and the responsibilities) of a person—the limited liability corporation. IBM, Amazon, Microsoft and McDonalds are all legal persons with rights similar to what you and I are granted under the law—the right to free speech, the right to defend ourselves from accusations, the right to religion, etc. If IBM is a legally recognized person with rights, it is possible that Watson—IBM’s AI—might also qualify for the same kind of status and protections. This is precisely what the book seeks to examine and investigate—whether this is possible, necessary and/or expedient.

Do you ultimately make a case for or against establishing some type of rights for robots?

The book is not a manifesto that ends with a call to arms for robots to have rights. What we need most at this point in time is a realistic assessment of the social opportunities and challenges that increasingly autonomous systems—like AI and robots—introduce into our world. These artifacts already occupy a weird position that strains against existing categorical distinctions. They are not persons like you and I, even if we are designing and developing artifacts like Sophia, the Hanson Robotics android that emulates both the appearance and behavior of human beings and that was recently granted honorary citizenship by the Kingdom of Saudi Arabia. At the same time, these things are not just things that we feel free to use and abuse without further consideration. The soldiers who work with EOD (explosive ordinance disposal) robots in the field name their robots, award them medals and promotions for valor, and even risk their own lives to protect the robot from harm. Something is happening that is slowly but surely changing the rules of the game. The book is an attempt to diagnose the opportunities and challenges that we now confront in the face—or the faceplate—of the robot.

People are more glued to their technology today than ever before. How is that affecting us?

The intimacy that we now share with our technology is altering both us and our tools. The book is as much about how we deal with the changing nature of technology as it is about how technology is involved in changing and evolving us. No matter how simple or sophisticated it is, technology is a mirror in which we can see a reflection of ourselves. The book, therefore, develops its argument along two different vectors. On the one hand, it attempts to use the traditions of moral philosophy and legal theory to investigate the social meaning and status of increasingly autonomous and sociable technology. On the other hand, it uses the confrontation with this technology as a way to critically examine some of our most deep-seated assumptions about moral conduct and legal status.

At the end of your book, you propose a different way to conceptualize the social situation of robots. Can you elaborate?

I introduce and prototype a new way to conceptualize the social position and status of technological artefacts. This way of thinking has been called “the relational turn,” which is a term I borrow from my friend and colleague Mark Coeckelbergh at the University of Vienna. Simply put, the relational turn flips the script on the usual way of deciding who (or what) is a legitimate moral and legal subject. Typically we make these decisions based on what something is—whether an entity is conscious, intelligent or can feel pain. In this transaction, what something is determines how it comes to be treated. Many of our moral and legal systems proceed and are organized in this fashion.

The relational turn puts the how before the what. As we encounter and interact with others—whether they are humans, animals, the natural environment or robots—these other entities are first and foremost situated in relationship to us. Consequently, the question of social and moral status does not necessarily depend on what the other is but on how she/he/it stands before us and how we decide, in “the face of the other,” to respond. Importantly, this alternative is not offered as the ultimate solution or as a kind of “moral theory of everything.” It is put forward as a kind of counterweight to provide new ways of thinking about our moral and legal obligations to others.