ALBERT

Description: Since its inception in 1997, RoboCup has developed into a truly unique and long-standing research community advancing robotics and artificial intelligence through various challenges, benchmarks, and test fields. The main purposes of this article are to evaluate the research and development achievements so far and to identify new challenges and related new research issues. Unlike other robot competitions and research conferences, RoboCup eliminates the boundaries between pure research activities and the development of full system designs with hardware and software implementations at a site open to the public. It also creates specific scientific and technological research and development challenges to be addressed. In this article, we provide an overview of RoboCup, including its league structure and related research issues. We also review recent studies across several research categories to show how participants (called RoboCuppers) address the research and development challenges before, during, and after the annual competitions. Among the diversity of research issues, we highlight two unique aspects of the challenges: the platform design of the robots and the game evaluations. Both of these aspects contribute to solving the research and development challenges of RoboCup and verifying the results from a common perspective (i.e., a more objective view). Finally, we provide concluding remarks and discuss future research directions.

Description: This article reviews the current state of the art in the development of modular reconfigurable robot (MRR) systems and suggests promising future research directions. A wide variety of MRR systems have been presented to date, and these robots promise to be versatile, robust, and low cost compared with other conventional robot systems. MRR systems thus have the potential to outperform traditional systems with a fixed morphology when carrying out tasks that require a high level of flexibility. We begin by introducing the taxonomy of MRRs based on their hardware architecture. We then examine recent progress in the hardware and the software technologies for MRRs, along with remaining technical issues. We conclude with a discussion of open challenges and future research directions.

Description: Advances in wired and wireless technology have necessitated the development of theory, models, and tools to cope with the new challenges posed by large-scale control and optimization problems over networks. The classical optimization methodology works under the premise that all problem data are available to a central entity (a computing agent or node). However, this premise does not apply to large networked systems, where each agent (node) in the network typically has access only to its private local information and has only a local view of the network structure. This review surveys the development of such distributed computational models for time-varying networks. To emphasize the role of the network structure in these approaches, we focus on a simple direct primal (sub)gradient method, but we also provide an overview of other distributed methods for optimization in networks. Applications of the distributed optimization framework to the control of power systems, least squares solutions to linear equations, and model predictive control are also presented.

Description: The field of socially assistive robotics (SAR) aims to supplement the efforts of clinicians, therapists, educators, and caregivers through individualized, socially mediated interventions with robots. SAR is faced with the interdisciplinary challenge to balance sensitive domain needs with current technical limitations. Many researchers in SAR and the broader human–robot interaction community overcome technical barriers by using a Wizard of Oz approach, or teleoperation of the robot or aspects of the interaction. Although Wizard of Oz is a well-established practice, it becomes intractable in critical SAR domains that require long-term, situated support, such as aging in place and special needs education. In this article, we define a set of autonomy-centric design properties for SAR interventions based on concepts from artificial intelligence and robotics. These properties structure a systematic review of the last decade of autonomous SAR research. From the review, we draw and discuss common computational methods, engineering practices, and design patterns that enable autonomy in SAR.

Description: Advanced measurement techniques and high-performance computing have made large data sets available for a range of turbulent flows in engineering applications. Drawing on this abundance of data, dynamical models that reproduce structural and statistical features of turbulent flows enable effective model-based flow control strategies. This review describes a framework for completing second-order statistics of turbulent flows using models based on the Navier–Stokes equations linearized around the turbulent mean velocity. Dynamical couplings between states of the linearized model dictate structural constraints on the statistics of flow fluctuations. Colored-in-time stochastic forcing that drives the linearized model is then sought to account for and reconcile dynamics with available data (that is, partially known statistics). The number of dynamical degrees of freedom that are directly affected by stochastic excitation is minimized as a measure of model parsimony. The spectral content of the resulting colored-in-time stochastic contribution can alternatively arise from a low-rank structural perturbation of the linearized dynamical generator, pointing to suitable dynamical corrections that may account for the absence of the nonlinear interactions in the linearized model.

Description: The areas of mechanics and control theory have a rich and productive history of interaction with the broad mathematical subject of differential geometry. This article provides an overview of these sorts of interplay in the areas of Riemannian and affine differential geometry and the geometry of vector distributions. It emphasizes areas where differential geometric methods have played a crucial role in solving problems whose solutions are difficult to achieve without access to these methods. It also emphasizes a concise and elegant presentation of the approach, rather than a detailed and concrete presentation. The results overviewed, while forming a coherent and elegant body of work, are limited in scope. The review closes with a discussion of why the approach is limited and a brief consideration of issues that must be resolved before the results of the type presented here can be extended.

Description: This article provides an introduction and overview of the inerter concept and device. Careful attention is given to the distinction between the inerter as an ideal modeling element and devices that approximate the ideal behavior. The background is given to the formal definition of the inerter as a mechanical one-port with terminal forces proportional to the relative acceleration between them. Four major methods of construction are described and modeled. The discussion focuses particularly on the notion of terminals, the distinction between a device and an effect, sign reversals, back driving in geared systems, the conceptual aspects of the modeling step for inerter embodiments, and the problem of reverse engineering to discover a purpose. The article includes an analysis and discussion of the rotational inerter, a brief review of the ideas of passive network synthesis that led to the inerter concept, and an analysis and discussion of several examples of integrated mechanical devices. It concludes with an imaginary dialogue between the author and an interlocutor on the understanding and purpose of the inerter.

Description: The increasingly tight coupling between humans and system operations in domains ranging from intelligent infrastructure to e-commerce has led to a challenging new class of problems founded on a well-established area of research: incentive design. There is a clear need for a new tool kit for designing mechanisms that help coordinate self-interested parties while avoiding unexpected outcomes in the face of information asymmetries, exogenous uncertainties from dynamic environments, and resource constraints. This article provides a perspective on the current state of the art in incentive design from three core communities—economics, control theory, and machine learning—and highlights interesting avenues for future research at the interface of these domains.

Description: Many modern dynamical systems, such as smart grids and traffic networks, rely on user data for efficient operation. These data often contain sensitive information that the participating users do not wish to reveal to the public. One major challenge is to protect the privacy of participating users when utilizing user data. Over the past decade, differential privacy has emerged as a mathematically rigorous approach that provides strong privacy guarantees. In particular, differential privacy has several useful properties, including resistance to both postprocessing and the use of side information by adversaries. Although differential privacy was first proposed for static-database applications, this review focuses on its use in the context of control systems, in which the data under processing often take the form of data streams. Through two major applications—filtering and optimization algorithms—we illustrate the use of mathematical tools from control and optimization to convert a nonprivate algorithm to its private counterpart. These tools also enable us to quantify the trade-offs between privacy and system performance.

Description: This article provides a concise summary of the basic ideas and concepts in port-Hamiltonian systems theory and its use in analysis and control of complex multiphysics systems. It gives special attention to new and unexplored research directions and relations with other mathematical frameworks. Emergent control paradigms and open problems are indicated, including the relation with thermodynamics and the question of uniting the energy-processing view of control, as emphasized by port-Hamiltonian systems theory, with a complementary information-processing viewpoint.

Description: In the context of robotics and automation, learning from demonstration (LfD) is the paradigm in which robots acquire new skills by learning to imitate an expert. The choice of LfD over other robot learning methods is compelling when ideal behavior can be neither easily scripted (as is done in traditional robot programming) nor easily defined as an optimization problem, but can be demonstrated. While there have been multiple surveys of this field in the past, there is a need for a new one given the considerable growth in the number of publications in recent years. This review aims to provide an overview of the collection of machine-learning methods used to enable a robot to learn from and imitate a teacher. We focus on recent advancements in the field and present an updated taxonomy and characterization of existing methods. We also discuss mature and emerging application areas for LfD and highlight the significant challenges that remain to be overcome both in theory and in practice.

Description: This article reviews recent work on surgical robots that have been used or tested in vivo, focusing on aspects related to human–robot interaction. We present the general design requirements that should be considered when developing such robots, including the clinical requirements and the technologies needed to satisfy them. We also discuss the human aspects related to the design of these robots, considering the challenges facing surgeons when using robots in the operating room, and the safety issues of such systems. We then survey recent work in seven different surgical settings: urology and gynecology, orthopedic surgery, cardiac surgery, head and neck surgery, neurosurgery, radiotherapy, and bronchoscopy. We conclude with the open problems and recommendations on how to move forward in this research area.

Description: Recent successes in the field of machine learning, as well as the availability of increased sensing and computational capabilities in modern control systems, have led to a growing interest in learning and data-driven control techniques. Model predictive control (MPC), as the prime methodology for constrained control, offers a significant opportunity to exploit the abundance of data in a reliable manner, particularly while taking safety constraints into account. This review aims at summarizing and categorizing previous research on learning-based MPC, i.e., the integration or combination of MPC with learning methods, for which we consider three main categories. Most of the research addresses learning for automatic improvement of the prediction model from recorded data. There is, however, also an increasing interest in techniques to infer the parameterization of the MPC controller, i.e., the cost and constraints, that lead to the best closed-loop performance. Finally, we discuss concepts that leverage MPC to augment learning-based controllers with constraint satisfaction properties.

Description: In this review, we discuss routing algorithms for the dynamic traffic assignment (DTA) problem that assigns traffic flow in a given road network as realistically as possible. We present a new class of so-called routing operators that route traffic flow at intersections based on either real-time information about the status of the network or historical data. These routing operators thus cover the distribution of traffic flow at all possible intersections. To model traffic flow on the links, we use a well-known macroscopic ordinary delay differential equation. We prove the existence and uniqueness of the solutions of the resulting DTA for a broad class of routing operators. This new routing approach is required and justified by the increased usage of real-time information on the network provided by map services, changing the laws of routing significantly. Because these map and routing services have a huge impact on the infrastructure of cities, a more precise mathematical description of the emerging new traffic patterns and effects becomes crucial for understanding and improving road and city conditions.

Description: A milestone in vertebrate evolution, the transition from water to land, owes its success to the development of a sprawling body plan that enabled an amphibious lifestyle. The body, originally adapted for swimming, evolved to benefit from limbs that enhanced its locomotion capabilities on submerged and dry ground. The first terrestrial animals used sprawling locomotion, a type of legged locomotion in which limbs extend laterally from the body (as opposed to erect locomotion, in which limbs extend vertically below the body). This type of locomotion—exhibited, for instance, by salamanders, lizards, and crocodiles—has been studied in a variety of fields, including neuroscience, biomechanics, evolution, and paleontology. Robotics can benefit from these studies to design amphibious robots capable of swimming and walking, with interesting applications in field robotics, in particular for search and rescue, inspection, and environmental monitoring. In return, robotics can provide useful scientific tools to test hypotheses in neuroscience, biomechanics, and paleontology. For instance, robots have been used to test hypotheses about the organization of neural circuits that can switch between swimming and walking under the control of simple modulation signals, as well as to identify the most likely gaits of extinct sprawling animals. Here, I review different aspects of amphibious and sprawling locomotion, namely gait characteristics, neurobiology, numerical models, and sprawling robots, and discuss fruitful interactions between robotics and other scientific fields.

Description: Micro- and nanorobots can perform a number of tasks at small scales, such as minimally invasive diagnostics, targeted drug delivery, and localized surgery. During the past decade, the field has been transformed in many ways, one of the most significant being a transition from hard and rigid micro- and nanostructures to soft and flexible architectures. Inspired by the dynamics of flexible microorganisms, researchers have focused on developing miniaturized soft components such as actuators, sensors, hinges, joints, and reservoirs to create soft micro- and nanoswimmers. The use of organic structures such as polymers and supramolecular ensembles as functional components has brought more complex features to these devices, such as advanced locomotion strategies and stimulus-triggered shape transformations, as well as other capabilities. A variety of microorganisms and contractile mammalian cells have also been utilized as microengines and integrated with functional synthetic materials, producing bending or deformation of the functional materials to initiate motion. In this review, we consider several types of soft micro- and nanorobots in terms of their architecture and design, and we describe their locomotion mechanisms and applications.

Description: Emerging paradigms furthering the reach of medical technology into human anatomy present unique modeling, control, and sensing problems. This review provides a brief history of medical robotics, leading to the current trend of minimally invasive intervention and diagnostics in confined spaces. We discuss robotics for natural orifice and single-port access surgery, capsule and magnetically actuated robotics, and microrobotics, with the aim of elucidating the state of the art. We also discuss works on modeling, sensing, and control of mechanical architectures of robots for natural orifice and single-port access surgery, followed by works on magnetic actuation, sensing, and localization for capsule robotics and microrobotics. Finally, we present challenges and open problems in each of these areas.

Description: This article surveys reinforcement learning from the perspective of optimization and control, with a focus on continuous control applications. It reviews the general formulation, terminology, and typical experimental implementations of reinforcement learning as well as competing solution paradigms. In order to compare the relative merits of various techniques, it presents a case study of the linear quadratic regulator (LQR) with unknown dynamics, perhaps the simplest and best-studied problem in optimal control. It also describes how merging techniques from learning theory and control can provide nonasymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. The article concludes with a discussion of some of the challenges in designing learning systems that safely and reliably interact with complex and uncertain environments and how tools from reinforcement learning and control might be combined to approach these challenges.

Description: In control theory, complicated dynamics such as systems of (nonlinear) differential equations are controlled mostly to achieve stability. This fundamental property, which can be with respect to a desired operating point or a prescribed trajectory, is often linked with optimality, which requires minimizing a certain cost along the trajectories of a stable system. In formal verification (model checking), simple systems, such as finite-state transition graphs that model computer programs or digital circuits, are checked against rich specifications given as formulas of temporal logics. The formal synthesis problem, in which the goal is to synthesize or control a finite system from a temporal logic specification, has recently received increased interest. In this article, we review some recent results on the connection between optimal control and formal synthesis. Specifically, we focus on the following problem: Given a cost and a correctness temporal logic specification for a dynamical system, generate an optimal control strategy that satisfies the specification. We first provide a short overview of automata-based methods, in which the dynamics of the system are mapped to a finite abstraction that is then controlled using an automaton corresponding to the specification. We then provide a detailed overview of a class of methods that rely on mapping the specification and the dynamics to constraints of an optimization problem. We discuss advantages and limitations of these two types of approaches and suggest directions for future research.

Description: The commercialization of lithium-ion batteries enabled the widespread use of portable consumer electronics and serious efforts to electrify trans-portation. Managing the potent brew of lithium-ion batteries in the large quantities necessary for vehicle propulsion is still challenging. From space applications a billion miles from Earth to the daily commute of a hybrid electric automobile, these batteries require sophisticated battery management systems based on accurate estimation of battery internal states. This system is the brain of the battery and is responsible for estimating the state of charge, state of health, state of power, and temperature. The state estimation relies on accurate prediction of complex electrochemical, thermal, and mechanical phenomena, which increases the importance of model and parameter accuracy. Moreover, as the batteries age, how should the parameters of the model change to accurately represent the performance, and how can we leverage the limited sensor information from the measured terminal voltage and sparse surface temperatures available in a battery system? With a frugal sensor set, what is the optimal sensor placement? This article reviews estimation techniques and error bounds regarding sensor noise and modeling errors, and concludes with an outlook on the research that will be necessary to enable fast charging, repurposing of batteries for grid energy storage, degradation prediction, and fault detection.

Description: In this review, we provide an overview of emerging trends and challenges in the field of intelligent and autonomous, or self-driving, vehicles. Recent advances in the field of perception, planning, and decision-making for autonomous vehicles have led to great improvements in functional capabilities, with several prototypes already driving on our roads and streets. Yet challenges remain regarding guaranteed performance and safety under all driving circumstances. For instance, planning methods that provide safe and system-compliant performance in complex, cluttered environments while modeling the uncertain interaction with other traffic participants are required. Furthermore, new paradigms, such as interactive planning and end-to-end learning, open up questions regarding safety and reliability that need to be addressed. In this survey, we emphasize recent approaches for integrated perception and planning and for behavior-aware planning, many of which rely on machine learning. This raises the question of verification and safety, which we also touch upon. Finally, we discuss the state of the art and remaining challenges for managing fleets of autonomous vehicles.

Description: Within the field of human rehabilitation, robotic machines are used both to rehabilitate the body and to perform functional tasks. Robotics autonomy that would enable perception of the external world and reasoning about high-level control decisions, however, is seldom present in these machines. For functional tasks in particular, autonomy could help to decrease the operational burden on the human and perhaps even increase access, and this potential only grows as human motor impairments become more severe. There are, however, serious and often subtle considerations for incorporating clinically feasible robotics autonomy into rehabilitation robots and machines. Today, the fields of robotics autonomy and rehabilitation robotics are largely separate, and the topic of this article is at the intersection of these fields: the incorporation of clinically feasible autonomy solutions into rehabilitation robots and the opportunities for autonomy within the rehabilitation domain.

Description: This article surveys manipulation, including both biological and robotic manipulation. Biology inspires robotics and demonstrates aspects of manipulation that are far in the future of robotics. Robotics develops concepts and principles that become evident only in the creative process. Robotics also provides a test of our understanding. As Richard Feynman put it: “What I cannot create, I do not understand.”

Description: Rationality principles such as optimal feedback control and Bayesian inference underpin a probabilistic framework that has accounted for a range of empirical phenomena in biological sensorimotor control. To facilitate the optimization of flexible and robust behaviors consistent with these theories, the ability to construct internal models of the motor system and environmental dynamics can be crucial. In the context of this theoretic formalism, we review the computational roles played by such internal models and the neural and behavioral evidence for their implementation in the brain.

Description: Robot control for tasks such as moving around obstacles or grasping objects has advanced significantly in the last few decades. However, controlling robots to perform complex tasks is still accomplished largely by highly trained programmers in a manual, time-consuming, and error-prone process that is typically validated only through extensive testing. Formal methods are mathematical techniques for reasoning about systems, their requirements, and their guarantees. Formal synthesis for robotics refers to frameworks for specifying tasks in a mathematically precise language and automatically transforming these specifications into correct-by-construction robot controllers or into a proof that the task cannot be done. Synthesis allows users to reason about the task specification rather than its implementation, reduces implementation error, and provides behavioral guarantees for the resulting controller. This article reviews the current state of formal synthesis for robotics and surveys the landscape of abstractions, specifications, and synthesis algorithms that enable it.

Description: The Kalman filter—or, more precisely, the extended Kalman filter (EKF)—is a fundamental engineering tool that is pervasively used in control and robotics and for various estimation tasks in autonomous systems. The recently developed field of invariant extended Kalman filtering uses the geometric structure of the state space and the dynamics to improve the EKF, notably in terms of mathematical guarantees. The methodology essentially applies in the fields of localization, navigation, and simultaneous localization and mapping (SLAM). Although it was created only recently, its remarkable robustness properties have already motivated a real industrial implementation in the aerospace field. This review aims to provide an accessible introduction to the methodology of invariant Kalman filtering and to allow readers to gain insight into the relevance of the method as well as its important differences with the conventional EKF. This should be of interest to readers intrigued by the practical application of mathematical theories and those interested in finding robust, simple-to-implement filters for localization, navigation, and SLAM, notably for autonomous vehicle guidance.

Description: This article begins with an introduction to the modeling of discrete event systems, a class of dynamical systems with discrete states and event-driven dynamics. It then focuses on logical discrete event models, primarily automata, and reviews observation and control problems and their solution methodologies. Specifically, it discusses diagnosability and opacity in the context of partially observed discrete event systems. It then discusses supervisory control for both fully and partially observed systems. The emphasis is on presenting fundamental results first, followed by a discussion of current research directions.

Description: Feedback is a key element of regulation, as it shapes the sensitivity of a process to its environment. Positive feedback upregulates, and negative feedback downregulates. Many regulatory processes involve a mixture of both, whether in nature or in engineering. This article revisits the mixed-feedback paradigm, with the aim of investigating control across scales. We propose that mixed feedback regulates excitability and that excitability plays a central role in multiscale neuronal signaling. We analyze this role in a multiscale network architecture inspired by neurophysiology. The nodal behavior defines a mesoscale that connects actuation at the microscale to regulation at the macroscale. We show that mixed-feedback nodal control provides regulatory principles at the network scale, with a nodal resolution. In this sense, the mixed-feedback paradigm is a control principle across scales.

Description: Planning and executing object manipulation requires integrating multiple sensory and motor channels while acting under uncertainty and complying with task constraints. As the modern environment is tuned for human hands, designing robotic systems with similar manipulative capabilities is crucial. Research on robotic object manipulation is divided into smaller communities interested in, e.g., motion planning, grasp planning, sensorimotor learning, and tool use. However, few attempts have been made to combine these areas into holistic systems. In this review, we aim to unify the underlying mechanics of grasping and in-hand manipulation by focusing on the temporal aspects of manipulation, including visual perception, grasp planning and execution, and goal-directed manipulation. Inspired by human manipulation, we envision that an emphasis on the temporal integration of these processes opens the way for human-like object use by robots.

Description: Historically, scalability has been a major challenge for the successful application of semidefinite programming in fields such as machine learning, control, and robotics. In this article, we survey recent approaches to this challenge, including those that exploit structure (e.g., sparsity and symmetry) in a problem, those that produce low-rank approximate solutions to semidefinite programs, those that use more scalable algorithms that rely on augmented Lagrangian techniques and the alternating-direction method of multipliers, and those that trade off scalability with conservatism (e.g., by approximating semidefinite programs with linear and second-order cone programs). For each class of approaches, we provide a high-level exposition, an entry point to the corresponding literature, and examples drawn from machine learning, control, or robotics. We also present a list of software packages that implement many of the techniques discussed in the review. Our hope is that this article will serve as a gateway to the rich and exciting literature on scalable semidefinite programming for both theorists and practitioners.

Description: Cyber-physical systems, in which computation and networking technologies interact with physical systems, have made great strides into manufacturing systems. From the early days, when electromechanical relays were used to automate conveyors and machines, through the introduction of programmable logic controllers and computer numerical control, computing and networking have become pervasive in manufacturing systems. By increasing the amount of automation at multiple levels within a factory and across the enterprise, cyber-physical manufacturing systems enable higher productivity and higher quality as well as lower costs.

Description: As robots move beyond manufacturing applications to less predictable environments, they can increasingly benefit, as animals do, from integrating sensing and control with the passive properties provided by particular combinations and arrangements of materials and mechanisms. This realization is partly responsible for the recent proliferation of soft and bioinspired robots. Tuned materials and mechanisms can provide several kinds of benefits, including energy storage and recovery, increased physical robustness, and decreased response time to sudden events. In addition, they may offer passive open-loop behaviors and responses to external changes in loading or environmental conditions. Collectively, these properties can also increase the stability of a robot as it interacts with the environment and allow the closed-loop controller to reduce the apparent degrees of freedom subject to control. The design of appropriate materials and mechanisms remains a challenging problem; bioinspiration, genetic algorithms, and numerical shape and materials optimization are all applicable. New multimaterial fabrication processes are also steadily increasing the range and magnitude of passive properties available for intrinsically responsive robots.

Description: The goal of this article is to provide a thorough introduction to the state of the art in magnetic methods for remote-manipulation and wireless-actuation tasks in robotics. The article synthesizes prior works using a unified notation, enabling straightforward application in robotics. It begins with a discussion of the magnetic fields generated by magnetic materials and electromagnets, how magnetic materials become magnetized in an applied field, and the forces and torques generated on magnetic objects. It then describes systems used to generate and control applied magnetic fields, including both electromagnetic and permanent-magnet systems. Finally, it surveys work from a variety of robotic application areas in which researchers have utilized magnetic methods, including microrobotics, medical robotics, haptics, and aerospace.

Description: Research toward small, autonomous, and mobile robots is inspired by both the insects we see around us and numerous applications, from inspection of jet engines and civil infrastructure to medical procedures. When comparing existing robots at small scales with their biological counterparts, the capability for autonomous operation is a glaring contrast. This review describes the state of the art in robotics at sub-gram scales along with the progress toward autonomy in power, mobility, and control at these small sizes. Metrics are described to both quantify the performance of existing sub-gram robots (e.g., speed and cost of transport) and define a more quantitative path toward autonomy (e.g., mass-specific run time and traversal probability). These metrics from existing robots are also compared with those of insects to identify significant performance gaps and highlight important areas for future study.

Description: This article reviews the technology behind creating artificial touch sensations and the relevant aspects of human touch. We focus on the design and control of haptic devices and discuss the best practices for generating distinct and effective touch sensations. Artificial haptic sensations can present information to users, help them complete a task, augment or replace the other senses, and add immersiveness and realism to virtual interactions. We examine these applications in the context of different haptic feedback modalities and the forms that haptic devices can take. We discuss the prior work, limitations, and design considerations of each feedback modality and individual haptic technology. We also address the need to consider the neuroscience and perception behind the human sense of touch in the design and control of haptic devices.

Description: Robotic micromanipulation is a relatively young field. However, after three decades of development and evolution, the fundamental physics; techniques for sensing, actuation, and control; tool sets and systems; and, more importantly, a research community are now in place. This article reviews the fundamentals of robotic micromanipulation, including how micromanipulators and end effectors are actuated and controlled, how remote physical fields are utilized for micromanipulation, how visual servoing is implemented under an optical microscope, how force is sensed and controlled at the micro- and nanonewton levels, and the similarities and differences between robotic manipulation at the micro- and macroscales. We also review representative milestones over the past three decades and discuss potential future trends of this field.

Description: A key goal of contemporary agriculture is to dramatically increase production of food, feed, fiber, and biofuel products in a sustainable fashion, facing the pressure of diminishing farm labor supply. Agricultural robots can accelerate plant breeding and advance data-driven precision farming with significantly reduced labor inputs by providing task-appropriate sensing and actuation at fine spatiotemporal resolutions. This article highlights the distinctive challenges imposed on ground robots by agricultural environments, which are characterized by wide variations in environmental conditions, diversity and complexity of plant canopy structures, and intraspecies biological variation of physical and chemical characteristics and responses to the environment. Existing approaches to address these challenges are presented, along with their limitations; possible future directions are also discussed. Two key observations are that biology (breeding) and horticultural practices can reduce variabilities at the source and that publicly available benchmark data sets are needed to increase perception robustness and performance despite variability.

Description: Biohybrid microrobots, composed of a living organism integrated with an artificial carrier, offer great advantages for the miniaturization of devices with onboard actuation, sensing, and control functionalities and can perform multiple tasks, including manipulation, cargo delivery, and targeting, at nano- and microscales. Over the past decade, various microorganisms and artificial carriers have been integrated to develop unique biohybrid microrobots that can swim or crawl inside the body, in order to overcome the challenges encountered by the current cargo delivery systems. Here, we first focus on the locomotion mechanisms of microorganisms at the microscale, crucial criteria for the selection of biohybrid microrobot components, and the integration of the selected artificial and biological components using various physical and chemical techniques. We then critically review biohybrid microrobots that have been designed and used to perform specific tasks in vivo. Finally, we discuss key challenges, including fabrication efficiency, swarm manipulation, in vivo imaging, and immunogenicity, that should be overcome before biohybrid microrobots transition to clinical use.

Description: Game theory is the study of decision problems in which there are multiple decision makers and the quality of a decision maker's choice depends on both that choice and the choices of others. While game theory has been studied predominantly as a modeling paradigm in the mathematical social sciences, there is a strong connection to control systems in that a controller can be viewed as a decision-making entity. Accordingly, game theory is relevant in settings with multiple interacting controllers. This article presents an introduction to game theory, followed by a sampling of results in three specific control theory topics where game theory has played a significant role: ( a) zero-sum games, in which the two competing players are a controller and an adversarial environment; ( b) team games, in which several controllers pursue a common goal but have access to different information; and ( c) distributed control, in which both a game and online adaptive rules are designed to enable distributed interacting subsystems to achieve a collective objective.

Description: This review surveys the current state of the art in the development of unmanned aerial vehicles, focusing on algorithms for quadrotors. Tremendous progress has been made across both industry and academia, and full vehicle autonomy is now well within reach. We begin by presenting recent successes in control, estimation, and trajectory planning that have enabled agile, high-speed flight using low-cost onboard sensors. We then examine new research trends in learning and multirobot systems and conclude with a discussion of open challenges and directions for future research.

Description: This article reports on the state of the art of artificial hands, discussing some of the field's most important trends and suggesting directions for future research. We review and group the most important application domains of robotic hands, extracting the set of requirements that ultimately led to the use of simplified actuation schemes and soft materials and structures—two themes that clearly emerge from our examination of developments over the past century. We provide a comprehensive analysis of novel technologies for the design of joints, transmissions, and actuators that enabled these trends. We conclude by discussing some important new perspectives generated by simpler and softer hands and their interaction with other aspects of hand design and robotics in general.

Description: Autonomous systems are becoming pervasive in everyday life, and many of these systems are complex and safety-critical. Formal verification is important for providing performance and safety guarantees for these systems. In particular, Hamilton–Jacobi (HJ) reachability is a formal verification tool for nonlinear and hybrid systems; however, it is computationally intractable for analyzing complex systems, and computational burden is in general a difficult challenge in formal verification. In this review, we begin by briefly presenting background on reachability analysis with an emphasis on the HJ formulation. We then present recent work showing how high-dimensional reachability verification can be made more tractable by focusing on two areas of development: system decomposition for general nonlinear systems, and traffic protocols for unmanned airspace management. By tackling the curse of dimensionality, tractable verification of practical systems is becoming a reality, paving the way for more pervasive and safer automation.

Description: Robots with many degrees of freedom (e.g., humanoid robots and mobile manipulators) have increasingly been employed to accomplish realistic tasks in domains such as disaster relief, spacecraft logistics, and home caretaking. Finding feasible motions for these robots autonomously is essential for their operation. Sampling-based motion planning algorithms are effective for these high-dimensional systems; however, incorporating task constraints (e.g., keeping a cup level or writing on a board) into the planning process introduces significant challenges. This survey describes the families of methods for sampling-based planning with constraints and places them on a spectrum delineated by their complexity. Constrained sampling-based methods are based on two core primitive operations: ( a) sampling constraint-satisfying configurations and ( b) generating constraint-satisfying continuous motion. Although this article presents the basics of sampling-based planning for contextual background, it focuses on the representation of constraints and sampling-based planners that incorporate constraints.

Description: The past decade has witnessed the rise of an exciting new field of engineering: synthetic biology. Synthetic biology is the application of engineering principles to the fundamental components of biology with the aim of programming cells with novel functionalities for utilization in the health, environment, and energy industries. Since its beginnings in the early 2000s, control design principles have been used in synthetic biology to design dynamics, mitigate the effects of uncertainty, and aid modular and layered design. In this review, we provide a basic introduction to synthetic biology, its applications, and its foundations and then describe in more detail how control design approaches have permeated the field since its inception. We conclude with a discussion of pressing challenges in this field that will require new control theory, with the hope of attracting researchers in the control theory community to this exciting engineering area.

Description: In autonomous systems, the ability to make forecasts and cope with uncertain predictions is synonymous with intelligence. Model predictive control (MPC) is an established control methodology that systematically uses forecasts to compute real-time optimal control decisions. In MPC, at each time step an optimization problem is solved over a moving horizon. The objective is to find a control policy that minimizes a predicted performance index while satisfying operating constraints. Uncertainty in MPC is handled by optimizing over multiple uncertain forecasts. In this case, performance index and operating constraints take the form of functions defined over a probability space, and the resulting technique is called stochastic MPC. Our research over the past 10 years has focused on predictive control design methods that systematically handle uncertain forecasts in autonomous and semiautonomous systems. In the first part of this article, we present an overview of the approach we use, its main advantages, and its challenges. In the second part, we present our most recent results on data-driven predictive control. We show how to use data to efficiently formulate stochastic MPC problems and autonomously improve performance in repetitive tasks. The proposed framework is able to handle a large set of predicted scenarios in real time and learn from historical data.

Description: This article surveys the use of natural language in robotics from a robotics point of view. To use human language, robots must map words to aspects of the physical world, mediated by the robot's sensors and actuators. This problem differs from other natural language processing domains due to the need to ground the language to noisy percepts and physical actions. Here, we describe central aspects of language use by robots, including understanding natural language requests, using language to drive learning about the physical world, and engaging in collaborative dialogue with a human partner. We describe common approaches, roughly divided into learning methods, logic-based methods, and methods that focus on questions of human–robot interaction. Finally, we describe several application domains for language-using robots.

Description: The control of mobile sensor networks uses sensor measurements to update a model of an unknown or estimated process, which in turn guides the collection of subsequent measurements—a feedback control framework called adaptive sampling. Applications for adaptive sampling exist in a wide range of settings, especially for unmanned or autonomous vehicles that can be deployed cheaply and in cooperative groups. The dynamics of mobile sensor platforms are often simplified to planar self-propelled particles subject to the ambient flow of the surrounding fluid. Sensor measurements are assimilated into continuous or discrete models of the process of interest, which in general can vary in space and time. The variability of the estimated process is one metric to score future candidate sampling trajectories, along with information- and uncertainty-based metrics. Sampling tasks are allocated to the network using centralized or decentralized optimization, in order to avoid redundant measurements and observational gaps.

Description: The concept of an artificial corporeal machine that can reproduce has attracted the attention of researchers from various fields over the past century. Some have approached the topic with a desire to understand biological life and develop artificial versions; others have examined it as a potentially practical way to use material resources from the moon and Mars to bootstrap the exploration and colonization of the solar system. This review considers both bodies of literature, with an emphasis on the underlying principles required to make self-replicating robotic systems from raw materials a reality. We then illustrate these principles with machines from our laboratory and others and discuss how advances in new manufacturing processes such as 3-D printing can have a synergistic effect in advancing the development of such systems.

Description: We review selected results related to the robustness of networked systems in finite and asymptotically large size regimes in static and dynamical settings. In the static setting, within the framework of flow over finite networks, we discuss the effect of physical constraints on robustness to loss in link capacities. In the dynamical setting, we review several settings in which small-gain-type analysis provides tight robustness guarantees for linear dynamics over finite networks toward worst-case and stochastic disturbances. We discuss network flow dynamic settings where nonlinear techniques facilitate understanding the effect, on robustness, of constraints on capacity and information, substituting information with control action, and cascading failure. We also contrast cascading failure with a representative contagion model. For asymptotically large networks, we discuss the role of network properties in connecting microscopic shocks to emergent macroscopic fluctuations under linear dynamics as well as for economic networks at equilibrium. Through this review, we aim to achieve two objectives: to highlight selected settings in which the role of the interconnectivity structure of a network in its robustness is well understood, and to highlight a few additional settings in which existing system-theoretic tools give tight robustness guarantees and that are also appropriate avenues for future network-theoretic investigations.

Description: Modern robots are increasingly capable of performing “basic” activities such as localization, navigation, and motion planning. However, for a robot to be considered intelligent, we would like it to be able to automatically combine these capabilities in order to achieve a high-level goal. The field of automated planning (sometimes called AI planning) deals with automatically synthesizing plans that combine basic actions to achieve a high-level goal. In this article, we focus on the intersection of automated planning and robotics and discuss some of the challenges and tools available to employ automated planning in controlling robots. We review different types of planning formalisms and discuss their advantages and limitations, especially in the context of planning robot actions. We conclude with a brief guide aimed at helping roboticists choose the right planning model to endow a robot with planning capabilities.

Description: We review the problem of defining and inferring a state for a control system based on complex, high-dimensional, highly uncertain measurement streams, such as videos. Such a state, or representation, should contain all and only the information needed for control and discount nuisance variability in the data. It should also have finite complexity, ideally modulated depending on available resources. This representation is what we want to store in memory in lieu of the data, as it separates the control task from the measurement process. For the trivial case with no dynamics, a representation can be inferred by minimizing the information bottleneck Lagrangian in a function class realized by deep neural networks. The resulting representation has much higher dimension than the data (already in the millions) but is smaller in the sense of information content, retaining only what is needed for the task. This process also yields representations that are invariant to nuisance factors and have maximally independent components. We extend these ideas to the dynamic case, where the representation is the posterior density of the task variable given the measurements up to the current time, which is in general much simpler than the prediction density maintained by the classical Bayesian filter. Again, this can be finitely parameterized using a deep neural network, and some applications are already beginning to emerge. No explicit assumption of Markovianity is needed; instead, complexity trades off approximation of an optimal representation, including the degree of Markovianity.

Description: Here, we review the role of control theory in modeling neural control systems through a top-down analysis approach. Specifically, we examine the role of the brain and central nervous system as the controller in the organism, connected to but isolated from the rest of the animal through insulated interfaces. Though biological and engineering control systems operate on similar principles, they differ in several critical features, which makes drawing inspiration from biology for engineering controllers challenging but worthwhile. We also outline a procedure that the control theorist can use to draw inspiration from the biological controller: starting from the intact, behaving animal; designing experiments to deconstruct and model hierarchies of feedback; modifying feedback topologies; perturbing inputs and plant dynamics; using the resultant outputs to perform system identification; and tuning and validating the resultant control-theoretic model using specially engineered robophysical models.

Description: Estimation of functions from sparse and noisy data is a central theme in machine learning. In the last few years, many algorithms have been developed that exploit Tikhonov regularization theory and reproducing kernel Hilbert spaces. These are the so-called kernel-based methods, which include powerful approaches like regularization networks, support vector machines, and Gaussian regression. Recently, these techniques have also gained popularity in the system identification community. In both linear and nonlinear settings, kernels that incorporate information on dynamic systems, such as the smoothness and stability of the input–output map, can challenge consolidated approaches based on parametric model structures. In the classical parametric setting, the complexity of the model (the model order) needs to be chosen, typically from a finite family of alternatives, by trading bias and variance. This (discrete) model order selection step may be critical, especially when the true model does not belong to the model class. In regularization-based approaches, model complexity is controlled by tuning (continuous) regularization parameters, making the model selection step more robust. In this article, we review these new kernel-based system identification approaches and discuss extensions based on nuclear and [Formula: see text] norms.

Description: While reducing anthropogenic greenhouse gas emissions remains the most essential element of any strategy to manage climate change risk, it is also in principle possible to directly cool the climate by reflecting some sunlight back to space. Such climate engineering approaches include adding aerosols to the stratosphere and marine cloud brightening. Assessing whether these ideas could reduce risk requires a broad, multidisciplinary research effort spanning climate science, social sciences, and governance. However, if such strategies were ever used, the effort would also constitute one of the most critical engineering design and control challenges ever considered: making real-time decisions for a highly uncertain and nonlinear dynamic system with many input variables, many measurements, and a vast number of internal degrees of freedom, the dynamics of which span a wide range of timescales. Here, we review the engineering design aspects of climate engineering, discussing both progress to date and remaining challenges that will need to be addressed.

Description: This article surveys recent advancements in strategy designs for persistent robotic surveillance tasks, with a focus on stochastic approaches. The problem describes how mobile robots stochastically patrol a graph in an efficient way, where the efficiency is defined with respect to relevant underlying performance metrics. We start by reviewing the basics of Markov chains, which are the primary motion models for stochastic robotic surveillance. We then discuss the two main criteria regarding the speed and unpredictability of surveillance strategies. The central objects that appear throughout the treatment are the hitting times of Markov chains, their distributions, and their expectations. We formulate various optimization problems based on the relevant metrics in different scenarios and establish their respective properties. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: This review examines the dichotomy between automatic and autonomous behaviors in surgical robots, maps the possible levels of autonomy of these robots, and describes the primary enabling technologies that are driving research in this field. It is organized in five main sections that cover increasing levels of autonomy. At level 0, where the bulk of commercial platforms are, the robot has no decision autonomy. At level 1, the robot can provide cognitive and physical assistance to the surgeon, while at level 2, it can autonomously perform a surgical task. Level 3 comes with conditional autonomy, enabling the robot to plan a task and update planning during execution. Finally, robots at level 4 can plan and execute a sequence of surgical tasks autonomously. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: The last decade has seen a flowering of applications driven by brain–machine interfaces (BMIs), particularly brain-actuated robotic devices designed to restore the independence of people suffering from severe motor disabilities. This review provides an overview of the state of the art of noninvasive BMI-driven devices based on 86 studies published in the last 15 years, with an emphasis on the interactions among the user, the BMI system, and the robot. We found that BMIs are used mostly to drive devices for navigation (e.g., telepresence mobile robots), with BMI paradigms based mainly on exogenous stimulation, and the majority of brain-actuated robots adopt a discrete control strategy. Most critically, in only a few works have disabled people evaluated a brain-actuated robot. The review highlights the most urgent challenges in the field, from the integration between BMI and robotics to the need for a user-centered design to boost the translational impact of BMIs. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: Physics-based simulation provides an accelerated and safe avenue for developing, verifying, and testing robotic control algorithms and prototype designs. In the quest to leverage machine learning for developing AI-enabled robots, physics-based simulation can generate a wealth of labeled training data in a short amount of time. Physics-based simulation also creates an ideal proving ground for developing intelligent robots that can both learn from their mistakes and be verifiable. This article provides an overview of the use of simulation in robotics, emphasizing how robots (with sensing and actuation components), the environment they operate in, and the humans they interact with are simulated in practice. It concludes with an overview of existing tools for simulation in robotics and a short discussion of aspects that limit the role that simulation plays today in intelligent robot design. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: Magnetic microrobotics has undergone approximately 20 years of development, and the robotics and control communities have contributed significant theoretical and practical results to the motion control aspects of this field. This article introduces fundamental motion principles covering individual, multiagent, and swarm control and critically reviews the state of the art along with representative results. It then describes closed-loop control (an important part of this field), including the system structure, current motion planning and control methods, and current feedback approaches. As the development of motion control in magnetic microrobotics is far from complete, especially for swarm control, its current limitations are discussed. Finally, we conclude with several challenges and future research directions. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: The human skin is a unique organ that embeds multiple functions that no artificial systems can currently replicate. Advances in materials science and engineering are driving the design of electronic skins—large-area sensor arrays that mimic some sensory modalities and have the soft, elastic form of natural skin. Here, we focus on electronic skins designed to be worn on the human body for healthcare monitoring or prosthetic applications. The primary sensing modalities are mechanical, thermal, and electrophysiological. We review key materials and associated designs needed to manufacture electronic devices that can conform to the human body and move along with it. Electronic skins offer exciting opportunities for human–machine interfaces. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: In this article, we first present some foundational results about the stability and positive stabilization of continuous-time positive systems. Necessary and sufficient conditions for achieving stability are provided, together with some desired performance in terms of disturbance attenuation. These conditions are expressed in terms of linear programming and scale well with the system size. We then discuss the interconnection of positive subsystems by means of a static output feedback that preserves positivity, and propose conditions to achieve both stability and the asymptotic alignment of the closed-loop output to a desired vector. Finally, we describe some results for a class of parameterized positive systems. The second part of the article presents some interesting applications of the results presented in the first part. Specifically, control problems for heating networks, formation control, power control in wireless communication, and the evolutionary dynamics of cancer and HIV are formalized and solved as optimal control problems for positive systems. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: This article reviews the literature on the design of robotic mechanical grippers, with a focus on the mechanical aspects, which are believed to be the main bottleneck for effective designs. Our discussion includes gripper architectures and means of actuation, anthropomorphism and grasp planning, and robotic manipulation, emphasizing the complementary concepts of intrinsic and extrinsic dexterity. We also consider interactions of robotic grippers with the environment and with the objects to be grasped and argue that the proper handling of such interactions is key to the development of grasping and manipulation tools and scenarios. Finally, we briefly present examples of recent designs to support the discussion. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: Dynamic walking on bipedal robots has evolved from an idea in science fiction to a practical reality. This is due to continued progress in three key areas: a mathematical understanding of locomotion, the computational ability to encode this mathematics through optimization, and the hardware capable of realizing this understanding in practice. In this context, this review outlines the end-to-end process of methods that have proven effective in the literature for achieving dynamic walking on bipedal robots. We begin by introducing mathematical models of locomotion, from reduced-order models that capture essential walking behaviors to hybrid dynamical systems that encode the full-order continuous dynamics along with discrete foot-strike dynamics. These models form the basis for gait generation via (nonlinear) optimization problems. Finally, models and their generated gaits merge in the context of real-time control, wherein walking behaviors are translated to hardware. The concepts presented are illustrated throughout in simulation, and experimental instantiations on multiple walking platforms are highlighted to demonstrate the ability to realize dynamic walking on bipedal robots that is both agile and efficient. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: The problem of planning for a robot that operates in environments containing a large number of objects, taking actions to move itself through the world as well as to change the state of the objects, is known as task and motion planning (TAMP). TAMP problems contain elements of discrete task planning, discrete–continuous mathematical programming, and continuous motion planning and thus cannot be effectively addressed by any of these fields directly. In this article, we define a class of TAMP problems and survey algorithms for solving them, characterizing the solution methods in terms of their strategies for solving the continuous-space subproblems and their techniques for integrating the discrete and continuous components of the search. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: It has been 10 years since the Fukushima Daiichi Nuclear Power Station (NPS) accident. This article begins by discussing the robots used during the responses to the Three Mile Island and Chernobyl nuclear accidents. It then reviews the robots used to respond to the Fukushima Daiichi NPS accident, while considering the lessons learned from the previous accidents. Such discussions will hopefully lead to the further development of robots for decommissioning the Fukushima Daiichi NPS. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: A common way to represent a system's dynamics is to specify how the state evolves in time. An alternative viewpoint is to specify how functions of the state evolve in time. This evolution of functions is governed by a linear operator called the Koopman operator, whose spectral properties reveal intrinsic features of a system. For instance, its eigenfunctions determine coordinates in which the dynamics evolve linearly. This review discusses the theoretical foundations of Koopman operator methods, as well as numerical methods developed over the past two decades to approximate the Koopman operator from data, for systems both with and without actuation. We pay special attention to ergodic systems, for which especially effective numerical methods are available. For nonlinear systems with an affine control input, the Koopman formalism leads naturally to systems that are bilinear in the state and the input, and this structure can be leveraged for the design of controllers and estimators. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: The air transportation system connects the world through the transport of goods and people. However, operational inefficiencies such as flight delays and cancellations are prevalent, resulting in economic and environmental impacts. In the first part of this article, we review recent advances in using network analysis techniques to model the interdependencies observed in the air transportation system and to understand the role of airports in connecting populations, serving air traffic demand, and spreading delays. In the second part, we present some of our recent work on using operational data to build dynamical system models of air traffic delay networks. We show that Markov jump linear system models capture many of the salient characteristics of these networked systems. We illustrate how these models can be validated and then used to analyze system properties such as stability and to design optimal control strategies that limit the propagation of disruptions in air traffic networks. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: Network systems consist of subsystems and their interconnections and provide a powerful framework for the analysis, modeling, and control of complex systems. However, subsystems may have high-dimensional dynamics and a large number of complex interconnections, and it is therefore relevant to study reduction methods for network systems. Here, we provide an overview of reduction methods for both the topological (interconnection) structure of a network and the dynamics of the nodes while preserving structural properties of the network. We first review topological complexity reduction methods based on graph clustering and aggregation, producing a reduced-order network model. Next, we consider reduction of the nodal dynamics using extensions of classical methods while preserving the stability and synchronization properties. Finally, we present a structure-preserving generalized balancing method for simultaneously simplifying the topological structure and the order of the nodal dynamics. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: Reachability analysis consists in computing the set of states that are reachable by a dynamical system from all initial states and for all admissible inputs and parameters. It is a fundamental problem motivated by many applications in formal verification, controller synthesis, and estimation, to name only a few. This article focuses on a class of methods for computing a guaranteed overapproximation of the reachable set of continuous and hybrid systems, relying predominantly on set propagation; starting from the set of initial states, these techniques iteratively propagate a sequence of sets according to the system dynamics. After a review of set representation and computation, the article presents the state of the art of set propagation techniques for reachability analysis of linear, nonlinear, and hybrid systems. It ends with a discussion of successful applications of reachability analysis to real-world problems. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: Biomimetic robots that replace living social interaction partners can help elucidate the underlying interaction rules in animal groups. Our review focuses on the use of interactive robots that respond dynamically to animal behavior as part of a closed control loop. We discuss the most influential works to date and how they have contributed to our understanding of animal sociality. Technological advances permit the use of robots that can adapt to the situations they face and the conspecifics they encounter, or robots that learn to optimize their social performance from a set of experiences. We discuss how adaptation and learning may provide novel insights into group sociobiology and describe the technical challenges associated with these types of interactive robots. This interdisciplinary field provides a rich set of problems to be tackled by roboticists, machine learning engineers, and control theorists. By cultivating smarter robots, we can usher in an era of more nuanced exploration of animal behavior. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Description: Optimal transport began as the problem of how to efficiently redistribute goods between production and consumers and evolved into a far-reaching geometric variational framework for studying flows of distributions on metric spaces. This theory enables a class of stochastic control problems to regulate dynamical systems so as to limit uncertainty to within specified limits. Representative control examples include the landing of a spacecraft aimed probabilistically toward a target and the suppression of undesirable effects of thermal noise on resonators; in both of these examples, the goal is to regulate the flow of the distribution of the random state. A most unlikely link turned up between transport of probability distributions and a maximum entropy inference problem posed by Erwin Schrödinger, where the latter is seen as an entropy-regularized version of the former. These intertwined topics of optimal transport, stochastic control, and inference are the subject of this review, which aims to highlight connections, insights, and computational tools while touching on quadratic regulator theory and probabilistic flows in discrete spaces and networks. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.