Matt Vollrath and Wojciech Ziniew work at an ecommerce consultancy called End Point, where they provide support for Liquid Galaxy; a product that’s almost as cool as it sounds. Originally an open source project begun by Google engineers on their twenty percent time, Liquid Galaxy is a data visualization system consisting of a collection of large vertical displays that wrap around you horizontally. The displays show an immersive (up to 270°) image that’s ideal for data presentations, virtual tours, Google Earth, or anywhere you want a visually engaging environment. Think events, trade shows, offices, museums, galleries, and the like.

Last year, End Point decided to take all of the ad hoc services and protocols that they’d been using to support Liquid Galaxy and move everything over to ROS. The primary reason to do this was ROS support for input devices: you can use just about anything to control a Liquid Galaxy display system, from basic touchscreens to Space Navigator 3D mice to Leap Motions to depth cameras. The modularity of ROS is inherently friendly to all kinds of different hardware.

Check out this week’s ROSCon15 video as Matt and Wojciech take a deep dive into their efforts in bringing ROS to bear for these unique environments.

Next up: Tom Moore (Clearpath Robotics)

ROS already comes with a fantastic built-in visualization tool called rviz, so why would you want to use anything else? At Southwest Research Institute, Jerry Towler explains how they’ve created a new visualization tool called Mapviz that’s specifically designed for the kind of large-scale outdoor environments necessary for autonomous vehicle development. Specifically, Mapviz is able to integrate all of the sensor data that you need on top of a variety of two-dimensional maps, such as road maps or satellite imagery.

As an autonomous vehicle visualization tool, Mapviz works just like you’d expect that it would, which Jerry demonstrated with several demos at ROSCon. Mapviz shows you a top-down view of where your vehicle is, and tracks it across a basemap that seamlessly pulls image tiles at multiple resolutions from a wide variety of local or networked map servers, including Open MapQuest and Bing Maps. Mapviz is, of course, very plugin-friendly. You can add things like stereo disparity feeds, GPS fixes, odometry, grids, pathing data, image overlays, projected laser scans, markers (including textured markers) from most sensor types, and more. It can’t really handle three dimensional data (although it’ll do two-and-a-half dimensions via color gradients), but for interactive tracking of your vehicle’s navigation and path planning behavior, Mapviz should offer most of what you need.

For a variety of non-technical reasons, SwRI hasn’t been able to release all of its tools and plugins as open source quite yet, but they’re working on getting approval as fast as they can. They’re also in the process of developing even more enhancements for Mapviz, and you can keep up to date with the latest version of the software on GitHub.

Next up: Matt Vollrath & Wojciech Ziniewicz (End Point)

BMW has been working on automated driving for the last decade, steadily implementing more advanced features ranging from emergency stop assistance and autonomous highway driving to fully automated valet parking and 360° collision avoidance. Several of these projects were presented at the 2015 Consumer Electronics Show, and as it turns out, the cars were running ROS for both environment detection and planning.

BMW, being BMW, has no problem getting new research hardware. Their latest development platform is the 335I G. This model comes with an advanced driver assistance system based around cameras and radar. The car has been outfitted with four low-profile laser scanners and one long-range radar, but otherwise, it’s pretty close (in terms of hardware) to what’s available in production BMWs.

Why did BMW choose to move from their internally developed software architecture to ROS? Michael explains how ROS’ reputation in the robotics research community prompted his team to give it a try, and they were impressed with its open source nature, distributed architecture, existing selection of software packages, as well as its helpful community. “A large user base means stability and reliability,” Michael says, “because somebody else probably already solved the problem you’re having.” Additionally, using ROS rather than a commercial software platform makes it much easier for BMW to cooperate with universities and research institutions.

Michael discusses the ROS software architecture that BMW is using to do its autonomous car development, and shows how the software interprets the sensor data to identify obstacles and lane markings and do localization and trajectory planning to enable full highway autonomy, based on a combination of lane keeping and dynamic cruise control. BMW also created their own suite of RQT and rviz plugins specifically designed for autonomous vehicle development.

After about two years of experience with ROS, BMW likes a lot of things about it, but Michael and his team do have some constructive criticisms: message transport needs more work (although ROS 2 should help with this), managing configurations for different robots is problematic, and it’s difficult to enforce compliance with industry standards like ISO and AUTOSAR, which will be necessary for software that’s usable in production vehicles.

Next up: Jerry Towler & Marc Alban (SwRI)

While Intel is best known for making computer processors, the company is also interested in how people interact with all of the computing devices that have Intel inside. In other words, Intel makes brains, but they need senses to enable those brains to understand the world around them. Intel has developed two very small and very cheap 3D cameras (one long range and one short range) called RealSense, with the initial intent of putting them into devices like laptops and tablets for applications such as facial recognition and gesture tracking.

Robots are also in dire need of capable and affordable 3D sensors for navigation and object recognition, and fortunately, Intel understands this, and they’ve created the RealSense Robotics Innovation Program to help drive innovation using their hardware. Intel itself isn’t a robotics company, but as Amit explains in his ROSCon talk, they want to be a part of the robotics future, which is why they prioritized ROS integration for their RealSense cameras.

A RealSense ROS package has been available since 2015, and Intel has been listening to feedback from roboticists and steadily adding more features. The package provides access to the RealSense camera data (RGB, depth, IR, and point cloud), and will eventually include basic computer vision functions (including plane analysis and blob detection) as well as more advanced functions like skeleton tracking, object recognition, and localization and mapping tools.

Intel RealSense 3D camera developer kits are available now, and you can order one for as little as $99.

Next up: Michael Aeberhard, Thomas Kühbeck, Bernhard Seidl, et al. (BMW Group Research and Technology)
Check out last week’s post: The Descartes Planning Library for Semi-Constrained Cartesian Trajectories

Descartes is a path planning library that’s designed to solve the problem of planning with semi-constrained trajectories. Semi-constrained means that the degrees of freedom of the path you need to plan are fewer than the degrees of freedom that your robot has. In other words, when planning a path, there are one or more “free” axes that your robot has to work with that can be moved any which way without disrupting the path. This can open up the planning space if you can utilize them creatively, which traditional robots (especially in the industrial space) usually can’t. This results in reduced workspaces and (most dangerous of all) increased reliance on human intuition during the planning process.

Descartes was designed to generate common sense plans, exhibiting similar characteristics to paths planned by a human. It can solve easy problems quickly, and difficult problems eventually, integrating hybrid trajectories and dynamic replanning. It’s easy to use, with a GUI that allows you to quickly set anchor points that the robot replans around, with visual confirmation of the new path. The second half of Shaun’s ROSCon talk is an in-depth explanation of Descartes’ interfaces and implementations intended for path planning fans (you know who you are).

As with many (if not most) of the projects being presented at ROSCon, Descartes is open source, and all of the development is public. If you’d like to try it out, the current stable release runs on ROS Hydro, and a tutorial is available on the ROS Wiki to help you get started.

Next up: Amit Moran & Gila Kamhi (Intel)
Check out last week’s post: Phobos — Robot Model Development on Steroids

The ROS Industrial Consortium was established four years ago as a partnership between Yaskawa Motoman Robotics, Southwest Research Institute (SwRI), Willow Garage, and Fraunhofer IPA. The idea was to provide a ROS-based open-source framework for robotics applications, designed to make it easy (or at least possible) to leverage advanced ROS capabilities (like perception and planning) in industrial environments. Basically, ROS-I adds models, libraries, drivers, and packages to ROS that are specifically designed for manufacturing automation, with a focus on code quality and end user reliability.

Mirko Bordignon from Fraunhofer IPA opened the final ROSCon 2016 keynote by pointing out that ROS is still heavily focused on research and service robotics. This isn’t a bad thing, but with a little help, there’s an enormous opportunity for ROS to transform industrial robotics as well. Over the past few years. The ROS Industrial Consortium has grown into two international consortia (one in America and one in Europe), comprising over thirty members that provide financial and managerial support to the ROS-I community.

To help companies get more comfortable with the idea of using ROS in their robots, ROS-I holds frequent training sessions and other outreach events. “People out there are realizing that at least they can’t ignore ROS, and that they actually might benefit from it,” Bordignon says. And companies are benefiting from it, with ROS starting to show up in a variety of different industries in the form of factory floor deployments as well as products.

Bordignon highlights a few of the most interesting projects that the ROS-I community is working on at the moment, including a CAD to ROS workbench, getting ROS to work on PLCs, and integrating the OPC data protocol, which is common to many industrial systems.

Before going into deeper detail on ROS-I’s projects, Shaun Edwards from SwRI talks about how the fundamental idea for a ROS-I consortium goes back to one of their first demos. The demo was of a PR2 using 3D perception and intelligent path planning to pick up objects off of a table. “[Companies were] impressed by what they saw at Willow Garage, but they didn’t make the connection: that they could leverage that work,” Edwards explains. SwRI then partnered with Yaskawa to get the same software running on an industrial arm, “and this alone really sold industry on ROS being something to pay attention to,” says Edwards.

Since 2014, ROS-I has been refining a general purpose Calibration Toolbox for industrial robots. The goal is to streamline an otherwise time-consuming (and annoying) calibration process. This toolbox covers robot-to-camera calibration (with both stationary and mobile cameras), as well as camera-to-camera calibration. Over the next few months, ROS-I will be releasing templates for common calibration use cases to make it as easy as possible.

Path planning is another ongoing ROS-I project, as is ROS support for CANOpen devices (to enable IoT-type networking), and integrated motion planning for mobile manipulators. ROS-I actually paid the developers of the ROS mobile manipulation stack to help with this. “Leveraging the community this way, and even paying the community, is a really good thing, and I’d like to see more of it,” Edwards says.

To close things out, Edwards briefly touches on the future of ROS-I, including the seamless fusion of 3D scanning, intelligent planning, and dynamic manipulation, which is already being sponsored by Boeing and Caterpillar. If you’d like to get involved in ROS-I, they’d love for you to join them, and even if you’re not directly interested in industrial robotics, there are still plenty of opportunities to be part of a more inclusive and collaborative ROS ecosystem.

Next up: Kai von Szadkowski (University of Bremen)
Check out last week’s post: MoveIt! Strengths, Weaknesses, and Developer Insight

Jackie Kay was upgraded from OSRF intern to full-time software engineer in 2014. Her background includes robotics education and path planning for autonomous lunar rovers. More recently, she’s been working on bringing real-time computing to ROS 2.

Real-time computing isn’t about computing at a certain speed— it’s about computing on schedule. It means that your system can return data reliably and on time, in situations where responding late is usually bad thing; and sometimes a really bad thing. Hard real-time computing is important in safety critical applications (like nuclear reactors, spacecraft, and autonomous vehicles), when taking too long thinking about something could result in a figurative or literal crash — or both. Soft real-time computing is a bit more forgiving, in that things running behind have a cost, but the data are still usable, as with packets arriving out of order while streaming video. And in between there’s firm real-time computing, where missing deadlines is definitely bad but nothing explodes (or things only explode a little bit), like on a robotic assembly line.

Making a system that’s adaptable and reliable, especially in the context of commercialization, often requires real-time computing, and this is why integrating real-time compatibility is one of the primary goals of ROS 2. Jackie’s keynote addresses many of the technical details underlying the ROS 2 real-time approach, including scheduling, memory management, node design, and communications strategies. To illustrate the improvements that ROS 2 has over ROS, Jackie shares benchmarking results of a ROS 2 demo running in real-time, showing that even under stress, implementing a high performance soft real-time system in ROS 2 looks promising.

To try real-time computing in ROS 2 for yourself, you can download an Alpha release and play around with a demo here: https://github.com/ros2/ros2/wiki/Real-Time-Programming

ROSCon 2015 Hamburg: Day 1 – Jackie Kay: Real-time Performance in ROS 2 from OSRF on Vimeo.

Next up: Dave Coleman (University of Colorado Boulder)
Check out last week’s post: State of ROS 2

ROS has been an enormously important resource for the robotics community. It turned eight years old at the end of 2015, and is currently on its ninth official release. As ROS adoption has skyrocketed (especially over the past several years), OSRF, together with the community, have identified many specific areas of the operating system that need major overhauls in order to keep pace with maturing user demand. Dirk Thomas, Esteve Fernandez, and William Woodall from OSRF gave a preview at ROSCon 2015 of what to expect in ROS 2, including multi-robot systems, commercial deployments, microprocessor compatibility, real time control, and additional platform support.

The OSRF team shows off many of the exciting new ROS 2 features in this demo-heavy talk, including distributed message passing through DDS (no ROS master required), performance boosts for communications within nodes, quality of service improvements, and ways of bridging ROS 1 and ROS 2 so that you don’t have to make the leap all at once. If you’d like to make the leap all at once anyway, the Alpha 1 release of ROS 2 has been available since last September, and Thomas ends the talk with a brief overview of the roadmap leading up to ROS 2’s Alpha 2 release. As of April 2016, ROS 2 is on release Alpha 5 (“Epoxy”), and you can keep up-to-date on the roadmap and release schedule here.

ROSCon 2015 Hamburg: Day 1 – Dirk Thomas: State of ROS 2 – demos and the technology behind from OSRF on Vimeo.

Next up: Jackie Kay (OSRF) & Adolfo Rodríguez Tsouroukdissian (PAL Robotics)
Check out last week’s post: Lightning Talk highlights