Credit: Fraunhofer IPA

As part of the “SeRoDi” project (“Service Robotics for Personal Services”), Fraunhofer IPA collaborated with other research and application partners to develop new service robotics solutions for the nursing sector. The resulting robots, the “intelligent care cart” and the “robotic service assistant”, were used in extensive real-world trials in a hospital and at two care homes.

Not enough nurses for too many patients or residents: this is a familiar problem in the nursing sector. Service robots have the potential to help maintaining an adequate quality of care also under these challenging conditions.

Intelligent care cart

Credit: Fraunhofer IPA

To cut down the legwork of the nursing staff and reduce the time spent keeping manual records of the consumption of medical supplies, Fraunhofer IPA in collaboration with the MLR Company developed the “intelligent care cart”. Using a smartphone, the care staff is able to summon the care cart to the desired room, whereupon it makes its own way there. A 3D sensor along with object recognition software enables the care cart to automatically register the consumption of medical supplies. Being of modular design, the care cart can be adapted to different application scenarios and practical requirements.

The care carts developed as part of the project were used in a hospital (stocked with wound treatment materials) and two nursing homes (stocked with laundry items). As the intelligent care cart is based on the navigation processes of a driverless transport vehicle, it travels primarily along fixed predefined paths. For use in public spaces, it is possible to make minor deviations from these paths in order, for example, to dynamically negotiate obstacles in the way. The real-world trials revealed that efficient navigation requires extensive knowledge of the internal processes in order, among other things, to guarantee that the desired destination is actually accessible.

The initial trials also showed that it makes a big difference whether the corridors have a single lane for both directions or separate lanes, i.e. one for each direction. For the residents and staff, using one lane made it clearer where the robot was going. In addition, restricting the care carts to a single lane ensured that they did not have to make major detours. Evaluating the real-world trials, the participating nursing staff confirmed that, by reducing the amount of legwork, along with the associated timesaving, the intelligent care cart represents a potential benefit in their day-to-day work. Also, the faster provision of care, with no interruptions for restocking the care cart, results in an improvement in quality for patients and residents.

Robotic service assistant serves drinks to residents

Credit: Fraunhofer IPA

At a nursing home, where the robotic service assistant was trialed for one week in a common room nursing home, it made for a welcome change, with many residents being both curious and interested. Using the robot’s touch screen, they were able to select from a choice of drinks, which were then served to them by the robot. Once all the supplies had been used up, the service assistant returned to the kitchen, where it was restocked by the staff before being sent back to the common room by the use of a smartphone. This robot, too, received great interest from the participating nursing staff. The synthesized voice of the robot was especially popular and even motivated the residents to converse with the robot.

Have a look at the YouTube video showing the project results.

The project received funding from the German Federal Ministry for Education and Research.

Paul, a member of the Care-O-bot 4 robot family, has been greeting customers in Saturn-Markt Ingolstadt since the end of October 2016 and directing them towards their desired products. Image: Saturn.

In January 2015, Fraunhofer IPA presented a prototype of the “Care-O-bot 4” service robot. The charming helper is now proving its worth in the real world. “Paul” the robot has been greeting customers in Saturn-Markt Ingolstadt since the end of October 2016 and directing them towards their desired products.

Care-O-bot 4, alias Paul, approaches Saturn customers and welcomes them to the store. If they ask him about a certain product, he accompanies the customer to the department and points them in the direction of the relevant shelf. As he indulges in small talk about the weather, or another subject, Paul turns out to be a most charming contact partner. However, he prefers to leave actual customer service to his human colleagues. Paul is able to call another member of staff for support via “Voice over IP”. Before Paul bids a fond farewell and returns to the store entrance, he still has time to ask for feedback. In this way, he discovers whether customers appreciated the interaction or not. Martin Wild, Chief Digital Officer at Media-Saturn-Holding: “With Paul, we are offering our customers the opportunity to get to know one of the most advanced robots in the world.”

Image: Fraunhofer IPA, Photographer: Rainer Bez (2015)

Modular construction facilitates a wide range of application areas

The interactive robot prototype was originally developed by Fraunhofer IPA as an assistant robot to actively support humans in areas such as the household, hotels, care homes and hospitals. IPA scientists worked together with the design studio Phoenix Design and Schunk over a period of three years to complete the fourth generation of the robot.  While the predecessors of Care-O-bot 4 were used primarily in the development of technological fundamentals from 1989 onwards, the latest version is a modular product family offering elementary commercial solutions for the first time. Fraunhofer IPA is now working with its spin-off company “Unity Robotics” to ensure the solution is suitable for use in various commercial application areas. Saturn is the first customer to make use of the robot platform in real-life scenarios. The project will initially run for six months, with practical findings from customer interactions and employees directly incorporated into the further development of robot technology.

Care-O-bot 4 providing assistance in the kitchen. Image: Phoenix Design 2015

For the assignment with Saturn, the Stuttgart-based scientists supplemented Care-O-bot 4’s software and hardware with additional features. Ulrich Reiser, IPA project manager, explains: “For example, we specified Paul’s navigation, dialog communication and facial expressions. Semvox and Phoenix Design helped implement these changes.” Laser scanners at shin level help with Paul’s orientation. Paul’s front camera and the “Shore” software developed by Fraunhofer IIS allow him to ascertain the mood of the person he is dealing with and express their emotional state. Microphones and cameras for speech and gesture recognition respectively allow Paul to communicate with those around him.

Care-O-bot 4 open for research and industry

Care-O-bot 4 is not only making a positive impression as a retail assistant. With its modular construction and open software interface, the system can be used in many different application areas. It could, for example, conceivably be used as a mobile information kiosk in shops and airports, for pick-up and delivery services in homes and offices, for security applications or as an attraction in a robotics museum. “We retain an interest in ensuring that as many scientists and companies as possible are able to work with the system in order to expand its potential areas of application”, Reiser explains. Within the framework of the Care-O-bot 4 research platform, experts from around the world can further develop the software and hardware components. Together with the spin-off company Unity Robotics, IPA experts are intending to adapt applications to the needs of companies, making Care-O-bot 4 available as a commercial product.

If you enjoyed this article, you might also be interested in:

• Robots Podcast #221: Ori Systems: Robotic Furniture, with Hasier Larrea
• Designing our home service robot from the ground up
• Robot companions are coming into our homes — how human should they be?
• Global and iRobot floor cleaning market
• Robots that may help you in your silver age

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Source: Max Planck Institute for Biological Cybernetics (MPI), Fraunhofer IPA

To date, cable-driven parallel robots have been used in production environments, where they meet high requirements. The systems surpass conventional industrial robots in size and payload by between one and two orders of magnitude. The end effector can be freely moved with high accuracy by up to eight cables and winches. Based on this technology and in a world first, the idea of a cable-driven motion simulator has now been realized under the lead of Professor Heinrich Bülthoff from MPI for Biological Cybernetics.

Technical innovations

In the cable-driven simulator, the motion of the simulator cabin is controlled by eight unsupported steel cables attached to winches. In contrast to conventional motion simulators, the use of cables makes it possible to reduce the moving mass and to scale the workspace to any required size. A total drive power of 348 kW allows the cabin to accelerate at 1.5 times gravitational acceleration along freely programmable paths inside a 5 x 8 x 4 m³ workspace. In addition, the cables can be reattached in under an hour to enable the simulator to be adapted to different cabins and thus used for a range of scenarios.

During the two-year collaboration between both Institutes, Philipp Miermeister, a member of Fraunhofer IPA’s Cable-driven Robotics working group headed by Junior Professor Andreas Pott, has contributed much know-how to driving forward the design and realization of the simulator. The scientists have not only implemented the control algorithms, but also developed a lightweight yet rugged carbon fibre cabin capable of withstanding the high dynamic loads during operation. Made entirely from carbon fibre tubes, the cabin frame maximizes the usable cabin volume with a diameter of 260 cm for projection surfaces and cockpit instrumentation. This allows it to be used for high-quality video projections and realistic operator interfaces. At the same time, the light 80 kg frame is capable of accelerating at high speed while also withstanding high forces, because, in operation, the cables pull on the outer structure with up to 1.5 tonnes.

The CableRobot Simulator as VR platform

Its large workspace and dynamic capabilities make the simulator suitable for a wide spectrum of VR (virtual reality) applications, including driving/flight simulation as well as investigation of basic perception processes in humans. “This simulator offers us entirely new possibilities for studying motion perception with possible applications in neurological research into balance disorders,” says Professor Bülthoff, who is a long-time perception researcher.

The CableRobot Simulator can be used in combination with different VR solutions such as Oculus Rift or HTC Vive. Depending on the simulation scenario and available space, the smallest version of the robot fits in a room with 5 x 5 x 5 m³. Optimal results will be achieved with a system of 10 to 15 m frame length. There are as good as no limits regarding the size and application range of our simulator. With the current system it is already possible to set up a system for rooms with more than 20 m edge length”, says Miermeister.

Source: Max Planck Institute for Biological Cybernetics (MPI), Fraunhofer IPA

The cabin uses a very flexible structure which provides a spherical internal space which currently is used to mount a race seat, computer systems for visualization, and an additional motion tracking system. The additional tracking system is used to separate the head acceleration signal of the Oculus Rift from the acceleration signal of the cabin.  “Currently we are implementing an additional rotation unit inside the cabin which allows us to rotate the seat independently and gives us more flexibility with regards to possible simulation scenarios” explains Philipp Miermeister. The first use case for the extended cabin design will be a realistic helicopter hover trainer, which allows 1:1 mapping of real flight data to the simulator workspace.

Specialized Software

The CableRobot simulator is controlled by a motion cueing software developed at the Max-Planck-Institute, which allows to replicate the motion from a simulation most accurately. It is also possible to use motion cueing and simulation software form third party providers, or to use the system for motion based cinematic experience. “Recently we started to work on motion reconstruction for 360° 3D footage using the optical flow information from the video stream”, mentions Miermeister. This technique allows reconstructing the camera trajectories for movies whose motion data is not available.

Through the linkage with computer games, such as race and flight simulations as well as 360° 3D videos, it is possible to use the CableRobot simulator for the presentation of highly diverse content. Currently a portable solution is developed which allows to set up the modular and lightweight motion simulator at different locations.

The developed robot system is capable of simulating thousands of typical usage scenarios on any kind of device with a human-machine interface, such as a touchscreen. (Source: Fraunhofer IPA/Photo: Rainer Bez)

Thousands of repetitions in the space of just a few days: while manually impossible, this can be accomplished by a newly developed robot system from Fraunhofer IPA. Whether it’s a matter of pressing keys, inputting data or using fingers or a stylus, e.g. for a signature pad – the robot system is capable of realistically simulating typical applications of human-machine interfaces. The system can test how long the materials human-machine-interfaces are made of and the associated functions remain fully functional. This provides device manufacturers with accurate knowledge about the quality of their products, which is of importance for possible warranty claims. In addition, customers can gain accurate information on durability.

Robot imitates typical usage with its forces and path

“For a device manufacturer, automated testing offers the advantage that the performed conditions are always identical and the test scenarios are reproducible and therefore comparable. The motion of the robot system can be precisely controlled to guarantee that the testing is completely traceable and verifiable,” says Geravand.

For each device, the researchers collaborate with the manufacturer and draw up the typical usage scenarios and loading profiles to set up the robot system. This includes, in particular, setting up the end effector, which is capable of holding a variety of tools – similar, for example, to a finger or stylus – and simulating various loading scenarios.

“We’ve succeeded in setting up the robot system so that it is capable of accurately simulating an application such as the use of a cash machine. Force sensors in a robot-guided finger-like end effector accurately measure and evaluate the force exerted by the user in a multiplicity of interaction cycles,” explains Milad Geravand, research assistant at Fraunhofer IPA. The accuracy of the tests is guaranteed by measuring systems as well as by regular checks by the employees.

The programming of the robot system depends on features that describe the interaction performed by the user on the devices with human-machine interfaces. The researchers measure and process these features in authentic situations with test subjects. That could include the duration and force registered during the interaction. If the device under test is a touchscreen, it might also be relevant where the action is typically performed. This data is used by the scientists to suitably configure the forces and path of the robot.

Uniform quality of testing

By the end of each intense test, device manufacturers are provided with a detailed report on the tests and results as well as with a brief summary, which can also be used as a reference for customers. The scientists are ready to employ this technology for life-cycle evaluation of human-machine-interfaces either in the labs at Fraunhofer IPA or on the manufacturer’s site by setting up the testing systems.