Smart robots to make a salad

Typical main-course salads you can buy in a supermarket contain lettuce, raw vegetables, pasta, potatoes, fruit, nuts, eggs, cheese, meat and fish. To date, there is no option to automatically gather and package all these different products without the input of human labour. WUR experts in computer vision, robotics and AI want to change that in the coming five years in collaboration with industrial partners. Successful development of such a machine would save a lot of line workers, which, in turn, saves costs in an increasingly shallow labour pool.

To automatically prepare and package a main-course salad, robots must perform all tasks performed when single products are packaged. Cameras take pictures for advanced computer programmes that learn to recognise products. Then, the same software is used to drive robotic arms that remove products rapidly without damaging them from the harvest crates and deposit them in custom packaging. The packaging is then sealed, and the product information is printed on it.

Artificial intelligence

The technology that the researchers will develop further is built upon the results from the ‘Pick’nPack’ project led by WUR from 2011 till 2016. ‘At the time, producers were not exactly lining up to obtain the Pick’nPack technology, but that is beginning to change’, says Paul Goethals, business development manager in the WUR Agro Food Robotics programme. Not only has the shortage of labour increased the demand, but the technology has also been developed further and is now cheaper, says Goethals. This is mainly due to the advancements in Artificial Intelligence (AI). ‘AI is making applications such as this more relevant for businesses.’

‘Before, we would have to feed the software with characteristics with which to analyse the camera images. Now, we use machine learning with training data. This serves as input for the software to autonomously recognise particular properties of vegetables’, Goethals explains. ‘That then helps the robots recognise and grab the products directly from the harvest crates, without humans first having to deposit the goods on the production line. The stacked products are often easily bruised.’ This means that the robots must be very precise.

Flexible robots

The ultimate goal is a flexible robot. After all, cucumbers are different from grapes. That requires different “grippers” or suction cups to prevent soft fruit from being damaged. The robot should select the appropriate device and activate the matching computer programme for the product in the crate. With the increasing variation in products – consider, for example, the many different types of tomatoes – a machine capable of rapidly switching saves time, money and labour. Moreover, different products are sometimes placed in the same package. Goethals: ‘In the future, we will be able to package mixed vegetables for soup with this technology.’

Goethals and his colleagues use the latest tools for artificial intelligence. ‘A sensor in the gripper is also interesting for quality control purposes. That makes it easy to instantly assess whether anything is wrong with the product. That information then enables the robot to sort products into various quality categories or a residue stream used for a different purpose.’ The data thus gathered makes it easier to retrace products back to their origin and print the product information on the packaging. Even cleaning the machine can be made ‘smarter’ using sensors that continuously measure how much dirt accumulates and that only clean the machine from within when needed.

Main-course salads as a test case

In the coming five years, Agro Food Robotics researchers and industry partners will merge their work and recent developments to design one flexible state-of-the-art production robot for main-course salads. This new knowledge will then be shared with other businesses in the overarching Nxt Gen HighTech programme. ‘With Pick’nPack, we were ahead of the developments in the field, but now, we can develop our prototypes and knowledge further and apply them. The time and technology are ripe.’