For humans, "touch" plays a vital role in daily activities. During tasks such as picking up objects, people can feel the hard or soft, light or heavy, warm or cold of objects through touch, and the combination of touch and vision allows us to avoid damaging them. Today, with the rapid development of science and technology, robots that can walk, run, see, speak, and listen have been born. But research on robotic haptics is relatively backward, and new technologies need to be developed to overcome some of these issues.

The multi-camera tactile sensor consists of four cameras under a soft, transparent material that contains embedded dispersed spherical particles. The camera tracks the motion of these spherical particles, which deform when an external force is applied to the material.

The researchers also developed a machine learning (ML) architecture that can analyze the motion of spherical particles in the material. By analyzing the motion of spherical particles, the system can reconstruct the forces that cause the material to deform, also known as the contact force distribution.

"Using relatively inexpensive cameras, we generate a large amount of high-resolution image information, providing a total of about 65,000 pixel images, which are important for data-driven tactile sensors," the researchers explained.

The multi-camera tactile sensor not only provides the total force value as standard force sensors used on most existing robots, but also provides feedback on the distribution of all forces applied to its soft surface, thereby decoupling the normal components and Tangential component. Due to the unique structural design, the new multi-camera tactile sensor has a larger contact surface and a thinner structure compared to other camera-based tactile sensors because it does not require the addition of other reflective components such as mirrors.

"Using multiple cameras allows the use of this type of tactile sensor to cover large areas of arbitrary shape. This work also shows how data obtained on a subset of cameras can be transferred to other cameras to obtain A way to scale data."

Robotic force sensors can be scaled to larger surfaces to create soft and senseable robotic skin. The researchers discussed how their machine learning architecture could be adapted and optimized to facilitate its application in robotics in the future.

For the future, the researchers say they now plan to expand the capabilities of the sensor to reconstruct information about contact with objects of complex and generic shapes. "We believe that sensing algorithms should always be developed with a data efficiency component in mind to facilitate widespread use in robotics, and we will therefore also move in this direction in future work."