Precise In-Hand Manipulation of Soft Objects with Soft Fingertips | RoboSoft 2020

Machine learning can be applied to soft robotic systems to bridge the gap between soft systems and robotic control. By using motion capture systems with cameras and point markers, neural networks can map finger motions to tendon actuation. This approach generates the Jacobian (the relationship between joint velocities and end-effector velocities) through machine learning, solving the challenge of controlling under-actuated soft systems where traditional kinematic approaches fail.

This section explores the convergence of mechanical design and machine learning in soft robotic manipulation. Soft robotic hands achieve distributed intelligence through mechanical compliance combined with computational strategies, as demonstrated by the Pisa-IIT SoftHand using only one motor to achieve 19 degrees of freedom through compliant mechanics. Deep neural networks recognize human grasp primitives from RGB images using transfer learning, enabling robots to select appropriate grasp strategies based on object appearance. Reactive grasp adjustment uses IMU feedback to complement feedforward planning, allowing adaptation to environmental conditions through two-phase approaches (feedforward planning plus sensory-triggered feedback). Recurrent neural networks with gated recurrent units predict grasp failure events from IMU data streams, enabling proactive re-grasping with approximately 85% success rate. The integration of model-based methods (biomechanical principles) with data-driven approaches (machine learning) enables unique computational power for designing next-generation human-robot interaction systems capable of handling complex manipulation tasks.

Soft materials are highly damping, which pre-processes mechanical information without requiring transmission to a central controller. This damping eliminates higher-order effects that complicate stiff systems, making machine learning training more efficient. Statistical methods like k-nearest neighbors can train soft robot manipulators quickly, and convolutional neural networks can accurately predict gestures from soft surface interactions. This compatibility makes soft robotics an effective approach for training robot 'brains' through machine learning.

Soft robots have substantially complex dynamics that make traditional rigid robot control methods unsuitable; machine learning algorithms provide adaptive autonomy that enables soft robots to perform sophisticated tasks like object manipulation by learning from experience rather than relying on pre-programmed instructions.

Manipulating soft objects like towels has been historically difficult for robots because traditional approaches require precise 3D models that don't exist for deformable items. Deep learning enables robots to process raw image data directly and predict future states, allowing real-time manipulation without pre-programmed trajectories. The system's internal model highlights relevant aspects of the environment while filtering out distractions. This top-down prediction approach enables robots to handle novel objects they have never seen before by recognizing similarities to previously learned examples.

Practical implementation of soft sensors requires advanced manufacturing techniques beyond simple deposition. Tapping extrusion creates structured sensor geometries by first depositing liquid metal nanoparticles via co-solvent methods, then using tools to tap out desired patterns (like spiral pressure sensors), followed by polymer extrusion to lock geometry. This enables creation of long wires and intricate patterns larger than printing stages. Laser sintering provides precision and speed through ablation and thermal contributions, achieving features as small as 5 microns—the smallest liquid metal traces ever fabricated. Multi-layer devices can be created by depositing, sintering, and stacking layers, enabling complex 3D sensor architectures with automated processing. These techniques bridge the gap between laboratory demonstrations and practical sensor fabrication for robotic skins.

iSoft is a customizable soft sensor that enables real-time continuous contact and unidirectional stretching sensing using piezoresistive elastomer materials. The system employs electrical impedance tomography (EIT) with dynamic baseline updates triggered by fingertip contact and movement detections to overcome the rebound elasticity issues of carbon elastomer, achieving real-time performance. The fabrication process uses carbon-filled silicone mixed with heat-stabilized film and cured in approximately 140 seconds using a heat press, with electronic components costing around $30. The sensor supports multimodal sensing including discrete touch, continuous contact localization, and stretching recognition up to 40% strain, with applications demonstrated on lamps, 3D objects, textiles, and volumetric fabrics.

Soft sensors can be fabricated using capacitive sensing instead of resistive sensing. Capacitive sensing offers several advantages: lower error rates after thousands of cycles, better performance across temperature ranges (since temperature affects resistance more than capacitance), and faster response times. A typical capacitive soft sensor consists of three layers: a passive silicone layer, a conductive silicone electrode layer, and another passive silicone layer. Electrodes are patterned using laser cutting to create interdigitated structures that enable precise strain measurement.

Sensor integration in soft robots requires balancing information richness with fabrication complexity. Starting with few sensors provides redundancy for mapping proprioceptive and exteroceptive feedback through signal analysis. Strategic sensor placement can mechanically guide actuator curvature. Acoustic sensing using microphones detects object shape by analyzing how filament contacts change vibration frequencies—identical-weight objects of different spatial densities produce distinguishable signatures. Optical fiber sensors embedded in soft sheets detect deformation and pressure through refractive index differences, with roughened sides providing directional bending sensitivity. For soft robotics, form defines function, meaning design necessarily involves designing the fabrication process itself. Applying fabrication-integrated design yields soft robots that are more durable, reliable, stronger, faster, and scalable in manufacturing. Geometric and material optimization is necessary but not sufficient; similarly, manufacturing alone is insufficient. A comprehensive framework requires multidisciplinary community effort integrating design and fabrication considerations throughout development.

PDMS-PDMS microfluidics eliminate glass substrates, providing complete flexibility essential for organ-on-a-chip applications where cells like lung and muscle cells require mechanical stimulation. This flexibility enables wearable sensors for sweat analysis on skin and advances soft robotics through layered materials with varying stiffness. Soft robotic actuators use air pressure to create bending motions without hard components, demonstrating practical indestructibility. 3D printing enables fabrication of microfluidic devices with complex three-dimensional geometries improving mixing efficiency over traditional planar channels. ESCARGOT (Embedded Scaffold Removing Open Technology) enables monolithic PDMS microfluidic devices without lithography, plasma treatment, or harsh chemicals, using standard FDM printing and acetone dissolution to create flexible blocks with integrated channels, embedded electronics, and heating coils impossible with conventional techniques.

Radioactive decay has important industrial and medical applications. Industrial applications include radiography for detecting material flaws, thickness gauges for manufacturing, and level gauges for tanks. Food irradiation uses gamma radiation from Cobalt-60 to kill bacteria and extend shelf life. Medical applications include imaging using Technetium-99m and treatment using Iodine-131 for thyroid cancer. Smoke detectors use alpha-emitting Americium-241 for ionization-based detection. Nuclear power generation uses fission of U-235 to produce electricity.

Microorganisms are extensively utilized in industrial and medical applications. Industrial acids include citric acid from Aspergillus niger, acetic acid from Acetobacter, butyric acid from Clostridium butyricum, and lactic acid from Lactobacillus. Enzymes like zymase (from Saccharomyces) enable ethanol production, streptokinase (from Streptococcus) acts as a clot-busting drug, lipase breaks down fats in detergents, and protease clarifies fruit juices. Pharmaceutical applications include cyclosporin A (from Trichoderma and Monascus) as an immunosuppressant for organ transplantation, and statins as cholesterol-lowering agents preventing cardiovascular diseases.

Adsorption has numerous important applications in industry and medicine. Industrial applications include synthesis of ammonia, methanol, hydrocarbons, manufacture of hydrogen, nitric acid, synthetic petrol, sulphuric acid, and vegetable oil. Medical applications include curing diseases by absorbing drugs or poisons onto germs, creating masks, achieving high vacuum, serving as indicators, checking evaporation of water, chromatography, recovering and concentrating vitamins, acting as lubricants, and softening water.

Microorganisms are used extensively in industrial and medical applications. In food processing, organic acids like lactic acid are produced for preservation and flavor. Enzymes from microorganisms speed up chemical reactions in industrial processes. Blood clotting factors are produced using microorganisms for treating bleeding disorders. Bioactive molecules influence metabolism and hormone regulation. Cholesterol management uses microorganisms to reduce levels and prevent cardiovascular diseases. Kidney transplantation relies on microorganisms for producing immunosuppressive drugs.

Radioisotopes have important medical applications: Iodine-131 for brain tumor detection and thyroid disorders, Sodium-24 for blood clots and circulation disorders, Iron-59 for anemia diagnosis, Cobalt-60 and Technetium-99m for cancer diagnosis, and Tritium for detecting water content in the human body. Industrial applications include: Americium-241 for uniform thickness measurement of steel and paper, Sodium-24 for oil well study and pipeline leak detection, Iridium-192 for testing boiler and aircraft part integrity, Uranium-235 for nuclear power plants and naval propulsion, and Californium-252 for determining soil moisture content in construction.