Integrating neuromorphic circuits in soft robotic systems via reinforcement learning to move towards combined intelligence

Besides drinking nectar, the hummingbird needs to hunt for insects to consume enough nutrients. However, its long beak would prevent the hummingbird from effectively catching its prey. Evolution’s solution is to allow the bottom part of the beak to curl up and snap shut. The inherent passive intelligence of a system, in this case the hummingbird’s beak, is what is often referred to as embodied intelligence. The field of soft robotics has approached embodied intelligence by designing inherently adaptive actuators, based on flexible or soft materials. These soft robotic systems are promising candidates for use in non-conventional environments, such as the exploration of coral reefs, and for complex applications, such as fruit picking. However, the behavior is typically pre-programmed and subsequently is not capable of actively varying the system’s behavior, yet the hummingbird needs to make decisions to be able to catch its prey.

As such, we want a centralized intelligent system to work in tandem with the embodied intelligence of our soft robot. Typical learning systems, such as conventional AI, are not suitable because they are too big and weigh too much, or require remote communication. Organic neuromorphic circuits could tackle this problem, for which organic electrochemical transistors (OECT) have shown to be promising components. These transistors have the ability to control stable conductance states over a large range, which can be used to emulate the synaptic weight in artificial neural networks. This ability is an important component in achieving local learning for circumventing the von Neumann bottleneck, reducing operation power and reducing the size and weight of the centralized intelligent system. Conversely, the embodied intelligent robot confines the extensiveness of its control, as some behavior is inherent to the robot and does not have to be learned.

Our goal is to integrate local neuromorphic circuits in soft robotic systems. We present a reinforcement learning algorithm that makes use of two networks: the actor and pseudo-environment. With this algorithm we create a tool that allows communication between an embodied intelligent and centralized intelligent system, thereby moving closer to combined intelligence that the hummingbird shows when hunting for its prey.