Explainable Causal Reinforcement Learning for Bio-Inspired Soft Robotics Maintenance

This article discusses the author's journey in developing explainable causal reinforcement learning systems for maintaining carbon-negative infrastructure using bio-inspired soft robotics.

Details

The author's initial experiment with a deep reinforcement learning agent to control a simulated soft robotic gripper for inspecting bio-concrete surfaces failed when deployed to a physical testbed. This experience led the author to study causal inference, structural causal models, and hybrid neuro-symbolic systems, discovering that traditional reinforcement learning approaches lack the fundamental understanding of why actions lead to outcomes. The author found that combining causal reasoning with reinforcement learning can create systems that not only perform maintenance tasks but understand the underlying physical and biological processes they're intervening upon. This is crucial in the delicate ecosystem of carbon-negative infrastructure, where bio-inspired soft robots maintain living building materials. The article also discusses the technical background, including the causal revolution in reinforcement learning, the advantages of bio-inspired soft robotics, and the unique maintenance requirements of carbon-negative infrastructure. The author's exploration of structural causal models (SCMs) revealed they provide the mathematical framework needed to encode domain knowledge about maintenance environments and enable explainable causal reinforcement learning.