Neural Cellular Automata

Growing Neural Cellular Automata

Differentiable Model of Morphogenesis, Feb. 11, 2020. doi: 10.23915/distill.00023

Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, Michael Levin

Neural Cellular Automata (NCA) are used here to simulate individual cells. In particular, the paper demonstrated the capacity of NCA to model multicellular morphogenesis, or how a single cell can split, differentiate, and become a complete living being. To achieve behaviour resembling real biology, the following techniques are used:

• Convolutional Network architecture: The neural net is trained such that all cells in the simulation have the same model, similar to how all biological cells in an organism have the same DNA.
• Hidden channels: The state of a cell is represented as a 16 channel vector, of which only the first 4 are visualised. The 12 other channels are intended to simulate inter-cell communication such as hormone signalling or electrical pulses
• Alive masking: the alpha value (4th channel) is used to encode whether the cell is alive. Cells deemed as "dead" do not update. The simulation begins with a single live cell.
• Stochastic updates: cell updates trigger randomly, mimicking how biological cells may split or change at different times.

Self-Organising Textures

Neural Cellular Automata Model of Pattern Formation, Feb. 11, 2021. doi: 10.23915/distill.00027.003

Eyvind Niklasson, Alexander Mordvintsev, Ettore Randazzo, Michael Levin

NCA are used to generate cohesive textures, rather than specific complex images. The results demonstrate how NCA are suitable for mimicking and replicating patterns. In particular, images generated by NCA share features and similarities with the original image without being pixel perfect copies. Further, due to the incremental updating nature of the simulation, NCA continuously generate smooth animations from a single image.

The unique contribution of this paper is primarily its success using pre-trained CNN (VGG-16) as the loss function for NCA.

Below are the results from our re-implementation:

NCA algorithm in live operation after training.

Targeted texture pattern for the NCA to replicate.

Mesh Neural Cellular Automata

Jul. 19, 2024. doi: 10.1145/3658127

Ehsan Pajouheshgar, Yitao Xu, Alexander Mordvintsev, Eyvind Niklasson, Tong Zhang, Sabine Süsstrunk

This paper focuses on 3D texture generation, moving NCA onto a mesh or graph. Each cell is now a node, with its neighbours defined by graph connections. This allows those textures to seamlessly wrap around 3D objects. In the demonstrations from this paper, there are still some flaws in the sharpest corners, however overall the resultant texture is smooth and continuous.

DyNCA

Dynamic Texture Synthesis Using Neural Cellular Automata, Mar. 30, 2023. doi: 10.48550/arXiv.2211.11417

Ehsan Pajouheshgar, Yitao Xu, Tong Zhang, Sabine Süsstrunk

This paper takes NCA into the video generation realm. Also based off Self Organising Textures this paper aimed to create dynamic directable video-like textures. While this technique was not suited for fast motion scenes, it performed well for scenarios such as fire and rising smoke, suitable for background textures.

Med-NCA

Robust and Lightweight Segmentation with Neural Cellular Automata, Feb. 7, 2023. doi: 10.48550/arXiv.2211.11417

John Kalkhof, Camila González & Anirban Mukhopadhyay

Med-NCA is an image segmentation model, which builds off another proceeding paper Image segmentation via Cellular Automata (Sandler M. et al) and is a part of a series of papers that investigate Neural Cellular Automata as a low-resource medical segmentation model. Med-NCA is the first paper to pose NCA as a low memory solution for image segmentation, due to the small model sizes, and demonstrates than NCA can effectively segmentation medical grade images while running on Raspberry Pi Zero.

Med-NCA's most notable contribution though is usage of a two stage architecture that includes a low and high resolution layer, which highlights the issue that NCAs struggle with long-distance communication of information across images, as this architectural change sped up the image segmentation process drastically.

The authors follow-up this paper by taking NCA to 3D medical scans in M3D-NCA, Robust 3D Segmentation with built in quality control.