Netherlands Plant Eco-phenotyping Centre Client
Automated Root Phenotyping & Inoculation
Built a computer-vision and reinforcement-learning system that segments plant roots and drives a liquid-handling robot to inoculate them, reaching 0.18mm targeting accuracy and cutting time per Petri dish 21×, from ~3.5 minutes to roughly 10 seconds.
Context
NPEC studies how plant genetics and environment interact, screening for robust, disease-resistant genotypes using automated growth modules. Those modules generate huge volumes of imagery, but the downstream analysis was still manual, creating a bottleneck where valuable genetic data was lost to fatigue, human error and sheer processing speed.
My task, as an end-to-end individual project, was to automate the inoculation step for Arabidopsis thaliana using computer vision, deep-learning segmentation and a simulated robotic workflow.
Approach
The system runs as a pipeline from raw image to robot movement:
- Segmentation: a SegFormer model classifies each pixel as root, shoot or seed, trained on hand-annotated, peer-reviewed labels with clean splits to avoid data leakage.
- Classical CV front-end: OpenCV colour-thresholding and contour detection isolate the Petri dish and standardise inputs before the model sees them.
- Robotic control: a Soft Actor-Critic agent, trained in a custom Gymnasium simulation of an Opentrons OT-2 robot, learns to drive the pipette to each target. Pixel coordinates from the vision model map to physical robot coordinates for end-to-end validation.
MLOps & experiment tracking
Every run was logged in Weights & Biases (loss curves, F1 scores and side-by-side model comparisons) on top of a baseline-first workflow, so each step was measured against the last and the whole development trail stayed reproducible.
Results
- Root segmentation F1 0.81, shoot segmentation F1 0.93.
- RL controller reached 0.18mm targeting accuracy at ~1.2s per plant, beating a PID baseline (0.32mm, ~3s).
- 21× faster per dish than a skilled scientist, with consistent, scalable throughput.
What I took from it
This project is the root of my interest in applied computer vision and ML that touches the physical world. It taught me how sensitive RL agents are to reward design, and how much careful annotation and integration logic matter once a model has to drive real hardware.