LILS: Physical-to-Digital Traffic Simulation

Context

The Places and Flows Lab at BUas simulates traffic on Wilhelminastraat in Breda, projected onto a scale-printed (~1:300) physical model of the actual street. The goal was a tangible, demonstrable planning tool: rather than reading numbers off a screen, a stakeholder can place a real object on the map and watch the simulated traffic respond to it in real time. It turns an abstract traffic model into something you can reach out and interact with by hand, which makes it far easier for a non-technical audience to grasp what the simulation is actually doing.

I owned the computer-vision side, turning the physical world into an input the simulation could act on, while a teammate built the simulation engine itself. The two halves meet at a single contract: the CV pipeline decides where the obstructions are, and the simulation decides what the traffic does about them.

The system

A depth camera looks down at the map and translates physical objects into simulation events on a continuous cycle:

• Baseline sensing captures a depth reference of the empty street, then subtracts it each cycle so that only objects raised above the surface show up, leaving the printed map itself invisible to the detector.
• Object localisation runs blob detection over the height difference to find the centroid of each placed object, and a calibration homography maps that sensor-space position into real street coordinates.
• Simulation hand-off writes detected objects out as blockages the simulation polls and treats as obstructions; traffic stops at the blockage and backs up behind it, so a physical object becomes a live disruption on the projected map.

Because the map is both the input surface and the display surface, the whole loop is closed: you place an object, the camera sees it, and within a couple of seconds the projected traffic in that exact spot responds, with no screen or keyboard in between.

Calibration and integration

Most of the real difficulty lived in the seam between sensor space and the physical map. The depth camera sees the world in its own distorted coordinate frame, and getting a detection to land on the right part of the street meant solving several problems at once:

• Lens distortion had to be modelled with the full rational distortion model rather than a simplified one; dropping the higher-order coefficients caused the correction to diverge instead of flattening the image.
• A four-corner homography maps the flattened sensor image onto street coordinates, so a blob detected in the camera frame resolves to a real position on Wilhelminastraat.
• A building mask excludes the printed structures flanking the road, so the detector reacts only to objects placed on the street surface and not to the permanent geometry of the model.
• Configuration ordering mattered more than expected: the camera refuses depth-mode changes while streaming, so the pipeline reads its resolution and mode before the stream opens, otherwise it silently falls back to the wrong resolution and every downstream coordinate is off.

The lesson that stuck

Raising the camera by a few centimetres broke everything. Detection resolution, the reference baseline, the building mask and the four-corner calibration were all tied to the exact physical geometry of the mount. Nothing in the code had changed, but the setup had, and that was enough. In camera-driven systems the hardware placement is part of the system, which is why the pipeline is built around an explicit recalibration workflow rather than assuming a fixed rig. A resolution change alone invalidates every geometry-dependent piece of state at once, so the recapture steps are treated as a deliberate sequence rather than something to patch one field at a time.

Designed for a live demo

Because the end product is something a person operates in front of an audience, the interface was built around restraint rather than completeness. Control was reduced to a small handful of physical buttons that toggle between simulation modes and overlays, and a debug view exposes the raw depth feed, detections and calibration overlays for setup without cluttering the presentation itself. The single most valuable piece of feedback from the client was about exactly this: in a project like it the instinct is almost always to surface as much data as possible, when in fact constraining what the demo can show is what keeps it clear and intuitive.

What I took from it

This is my clearest example of computer vision touching the physical world in real time: sensing, calibration and integration under real hardware constraints, not a clean dataset in a notebook. It taught me how tightly a vision system is coupled to its physical setup, how much of the work is in the seams between camera, model and display rather than in any single algorithm, and how much a deliberately constrained interface can do for something that has to be understood at a glance.