Home

Chapter 01 / 8Four cooperating subsystems

Implemented

1019 words · 6 min read · Arsh Shah

System

A system that listens to the stroke

RowSim is not a screen attached to a rowing machine. It is a continuous loop: motion becomes data, data becomes force, and force becomes a visible wake beneath the athlete. The installation is organised as four cooperating subsystems—GPU fluid and wave simulation, host-side sensor fusion and rowing physics, transport-agnostic device ingestion, and perceptually oriented visualization—so that sensing complexity never rewrites the fluid core.

Rendering and orchestration stay apart

The shared Renderer owns Metal resources and performs one simulation-and-draw cycle per frame. It contains no application logic beyond the simulation and presentation passes. The macOS RenderViewController owns the higher-level meaning of the session: rowing mode, stroke arbitration, PM5 coupling, and distance tracking—everything that decides what a stroke means before the GPU is asked to paint water. That split is the difference between a maintainable real-time installation and a renderer that has absorbed every sensor quirk in the building.

Input arrives through an InputRouter that fans in one or more InputProvider implementations and forwards a single typed InputEvent stream to the view controller, which implements the InputEventSink protocol in SensorInput. Providers converge through that vocabulary; the renderer never reaches back into the application. It receives uniform buffers written on the main thread before each frame, then walks its seventeen-stage graph without asking how a packet arrived or which BLE characteristic last fired.

That one-way dependency—providers → router → sink → Renderer—is what keeps sensing complexity from contaminating the fluid core. Adding a sensor becomes a provider problem, not a rewrite of the solver or the visualization pass. Directionality is the primary architectural claim: the wake stays continuous because ownership stays boring.

Dependency direction: providers → router → sink → renderer.

One fluid core, two Apple hosts

The software is organised as a shared Metal rendering core that runs on both macOS and iOS, a macOS host that owns sensor ingestion and rowing physics, a thin iOS host that supports direct touch interaction only, and a Python telemetry bridge that permits libmapper-driven input without the native stack. On macOS, the full input stack is present: InputRouter plus native libmapper or UDP, with optional PM5 BLE wired beside the mapper path so ergometer telemetry can own the stroke when connected.

iOS is intentionally thinner. Touch interaction—up to five simultaneous touches—drives the same shared Metal renderer without the ergometer, mapper, or UDP provider stacks. The iOS RenderViewController automatically pauses and resumes rendering on background and foreground transitions, which is the right lifecycle contract for a handheld host, but it does not pretend to be a second installation stack. Deployment floors differ as well: macOS 26.0 and iOS 17.0, with Metal required on iOS.

This split keeps platform-specific sensing outside the simulation while preserving a consistent visual system. The wake looks like the same water whether the disturbance arrives from a Concept2 stroke or a fingertip, because both hosts ultimately write into the same renderer uniforms and slabs. The architectural bet is that shared fluid truth plus host-local orchestration is more honest than a single binary that quietly disables half its sensing path at runtime.

Hosts · local sensingShared fluid coreshared uniforms ↓01macOS hostSensing · arbitration02iOS hostTouch · lifecycle03Shared Metal coreOne sim-and-draw cycle

Platform split

One fluid core, two hosts.

Sibling hosts write shared uniforms into one Metal core — not a macOS→iOS pipeline.

macOS hostOwns sensing, stroke meaning, and session orchestration before uniforms reach Metal.

Fig. 01.2

Sibling macOS and iOS hosts write shared uniforms into one Metal fluid core; sensing stays host-local.

Sensing, host physics, Metal, projection

The implementation can be understood as four cooperating subsystems. A GPU fluid and wave simulation pipeline built on Apple’s Metal API owns the velocity, density, ripple, bloom, and particle fields and advances them through a fixed per-frame order. A host-side input, sensor-fusion, and rowing-physics layer turns heterogeneous streams into stroke meaning, displacement-trail geometry, boat velocity targets, and the uniform payload the renderer will consume on the next draw.

A networking and device-ingestion layer normalises libmapper, UDP, and PM5-BLE feeds into one event vocabulary before they touch arbitration. Native mapper polling, BLE callbacks, and bridge activity may run on distinct queues, but every event is copied, normalised, and re-dispatched onto the main thread before it mutates simulation state. The component diagram makes the same claim with a dashed Python-bridge edge that still ends in the host vocabulary.

A perceptually oriented visualization and post-processing stage then composes obstacle masks, base fluid shading, optional psychedelic palette transforms, refraction, background blend, azimuth brightness bias, bloom, particles, motion blur, and the boat overlay into the drawable that the projector puts underfoot. Floor projection and the typed event boundary are the product claims; psychedelic palettes, particles, background image, and motion blur remain optional layers. Remove the floor, and the system collapses back into a dashboard. Remove the typed event boundary, and sensing begins to rewrite the solver.

InputRouteHostMetalFloorPM5 BLELocked snapshotslibmapperNative · macOSUDP bridgeSame eventsPython bridgeInputRouterFan-in · typed streamHost physicsStroke · uniformsMetal rendererSim-and-draw cycleFloor projectionEmbodied feedback

Live architecture map

From signal to floor.

Providers → router → host → Metal → floor. Dashed satellite: Python bridge into UDP.

InputPM5, native libmapper, and UDP (fed by the Python bridge) normalize into one event vocabulary.

Fig. 01.3

Three providers fan into InputRouter → host physics → Metal → floor projection. The dashed satellite is the Python bridge feeding UDP.

The display is the floor

RowSim projects onto the ground plane around the ergometer so the athlete’s body remains inside the feedback. Numbers migrate off a dashboard and into ambient light that moves with the stroke. Force and rhythm become field perturbations; the visual response is spatially distributed and temporally persistent. Users perceive kinesthetic-to-visual causality through fluid advection and wake traces rather than through discrete command acknowledgements on a monitor above the machine.

The architectural bet is simple: if attention stays in the act of rowing, the simulation can teach without demanding constant visual checkout. Displacement and velocity become oar-trail wakes; PM5 drive and recovery become gated propulsion; IMU roll and azimuth become ripple impulses and brightness bias. On the study path, watts scaled trail ink and peak force scaled impulse; radius was computed from watts and then clamped to 95, so it did not vary. Those mappings only make sense if the display plane is the floor rather than a secondary screen competing for gaze.

Because the display is the floor, evaluating RowSim means evaluating an embodied mixed-reality system—not a 2D UI prototype. The product claim stays the continuous loop above, not a feature list.

One continuous wake

The product claim is the loop itself: providers emit events; the sink mutates meaning on the main thread; the renderer consumes prepared uniforms without reaching back into sensing; the floor keeps the athlete inside the feedback. Even the Python ingestion path ends at UDP into the host vocabulary—not a second fluid authority. Later chapters unpack each contract without restating the whole stack.

Works cited

Numbers match the thesis bibliography. Locators such as ch. 4–5 name chapters of that argument.

  1. [127]

    Mark Weiser (1991).

    The computer for the 21st century.

    Scientific American, 265(3), 94–104.

    doi:10.1038/scientificamerican0991-94

  2. [128]

    Mark Weiser and John Seely Brown (1996).

    Designing calm technology.

    Technical report, Xerox PARC.

  3. [130]

    Craig Wisneski, Hiroshi Ishii, Andrew Dahley, Matt Gorbet, Scott Brave, Brygg Ullmer, and Paul Yarin (1998).

    Ambient displays: Turning architectural space into an interface between people and digital information.

    Cooperative Buildings (CoBuild ’98). Springer.

    doi:10.1007/3-540-69706-3_4

  4. [081]

    Paul Milgram and Fumio Kishino (1994).

    A taxonomy of mixed reality visual displays.

    IEICE Transactions on Information and Systems, E77-D(12), 1321–1329.

  5. [Th.]

    Arsh Shah (2026).

    RowSim: Designing and Evaluating Ambient Interaction in Mixed-Reality Rowing.

    Master’s thesis, Dalhousie University, Halifax, NS.

    ch. 3Thesis PDF