Home
ROWSIM

HomeChapter 01 / 8Spatial augmented reality on the floor

Implemented

284 words · 2 min read · Arsh Shah

System

A system that listens to the stroke

RowSim belongs to spatial augmented reality: imagery on the physical surfaces of a room, not through a worn display.[013][097] An abstract, fluid-like field is projected onto the floor in the rower’s lower periphery, coupled to the stroke — one fluid core, two Apple hosts, four subsystems.

Effort without a changing scene

A rowing ergometer keeps the work of locomotion and removes the changing scene. Into that emptiness the machine inserts a console — split, watts, rate — an encoding that must be read.[048][128] RowSim asks whether a floor-projected fluid can occupy the periphery instead, without becoming another task inside the stroke.[128][Th.]

Hosts · local sensingShared fluid coreshared uniforms ↓01macOS hostSensing · arbitration02iOS hostTouch · lifecycle03Shared Metal coreOne sim-and-draw cycle

Platform split

One fluid core, two hosts.

macOS hostSensors, stroke meaning, and the rowing-physics host stay here. Uniforms leave for Metal.

Fig. 01.1

Sibling macOS and iOS hosts write shared uniforms into one Metal fluid core; sensing stays host-local.

One fluid core, two Apple hosts

The two hosts are siblings, not a pipeline. macOS listens: sensors, stroke meaning, the rowing-physics host. iOS is touch and lifecycle only. Both write the same Metal uniforms into one shared fluid core, so the field is one fluid even when the machines stay local. Whoever hears the stroke does not also have to paint it.

Sensing, host physics, Metal, projection

Providers converge on a typed event, the host decides what a stroke means, Metal paints the field, and two projectors put it on the floor. Directionality is the claim: sensing never rewrites the solver. Wisneski et al. turned architectural space into an interface;[130] here the architecture is a blackout room and a sheet, and the interface is on the floor rather than in the focal cone.[081][013][097]

InputRouteHostMetalFloorPM5 BLELocked snapshotslibmapperNative · macOSUDP bridgeSame eventsPython bridgeInputRouterFan-in · typed streamHost physicsStroke · uniformsMetal rendererSim-and-draw cycleFloor projectionEmbodied feedback

Live architecture map

From signal to floor.

Providers → router → host → Metal → floor. Dashed satellite: Python bridge into UDP.

Input — PM5, native libmapper, and UDP (fed by the Python bridge) normalize into one event vocabulary.

Fig. 01.3

Three providers fan into InputRouter → host physics → Metal → floor projection. The dashed satellite is the Python bridge feeding UDP.

Five parts of one floor

The fluid is on the sheet. The hull is the unlit cutout. The ergometer is the boat. The host sits to the side. Two projectors hang above. That is the mixed-reality register: not a headset, a floor.[081]

Fig. 01.4

Five parts of the floor-MR room: floor projection, hull mask, ergometer, host, and the two projectors.

Works cited

Superscripts use the thesis bibliography number. Click a number to return to the first mention.

  1. [048]

    James J. Gibson (1979).

    The Ecological Approach to Visual Perception.

    Houghton Mifflin.

  2. [128]

    William H. Warren (2006).

    The dynamics of perception and action.

    Psychological Review, 113(2), 358–389.

    doi:10.1037/0033-295x.113.2.358

  3. [127]

    Mark Weiser (1991).

    The computer for the 21st century.

    Scientific American, 265(3), 94–104.

    doi:10.1038/scientificamerican0991-94

  4. [128]

    Mark Weiser and John Seely Brown (1996).

    Designing calm technology.

    Technical report, Xerox PARC.

  5. [025]

    William Buxton (1995).

    Integrating the periphery and context: A new model of telematics.

    Proceedings of Graphics Interface ’95, pp. 239–246.

  6. [009]

    Saskia Bakker, Elise van den Hoven, and Berry Eggen (2015).

    Peripheral interaction: characteristics and considerations.

    Personal and Ubiquitous Computing, 19, 239–254.

    doi:10.1007/s00779-014-0775-2

  7. [130]

    Craig Wisneski, Hiroshi Ishii, Andrew Dahley, Matt Gorbet, Scott Brave, Brygg Ullmer, and Paul Yarin (1998).

    Ambient displays: Turning architectural space into an interface between people and digital information.

    Cooperative Buildings (CoBuild ’98). Springer.

    doi:10.1007/3-540-69706-3_4

  8. [081]

    Paul Milgram and Fumio Kishino (1994).

    A taxonomy of mixed reality visual displays.

    IEICE Transactions on Information and Systems, E77-D(12), 1321–1329.

  9. [013]

    Oliver Bimber and Ramesh Raskar (2005).

    Spatial Augmented Reality: Merging Real and Virtual Worlds.

    A K Peters / CRC Press.

  10. [097]

    Ramesh Raskar, Greg Welch, Matt Cutts, Adam Lake, Lev Stesin, and Henry Fuchs (1998).

    The office of the future: a unified approach to image-based modeling and spatially immersive displays.

    Proceedings of SIGGRAPH ’98, pp. 179–188. ACM.

    doi:10.1145/280814.280861

  11. [Th.]

    Arsh Shah (2026).

    RowSim: Designing and Evaluating Ambient Interaction in Mixed-Reality Rowing.

    Master’s thesis, Dalhousie University, Halifax, NS.

    ch. 1, 3hdl.handle.net/10222/86331PDF