RoadReady

A phone-in-your-hands, AI-assisted mock driving test for practising how you read and respond to a real UK road.

Built for OpenAI Build Week with OpenAI Codex using GPT-5.6.

RoadReady turns a laptop and a phone into a lightweight driving-practice setup. The learner follows one uninterrupted two-minute front-POV route, holds the phone like a steering wheel, and performs observations, signalling and braking naturally. Browser-based computer vision and controller inputs are assessed silently against events in the route; feedback appears after the drive instead of giving away answers while it is happening.

Hackathon MVP: one route, one focused assessment loop, no special simulator hardware.

Learner drivers can memorise the theory but still struggle to coordinate observation, signalling and vehicle control at the right moment. Full simulators are expensive, ordinary driving videos are passive, and a checklist shown on screen can teach the answer rather than test whether the learner actually read the road.

RoadReady makes that practice more active and accessible:

• a real road video supplies the visual context;
• the learner responds with their head, hands and phone;
• the assessment runs in the background without on-screen instructions;
• the final report shows whether the expected safety actions were detected.

It is designed as a repeatable practice aid between lessons—not as a replacement for an approved driving instructor or an official test.

• One continuous UK road route — a two-minute front-facing Sidcup driving sequence embedded from YouTube, without procedural graphics or user-facing stages.
• Silent route-aware assessment — fixed time windows correspond to known events in the selected footage. During the drive, RoadReady checks the learner's actions without displaying prompts, answers or assessment criteria.
• On-device observation detection — MediaPipe Face Landmarker and Pose Landmarker run in the browser to detect head direction and simple movement signals. Webcam frames are not uploaded to an AI service.
• Phone-as-wheel controller — hold a phone in landscape and rotate it like a steering wheel. Swipe down for the left indicator, swipe up for the right indicator, and hold both lower grips to apply the brake. Touch controls are available as a fallback.
• End-of-route feedback — observations, steering, indicators and braking are combined into a score, driving faults and serious faults.
• Optional voice — browser speech guidance is off by default and can be enabled when accessibility or demo conditions call for it.
• Demo assist — an explicit presentation fallback that feeds successful inputs through the same scoring path when camera permission, phone sensors or venue connectivity are unreliable. It also compresses the five hidden assessment windows to about 40 seconds so a complete result can be shown inside a 90-second hackathon demo; the normal learner route follows the full two-minute footage.

The submitted video should remain a screen recording of the working product—no slides. Turn on the visibly labelled Demo assist before starting; it compresses the assessment timing so the complete working flow fits below 90 seconds without cutting away from the product.

Phone controller ── POST /api/controller/:session ──┐
                                                    ├── Assessment state machine ── Final report
Webcam ── MediaPipe face + pose detection ──────────┤
                                                    │
YouTube UK front-POV route ── fixed event windows ──┘

1. The desktop creates a short controller session and displays its URL as a QR code.
2. The phone sends steering angle, indicator state and brake state to the same Next.js app. The desktop polls the current in-memory controller state.
3. MediaPipe models process webcam frames locally and emit observation signals such as left/right head direction.
4. A small state machine divides the continuous two-minute video into hidden event windows. Only actions relevant to the active window can satisfy its assessment criteria.
5. At the end, missed expectations are classified and converted into a simple prototype score and report.

• Next.js 16 App Router, React 19 and TypeScript
• Tailwind CSS
• MediaPipe Tasks Vision (Face Landmarker and Pose Landmarker)
• Device Orientation API for motion steering
• Browser SpeechSynthesis for optional voice
• Same-origin Next.js route handlers and an in-memory session store for controller relay
• YouTube iframe playback for the front-facing road sequence

No hosted generative-AI API is called while a learner is driving. The runtime “AI” component is the browser-based vision model used to interpret observations and movement.

RoadReady was built inside OpenAI Codex with GPT-5.6 as the primary coding model. GPT-5.6 reasoned across the desktop experience, phone controller, camera pipeline and assessment state machine; Codex provided the repository inspection, editing, browser testing and shell-validation loop that turned those decisions into a working product. Together they materially helped to:

• turn the product idea into an end-to-end Next.js prototype spanning desktop, camera analysis and a separate mobile controller;
• implement and iterate the MediaPipe face/pose pipeline, including camera lifecycle handling and GPU-to-CPU fallback;
• build the phone interaction model: orientation steering, swipe indicators, two-grip braking and touch fallbacks;
• diagnose a reversed phone-steering direction and add regression coverage for orientation maths, including the 0°/360° boundary;
• repeatedly compare the route footage with assessment timing, then simplify the MVP from multiple scenes to one continuous route with hidden scoring windows;
• refine the demo UX, camera failure messages, privacy copy and production-ready documentation;
• run linting, tests and production builds while iterating on reported issues.

The final product decisions—especially the single-route scope, silent assessment and physical phone gestures—were directed through rapid human feedback and tested in the working interface.

GPT-5.6 is used in the build workflow rather than called by learners at runtime. During a drive, the only ML inference is the local MediaPipe vision pipeline described above; this keeps webcam frames on the learner's device and makes the prototype runnable without an API key.

• Node.js 20.9 or newer
• A modern browser with camera support
• Optional: a phone on the same network for the controller

npm install
npm run dev

Open http://localhost:3000/?variant=A, select Enable camera, pair a phone if available, and select Start route.

To expose the development server on your local network:

npm run dev -- --hostname 0.0.0.0

The desktop setup screen generates a QR code for the controller. A laptop-only fallback is also available: use the on-screen controls, or use the arrow keys to steer, Q / E for indicators and Space to hold the brake.

npm run lint
npm run build
npm run start

Mobile motion sensors require a secure browser context. Opening a controller at a plain LAN address such as http://192.168.x.x:3000 may load the touch UI, but iOS and other browsers will block orientation steering.

For full phone controls:

1. Serve the app from a trusted HTTPS origin (for example, an HTTPS deployment or tunnel).
2. Open that same HTTPS origin on the desktop so the QR code also points to HTTPS.
3. Hold the phone in landscape.
4. Tap Enable motion steering—iOS requires this user gesture before it shows the sensor-permission request.
5. Keep both devices on a stable connection and recalibrate with the phone centred if necessary.

localhost is treated as secure for desktop development, but that exception does not make a phone's LAN URL secure. When HTTPS or sensor permission is unavailable, use the controller's touch-steering fallback or Demo assist.

• Webcam analysis happens locally in the learner's browser using bundled MediaPipe models.
• RoadReady does not record or intentionally upload webcam frames.
• Controller inputs are relayed to the same app and held only in an in-memory session; the prototype does not persist a driving history or user profile.
• Voice guidance uses the browser's speech synthesis and is off by default.
• The road sequence is played through a YouTube embed, so YouTube receives the normal requests associated with embedded playback.
• Demo assist is visibly labelled and intended for presentation resilience, not for producing a genuine learner assessment.

• The MVP is calibrated to one specific two-minute video sequence. Assessment windows are authored against its timeline; RoadReady does not yet understand arbitrary road video in real time.
• Vision signals are heuristic and can be affected by lighting, camera angle, occlusion, glasses, movement speed and individual posture.
• Phone orientation behaviour and permission flows vary by device and browser; motion steering requires HTTPS.
• The controller relay uses one fixed demo session and an in-memory store. It is suitable for a hackathon demo, not concurrent production users.
• Browser autoplay policy or YouTube availability may affect playback; the learner starts the route with a user gesture and the embed is muted by default.
• The phone simulates intent and control inputs; it is not a physics-accurate vehicle simulator and provides no force feedback.
• Scoring is a prototype teaching signal, not a validated measure of driving competence.

RoadReady is an experimental educational prototype. It is not approved by the DVSA, does not reproduce an official UK practical driving test, and must not be used to decide whether someone is safe or legally qualified to drive. Learners should follow the Highway Code and practise on public roads only with the legally required supervision, insurance and suitable professional instruction.

• The prototype embeds a time-limited excerpt from this front-facing UK driving video on YouTube.
• RoadReady does not bundle or redistribute that source video. The footage remains owned by its publisher and subject to YouTube's terms.

RoadReady's source code is released under the MIT License. The third-party video shown through the YouTube embed is not part of the repository and is not covered by that license.