COZMOBOT
Technical brief

The layer between a brain and a body.

nex-ON turns “understand the goal” into perceive, choose a tool, and act. Because perception, tooling and control are interfaces rather than fixed implementations, a new robot or a new skill is a registration — not a rebuild.

01 / architecture

Three layers, one contract between them.

Layer 01 — brain
Reasoning

A large language model reasons about the goal and orchestrates the work, deciding mid-conversation which capability to call next.

Claude Opus via LangChain
agentic multi-tool loop
spoken summary per action
Layer 02 — capabilities
Modular tools

Vision, measurement, motion, tooling and sensor feeds are swappable tools registered against an interface, not hard wiring.

swappable detector backend
tool-based capabilities
abstracted motion
Layer 03 — body
Any robot

Today a collaborative arm. The same orchestration is architected for welding cobots, humanoids, AMRs and mixed fleets.

live: collaborative arm
roadmap: humanoid · AMR
new body = integration
02 / tool registry

The capabilities the brain can call.

Class
Capability
Status
vision
Open-vocabulary detection — ask for any object in plain words, no per-class training
Live
vision
Depth-fused dimensioning — length, width and distance in millimetres
Live
vision
Seam profiling — depth-and-image scan inside an operator-drawn area of interest
Live
motion
Linear and joint motion with separate working and positioning speeds
Live
motion
Dry-run reachability check — feasibility reported before the arm moves
Live
motion
Per-axis motion locks — e.g. hold height constant
Live
tooling
Arc welding pass with weave: triangle, sine, circular, vertical
Live · gated
tooling
Colour-guided pathing — markers, dots and taped lines, shortest-path routes
Live
calib
Umeyama-fit hand-eye transform, camera to robot base
Live
voice
Locked-language speech in and out, optional barge-in
Live
Additional end-effectors, sensor classes and robot bodies
Architected
03 / safety model

Dangerous by exception, never by default.

The safety model is a property of the platform, not of the welding demo — it transfers to any actuated hardware we put underneath it.

Welding defaults to a dry pass — motion identical, nothing energised.
A live arc must be deliberately armed each session and never persists across a restart.
Reachability is checked as a dry run before any move executes.
Working and positioning speeds are separate, so a fast work speed cannot leak into a jog.
Speeds default low; per-axis locks constrain what the robot is allowed to change.
04 / stack

What it is built from.

Orchestration
Claude Opus via LangChain, agentic multi-tool loop
Voice
ElevenLabs Scribe speech-to-text and Flash TTS, streamed and sentence-buffered
Vision
Orbbec Gemini 336L RGB-D; Grounding DINO open-vocabulary detection, swappable
Geometry
Depth + PCA dimensioning, RGB+depth seam profiling
Robotics
Fairino collaborative arm via vendored SDK; IK reachability, torch-down orientation solving
Calibration
Umeyama-fit hand-eye camera-to-robot-base transform
Runtime
Python, standard USB depth camera and audio
Footprint
A standard USB depth camera, a mic and speakers. No training pipeline.
05 / operator runtime

A clean conversation, with the diagnostics behind it.

The operator sees only the exchange. Everything else — tool arguments, depth profiles, IK results, arc state — lands in timestamped logs for the engineer.

[ terminal / log screenshot ]
operator view beside the timestamped diagnostic log

Put it on a robot you already own.