OPENGLASSUI OPENGLASSUI / GETTING STARTED
Your setup.
In view.
Connect wearables and desktop clients to coding-agent harnesses running on your computer or server.
Some integrations and physical-device flows still need end-to-end testing. The public demo uses fictional data and cannot connect to your agents.
01 / BEFORE YOU START
Requirements
- A computer or server with Node.js 20 or later, npm, and a project folder.
- At least one configured coding agent, installed and authenticated with its provider.
- For the tested glasses path: Meta Ray-Ban Display, a compatible phone with Meta AI, and access to Meta's web-app developer settings.
- A reachable HTTPS address for the gateway. A tunnel client such as Cloudflare is one option.
Availability of Meta features can vary by device, account, region and software version. Check Meta's web-app documentation for current requirements.
02 / COMPANION
Install on your computer
Clone the repository, install dependencies and create your local configuration:
git clone https://github.com/weisser-dev/openglassui.git
cd openglassui
npm ci
cp config.example.json config.json
npm startOn Windows, copy config.example.json to config.json using Explorer or your shell. You can also use setup.sh on macOS/Linux or setup.bat on Windows. These are setup scripts, not signed installers.
Open http://127.0.0.1:4173 on the computer running the companion. Edit config.json to set your workspace and agent executables. Agent credentials and permissions are managed by the agent and its provider.
03 / CONTROL CENTER
Complete the six-step setup
- Agents. Select one or more configured agents. An executable check does not verify your sign-in.
- Projects. For each agent, import selected discovered projects or add a project path manually. Only approved projects are made available to paired devices.
- Connection. Follow the selected provider’s installation instructions, start it or enter an existing HTTPS address pointing at the gateway. On macOS, setup can install cloudflared with Homebrew. Automatic startup is optional and disabled by default.
- Glasses. Generate the pairing QR code. It is single-use and expires after five minutes.
- Startup. Optionally start OpenGlassUI at login on macOS and start the selected tunnel automatically. Public tunnels expose the gateway; Quick Tunnel addresses change on restart.
- Finish. Open the app on your device, check the pairing, and open the dashboard. The setup can be reopened later.
Discovered sessions depend on what each agent exposes. Avoid resuming the same session concurrently in another client; live takeover is not currently reliable.
04 / PAIR A DEVICE
Connect the glasses
Generate the QR in your local setup. The GitHub Pages demo is not the installable glasses app.
- Pair your display glasses and phone in the Meta AI app and enable the developer/web-app options available to your account. Consult Meta's current web-app instructions.
- Start the companion and a public HTTPS tunnel to its gateway. In setup, choose Cloudflare Quick Tunnel or another available provider, or enter an existing HTTPS address.
- Generate the QR in OpenGlassUI, scan it with your phone and follow the Meta AI add-web-app flow. Open the app within five minutes.
- Keep the companion and tunnel running. Test navigation, text entry and the connection on the glasses; a successful phone test alone does not prove glasses access.
Manual Cloudflare example
cloudflared tunnel --url http://127.0.0.1:4174Enter the HTTPS address provided by cloudflared in setup. Quick Tunnel addresses are temporary and change when restarted, so the glasses app may need the new address and pairing. Forward only the gateway port, normally 4174—never the dashboard administration port 4173. See Cloudflare Quick Tunnels.
Other connection options
ngrok and Tailscale Funnel can be selected if installed and configured. ngrok requires an account and auth token; follow ngrok setup. Install and sign in to Tailscale before enabling Funnel. Test the public address on the glasses; access from a phone alone does not prove glasses connectivity. Tailscale is not required for Cloudflare or ngrok.
05 / COMPATIBILITY
Agents & model providers
| Agent | Available integration | Current limits |
|---|---|---|
| Codex | App-server adapter, sessions, supported decisions and model catalog. | Not every request type is supported; model access depends on your account. |
| Claude Code | Agent SDK, sessions and supported questions/approvals. | Real end-to-end authentication and model switching need further verification. |
| Kiro CLI | ACP V3, sessions, tool events and one-time permissions. | Reliable takeover of a running Kiro IDE session is not established. |
| OpenCode | Experimental ACP adapter and session discovery. | Real task execution, permissions, model changes and continuation need end-to-end verification. |
| Custom CLI | Configured executable receives prompts through stdin. | Raw output; no generic session discovery or interactive approvals. |
Agent Client Protocol: built in
ACP v1 support is included by default; no additional OpenGlassUI plugin is required. In setup, choose Add an ACP-compatible agent and enter the installed executable and its arguments as a JSON array. OpenGlassUI stores the entry locally and runs the executable without a shell. Only configure agents you trust, and do not place API keys in arguments. Session browsing and model selection depend on the agent advertising those capabilities; filesystem and terminal client RPCs are not implemented. Additional dedicated adapters can be contributed by pull request. See the agent-client interface.
LiteLLM and model providers
LiteLLM AI Gateway can route compatible agent requests to multiple configured model providers. Configure the gateway in your coding agent, then connect that agent to OpenGlassUI. LiteLLM does not by itself replace the agent harness or grant local tools, and OpenGlassUI does not currently configure LiteLLM or its credentials.
OpenCode can use providers configured within OpenCode, including OpenAI, Anthropic, Google, DeepSeek and compatible services. A model provider is not itself a coding-agent integration. See OpenCode provider setup.
06 / WEARABLES
Devices & input
| Device | Status |
|---|---|
| Meta Ray-Ban Display | Experimental target. Pairing and basic band navigation have been exercised; full functionality is not verified. |
| Meta Neural Band | Directional navigation and Select are supported; thumb + middle finger Back has been confirmed by a wearer. Test on your device. |
| Desktop browser | Use the fictional-data demo or the local companion's simulator. |
| Rokid and other glasses | No dedicated integration or device validation yet. |
Select a text field to open Meta's system composer for dictation or handwriting. OpenGlassUI cannot currently force the composer directly into dictation mode. Standard Web Push was unavailable on the tested glasses browser; in-app notices work while the app is open.
07 / TRUST & DATA
Privacy & security
OpenGlassUI is a local control surface. The connected agent manages its own models, authentication and permissions. OpenGlassUI also stores configuration, paired-device records, prompts, task output and conversation history locally in .openglass/ by default; this data is not encrypted at rest.
- Treat pairing QR codes and links like passwords. A paired device can access registered projects and tasks.
- Forward only the gateway, and keep the dashboard administration endpoint local.
- Review the agent's own permissions before using it remotely. Approval prompts are not guaranteed for every operation.
- The tunnel provider can carry task traffic. The current design is an experimental single-user setup, not a hardened multi-user service.
See the privacy overview and security and privacy details for the current limitations.
08 / TROUBLESHOOTING
When something does not connect
App does not load on the glasses
Check that the companion and tunnel are running. Open the HTTPS app URL on the phone, then test on the glasses. For Tailscale, phone access does not guarantee glasses access; try a public HTTPS tunnel for an initial test.
An agent or session is missing
Check that the executable is installed and authenticated on the companion. Select the agent in setup, then discover or approve its project. Existing clients and agent versions expose different session histories.
Background alerts are unavailable
The tested glasses browser did not expose the Notification and Push APIs. This does not prevent in-app notices while OpenGlassUI is open.
Need more detail?
See the README, architecture or open an issue.