shawnpana/phone-harness

Phone Harness lets an AI agent like Claude Code or Codex actually control a real iPhone through your Mac, without jailbreaking the phone or installing any special developer tools on it. It works by taking over the Mac's built-in iPhone Mirroring window, which shows your phone's screen live and forwards mouse and keyboard input to it as real touches. The agent sees the phone by taking a screenshot of that mirroring window and running Apple's Vision framework on it to read every piece of visible text along with its exact screen position, similar to how a web agent might read a page's structure. It acts on the phone by sending low-level input events, taps, long presses, drags, scrolls, and typed text, directly to that same window. After acting, it takes another screenshot to confirm the change actually happened, since there is no other way to inspect the phone's state. Setting it up mostly means running a guided prompt inside an agent like Claude Code, which walks you through cloning the project, installing it as a command, and registering it so the agent automatically reaches for it. Two steps require you personally: pairing iPhone Mirroring with your physical phone once, and granting the Mac's terminal permission for Accessibility and Screen Recording in System Settings. A built-in doctor command checks that everything is connected correctly. The project is honest about what does not work. Older automation approaches like AppleScript clicks are silently ignored because the mirrored window has no accessibility information, only a video feed. Typed text has to use raw keyboard codes rather than Unicode characters, slow drags barely register on iOS lists so scrolling needs a proper flick or wheel gesture, and input is ignored if the mirroring window is not the frontmost window on screen. It has real limits too: only one phone and one session at a time, no multi-touch gestures like pinching, no camera or Face ID flows, and DRM protected video shows up as a black screen since it cannot be captured.