Infron Open-Sources iPhone Use with Codex

AI coding tools can generate an iOS view quickly. That is only the beginning of a working application.

A complete development loop must still answer practical questions. Does the project compile with the current Xcode toolchain? Is signing configured correctly? Can the app be installed and launched on a physical device? Can an agent inspect the current UI, perform an interaction and verify the resulting state?

Infron built and open-sourced iPhone Use with Codex to make that loop observable, executable and repeatable.

The physical-device loop

The repository follows one complete sequence:

write code → build and sign → install and launch → inspect the UI → tap, drag and type → verify the result

Codex edits and coordinates the project. Xcode’s command-line tools build, sign and deploy it. XCTest supplies the accessibility hierarchy, screenshots and UI interactions used to observe and control the physical iPhone.

The repository packages those steps behind scripts/iphone.py. A first validation run uses:

Before running these commands, complete the first-run setup, including device pairing, Developer Mode, local team and bundle configuration, development signing and keychain authorization.

A successful build is not treated as proof that the application was installed or that an interaction succeeded. Each stage has its own checkpoint and evidence.

BUILD SUCCEEDED alone is not proof of installation or interaction. The workflow records separate checkpoints: engine checks pass; the app and UI tests compile; signed deployment installs and launches the app; and the physical-device regression reports one test, zero failures and three screenshot attachments. The published validation record names the tested environment, actual executions and limits; each reader’s logs and screenshots remain local under build/.

The published results come from one physical iPhone 13 mini running iOS 26.7.1 with Xcode 27.1 (27A9275). They do not establish support for every iPhone, iOS release, Xcode version or app.

A reusable XCTest control bridge

For interactive control, a persistent XCTest bridge receives JSON actions from Python. It can inspect the accessibility hierarchy, collect screenshots and perform supported taps, drags and text input.

The protocol includes session identifiers, sequential action identifiers and pending-state handling. These controls matter when an action times out: an unknown result must be collected before deciding whether a retry is safe. Blindly resending a UI action can produce duplicate or unintended effects.

This is why we describe the project as a physical-device automation harness or toolchain. It combines an agent, build tooling, a device runner and explicit verification boundaries.

It is not a hosted device-control API or a standalone MCP server.

Xcode MCP is optional

Xcode MCP can provide additional IDE-facing tools, and the documentation explains how to connect it separately. It is not required for the core workflow.

The shell and XCTest route works with Codex, the Xcode command-line tools and the Python standard library. No jailbreak, Appium or WebDriverAgent is required for the documented path.

That separation is useful for reproducibility. The build, deployment and UI-test commands remain inspectable even when an IDE integration is unavailable.

What the project includes

The repository contains:

• a complete Xcode project and physical-device UI test;

• reusable build, deployment and interaction commands;

• a persistent XCTest bridge template for cross-app control;

• offline protocol and deterministic game checks;

• screenshots and logs for validation;

• English and Chinese setup, control, troubleshooting and first-run documentation.

The included example evolved from a SwiftUI Hello World application into a small game, followed by physical-device UI automation. The example exists to demonstrate the workflow; the reusable value is the control and verification structure around it.

The limits are part of the design

“Control” does not mean unrestricted access to iOS.

Initial device pairing, Developer Mode, Apple Account access, signing and keychain authorization require the device owner. The phone must be connected, prepared and unlocked when the relevant step requires it. Protected screens or applications may restrict automation, and behavior can depend on the iOS, Xcode, device and target-app versions.

Those boundaries are documented because a trustworthy agent workflow must distinguish between actions it completed, actions it could not perform and actions that require human authorization.

Closing the gap between generation and verification

The useful unit of AI-assisted development is not a generated code block. It is a verified change in a real environment.

iPhone Use with Codex provides an open implementation of that idea for physical iOS development. It connects coding, build tooling, deployment, observation, interaction and verification without hiding the points where the device owner must remain in control.

Explore the code, bilingual documentation and complete first-run walkthrough:

github.com/InfronAI/iPhone-Use-with-Codex

Claim → evidence → reproducible outcome: start with the open-source repository, complete the setup guide, and run the exact first-run walkthrough on a compatible spare iPhone. Compare every stated result with fresh local evidence rather than a plan, an old screenshot or a successful build alone.

FAQ

What environment did Infron validate?

The published validation used one physical iPhone 13 mini running iOS 26.7.1 with Xcode 27.1 (27A9275). Eight offline protocol checks passed; one physical-device UI test completed with zero failures and three screenshot attachments. These results do not establish support for every iPhone, iOS release, Xcode version or app.

Is iPhone Use with Codex a standalone MCP server?

No. It is a physical-device automation harness built around Codex, Xcode command-line tools, XCTest and the Python entry point scripts/iphone.py. It is not a hosted device-control API.

Is Xcode MCP required?

No. Xcode MCP is an optional enhancement for IDE-facing tools. The core workflow uses a full Xcode installation, Xcode command-line tools, XCTest and the Python standard library.

What must the device owner configure manually?

The device owner must complete initial pairing, enable Developer Mode, provide Apple Account access, configure development signing and approve keychain authorization. The iPhone must also be connected, prepared and unlocked when the relevant step requires it.

What can the workflow do on a physical iPhone?

It can build, sign, install and launch the included app; inspect the accessibility hierarchy and screenshots; perform supported taps, drags and text input; activate selected apps where iOS and XCTest permit; and return structured evidence for verification.

Can it control every iOS screen or application?

No. Protected system screens and some applications may restrict automation, and behavior depends on the device, iOS, Xcode and target-app versions. The project does not provide unrestricted access to iOS.

Where should I report setup friction?

Run the first physical-device walkthrough on a compatible spare iPhone, then open a GitHub issue with the step that still causes friction.