The iOS Simulator feedback loop: build, see, point, verify with Claude Code (2026)

Claude Code just told you the SwiftUI fix is in and the build is green — and the button is still eight points off. Nothing failed, because nothing was looking: by default your agent has never seen the iOS Simulator, so "done" means "compiles", not "looks right".

This post maps the full feedback loop for iOS work with a coding agent — build, see, point, verify — and compares the tools that exist for each stage: ios-simulator-mcp, XcodeBuildMCP, and Claude Code's built-in computer use. Then it covers the part none of them do: letting you point at the element that's wrong.

"It compiles" is a claim about your code. "It looks right" is a claim about pixels — and an agent that never sees the Simulator can only verify the first.

Why "it compiles" isn't "it looks right"

SwiftUI moved most layout bugs out of the compiler's jurisdiction. Modifier order silently changes meaning — .padding().background(...) and .background(...).padding() both compile and render differently. A Spacer in the wrong stack, a frame fighting a GeometryReader, a text style that truncates only with dynamic type cranked up: all of it builds clean.

Xcode's answer is Previews — a live render loop for humans. But your agent doesn't sit in front of Xcode. When Claude Code edits a view, its natural definition of done is "the build succeeded and the code looks plausible". It will happily iterate three times on the wrong fix because nothing in its loop ever renders a frame.

That's uniquely painful for SwiftUI because the framework is about appearance. An off-by-one in a parser shows up in a failing test. An off-by-eight in an HStack shows up only on screen, and the screen is exactly what the agent can't see.

So the real loop for UI work has four stages: build (does it compile and launch), see (what did it render), point (which element is wrong), verify (did the fix change what I pointed at). Most setups automate stage one and leave you doing stages two through four by hand.

The manual loop today: screenshot, drag, describe

Here's the workaround every iOS dev using Claude Code has invented independently. Something looks wrong in the Simulator, so:

xcrun simctl io booted screenshot ~/Desktop/sim.png

Then you drag sim.png into the terminal, and then you write a paragraph: "the pay button in the checkout screen — the blue one, third element down — isn't aligned with the card fields above it, it's shifted right." The agent squints at pixels and greps for anything named "pay".

Three costs stack up:

Time. Screenshot, locate file, drag, describe, wait. A minute or more per round trip, and a UI-heavy session has many round trips.

Ambiguity. "The third cell" is a spatial claim rendered in prose. The agent has to reverse it back into space, then into code. Every hop loses precision — we've written about the general version of this problem in how to give Claude Code eyes.

Resolution. A Retina Simulator screenshot at 2x or 3x scale comfortably exceeds the ~1,568px long edge that Claude's vision system works best at, so it gets downscaled before the model sees it, and costs roughly width×height/750 tokens either way. The model is reasoning about a shrunken copy of a scaled window — fine for "there's a button", shaky for "it's 8 points off".

The manual loop works. It's just slow, lossy, and entirely on you. The obvious next step is giving the agent tools.

The tool landscape compared: ios-simulator-mcp vs XcodeBuildMCP vs computer use

Three tools dominate the "let Claude Code touch the Simulator" conversation in 2026. They're less interchangeable than they look.

ios-simulator-mcp (Joshua Yoes, MIT) is the focused option: an MCP server that inspects and drives a booted simulator. It exposes tools like ui_describe_all and ui_describe_point for accessibility info, ui_tap, ui_type, ui_swipe for interaction, plus screenshot and record_video. Setup is one line — claude mcp add ios-simulator npx ios-simulator-mcp — but it requires Facebook's idb installed as a prerequisite, and it does not build anything. Your app has to get onto the simulator some other way.

XcodeBuildMCP (now maintained under Sentry's GitHub org) is the heavyweight: 82 tools covering the whole Xcode workflow — building, running, and testing on simulators, physical devices over USB/Wi-Fi, and macOS, plus log capture and LLDB debugging. Its UI layer bundles AXe, a CLI for simulator automation: the snapshot_ui tool runs AXe's describe-ui under the hood and returns the simulated app's accessibility hierarchy with frame coordinates, alongside tap, swipe, type_text, gesture, and screenshot. No idb needed.

Claude Code's built-in computer use (research preview, March 2026) is the generalist: Claude sees your whole macOS desktop — Simulator window included — and moves the mouse itself, with per-app approval tiers and the terminal excluded. It knows nothing about iOS specifically: no build tools, no accessibility hierarchy for the simulated app, and it reasons over an auto-downscaled desktop (a 16" MacBook Pro's 3456×2234 becomes ~1372×887), so your iPhone frame is a small region of a small image.

| | ios-simulator-mcp | XcodeBuildMCP | Claude Code computer use | |---|---|---|---| | Builds, runs, tests the app | No | Yes — simulator, device, macOS | No | | Simulator screenshots | Yes | Yes | Whole desktop, downscaled | | UI / accessibility hierarchy | Yes — ui_describe_all, ui_describe_point (idb) | Yes — snapshot_ui (AXe describe-ui) | No — pixels only | | Taps, typing, swipes | Yes (idb) | Yes (bundled AXe) | Yes — desktop-wide mouse and keyboard | | Extra dependency | Facebook idb | None beyond Xcode (AXe bundled) | None (macOS research preview) | | You point, agent resolves the element | No | No | No | | Best for | Light inspect-and-poke on an already-built sim | The full build → run → drive → debug loop | Desktop tasks beyond the Simulator |

The honest framing: XcodeBuildMCP is complementary, not competing, with the "seeing" problem. It closes the build and drive stages better than anything else — but its eyes are agent-initiated. The agent screenshots when it thinks to, describes UI when it thinks to. When you are the one who spotted the misalignment, you're back to prose: "no, the other button." It builds; you still need eyes and pointing. (For screenshot-focused options beyond iOS, see our survey of screenshot MCP servers.)

Why coordinate taps fail and accessibility APIs fix them

Every pixels-only approach eventually hits the same wall: coordinates are a terrible interface between a model and a screen.

Count the coordinate systems in play. The app lays out in points; the simulated device renders at 2x or 3x backing scale; the Simulator window can display that at reduced physical size; the window sits somewhere on a Retina desktop; and the screenshot the model saw was downscaled again before inference. A tap target guessed from that image has to survive four transforms. Off by a little, and the agent taps the row above the one you meant — silently, confidently.

Accessibility APIs dissolve the whole problem, because they deal in elements, not locations. An element has a role, a label, a frame, an identifier. You don't estimate where it is; you ask the OS what is at a point, or you address it by identifier and read its frame back. The evidence for hybrid approaches is strong: the OSWorld ablations put accessibility tree plus screenshot at 69.2–71.8% relative task success versus 56.4% for annotated screenshots alone — we dug into why in accessibility trees vs screenshots.

Here's the fact that makes this work for iOS without any in-app agent or idb daemon: the Simulator bridges the simulated app's accessibility elements into the macOS accessibility tree. Your SwiftUI button, running on simulated iOS, is visible to the same AXUIElementCopyElementAtPosition calls that resolve native Mac buttons. You can prove it from your own terminal with xctree, a small CLI that dumps a simulated app's accessibility tree using the standard macOS AX API:

brew tap ldomaradzki/xctree && brew install xctree
xctree --format json

This is the mechanism we build on. We make SupaMaus Lite, a macOS menu-bar app that turns pointing into agent context: gesture over any element — including one inside the Simulator — and it hit-tests the AX tree and hands your agent the resolved element with role, name, and bounds. As of v1.0.69 (July 2026) that resolution reaches into simulated apps, including the accessibilityIdentifier strings you set in your own code. Which means the element you circle resolves not just to "a button", but to checkout.payButton — a string that greps.

The complete loop with spatial context

Here's what the four-stage loop looks like when every stage has a tool. Assume Claude Code with XcodeBuildMCP (or plain xcodebuild) for building, and SupaMaus Lite running in the menu bar as a local MCP server (127.0.0.1, token-authenticated, on-device).

1. Build. The agent builds and launches on the simulator itself — XcodeBuildMCP's build_run_sim or your own script. No human in this stage.

2. See and point. The app renders; the pay button is misaligned. Hold Control+Option and trace over it in the Simulator window. Release. That one gesture captures an annotated screenshot with your trail redrawn on it — and, because the Simulator bridges its AX elements, the resolved element underneath: role, name, bounds in global points, center, window-relative position, and your accessibilityIdentifier.

3. Land it in Claude Code. Type /mcp__supamaus__look, or let the agent call get_context over MCP. Every capture also mirrors to ~/.supamaus/latest/ as capture.png + context.md, so the same context pastes into Codex or any other composer. No screenshot dragging, no descriptive paragraph.

4. Fix and verify. The agent greps the identifier, lands in the exact view, sees from the bounds that the button's x-origin disagrees with the fields above it, and edits the layout. It rebuilds, relaunches — and then you (or it) capture the same region again. The verification is a comparison of fresh element bounds against the complaint, not a vibe check on a compile log.

The point isn't any single step; it's the cycle time. The describe-by-prose loop costs a minute or more per iteration and degrades with your patience. The point-based loop costs a two-second gesture, and the context that arrives is structured — which is exactly the argument we make in why coding agents need spatial context. For longer flows — a bug that only appears three screens into onboarding — triple-tap Control starts a recording session: narrate with your voice, draw regions across screens, triple-tap again, and the agent gets one multi-region context with a shared transcript and frames it can pull by timestamp.

SwiftUI-specific tips

A few things make the loop meaningfully better on the SwiftUI side.

Set accessibilityIdentifier on anything you'll iterate on. Labels are for users and VoiceOver; identifiers are for tooling — never localized, never restyled:

Button("Pay now") { submit() }
    .accessibilityIdentifier("checkout.payButton")

When a pointed element carries checkout.payButton, the agent's first grep lands in the right file. Without it, resolution still works — role "button", label "Pay now", bounds — but the mapping back to code is one inference hop less direct.

One identifier, three consumers. The same strings power XCUITest queries, AXe/idb automation from the MCP servers above, and pointed captures. It's the highest-leverage metadata you can add to a SwiftUI view, and it costs one modifier.

Give repeated rows stable identities. In a List or ForEach, derive identifiers from your model ("order.\(order.id)") rather than position, so "this row" resolves to the same element after a re-sort.

Know the bridging gap — an honest caveat. The Simulator's AX bridge is good but not complete: on certain Xcode versions, some elements don't make it into the macOS tree — SwiftUI containers occasionally collapse or omit children that Xcode's own Accessibility Inspector can see from inside. If pointing at something resolves to its parent, dump the bridged tree with xctree and check whether the element is there at all. When it isn't, hybrid capture still carries you: the agent gets the annotated screenshot, the trail, and the region bounds even without an element resolution — degraded, not blind.

FAQ

Can Claude Code see the iOS Simulator?

Not by default — Claude Code reads files and runs commands; it has no camera. You add sight one of three ways: manually (screenshot with xcrun simctl io booted screenshot, drag it in), via MCP servers (ios-simulator-mcp, XcodeBuildMCP, or SupaMaus Lite), or with the built-in computer use research preview, which sees the whole desktop including the Simulator window.

ios-simulator-mcp vs XcodeBuildMCP — which one?

They overlap on the UI layer but sit at different weights. XcodeBuildMCP is the superset: it builds, tests, runs, and debugs, and its AXe-backed snapshot_ui/tap/type_text tools cover what ios-simulator-mcp does — without the idb prerequisite. ios-simulator-mcp is smaller and simpler if your build already happens elsewhere and you just want inspect-and-poke (plus record_video, which is genuinely handy). If you're choosing one for daily iOS work, XcodeBuildMCP; neither one lets you point at an element yourself.

Do I need idb?

Only for ios-simulator-mcp, which lists Facebook's idb as a required prerequisite. XcodeBuildMCP bundles AXe instead. SupaMaus Lite needs neither — it resolves Simulator elements through the macOS accessibility tree the Simulator already bridges into.

Does this work with UIKit?

Yes. The bridge operates at the accessibility layer, not the framework layer: standard UIKit controls expose accessibility by default, and view.accessibilityIdentifier = "checkout.payButton" works the same as the SwiftUI modifier. Anything visible to Accessibility Inspector is a candidate for resolution.

Claude Code has built-in computer use now — do I still need any of this?

For iOS work, yes. Computer use gives desktop-wide eyes and hands, but pixels only: no accessibility hierarchy for the simulated app, coordinates guessed from a downscaled desktop image, and no build integration. The AX-backed tools are more precise and cheaper per iteration inside the Simulator; computer use earns its keep on everything around it.