Android UI
Analyser

Mobile testing infrastructure for AI agents

AUA helps AI agents understand, navigate, and test mobile apps without relying on a loop of screenshots and guessed coordinates.

Explore on GitHub ↗Read the launch post ↗

Type: Open-source CLI & MCP tool
Platforms: Android · iOS Simulator
Interfaces: CLI · MCP · Agent plugins
Built at: Luzia

The problem

A mobile agent needs two things on every step: a dependable view of the current screen and a safe way to act on it.

The usual screenshot-first loop is expensive and fragile. The agent has to process an image, infer where a control sits, tap raw coordinates, wait, and take another screenshot. As the flow gets longer, the same visual context is processed again and again.

AUA replaces that loop with structured observations and tracked elements. It lets the agent reason about the interface as controls, text, states, and bounds, then act on an element by ID.

How it works

Read the hierarchy

Inspect accessibility and view data first, returning element IDs, types, text, states, and bounds.

Act by element

Resolve the selected ID on the current screen instead of asking the agent to guess a pixel coordinate.

Return the result

Wait for the UI to settle and include the next structured observation in the same interaction.

Hierarchy first, vision when needed.

OCR, image detection, and optional visual grounding cover screens whose accessibility data is incomplete. The common path stays lean; heavier perception is a fallback.

Screenshot-first loop

Screenshot → analyse → find coordinates → tap → wait → screenshot → analyse again

With AUA

tap-and-analyze(target) → act → settle → return the next screen as structured data

Processed tokens at step 10~187K → ~20K

Time at the same point~10s → 2.5–2.7s

Figures are from the example workflow shared in Eiliya’s launch post; they describe that run, not a universal benchmark.

Beyond a single test

Parallel execution

Automatic device selection and leasing let multiple agents test without competing for the same target.

Reusable knowledge

Flows, screen maps, routes, transitions, and app memory let future agents reuse paths that have already been discovered.

Evidence and control

Goal-based testing, semantic assertions, screenshots, recordings, crash evidence, network mocking, and guarded database operations.

Agent-ready access

A CLI, MCP server, and plugins for Codex and Claude Code make the same operating model available in different workflows.

In use

AUA started as an internal answer to a real delivery problem at Luzia, then became an open-source project.

The Luzia Android team uses it to verify features after implementation and before release. It also supports agent-powered QA scenarios running in parallel. Early testing and feedback from the team shaped the tool, and outside contributions are now part of its development.

The project is public for teams and individual developers who want mobile agents to work from stronger UI context and leave behind reusable knowledge.

Open source

Try it, inspect it, or contribute.

The repository includes installation paths for the CLI, Codex, and Claude Code, plus the full documentation and release history.

Open the repository ↗Back to selected work