Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion codex-public-marketplace-submission/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
"name": "android-cli",
"version": "1.0.0",
"version": "1.0.1",
"description": "Use the Android CLI to create Android projects, run apps on devices and emulators, manage the Android SDK, and search official Android documentation.",
"author": {
"name": "Google / Android",
Expand Down
588 changes: 152 additions & 436 deletions codex-public-marketplace-submission/skills/android-cli/SKILL.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
Run `android layout --help` and `android screen --help`.

## UI Dump

`android layout` returns a flat JSON list of the UI elements on screen. `android
layout --diff` returns a flat JSON list of the UI elements that have changed
since the last call to `layout` or `layout --diff`

Each JSON object represents a UI element in the Android app. The following
properties may be present:

- `text` - any literal text the element contains
- `resourceId` - the Android resource id used to refer to the element
- `contentDesc` - a description of a UI element for use by accessibility tools
- `interactions` - the set of user interactions the element supports. May contain one or more of: `checkable`, `clickable`, `focusable`, `scrollable`, `long-clickable`, `password`
- `state` - the set of states the element is in. May contain one or more of `checked`, `focused`, `selected`
- `bounds` - the screen coordinates of the bounding rectangle of the element, in the format `[min X,min Y][max X, max Y]`
- `center` - the screen coordinates of the center of the element, in the format `[x,y]`
- `off-screen` - if true, the element is in the UI hierarchy but not visible; it may require scrolling to view.

Use `layout` as a primary means of examining an Android app. Use `layout --diff`
to focus on changes and to keep your context small. Example: When entering
digits into a calculator, use `layout --diff` to output only the digit readout
element.

`layout` may fail due to the app displaying a WebView or animation; in these
cases, use `android screen capture --annotate` to inspect the app. This failure
will likely resolve after navigating away from the current screen.

## Screenshot

`android screen capture -o <file path>` saves a PNG of the current device screen
to `<file path>`

Use `screen capture` as a secondary means of examining an Android app
Examples:

- Understanding the content of an on-screen image
- Looking at a `WebView` (web content does not always appear in the ui dump)
- Trying to find a UI element by its visual appearance

**IMPORTANT** : Always *VISUALLY* examine the PNG image returned from `android
screen` BEFORE doing anything else.

## Annotated Screenshot

`android screen capture --annotate -o <file path>`
`android screen resolve --screen <path> --string <string>`

The `--annotate` command adds numerical labels and bounding boxes around UI
elements. Use this command to locate UI elements that cannot be located in the
`layout` output.

**IMPORTANT** : When using `android screen --annotate`, always *VISUALLY* examine
the resulting PNG file.

To refer to these labels in input commands, use `screen resolve` to convert
labels into coordinates:

`android screen resolve --screen <file path> --string "#3"` returns `<x coord of
region 3> <y coord of region 3>`

To save turns, you can combine shell commands:

`adb shell input $(android screen resolve --screen screen.png --string "tap #34")`

This command taps on region #34 from `screen.png`

## Input

Use `adb shell input` for interacting with Android devices. Refer to the
`"interactions"` property of an element for what interactions can be performed
on a particular element.

Interact with UI elements with their `center` coordinate or their `bounds`
coordinates:
`json
{
"key": -248568265,
"class": "android.widget.Button",
"bounds": "[138,9][167,38]",
"center": "[152,23]"
}`

To tap on this button, you would execute `adb shell input tap 152 23`. This taps
the center.

{
"key": 12487234,
"class": "com.example.ui.ScrollableList",
"bounds": "[100,200][400,600]",
"center": "[250,400]"
}

To scroll down on this list, you would execute `adb shell input swipe 250 400
250 200 500`. This swipes from the center to the top over 500ms.

### Text Input

To enter text, ensure the field is focused and execute `adb shell input text
"<text>"`.

- **Spaces (`%s`):** Replace spaces with `%s` (Android ignores text after raw spaces): `adb shell input text "Hello%sworld"` (enters `"Hello world"`).
- **Special characters:** Escape shell metacharacters (`&`, `$`, `(`, `)`, `!`): `adb shell input text "AT\&T"` or `adb shell input text 'Price:\$10'`.
- **Submit / Enter:** `adb shell input keyevent 66`

## Android Interaction Rules

1. Always ensure text input fields have `"focused"` in their `"state"` list before entering text.
2. If an element has `"scrollable"` in its `"interactions"` list, try scrolling it when looking for missing UI elements.
3. Always scroll slowly when executing scroll inputs. The 5th argument to `adb
shell input swipe` controls scroll duration.
4. Content may take time to load; if a `layout` is missing information after you take an action, wait a few seconds, then perform `layout --diff` to see if anything changes.
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
A journey is an XML-specified test of an Android app's behavior. It consists of
a list of `<action>` elements. For example:
`xml
<journey name="My Journey">
<description>
A sample journey to illustrate the format
</description>
<actions>
<action>
Tap the "Home" icon
</action>
<action>
Verify that the app is on its Home screen
</action>
</actions>
</journey>`

Evaluate a journey by proceeding through the `<actions>` list in sequential
order. Evaluate each `<action>` block individually.
A journey succeeds if all elements in the `<actions>` list succeed.

A journey is a test case for an app. The journey XML is the source of truth; if
the app disagrees with the journey, the app has failed.
Additionally, if the app exits, crashes, or freezes, journey evaluation stops
and the journey fails.

**IMPORTANT** - Execute each step EXACTLY as written, and independently of other
steps! If an action says to `"tap the first search result"`,
you MUST find the search results and tap the first one. Do this even if you
believe you know the intent behind the action.

## Taking Actions

Some `<action>` elements specify UI interactions to perform on the running
Android app. Perform the interaction and verify that the app does
not crash or behave in an unexpected manner. This is the *only* verification
you should perform for an `<action>`.

If the interaction cannot be performed as specified, the journey fails.
Example:
`<action>Click the red button</action>`
If you determine a red button is not present in the UI, the journey fails.

If the text of an `<action>` specifies a list of actions, break it into
sub-actions and evaluate them individually:
Example:
`<action>Search for soda and add the first result to the cart</action>`
This should be evaluated as:
`<action>Search for soda</action>
<action>Add the first result to the cart</action>`

If an `<action>` contains something that is not a specification for a UI
interaction, alert the user that the journey is malformed and exit
early, specifying the error in question.

## Verifying Expectations

`<action>` elements that begin with "check" or "verify" specify expectations
for the current state of the Android app. Determine the current
state of the app and check if the expectations are met.

Determine the current state of the app by inspecting the current screen of the
device without interacting with it.
Example:
`<action>Check if "Switch 2" is visible on the screen</action>`
This requires only inspecting the current screen, not scrolling or interacting.
If "Switch 2" is not currently visible, the action fails.

If the expectations are not met, mark the `<action>` as a failure and the
journey evaluation ends. A single `<action>` may contain
multiple expectations.
Example:
`<action>Verify that the app is on the Home screen, the Home icon is blue, and the temperature is displayed</action>`
This `<action>` fails if ANY of the following are false:

- The app is on the Home screen
- There is a Home icon, and it is blue
- A temperature is displayed

## Handling failure

When running a journey, evaluate it as a test. Failure is acceptable, and often
expected. Proper reporting of failures is the priority.

Keep debugging and troubleshooting to a minimum; assume that tools are showing
you the correct output every time. The goal is to determine
if the *current* Android app can correctly handle the *current* steps outlined
in the journey. Suggestions for bug fixes, clarification, or
other improvements should be kept to journey evaluation summary at the end.

## Summarizing

After evaluating the whole journey, output a markdown (.md) file describing the
results.

The file should have the following format:

# Journey: <Name of the journey>

## Results

<!-- Append "✅" to the action header if the instruction was evaluated or "❌" if the instruction could not be evaluated -->
### Action: Click the blue button ✅
- **Commands**:
<!-- A list of the ADB commands executed while evaluating the instruction -->
- `adb shell input swipe 490 200 500 500 500`
- `adb shell input tap 45 920`

<!-- Failure reasons, feedback, or other useful information -->
- **Comment**: The journey step doesn't specify that the button requires scrolling to see

### Action: Check if "Switch 2" is visible on the screen ✅
- **Screenshot**: [switch_2_screenshot.png](path/to/screenshot)
- **Comment**: Confirmed both in layout and visually

### Action: The home screen is shown ❌
- **Comment**: The settings page was shown

<!-- Append nothing to the action header if the action was not evaluated because the journey execution ended early -->
### Action: Swipe down to view active notifications
10 changes: 8 additions & 2 deletions plugins/android-cli/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "android-cli",
"version": "1.0.0",
"version": "1.0.1",
"displayName": "Android CLI",
"description": "Use the Android CLI (android) to create Android projects, run apps on devices and emulators, manage the SDK, look up official Android docs and install Android skills.",
"author": {
Expand All @@ -10,5 +10,11 @@
"homepage": "https://developer.android.com/tools/agents/android-cli",
"repository": "https://github.com/android/android-cli-plugin",
"license": "Apache-2.0",
"keywords": ["android", "cli", "kotlin", "emulator", "sdk"]
"keywords": [
"android",
"cli",
"kotlin",
"emulator",
"sdk"
]
}
2 changes: 1 addition & 1 deletion plugins/android-cli/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "android-cli",
"version": "1.0.0",
"version": "1.0.1",
"description": "Use the Android CLI (android) to create Android projects, run apps on devices and emulators, manage the SDK, look up official Android docs and install Android skills.",
"author": {
"name": "Google / Android",
Expand Down
Loading
Loading