mcp-tools-vs-raw-shell.md 4.31 KB

MCP Tools vs Raw Shell Commands — Decision Matrix

Core Principle

MCP Android phone tools (mcp_android_phone_*) already handle root/su elevation internally. Do NOT wrap them with su -c in terminal commands. Use MCP tools directly for UI operations; use su -c only for raw shell commands that are NOT covered by MCP tools.

Decision Matrix

Operation Preferred Tool Why
Screenshot mcp_android_phone_screenshot Handles encoding, path resolution, error reporting automatically
Tap mcp_android_phone_tap Validates coordinates, handles screen state
Swipe mcp_android_phone_swipe Validates coordinates, handles duration
Input text mcp_android_phone_input_text Handles CJK, IME, focus management
Launch app mcp_android_phone_open_app Resolves activity, handles errors
Get UI tree mcp_android_phone_get_ui_tree XML parsing, package filtering
Find element mcp_android_phone_find_element OCR fallback, coordinate extraction
Screen lock status mcp_android_phone_get_lock_status Direct system query
File read/write on /sdcard/ terminal + su -c 'cat ...' or su -c 'echo ... > ...' MCP tools don't handle arbitrary file I/O
Process check (ps) terminal + su -c 'ps ...' No MCP equivalent
Directory listing terminal + su -c 'ls -la ...' No MCP equivalent
JSON parsing terminal + python3 -c '...' or jq No MCP equivalent
Complex conditional logic terminal + shell script MCP tools are single-operation

Anti-Patterns

❌ Double-Wrapping MCP Tools

# WRONG — don't do this
su -c "mcp_android_phone_screenshot ..."  # This is NOT a shell command!

MCP tools are Hermes-native functions, not shell binaries. They cannot be invoked via su -c. Call them directly in your tool call block.

❌ Using Raw input for Text Entry

# WRONG — fragile, no CJK support, no focus management
su -c "input text 'hello world'"

Use mcp_android_phone_input_text instead. It handles CJK, IME composition, and focus validation.

❌ Using Raw screencap for Screenshots

# WRONG — manual path handling, no error reporting
su -c "screencap -p /sdcard/screenshot.png"

Use mcp_android_phone_screenshot instead. It returns structured data (path, size) and handles errors gracefully.

When Raw Shell IS Correct

✅ File Operations on /sdcard/

# CORRECT — MCP tools don't do arbitrary file I/O
su -c "cat /sdcard/VisualAgent/results/result.json"
su -c "echo '{\"status\":\"ok\"}' > /data/local/tmp/result.json"

✅ Process Management

# CORRECT — no MCP equivalent for process checks
su -c "ps | grep visual_agent"
su -c "kill -9 <pid>"

✅ Directory Management

# CORRECT — no MCP equivalent
su -c "mkdir -p /data/local/tmp/VisualAgent/commands/"
su -c "find /data/local/tmp/VisualAgent/commands/ -name '*.json'"

Hybrid Workflows

Most VisualAgent tasks require both MCP tools and raw shell commands in sequence:

1. terminal: su -c "find /data/local/tmp/VisualAgent/commands/ -name '*.json'"
2. terminal: su -c "cat /data/local/tmp/VisualAgent/commands/cmd_001.json"
3. mcp_android_phone_screenshot (path: /data/local/tmp/VisualAgent/results/...)
4. terminal: su -c "echo '{\"status\":\"success\"}' > /data/local/tmp/VisualAgent/results/..."
5. terminal: su -c "mv /data/local/tmp/VisualAgent/commands/cmd_001.json /data/local/tmp/VisualAgent/processed/"

Path Parameter Clarification

The path parameter in mcp_android_phone_screenshot is a device-local path (on the Android device), not a host path. The tool saves the screenshot directly to the device at that path.

# CORRECT — device-local path
mcp_android_phone_screenshot(path="/data/local/tmp/VisualAgent/results/screenshot.png")

# WRONG — host path (will fail or save to non-existent dir on device)
mcp_android_phone_screenshot(path="/Users/alice/screenshot.png")

If you need the file on the host, use the tool's return value and then pull it:

# After mcp_android_phone_screenshot returns path and size
terminal: adb pull /data/local/tmp/VisualAgent/results/screenshot.png ./screenshot.png

Or read it back via base64:

terminal: su -c "base64 /data/local/tmp/VisualAgent/results/screenshot.png"