session-mcp-screenshot-nested-result.md 2.16 KB

Session: MCP Screenshot Nested Result Structure

Session Date

2026-06-26

Scenario

Cron job monitoring /sdcard/VisualAgent/commands/ for JSON command files. The directory was empty (only myfile.dat and test.txt). The agent took a screenshot using mcp_android_phone_screenshot and needed to understand the tool's response structure.

Tool Response Structure

When calling mcp_android_phone_screenshot with path="/sdcard/VisualAgent/screenshots/service_check.png", the tool returned:

{
  "result": "{\"ok\": true, \"path\": \"/sdcard/VisualAgent/screenshots/service_check.png\", \"size_bytes\": 15350}"
}

Note that result is a JSON string, not an object. The actual fields (ok, path, size_bytes) are inside that string and must be parsed with a second JSON.parse.

The Trap

A naive attempt to access result.path would fail because result is a string, not an object. The correct access pattern is:

import json

# First parse: the outer tool response
outer = json.loads(tool_response)

# Second parse: the inner result string
inner = json.loads(outer['result'])

# Now you can access the fields
path = inner['path']        # "/sdcard/VisualAgent/screenshots/service_check.png"
size_bytes = inner['size_bytes']  # 15350
ok = inner['ok']            # True

Why This Matters

When writing VisualAgent result files that include screenshot metadata, the result needs to be properly structured. If you write the raw result string (the outer object's result field), you'll serialize a JSON string inside JSON, which is valid but awkward:

{
  "status": "success",
  "screenshot_result": "{\"ok\": true, \"path\": \"...\", \"size_bytes\": 15350}"
}

Better to parse the inner JSON and write the actual fields:

{
  "status": "success",
  "screenshot": {
    "ok": true,
    "path": "/sdcard/VisualAgent/screenshots/service_check.png",
    "size_bytes": 15350
  }
}

Cross-References

  • references/session-minimal-screenshot-before-silent.md — The session where this nested result structure was discovered
  • references/mcp-tools-vs-raw-shell.md — Decision matrix for when to use MCP tools vs raw shell commands