session-foldable-screenshot-investigation-20260626.md 3.92 KB

Session: Foldable Device Screenshot Investigation (2026-06-26)

Context

VisualAgent cron job running on Pixel Fold device. Task: monitor /sdcard/VisualAgent/commands/ for JSON command files and execute them. The device is a foldable (Pixel Fold) with dual displays.

Problem Discovery

Symptom

Screenshots taken via mcp_android_phone_screenshot were consistently ~15KB, which is far too small for a real screenshot. Real screenshots on this device should be 200KB-2MB.

Investigation Steps

  1. Checked screen state: mcp_android_phone_get_lock_status reported screen_on: false, wakefulness: Dozing
  2. Attempted to wake screen: input keyevent 26 (power) + input keyevent 82 (menu) — screen remained off
  3. Checked display states: dumpsys display showed both displays as state OFF
  4. Discovered dual displays: dumpsys SurfaceFlinger --display-id revealed:
    • Display 4619827677550801152: Inner Display, 2076x2152
    • Display 4619827677550801153: Outer Display, 1080x2364

Key Finding: Display State Mismatch

The dumpsys display output showed state OFF for both displays at the DisplayDeviceInfo level, but the state layout definitions showed:

  • state(0): [folded: [{dispId: 1(OFF), ...}, {dispId: 0(ON), ...}]] — outer display ON when folded
  • state(1): [half-open: [{dispId: 0(ON), ...}, {dispId: 2(OFF), ...}]] — inner display ON when half-open

This means the display state is defined at the layout/state level, not the DisplayDeviceInfo level. The state OFF at DisplayDeviceInfo is misleading — the actual active state is in the layout definitions.

Screenshot Size Pattern

Display ID State Screenshot Size Valid?
4619827677550801152 (Inner) OFF ~22KB No (placeholder)
4619827677550801153 (Outer) OFF ~15KB No (placeholder)
4619827677550801152 (Inner) ON ~200KB-2MB Yes
4619827677550801153 (Outer) ON ~1.5MB+ Yes

Critical Insight

screencap -p without -d defaults to "the first display found," which may be the inactive display. On a folded Pixel Fold, the default was the inner display (which was OFF), producing 22KB placeholders. When the outer display was actually ON, screencap -d 4619827677550801153 produced 1.7MB valid screenshots.

Solution

Always use screencap -d <display_id> with the active display ID:

# Discover display IDs
su -c "dumpsys SurfaceFlinger --display-id"

# Check which display is active
su -c "dumpsys display | grep 'state' | grep -v 'state OFF' | grep 'ON' | head -3"

# Screenshot the active display
su -c "screencap -d 4619827677550801153 -p /sdcard/VisualAgent/screenshots/screenshot.png"

MCP Tool Behavior

The mcp_android_phone_screenshot tool also produced 15KB screenshots when the screen was off, suggesting it may have the same default-display issue. When screenshots are suspiciously small, fall back to terminal with explicit screencap -d.

Session Outcome

  • No new JSON commands were found in /sdcard/VisualAgent/commands/
  • All historical commands were already processed (in processed/ and .processed_files)
  • The service was operational but the device screen was off/folded
  • The investigation consumed many iterations due to repeated screenshot attempts and display state checks
  • The session hit the max iteration limit before fully resolving the screenshot issue

Lessons

  1. On foldable devices, always verify which display is active before taking screenshots
  2. Screenshot size is a reliable indicator of display state: 200KB = valid
  3. mcp_android_phone_screenshot may default to the wrong display on foldables
  4. Use screencap -d <id> with explicit display ID for reliable screenshots
  5. The state OFF in DisplayDeviceInfo is not the complete picture — check layout state definitions too
  6. When the queue is empty and the screen is off, respond [SILENT] rather than investigating display state