fuse-filesystem-quirks.md 8.91 KB

Android FUSE Filesystem Quirks for VisualAgent Backends

The Problem

On Android, /sdcard/ and /storage/emulated/0/ are FUSE (Filesystem in Userspace) mounts. When you write a file via su in one shell session, it may not be immediately visible in another session — or in a Python subprocess.run("su -c 'ls /sdcard/...'") call. The file appears to "disappear" or was "never created" even though ls in the same su session shows it.

Symptoms

  • su -c "echo '{}' > /sdcard/.../file.json && ls /sdcard/.../" shows the file
  • Immediately running python3 -c "import subprocess; subprocess.run('su -c ls /sdcard/...', shell=True)" does NOT show the file
  • The file may reappear later, or may never appear from the non-su perspective
  • This affects both file creation AND file reading

Root Cause

FUSE filesystems on Android cache directory listings and may have separate caches for different UID contexts. When su writes as root, the FUSE cache for the normal app UID doesn't invalidate. The file exists on the underlying filesystem but the FUSE layer doesn't show it to non-root processes.

Solutions

/data/local/tmp/ is a real filesystem (tmpfs or ext4), not FUSE. Files written here are immediately visible to all processes.

# Create directories
su -c "mkdir -p /data/local/tmp/VisualAgent/commands"
su -c "mkdir -p /data/local/tmp/VisualAgent/results"
su -c "mkdir -p /data/local/tmp/VisualAgent/processed"

# Write command file — immediately visible
su -c "echo '{\"command\":\"screenshot\"}' > /data/local/tmp/VisualAgent/commands/test.json"

# Read from Python — works reliably
python3 -c "import subprocess; r=subprocess.run('su -c ls /data/local/tmp/VisualAgent/commands/', shell=True, capture_output=True, text=True); print(r.stdout)"

Solution 2: Use sync After Write

Sometimes sync helps flush FUSE caches:

su -c "echo '{}' > /sdcard/VisualAgent/commands/test.json && sync"

This is unreliable — works on some devices/Android versions, not on others.

Solution 3: Access via /data/media/0/ (Advanced)

The underlying real filesystem path for /sdcard/ is often /data/media/0/. This bypasses FUSE entirely:

su -c "ls /data/media/0/VisualAgent/commands/"

Warning: /data/media/0/ requires root to access. Only use this if you already have su.

Base64 Encoding for JSON Through su

When passing JSON through su -c 'echo ... > file', shell quoting becomes extremely fragile. The {, }, ", and ' characters all have special meaning in shell contexts.

The Problem

# This breaks because " inside '...' is literal, but the shell may still parse it
su -c 'echo "{\"type\":\"screenshot\"}" > /tmp/test.json'

# This breaks because ' inside "..." ends the string
su -c "echo '{\"type\":\"screenshot\"}' > /tmp/test.json"

The Solution: Base64 Encode

import base64
import json
import subprocess

def write_json_via_su(data, filepath):
    """Write JSON to a file via su, using base64 to avoid quoting issues."""
    json_str = json.dumps(data, indent=2, ensure_ascii=False)
    json_bytes = json_str.encode('utf-8')
    b64 = base64.b64encode(json_bytes).decode('ascii')

    cmd = f"su -c 'echo {b64} | base64 -d > {filepath}'"
    subprocess.run(cmd, shell=True, check=True)

# Example usage
data = {"command": "screenshot", "filename": "test.png"}
write_json_via_su(data, "/data/local/tmp/VisualAgent/commands/test.json")

For reading JSON via su:

def read_json_via_su(filepath):
    """Read JSON from a file via su, using base64 to avoid quoting issues."""
    cmd = f"su -c 'cat {filepath} | base64'"
    result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
    if result.returncode != 0:
        raise RuntimeError(f"Failed to read {filepath}: {result.stderr}")

    json_bytes = base64.b64decode(result.stdout.strip())
    return json.loads(json_bytes.decode('utf-8'))

Complete VisualAgent Service Pattern with /data/local/tmp/

#!/usr/bin/env python3
"""VisualAgent service using /data/local/tmp/ to avoid FUSE issues."""

import os
import json
import time
import base64
import subprocess

COMMANDS_DIR = "/data/local/tmp/VisualAgent/commands"
RESULTS_DIR = "/data/local/tmp/VisualAgent/results"
PROCESSED_DIR = "/data/local/tmp/VisualAgent/processed"
PROCESSED_LOG = "/data/local/tmp/VisualAgent/.processed_files"

def run_as_root(cmd):
    """Execute command via su."""
    full = f"su -c '{cmd}'"
    result = subprocess.run(full, shell=True, capture_output=True, text=True, timeout=60)
    return {
        "success": result.returncode == 0,
        "stdout": result.stdout,
        "stderr": result.stderr,
    }

def list_commands():
    """List command files using su."""
    result = run_as_root(f"ls '{COMMANDS_DIR}'")
    if not result["success"]:
        return []
    return [f.strip() for f in result["stdout"].split("\n") if f.strip()]

def read_command(filepath):
    """Read command file via su."""
    result = run_as_root(f"cat '{filepath}'")
    return json.loads(result["stdout"])

def write_result(result, filepath):
    """Write result file via su using base64 encoding."""
    json_str = json.dumps(result, indent=2, ensure_ascii=False)
    b64 = base64.b64encode(json_str.encode('utf-8')).decode('ascii')
    run_as_root(f"echo '{b64}' | base64 -d > '{filepath}'")

def move_to_processed(filepath, filename):
    """Move processed command to archive."""
    dest = os.path.join(PROCESSED_DIR, filename)
    run_as_root(f"mv '{filepath}' '{dest}'")

def execute_screenshot(params):
    filename = params.get("filename", f"screenshot_{int(time.time())}.png")
    filepath = os.path.join(RESULTS_DIR, filename)
    result = run_as_root(f"screencap -p '{filepath}'")
    return {"success": result["success"], "saved_to": filepath if result["success"] else None}

def execute_tap(params):
    x, y = params.get("x", 0), params.get("y", 0)
    result = run_as_root(f"input tap {x} {y}")
    return {"success": result["success"]}

def execute_swipe(params):
    x1, y1 = params.get("x1", 0), params.get("y1", 0)
    x2, y2 = params.get("x2", 0), params.get("y2", 0)
    duration = params.get("duration", 300)
    result = run_as_root(f"input swipe {x1} {y1} {x2} {y2} {duration}")
    return {"success": result["success"]}

def execute_input(params):
    text = params.get("text", "")
    escaped = text.replace("'", "'\\''")
    result = run_as_root(f"input text '{escaped}'")
    return {"success": result["success"]}

def execute_start_app(params):
    package = params.get("package", "")
    result = run_as_root(f"monkey -p {package} -c android.intent.category.LAUNCHER 1")
    return {"success": result["success"]}

HANDLERS = {
    "screenshot": execute_screenshot,
    "tap": execute_tap,
    "swipe": execute_swipe,
    "input": execute_input,
    "start_app": execute_start_app,
}

def process_command(filename):
    filepath = os.path.join(COMMANDS_DIR, filename)
    try:
        cmd_data = read_command(filepath)
        cmd_type = cmd_data.get("command", "")
        handler = HANDLERS.get(cmd_type)
        if not handler:
            return False

        exec_result = handler(cmd_data)
        result = {
            "command": cmd_type,
            "params": cmd_data,
            "success": exec_result["success"],
            "timestamp": int(time.time()),
            **exec_result,
        }

        result_file = os.path.join(RESULTS_DIR, f"{os.path.splitext(filename)[0]}_result.json")
        write_result(result, result_file)
        move_to_processed(filepath, filename)

        # Track processed file
        with open(PROCESSED_LOG, "a") as f:
            f.write(filename + "\n")

        return True
    except Exception as e:
        print(f"Error processing {filename}: {e}")
        return False

def main():
    # Ensure directories exist
    for d in [COMMANDS_DIR, RESULTS_DIR, PROCESSED_DIR]:
        os.makedirs(d, exist_ok=True)

    # Load processed files
    processed = set()
    if os.path.exists(PROCESSED_LOG):
        with open(PROCESSED_LOG) as f:
            processed = set(line.strip() for line in f if line.strip())

    # Process new commands
    files = list_commands()
    count = 0
    for filename in files:
        if not filename.endswith(".json") or filename in processed:
            continue
        if process_command(filename):
            count += 1

    print(f"Processed {count} commands")
    return count

if __name__ == "__main__":
    main()

Key Takeaways

  1. Always use /data/local/tmp/ for automation backends — avoid FUSE filesystem issues entirely
  2. Use base64 encoding when passing JSON through su -c — eliminates all quoting problems
  3. Never rely on /sdcard/ for inter-process file visibility — the FUSE layer may hide files across UID boundaries
  4. Test file visibility from Python immediately after creation — don't assume su -c ls in one shell means Python can see it