# VisualAgent Executor Bugs and Workarounds

## Background

This reference documents specific bugs found in existing VisualAgent executor implementations (both shell and Python versions) and provides workarounds. These bugs were discovered during real-world debugging sessions on Android devices with Termux.

## Bug 1: Shell Executor's Fragile JSON Parsing (visual_agent_executor.sh)

### The Problem

The shell executor uses `grep`/`cut` to parse JSON command files:

```bash
cmd_type=$(su -c "cat '$cmd_file'" | grep -o '"type"[[:space:]]*:[[:space:]]*"[^"]*"' | cut -d'"' -f4)
```

This pattern breaks when:
- JSON contains nested quotes (e.g., `"text": "Say \"hello\""`)
- Values contain colons (e.g., `"path": "/sdcard/VisualAgent/screenshot_10:00.png"`)
- Whitespace varies (e.g., `"type" : "screenshot"` vs `"type":"screenshot"`)
- Unicode characters are present in values

### Real-World Impact

In this session, a `test_tap.json` file containing `{"type": "tap", ...}` was processed but the shell executor reported:
```
{"status":"error","message":"未知命令类型"}
```

The `grep`/`cut` pattern failed to extract the `type` value correctly, causing the command to be rejected as "unknown type".

### Workaround

**Never use `grep`/`cut` for JSON parsing.** Use Python or `jq` instead:

```bash
# Correct: Use Python for reliable JSON parsing
cmd_type=$(su -c "cat '$cmd_file'" | python3 -c "import json,sys; print(json.load(sys.stdin).get('type',''))")

# Alternative: Use jq if available
cmd_type=$(su -c "cat '$cmd_file'" | jq -r '.type // .command // empty')
```

### Fix for Shell Executor

Replace the fragile `grep`/`cut` patterns with Python one-liners:

```bash
# Before (fragile):
cmd_type=$(su -c "cat '$cmd_file'" | grep -o '"type"[[:space:]]*:[[:space:]]*"[^"]*"' | cut -d'"' -f4)

# After (robust):
cmd_type=$(su -c "cat '$cmd_file'" | python3 -c "import json,sys; print(json.load(sys.stdin).get('type',''))")
```

Similarly for coordinate extraction:

```bash
# Before (fragile):
x=$(su -c "cat '$cmd_file'" | grep -o '"x"[[:space:]]*:[[:space:]]*[0-9]*' | grep -o '[0-9]*$')

# After (robust):
x=$(su -c "cat '$cmd_file'" | python3 -c "import json,sys; print(json.load(sys.stdin).get('x',0))")
```

## Bug 2: Python Executor's Broken File Listing (visual_agent.py)

### The Problem

The Python executor's `get_pending_commands()` function has a bug:

```python
def get_pending_commands():
    success, stdout, stderr = run_su(f"ls -1 '{CMD_DIR}'/*.json 2>/dev/null")
    if not success or not stdout.strip():
        return []
    return [line.strip() for line in stdout.strip().split('\n') if line.strip()]
```

The `run_su()` function wraps commands with `su -c '...'`, but the glob pattern `*.json` is expanded by the shell BEFORE `su` runs. If the parent shell (running as the Termux user) doesn't have permission to read the directory, the glob fails even though `su` would have permission.

### Real-World Impact

In this session, command files existed in `/sdcard/VisualAgent/commands/` (visible via `su -c "ls -la ..."`), but the Python executor reported "没有待执行的命令" (no pending commands) because the glob expansion failed.

### Workaround

Use `find` instead of shell glob, or ensure the glob is expanded inside the `su` context:

```bash
# Correct: Use find (no glob expansion issues)
su -c "find /sdcard/VisualAgent/commands/ -maxdepth 1 -name '*.json' -type f"

# Alternative: Use ls without glob, then filter
su -c "ls -1 /sdcard/VisualAgent/commands/" | grep '\.json$'
```

### Fix for Python Executor

Replace the glob-based listing with `find`:

```python
def get_pending_commands():
    # Use find instead of glob to avoid expansion issues
    success, stdout, stderr = run_su(f"find '{CMD_DIR}' -maxdepth 1 -name '*.json' -type f 2>/dev/null")
    if not success or not stdout.strip():
        return []
    return [line.strip() for line in stdout.strip().split('\n') if line.strip()]
```

## Bug 3: Shell Executor's Duplicate Code

### The Problem

The shell executor (`visual_agent_executor.sh`) has two nearly identical code blocks:
1. One for non-daemon mode (lines 11-103)
2. One for daemon mode (lines 112-203)

The daemon mode block is a copy-paste of the non-daemon block with only the `sleep 5` added. This means:
- Any bug fix must be applied in two places
- The file is twice as long as necessary
- Maintenance is error-prone

### Real-World Impact

In this session, the duplicate code made it harder to understand the executor's behavior and increased the risk of inconsistencies between daemon and non-daemon modes.

### Fix

Refactor to use a single processing function:

```bash
#!/system/bin/sh

CMD_DIR="/sdcard/VisualAgent/commands"
RESULT_DIR="/sdcard/VisualAgent/results"
PROCESSED_DIR="/sdcard/VisualAgent/processed"

process_commands() {
    for cmd_file in $(su -c "ls -1 '$CMD_DIR'/*.json 2>/dev/null"); do
        [ -e "$cmd_file" ] || continue
        
        filename=$(basename "$cmd_file")
        result_file="$RESULT_DIR/${filename%.json}_result.json"
        
        echo "[$(date)] 处理命令: $filename"
        
        # Use Python for reliable JSON parsing
        cmd_type=$(su -c "cat '$cmd_file'" | python3 -c "import json,sys; print(json.load(sys.stdin).get('type',''))")
        
        # Execute command (same logic for both modes)
        case "$cmd_type" in
            "screenshot")
                # ... screenshot logic
                ;;
            "tap")
                # ... tap logic
                ;;
            # ... other cases
        esac
        
        # Save result and move processed file
        su -c "echo '$result' > '$result_file'"
        su -c "mv '$cmd_file' '$PROCESSED_DIR/$filename'"
    done
}

# Non-daemon mode
if [ "$1" != "--daemon" ]; then
    su -c "mkdir -p '$RESULT_DIR' '$PROCESSED_DIR'"
    process_commands
    echo "[$(date)] 处理完成"
    exit 0
fi

# Daemon mode
echo "[$(date)] 守护模式运行，每 5 秒检查一次..."
while true; do
    sleep 5
    process_commands
done
```

## Bug 4: Shell Executor's `type` vs `command` Field Name Mismatch

### The Problem

The shell executor only checks for `"type"` field:

```bash
cmd_type=$(su -c "cat '$cmd_file'" | grep -o '"type"[[:space:]]*:[[:space:]]*"[^"]*"' | cut -d'"' -f4)
```

But some command generators use `"command"` instead of `"type"`:

```json
{
  "command": "tap",
  "x": 500,
  "y": 1000
}
```

### Real-World Impact

Commands using `"command"` instead of `"type"` are rejected as "unknown type" even though they are valid commands.

### Fix

Check both field names:

```bash
# Use Python to check both fields
cmd_type=$(su -c "cat '$cmd_file'" | python3 -c "
import json,sys
cmd = json.load(sys.stdin)
print(cmd.get('type') or cmd.get('command', ''))
")
```

## Bug 5: Shell Executor's `input text` Quoting Issues

### The Problem

The shell executor passes text directly to `input text`:

```bash
su -c "input text '$text'"
```

This breaks when:
- Text contains single quotes (e.g., `It's working`)
- Text contains special shell characters (e.g., `;`, `&`, `|`)
- Text contains spaces (may be split incorrectly)

### Fix

Use `printf` with proper escaping, or use ADBKeyboard for reliable text input:

```bash
# Safer approach using printf
safe_text=$(printf '%s' "$text" | sed "s/'/'\\''/g")
su -c "input text '$safe_text'"
```

Or use ADBKeyboard (if installed):
```bash
su -c "am broadcast -a ADB_INPUT_TEXT --es msg '$text' -n com.android.adbkeyboard/.AdbIME"
```

## Summary of Recommended Fixes

| Bug | Location | Fix |
|-----|----------|-----|
| Fragile JSON parsing | `visual_agent_executor.sh` | Replace `grep`/`cut` with `python3 -c` |
| Broken file listing | `visual_agent.py` | Replace glob with `find` |
| Duplicate code | `visual_agent_executor.sh` | Extract `process_commands()` function |
| Field name mismatch | Both executors | Check both `type` and `command` fields |
| Text input quoting | `visual_agent_executor.sh` | Use `printf` escaping or ADBKeyboard |

## Testing After Fixes

After applying fixes, verify with these test commands:

```bash
# Test 1: Screenshot with path containing spaces
su -c "printf '%s\n' '{\"type\":\"screenshot\",\"path\":\"/sdcard/VisualAgent/test screenshot.png\"}' > /sdcard/VisualAgent/commands/test1.json"

# Test 2: Tap with unicode text
su -c "printf '%s\n' '{\"type\":\"input\",\"text\":\"Hello 世界\"}' > /sdcard/VisualAgent/commands/test2.json"

# Test 3: Command using "command" field instead of "type"
su -c "printf '%s\n' '{\"command\":\"tap\",\"x\":500,\"y\":1000}' > /sdcard/VisualAgent/commands/test3.json"

# Run executor and check results
sh /data/data/com.termux/files/home/visual_agent_executor.sh
```

## Related References

- `visual-agent-service-patterns.md` — General VisualAgent service patterns and cron job workflows
- `visual-agent-daemon-lifecycle.md` — Daemon health check and restart procedures
- `pre-existing-visualagent-infrastructure.md` — Discovery checklist for existing VisualAgent installations
- `termux-sdcard-permission-patterns.md` — Detailed patterns for su-wrapping file operations