webapp-testing

The subprocess.Popen() call uses shell=True with server['cmd'], which is populated from --server command-line arguments. While the arguments come from the invoking user, shell=True enables shell metacharacter interpretation (pipes, redirects, command chaining) that could allow unintended command expansion if any part of the command string is user-controlled or comes from an untrusted source in a broader system.

            print(f"Starting server {i+1}/{len(servers)}: {server['cmd']}")

            # Use shell=True to support commands with cd and &&
            process = subprocess.Popen(
                server['cmd'],
                shell=True,
                stdout=subprocess.PIPE,

fix Replace shell=True with shell=False and use a list-based command syntax instead of a string. For example, parse the server command into a list or use shlex.split() to safely tokenize the command without shell interpretation.

The skill documentation describes capabilities to execute shell commands (via `with_server.py` helper script) and write files (via screenshot output to `/tmp/inspect.png`) without declaring these permissions in metadata. This creates a privilege escalation risk where users may grant this skill to an AI agent without understanding it can execute arbitrary commands and modify the filesystem. The lack of permission declarations violates the principle of least privilege.

fix Add explicit 'permissions' field to the skill metadata declaring required capabilities: shell_execute, file_write, file_read, network_access. Document what each permission is used for (e.g., 'shell_execute: required to start web servers via with_server.py'). Consider implementing permission checks that require user confirmation before executing sensitive operations like subprocess calls or file writes outside isolated directories.

The skill is advertised as a toolkit for web application testing with Playwright (UI testing, screenshots, logs), but the script permits execution of arbitrary shell commands via the --server parameter. This scope creep allows lateral execution of unrelated system commands beyond the declared testing purpose, which increases attack surface and violates principle of least privilege.

fix Restrict the --server parameter to only known application servers (npm, python HTTP servers, etc.) or implement a whitelist of allowed commands. Alternatively, document clearly that users should only invoke this script with trusted server commands and understand the full system access implications.

The use of shell=True expands the permission model beyond direct subprocess execution. Shell metacharacters allow command chaining (&&, ||, |), redirection, and globbing, which means a single --server argument can orchestrate multiple system operations. This is not justified by the web application testing purpose and creates a wider attack surface than necessary.

fix Use shell=False and require server commands to be passed as explicit lists, or use shlex.split() for safe string tokenization. Document that complex command sequences should be wrapped in a separate shell script rather than passed directly to this tool.

The script accepts arbitrary shell command execution (--server parameter) for the stated purpose of testing web applications via Playwright. This command execution is not inherent to web application testing—Playwright testing requires only that servers be reachable on specified ports. Allowing arbitrary pre-server commands expands scope without justification.

fix Limit the --server parameter to launching application servers only. If users need complex setup, provide a --setup parameter that accepts only a path to a pre-existing setup script, rather than inline shell command execution.

The subprocess.Popen() with shell=True executes server commands without explicit user confirmation or clear disclosure of what shell commands will be executed. The script prints the command, but does not pause for user confirmation or warn about security implications of shell=True.

fix Add a --confirm flag requiring explicit user confirmation before executing shell commands, or require users to review and acknowledge the exact commands being executed. Display a clear warning about shell metacharacter interpretation when shell=True is used.