A safety system designed to prevent AI coding agents from running dangerous commands cleared as many as 82% of shell instructions without requiring an additional language model call, according to a blog post published Thursday by LM Studio. The company built Auto Review for its Bionic coding agent to catch risky operations—like a variable in `git diff $base` that could resolve to `--output=/some/file` and write data to the filesystem—before they execute. LM Studio described the 82% figure as anecdotal rather than a formal benchmark.
Bionic's Shell Judge analyzes commands by converting them into abstract syntax trees rather than searching for dangerous strings, following variables and nested instructions to determine what they might affect. The system tracks values across multiple commands—if an agent runs `git merge-base` to locate a shared ancestor commit and pipes the output into `git diff`, the Judge monitors that result when assessing the second instruction. When multiple potential values exist, Bionic traces up to 1,000 of them before halting its analysis. LM Studio has assembled 11,651 test cases to address the numerous ways this parsing can fail, covering malformed commands and the idiosyncrasies of how individual tools interpret their arguments—`ls -la` treats `-la` as bundled flags, while TypeScript's `tsc -vh` doesn't behave the same as running it with `-v` and `-h`.
Commands the Shell Judge can't approve go to the Shell Reviewer, a separate AI agent that evaluates the instruction within the context of the conversation. LM Studio found that simply asking the reviewer whether a command should execute didn't perform well, because the model occasionally greenlit risky operations that appeared necessary to fulfill the user's task. The reviewer now assigns each command scores for risk, authorization, and correctness without knowing which thresholds it must meet to pass. LM Studio excludes tool results from the reviewer's input to prevent instructions hidden in webpages or files from reaching it directly, though the reviewer still sees assistant messages—meaning compromised messages could carry malicious directions.
The approach has built-in vulnerabilities that become more significant as coding agents gain autonomy. The Shell Judge assumes executables like `git` haven't been tampered with and doesn't account for malicious configuration that could alter a command's behavior, while the Shell Reviewer's need to see enough conversation history to verify user authorization creates another avenue for prompt injection. LM Studio notes that a recent npm supply chain attack demonstrated how legitimate-looking provenance signals can conceal harmful payloads. These constraints matter more as tools like Google's Gemini coding agent expand beyond IDE boundaries, giving agents additional opportunities to execute commands and modify systems without developers manually approving each step. Organizations deploying AI coding assistants will need to weigh the efficiency gains from automation against the residual risk that structural parsing and contextual review can't fully eliminate, particularly in environments where configuration integrity can't be guaranteed.