Security Auditor
SkillForge includes an integrated security auditor and static analysis engine (skillforge audit) designed specifically for AI agent skills and instruction files.
The Threat Vector: Skill Supply-Chain Attacks
Unlike regular text files, SKILL.md documents are instruction sources. When an autonomous coding assistant loads a third-party skill, it follows the procedural steps and may execute companion scripts in its sandboxed or local terminal.
Common attack vectors in community-shared skills include:
- Hidden Zero-Width Payloads: Invisible unicode codepoints embedded between words that are invisible in GitHub code review but parsed by LLMs as execution instructions.
- Indirect Prompt Injection: Directives instructing the model to ignore user constraints, disable safety boundaries, or leak previous conversation turns.
- Script Exfiltration (SkillJect): Benign-looking documentation that points to a helper script in scripts/ containing unauthorized curl, wget, or credential-harvesting commands.
Audit Rule Reference
| Rule ID | Severity | Focus Area | Description |
|---|---|---|---|
SEC-001 |
CRITICAL | Hidden Unicode | Detects zero-width spaces (\u200B, \u200C, \u200D, \uFEFF), bidirectional overrides, and unicode tags used to conceal prompt instructions. |
SEC-002 |
CRITICAL | Prompt Injection | Scans for jailbreak patterns (ignore previous instructions, system override, developer mode, DAN mode). |
SEC-003 |
HIGH | Dangerous Shell | Identifies risky execution patterns in scripts and markdown blocks (curl \| bash, base64 piping to shell, outbound data exfiltration via POST). |
SEC-004 |
MEDIUM | Credential Harvesting | Detects unprompted direct reads to sensitive user paths (SSH private keys, AWS credentials, gcloud configs). |
SEC-005 |
HIGH | Exfiltration Endpoints | Detects outbound data transmission to webhook or request-capture domains (webhook.site, pipedream.net, requestbin.net). |
SEC-006 |
CRITICAL / HIGH | Hardcoded Secrets | Detects hardcoded API keys and provider tokens (ghp_, github_pat_, sk-ant-, AIzaSy, AKIA). |
SEC-007 |
HIGH | Markdown Link Spoofing | Detects deceptive links where the visible URL host differs from the target destination host. |
SEC-008 |
CRITICAL / HIGH | Python AST, Taint & Sandbox | Inspects bundled .py scripts via Python AST for eval(), exec(), subprocess(..., shell=True), unsafe deserialization, environment-to-network taint flows, and runtime sandbox policy violations (--sandbox). |
SEC-009 |
MEDIUM | Unrestricted Tool Scope | Flags wildcard tool permissions (*, Bash(*)) in allowed-tools YAML frontmatter. |
SEC-010 |
HIGH | Deceptive Installer Pattern | Flags prose instructions guiding users or agents to execute piped shell installers (curl ... \| bash, irm ... \| iex). |
SEC-011 |
HIGH / MEDIUM | Executable Payload in Assets | Flags native executables (ELF, Mach-O, PE headers) as HIGH and script or binary file types (.sh, .ps1, .bat, .exe, .dll, .so, .jar, ...) under assets/ as MEDIUM. Runnable code belongs in scripts/, where it gets audited as code. Text assets are also scanned for prompt injection and hidden unicode. |
SEC-012 |
HIGH | Allowed-Tools Scope Violation | Flags shell execution blocks or bundled scripts when frontmatter allowed-tools excludes command execution. |
SEC-013 |
HIGH | Memory & Hook Poisoning | Detects shell redirection or append operations (>>, >, tee -a) targeting CLAUDE.md, AGENTS.md, MEMORY.md, .cursorrules, .windsurfrules, or .git/hooks/* (OWASP ASI06). |
SEC-014 |
CRITICAL | Lethal Trifecta Compound Risk | Flags skills combining private credential/env access, untrusted external content ingestion, and outbound network egress (OWASP AST10). |
SEC-015 |
CRITICAL / HIGH | Obfuscation & Encoded Payloads | Detects base64 pipelines, python b64decode execution, powershell encoded commands, and hex escapes (Cisco obfuscation). |
SEC-016 |
CRITICAL | Destructive Resource Abuse | Detects bash fork bombs (:(){ :|:& };:), disk exhaustion (dd if=/dev/urandom), and raw disk formatting (Cisco resource_abuse). |
SEC-017 |
HIGH | Unpinned Package Installation | Detects unpinned runtime package installations from public registries (pip install, npm install -g) without lockfiles or version constraints (Cisco supply_chain). |
MCP-001 |
CRITICAL / HIGH | MCP Plaintext Secret | Detects hardcoded API tokens or embedded secret strings in MCP configuration environment variables. |
MCP-002 |
HIGH | MCP Shell Wrapper | Flags arbitrary shell interpreter wrappers (bash -c, sh -c, powershell) declared in MCP server command fields. |
MCP-003 |
MEDIUM | MCP Unpinned Package | Flags unpinned package executions (npx, uvx) lacking exact @version tags in MCP server arguments. |
MCP-004 |
HIGH | MCP Root Filesystem Grant | Flags permissive root filesystem (/, C:\, ~, $HOME) directory exposure in MCP arguments. |
QUAL-001 |
LOW | Trigger Quality | Verifies whether the skill description includes explicit activation triggers (when, use when, for) necessary for token-efficient progressive disclosure. |
QUAL-002 |
MEDIUM | Spec Compliance | Ensures directory name matches the name field in YAML frontmatter and validates identifier safety. |
QUAL-003 |
LOW | Context Budget Limit | Flags SKILL.md bodies exceeding 500 lines or ~5,000 tokens so reference docs are extracted into references/. |
QUAL-004 |
HIGH | Broken Resource Pointer | Verifies that every references/..., scripts/..., or assets/... path cited in SKILL.md exists in the package. |
QUAL-006 |
MEDIUM | Applicability Hijacking | Flags overbroad trigger descriptions that attempt to force activation on every user prompt. |
QUAL-008 |
LOW | Metadata Value Types | The agentskills.io spec maps string keys to string values in metadata. Flags numbers and booleans such as version: 1.0; quote them (version: "1.0"). |
QUAL-007 |
LOW | agentskills.io Name Format | Validates strict naming compliance (^[a-z0-9]+(-[a-z0-9]+)*$) for portability across all runtimes. |
QUAL-009 |
LOW | Nested Reference Chain | Flags reference files that link to secondary references/... files instead of keeping progressive disclosure one-level-deep from SKILL.md. |
CLI Usage
Basic Audit
skillforge audit ./skills/my-skill
Strict Mode for CI/CD
Fail the command with exit code 1 if any warning or low-severity finding is detected:
skillforge audit ./skills/my-skill --strict
Offline Sandbox Execution
Execute bundled Python scripts in an isolated subprocess to verify runtime network and process boundaries:
skillforge audit ./skills/my-skill --sandbox
Machine-Readable JSON
Emit structured audit reports for integration into pre-commit hooks or automated review bots:
skillforge audit ./skills/my-skill --json
Example JSON Output:
{
"target": "my-skill",
"passed": true,
"score": 100,
"findings_count": 0,
"findings": []
}
Regression Corpus
tests/test_corpus.py holds known-malicious and known-benign skill snippets. Each malicious case names the rule that must fire, and benign cases must produce no CRITICAL or HIGH finding. The corpus is small and hand-written, so it guards against regressions in the existing patterns; it does not measure real-world detection rates. Add a case whenever you fix a missed detection or a false positive.
Honest Static Scanner Scope & Limitations
Static analysis is an essential first line of defense for agent skill packages, but it has defined boundaries:
- What Static Analysis Guarantees:
- Deterministic discovery of malicious shell pipelines, reverse shells, and untrusted network endpoints.
- AST taint tracking in bundled Python scripts to prevent environment variable leaks and code injection.
- Zero-width unicode sanitization and prompt injection pattern detection.
-
Resource pointer integrity and identifier formatting verification.
-
What Requires Human-in-the-Loop & Runtime Sandboxing:
- Semantic Policy Evasion: Subtly biased or evasive natural language instructions that do not contain explicit injection phrases or code execution payloads cannot be definitively categorized by pattern matching alone.
- Dynamic Server Payloads: If a skill instructs an agent to call an external API that later changes its payload response, static analysis cannot inspect runtime remote data.
- Complementary Defenses: We recommend pairing SkillForge static audits with runtime container sandboxing (e.g. gVisor, Docker, or Windows sandbox environments) and requiring explicit user approval for destructive commands.