diff --git a/.github/workflows/wiki-sync.yml b/.github/workflows/wiki-sync.yml new file mode 100644 index 0000000..22e0ca7 --- /dev/null +++ b/.github/workflows/wiki-sync.yml @@ -0,0 +1,64 @@ +name: Sync Wiki + +on: + push: + branches: [main] + paths: + - 'wiki/**' + workflow_dispatch: + +permissions: + contents: write + +concurrency: + group: wiki-sync + cancel-in-progress: false + +jobs: + sync-wiki: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + + - name: Sync wiki folder to repository wiki + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + + if [ ! -d wiki ]; then + echo "::error::wiki/ directory not found" + exit 1 + fi + + WIKI_REMOTE="https://x-access-token:${GITHUB_TOKEN}@github.com/${{ github.repository }}.wiki.git" + if ! git ls-remote "$WIKI_REMOTE" >/dev/null 2>&1; then + echo "::warning::Wiki remote unavailable (repository wiki may be disabled). Skipping sync." + exit 0 + fi + + WIKI_TMP="$(mktemp -d)" + trap 'rm -rf "$WIKI_TMP"' EXIT + + git clone --depth 1 "$WIKI_REMOTE" "$WIKI_TMP" + rsync -a --delete --exclude '.git/' wiki/ "$WIKI_TMP/" + + cd "$WIKI_TMP" + if [ -z "$(git status --porcelain)" ]; then + echo "No wiki changes to sync." + exit 0 + fi + + WIKI_HEAD_REF="$(git symbolic-ref --short refs/remotes/origin/HEAD 2>/dev/null || true)" + if [ -n "$WIKI_HEAD_REF" ]; then + WIKI_BRANCH="${WIKI_HEAD_REF#origin/}" + else + WIKI_BRANCH="master" + fi + + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add -A + git commit -m "docs(wiki): sync from ${GITHUB_SHA}" + git push origin HEAD:"$WIKI_BRANCH" diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 26a05fe..a3a0379 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,8 +2,13 @@ Thank you for your interest in contributing security skills to the ClawSec ecosystem! This guide will walk you through creating, testing, and submitting new skills. +## Wiki Documentation Source of Truth + +For contributor-facing wiki docs, treat `wiki/` in this repository as the single source of truth. Do not edit the GitHub Wiki directly; `.github/workflows/wiki-sync.yml` publishes `wiki/` to `.wiki.git` when `wiki/**` changes on `main`. + ## Table of Contents +- [Wiki Documentation Source of Truth](#wiki-documentation-source-of-truth) - [Getting Started](#getting-started) - [Skill Structure](#skill-structure) - [Creating a New Skill](#creating-a-new-skill) diff --git a/README.md b/README.md index 972bf6e..849b6d5 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,12 @@ ClawSec is a **complete security skill suite for AI agent platforms**. It provid --- +## 📚 Documentation Source of Truth + +For all wiki content, edit files under `wiki/` in this repository. The GitHub Wiki (`.wiki.git`) is synced from `wiki/` by `.github/workflows/wiki-sync.yml` when `wiki/**` changes on `main`. + +--- + ## 🚀 Quick Start ### For AI Agents @@ -90,6 +96,20 @@ For Windows (PowerShell): Troubleshooting: if you see directories such as `~/.openclaw/workspace/$HOME/...`, a home variable was passed literally. Re-run using an absolute path or an unquoted home expression. +### Product Demos + +#### Install Demo (`clawsec-suite`) + + + +Direct link: [install-demo.mp4](https://raw.githubusercontent.com/prompt-security/clawsec/main/public/video/install-demo.mp4) + +#### Drift Detection Demo (`soul-guardian`) + + + +Direct link: [soul-guardian-demo.mp4](https://raw.githubusercontent.com/prompt-security/clawsec/main/public/video/soul-guardian-demo.mp4) + --- ## 📱 NanoClaw Platform Support diff --git a/public/video/install-demo.mp4 b/public/video/install-demo.mp4 new file mode 100644 index 0000000..a064a07 Binary files /dev/null and b/public/video/install-demo.mp4 differ diff --git a/public/video/soul-guardian-demo.mp4 b/public/video/soul-guardian-demo.mp4 new file mode 100644 index 0000000..6fb6c04 Binary files /dev/null and b/public/video/soul-guardian-demo.mp4 differ diff --git a/wiki/GENERATION.md b/wiki/GENERATION.md new file mode 100644 index 0000000..b164c52 --- /dev/null +++ b/wiki/GENERATION.md @@ -0,0 +1,26 @@ +# Wiki Generation Metadata + +- Commit hash: `448aed326192d38812cb508820f967cb74e77ae9` +- Branch name: `main` +- Generation timestamp (local): `2026-02-25T20:59:57+0200` +- Generation mode: `initial` +- Output language: `English` +- Assets copied into `wiki/assets/`: + - `overview_img_01_prompt-security-logo.png` (from `img/Black+Color.png`) + - `overview_img_02_clawsec-mascot.png` (from `public/img/mascot.png`) + - `architecture_img_01_prompt-line.svg` (from `public/img/prompt_line.svg`) + +## Notes +- This is a first-time generation (`wiki/` did not exist before this run). +- Index sections were generated from repository structure and created wiki pages. +- Future updates should preserve existing headings and append `Update Notes` sections when making deltas. + +## Source References +- README.md +- package.json +- AGENTS.md +- wiki/overview.md +- wiki/architecture.md +- wiki/dependencies.md +- wiki/data-flow.md +- wiki/glossary.md diff --git a/wiki/INDEX.md b/wiki/INDEX.md new file mode 100644 index 0000000..7a59ed7 --- /dev/null +++ b/wiki/INDEX.md @@ -0,0 +1,43 @@ +# Wiki Index + +## Summary +- Purpose: Document ClawSec as a combined web catalog, signed advisory channel, and multi-skill security distribution system. +- Tech stack: React 19 + Vite + TypeScript frontend, Node/ESM scripts, Python utilities, Bash automation, GitHub Actions pipelines. +- Entry points: `index.tsx`, `App.tsx`, `scripts/prepare-to-push.sh`, `scripts/populate-local-feed.sh`, `scripts/populate-local-skills.sh`, workflow files under `.github/workflows/`. +- Where to start: Read [Overview](overview.md), then [Architecture](architecture.md), then module pages for the area you are editing. +- How to navigate: Use Guides for cross-cutting concerns, Modules for implementation boundaries, and Source References at the end of each page to jump into code. + +## Start Here +- [Overview](overview.md) +- [Architecture](architecture.md) + +## Guides +- [Dependencies](dependencies.md) +- [Data Flow](data-flow.md) +- [Configuration](configuration.md) +- [Testing](testing.md) +- [Workflow](workflow.md) +- [Security](security.md) + +## Modules +- [Frontend Web App](modules/frontend-web.md) +- [ClawSec Suite Core](modules/clawsec-suite.md) +- [NanoClaw Integration](modules/nanoclaw-integration.md) +- [Automation and Release Pipelines](modules/automation-release.md) +- [Local Validation and Packaging Tools](modules/local-tooling.md) + +## Glossary +- [Glossary](glossary.md) + +## Generation Metadata +- [Generation Metadata](GENERATION.md) + +## Source References +- README.md +- App.tsx +- package.json +- scripts/prepare-to-push.sh +- scripts/populate-local-feed.sh +- scripts/populate-local-skills.sh +- skills/clawsec-suite/skill.json +- .github/workflows/ci.yml diff --git a/wiki/architecture.md b/wiki/architecture.md new file mode 100644 index 0000000..2bf2a05 --- /dev/null +++ b/wiki/architecture.md @@ -0,0 +1,129 @@ +# Architecture + +## System Context +- This page appears under the `Start Here` section in `INDEX.md`. +- ClawSec sits between upstream intelligence sources (NVD + community issues), GitHub automation, and runtime agent environments. +- The repository publishes both static site content and signed artifacts that runtime skills verify before using. +- External actor groups: + - GitHub Actions runners executing CI, release, and feed workflows. + - OpenClaw/NanoClaw agents consuming skills, advisories, and verification scripts. + - Repository maintainers approving advisory issues and merging release/tag changes. + +## Components +| Component | Location | Responsibility | +| --- | --- | --- | +| Web UI | `App.tsx`, `pages/`, `components/` | Renders skills catalog and advisory detail experiences. | +| Advisory Feed Core | `advisories/feed.json*`, `skills/clawsec-suite/.../feed.mjs` | Stores, verifies, and parses advisories with detached signatures/checksums. | +| Skill Packages | `skills/*/` | Distributes installable security capabilities with SBOM metadata. | +| Local Automation Scripts | `scripts/*.sh` | Build local mirrors, pre-push checks, and manual release helpers. | +| CI/CD Workflows | `.github/workflows/*.yml` | Linting, tests, NVD polling, release packaging, and Pages deploy. | +| Python Utility Layer | `utils/*.py` | Skill metadata validation and checksum generation. | + +## Key Flows +- Skill catalog flow: + 1. Release/tag workflows publish skill assets. + 2. Deploy workflow discovers release assets and builds `public/skills/index.json`. + 3. UI fetches `public/skills/index.json` and skill docs for `/skills` pages. +- Advisory feed flow: + 1. `poll-nvd-cves.yml` and `community-advisory.yml` update `advisories/feed.json`. + 2. Feed is signed and mirrored to public paths. + 3. Runtime hooks/scripts load remote feed and fallback to local signed copies. +- Guarded install flow: + 1. Installer requests target skill + version. + 2. Advisory matcher checks affected specifiers and severity/risk hints. + 3. Exit code 42 enforces second confirmation when advisories match. + +## Diagrams +```mermaid +flowchart TD + A["NVD + Community Inputs"] --> B["Feed Workflows\n(poll/community)"] + B --> C["advisories/feed.json + signatures"] + C --> D["Deploy Workflow Mirrors to public/"] + D --> E["React UI (catalog/feed pages)"] + C --> F["clawsec-suite hook + installers"] + F --> G["Agent advisory alerts / gated install"] +``` + +![Prompt Line Motif](assets/architecture_img_01_prompt-line.svg) + +## Interfaces and Contracts +| Interface | Contract Form | Validation | +| --- | --- | --- | +| Skill metadata | `skills/*/skill.json` | Validated by Python utility + CI version-parity checks. | +| Advisory feed | JSON + Ed25519 detached signature | Verified by `feed.mjs` and NanoClaw signature utilities. | +| Checksums manifest | `checksums.json` (+ optional `.sig`) | Parsed and hash-matched before trusting payloads. | +| Hook event interface | `HookEvent` (`type`, `action`, `messages`) | Runtime handler only processes selected event names. | +| Workflow release naming | Tag pattern `-vX.Y.Z` | Parsed in release/deploy workflows to discover skills. | + +## Key Parameters +| Parameter | Default | Effect | +| --- | --- | --- | +| `CLAWSEC_FEED_URL` | `https://clawsec.prompt.security/advisories/feed.json` | Remote advisory source for suite scripts/hooks. | +| `CLAWSEC_ALLOW_UNSIGNED_FEED` | `0` | Enables temporary unsigned fallback compatibility. | +| `CLAWSEC_VERIFY_CHECKSUM_MANIFEST` | `1` | Requires checksum manifest verification where available. | +| `CLAWSEC_HOOK_INTERVAL_SECONDS` | `300` | Scan throttling window for advisory hook. | +| `CLAWSEC_SKILLS_INDEX_TIMEOUT_MS` | `5000` | Remote skill index fetch timeout for catalog discovery. | +| `PROMPTSEC_GIT_PULL` | `0` | Optional auto-pull before watchdog audit runs. | + +## Error Handling and Reliability +- Feed fetching is fail-closed for invalid signatures and malformed manifests. +- Remote fetch failures gracefully fall back to local signed feeds. +- Hook state uses atomic file writes with strict mode where supported. +- UI pages detect HTML fallbacks served as JSON and avoid rendering corrupted data. +- Workflow steps enforce key-fingerprint consistency to avoid split-key drift. + +## Example Snippets +```tsx +// Route topology in the web app + + } /> + } /> + } /> + } /> + } /> + +``` + +```ts +// Guarded feed loading contract in advisory hook +const remoteFeed = await loadRemoteFeed(feedUrl, { + signatureUrl: feedSignatureUrl, + checksumsUrl: feedChecksumsUrl, + checksumsSignatureUrl: feedChecksumsSignatureUrl, + publicKeyPem, + checksumsPublicKeyPem: publicKeyPem, + allowUnsigned, + verifyChecksumManifest, +}); +``` + +## Runtime and Deployment +| Runtime Surface | Execution Model | Output | +| --- | --- | --- | +| Vite app (`npm run dev`) | Local frontend server | Interactive web app for feed/skills. | +| GitHub CI | Multi-OS matrix + dedicated jobs | Lint/type/build/security and test confidence. | +| Skill release workflow | Tag-driven | Release assets, signed checksums, optional ClawHub publish. | +| Pages deploy workflow | Triggered by CI/Release success | Static site + mirrored advisories/releases. | +| Runtime hooks | OpenClaw event hooks / NanoClaw IPC | Advisory alerts, gating decisions, integrity checks. | + +## Scaling Notes +- Advisory volume scales with keyword set in NVD polling; dedupe and post-filtering control noise. +- Deploy workflow processes release lists and keeps newest skill versions in index output. +- Module boundaries by skill folder allow adding new security capabilities without changing frontend structure. +- Signature verification paths remain lightweight because payload sizes (feed/manifests) are small. + +## Source References +- App.tsx +- pages/SkillsCatalog.tsx +- pages/FeedSetup.tsx +- pages/AdvisoryDetail.tsx +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/handler.ts +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/feed.mjs +- skills/clawsec-suite/scripts/guarded_skill_install.mjs +- skills/clawsec-suite/scripts/discover_skill_catalog.mjs +- skills/clawsec-nanoclaw/lib/advisories.ts +- skills/clawsec-nanoclaw/lib/signatures.ts +- .github/workflows/poll-nvd-cves.yml +- .github/workflows/community-advisory.yml +- .github/workflows/deploy-pages.yml +- .github/workflows/skill-release.yml diff --git a/wiki/assets/architecture_img_01_prompt-line.svg b/wiki/assets/architecture_img_01_prompt-line.svg new file mode 100644 index 0000000..3b9307c --- /dev/null +++ b/wiki/assets/architecture_img_01_prompt-line.svg @@ -0,0 +1,20 @@ + + + + + + + + + + \ No newline at end of file diff --git a/wiki/assets/overview_img_01_prompt-security-logo.png b/wiki/assets/overview_img_01_prompt-security-logo.png new file mode 100644 index 0000000..57383e0 Binary files /dev/null and b/wiki/assets/overview_img_01_prompt-security-logo.png differ diff --git a/wiki/assets/overview_img_02_clawsec-mascot.png b/wiki/assets/overview_img_02_clawsec-mascot.png new file mode 100644 index 0000000..cc859c6 Binary files /dev/null and b/wiki/assets/overview_img_02_clawsec-mascot.png differ diff --git a/wiki/configuration.md b/wiki/configuration.md new file mode 100644 index 0000000..727bbb5 --- /dev/null +++ b/wiki/configuration.md @@ -0,0 +1,88 @@ +# Configuration + +## Scope +- Configuration spans frontend build settings, runtime feed paths, workflow triggers, and skill metadata contracts. +- Most runtime-sensitive controls are environment variables prefixed with `CLAWSEC_` or `OPENCLAW_`. +- Path normalization is security-sensitive and intentionally rejects unresolved home-token literals. + +## Core Runtime Variables +| Variable | Default | Used By | +| --- | --- | --- | +| `CLAWSEC_FEED_URL` | Hosted advisory URL | Suite hook and guarded installer feed loading. | +| `CLAWSEC_FEED_SIG_URL` | `.sig` | Detached signature source. | +| `CLAWSEC_FEED_CHECKSUMS_URL` | `checksums.json` near feed URL | Optional checksum-manifest source. | +| `CLAWSEC_FEED_PUBLIC_KEY` | Suite-local PEM file | Feed signature verification. | +| `CLAWSEC_ALLOW_UNSIGNED_FEED` | `0` | Temporary migration bypass flag. | +| `CLAWSEC_VERIFY_CHECKSUM_MANIFEST` | `1` | Enables checksum-manifest verification. | +| `CLAWSEC_HOOK_INTERVAL_SECONDS` | `300` | Advisory hook scan throttle. | + +## Path Resolution Rules +| Rule | Behavior | Enforcement Location | +| --- | --- | --- | +| `~` expansion | Resolved to detected home directory | Shared path utility functions in suite/watchdog scripts. | +| `$HOME` / `${HOME}` expansion | Resolved when unescaped | Same utilities. | +| Windows home tokens | `%USERPROFILE%`, `$env:USERPROFILE` normalized | Same utilities. | +| Escaped tokens (`\$HOME`) | Rejected with explicit error | Prevents accidental literal directory creation. | +| Invalid explicit path | Can fallback to default path with warning | `resolveConfiguredPath` helpers. | + +## Frontend and Build Configuration +- `vite.config.ts` defines port (`3000`), host (`0.0.0.0`), and path alias (`@`). +- `index.html` provides Tailwind runtime config, custom fonts, and base color tokens. +- `tsconfig.json` uses bundler module resolution, `noEmit`, and JSX runtime configuration. +- `eslint.config.js` applies TS, React, hooks, and script-specific lint rules. + +## Skill Metadata Configuration +| Field Group | Location | Function | +| --- | --- | --- | +| Core skill identity | `skills/*/skill.json` | Name/version/author/license/description metadata. | +| SBOM file list | `skill.json -> sbom.files` | Declares release-required artifacts. | +| Platform metadata | `openclaw` or `nanoclaw` blocks | CLI requirements, triggers, platform capability hints. | +| Suite catalog metadata | `skills/clawsec-suite/skill.json -> catalog` | Integrated/default/consent behavior for suite members. | + +## Workflow Configuration +- Schedule configuration exists in workflow `cron` entries (`poll-nvd-cves`, `codeql`, `scorecard`). +- Release workflow expects tag naming pattern `-v`. +- Deployment workflow is triggered by successful CI/release `workflow_run` events and manual dispatch. +- Composite signing action requires private key inputs and verifies signatures immediately after signing. + +## Example Snippets +```bash +# run guarded install with explicit local signed feed paths +CLAWSEC_LOCAL_FEED="$HOME/.openclaw/skills/clawsec-suite/advisories/feed.json" \ +CLAWSEC_LOCAL_FEED_SIG="$HOME/.openclaw/skills/clawsec-suite/advisories/feed.json.sig" \ +CLAWSEC_FEED_PUBLIC_KEY="$HOME/.openclaw/skills/clawsec-suite/advisories/feed-signing-public.pem" \ +node skills/clawsec-suite/scripts/guarded_skill_install.mjs --skill clawtributor --dry-run +``` + +```json +{ + "name": "example-skill", + "version": "1.2.3", + "sbom": { + "files": [ + { "path": "SKILL.md", "required": true, "description": "Install docs" } + ] + } +} +``` + +## Operational Notes +- Keep signing keys outside the repository and inject via GitHub Secrets only. +- Prefer absolute paths or unescaped home expressions in local environment variable overrides. +- Treat unsigned feed mode as temporary migration support, not normal operation. +- Re-run release-link validation when editing `SKILL.md` URLs to avoid broken artifact references. + +## Source References +- vite.config.ts +- index.html +- tsconfig.json +- eslint.config.js +- skills/clawsec-suite/skill.json +- skills/clawsec-nanoclaw/skill.json +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/utils.mjs +- skills/openclaw-audit-watchdog/scripts/load_suppression_config.mjs +- skills/clawsec-suite/scripts/guarded_skill_install.mjs +- scripts/validate-release-links.sh +- .github/workflows/poll-nvd-cves.yml +- .github/workflows/skill-release.yml +- .github/actions/sign-and-verify/action.yml diff --git a/wiki/data-flow.md b/wiki/data-flow.md new file mode 100644 index 0000000..a459980 --- /dev/null +++ b/wiki/data-flow.md @@ -0,0 +1,98 @@ +# Data Flow + +## Primary Flows +- `Advisory ingestion`: NVD/community inputs are transformed into a normalized advisory feed, signed, then mirrored for clients. +- `Skill catalog publication`: release assets are discovered and converted into `public/skills/index.json` plus per-skill docs/checksums. +- `Runtime enforcement`: suite and nanoclaw consumers load advisory data, match against skills, and emit alerts or confirmation gates. +- This page appears under the `Guides` section in `INDEX.md`. + +## Step-by-Step +1. Feed producer workflow/script fetches source data (`NVD API` or issue payload). +2. JSON transform logic normalizes severity/type/affected fields and deduplicates by advisory ID. +3. Signature/checksum steps generate detached signatures and checksum manifests. +4. Deploy workflow mirrors signed artifacts under `public/` and `public/releases/latest/download/`. +5. UI and runtime consumers fetch feed/index files and validate format/signatures before use. +6. Matchers compare `affected` specifiers to skill names/versions and emit alerts or enforce confirmation. + +## Inputs and Outputs +Inputs/outputs are summarized in the table below. + +| Type | Name | Location | Description | +| --- | --- | --- | --- | +| Input | CVE payloads | `services.nvd.nist.gov/rest/json/cves/2.0` | Source vulnerabilities filtered by ClawSec keywords. | +| Input | Community advisory issue | `.github/workflows/community-advisory.yml` event payload | Maintainer-approved issue transformed into advisory record. | +| Input | Skill release assets | GitHub Releases API + assets | Used to build web catalog and mirror downloads. | +| Input | Local config/env | `OPENCLAW_AUDIT_CONFIG`, `CLAWSEC_*` vars | Controls feed pathing, suppression, and verification behavior. | +| Output | Advisory feed | `advisories/feed.json` | Canonical repository feed. | +| Output | Advisory signature | `advisories/feed.json.sig` | Detached signature for feed authenticity. | +| Output | Skill catalog index | `public/skills/index.json` | Runtime web catalog used by `/skills` pages. | +| Output | Release checksums/signatures | `release-assets/checksums.json(.sig)` | Integrity manifest for release consumers. | +| Output | Hook state | `~/.openclaw/clawsec-suite-feed-state.json` | Tracks scan timing and notified matches. | + +## Data Structures +| Structure | Key Fields | Purpose | +| --- | --- | --- | +| Advisory feed record | `id`, `severity`, `type`, `affected[]`, `published` | Unit of risk data used by UI and installers. | +| Skill metadata record | `id`, `name`, `version`, `emoji`, `tag` | Catalog row for web browsing and install commands. | +| Checksums manifest | `schema_version`, `algorithm`, `files` | Maps file names to expected digests. | +| Advisory state | `known_advisories`, `last_hook_scan`, `notified_matches` | Prevents repeated alerts and throttles scans. | +| Suppression config | `enabledFor[]`, `suppressions[]` | Targeted skip list by `checkId` + `skill`. | + +## Diagrams +```mermaid +flowchart LR + A["NVD + Issue Inputs"] --> B["Transform + Deduplicate"] + B --> C["advisories/feed.json"] + C --> D["Sign + checksums"] + D --> E["public/advisories + releases/latest"] + E --> F["Web UI fetch"] + E --> G["Suite/NanoClaw verification"] + G --> H["Match skills + emit alerts/gates"] +``` + +## State and Storage +| Store | Path/Scope | Write Path | +| --- | --- | --- | +| Canonical advisories | `advisories/` | NVD + community workflows and local populate script. | +| Embedded advisory copies | `skills/clawsec-feed/advisories/` and `skills/clawsec-suite/advisories/` | Sync/packaging processes and release workflow. | +| Public mirrors | `public/advisories/`, `public/releases/` | Deploy workflow. | +| Runtime state | `~/.openclaw/clawsec-suite-feed-state.json` | Advisory hook state persistence. | +| NanoClaw cache | `/workspace/project/data/clawsec-advisory-cache.json` | Host-side advisory cache manager. | +| Integrity state | `/workspace/project/data/soul-guardian/` (NanoClaw) | Integrity monitor baseline/audit storage. | + +## Example Snippets +```bash +# Local feed flow (NVD fetch -> transform -> sync) +./scripts/populate-local-feed.sh --days 120 +jq '.updated, (.advisories | length)' advisories/feed.json +``` + +```bash +# Runtime guarded install uses signed feed paths +CLAWSEC_LOCAL_FEED=~/.openclaw/skills/clawsec-suite/advisories/feed.json \ +CLAWSEC_FEED_PUBLIC_KEY=~/.openclaw/skills/clawsec-suite/advisories/feed-signing-public.pem \ +node skills/clawsec-suite/scripts/guarded_skill_install.mjs --skill test-skill --dry-run +``` + +## Failure Modes +- NVD rate limits (`403/429`) can delay feed refresh and require retries/backoff. +- Missing or invalid detached signatures cause feed rejection in fail-closed mode. +- HTML fallback responses for JSON endpoints can produce false positives unless explicitly filtered. +- Path-token misconfiguration (`\$HOME`) can break local fallback path resolution. +- Mismatched public key fingerprints in workflows trigger hard CI failure. + +## Source References +- advisories/feed.json +- advisories/feed.json.sig +- scripts/populate-local-feed.sh +- scripts/populate-local-skills.sh +- .github/workflows/poll-nvd-cves.yml +- .github/workflows/community-advisory.yml +- .github/workflows/deploy-pages.yml +- .github/workflows/skill-release.yml +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/feed.mjs +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/state.ts +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/matching.ts +- skills/clawsec-suite/scripts/guarded_skill_install.mjs +- skills/clawsec-nanoclaw/lib/advisories.ts +- skills/clawsec-nanoclaw/host-services/advisory-cache.ts diff --git a/wiki/dependencies.md b/wiki/dependencies.md new file mode 100644 index 0000000..1b702b8 --- /dev/null +++ b/wiki/dependencies.md @@ -0,0 +1,101 @@ +# Dependencies + +## Build and Runtime +| Layer | Primary Dependencies | Why It Exists | +| --- | --- | --- | +| Frontend runtime | `react`, `react-dom`, `react-router-dom`, `lucide-react` | UI rendering, routing, iconography. | +| Markdown rendering | `react-markdown`, `remark-gfm` | Render skill docs/readmes in detail pages. | +| Build tooling | `vite`, `@vitejs/plugin-react`, `typescript` | Fast TS/TSX bundling and production builds. | +| Python utilities | stdlib + `ruff`/`bandit` policy from `pyproject.toml` | Validate/package skills and run static checks. | +| Shell automation | `bash`, `jq`, `curl`, `openssl`, `sha256sum`/`shasum` | Feed polling, signing, checksum generation, release checks. | + +## Dependency Details +| Package | Version Constraint | Scope | +| --- | --- | --- | +| `react` / `react-dom` | `^19.2.4` | Frontend runtime | +| `react-router-dom` | `^7.13.1` | Frontend routing | +| `lucide-react` | `^0.575.0` | UI icon set | +| `vite` | `^7.3.1` | Dev server + build | +| `typescript` | `~5.8.2` | Type checking | +| `eslint` | `^9.39.2` | JS/TS linting | +| `@typescript-eslint/*` | `^8.55.0` / `^8.56.0` | TS lint parser/rules | +| `fast-check` | `^4.5.3` | Property/fuzz style tests | + +| Override | Pinned Version | Rationale | +| --- | --- | --- | +| `ajv` | `6.14.0` | Security and compatibility stabilization. | +| `balanced-match` | `4.0.3` | Transitive vulnerability control. | +| `brace-expansion` | `5.0.2` | Transitive dependency hardening. | +| `minimatch` | `10.2.1` | Deterministic dependency behavior. | + +## External Services +| Service | Used By | Function | +| --- | --- | --- | +| NVD API (`services.nvd.nist.gov`) | `poll-nvd-cves` workflow + local feed script | Pull CVEs by keyword/date window. | +| GitHub API | Deploy/release workflows | Discover releases, download assets, publish outputs. | +| GitHub Pages | Deploy workflow | Serve static site and mirrored artifacts. | +| ClawHub CLI/registry | Install scripts + optional publish jobs | Install and publish skills. | +| Optional local SMTP/sendmail | `openclaw-audit-watchdog` scripts | Deliver audit reports by email. | + +## Development Tools +| Tool | Invocation | Coverage | +| --- | --- | --- | +| ESLint | `npx eslint . --ext .ts,.tsx,.js,.jsx,.mjs --max-warnings 0` | Frontend and script linting. | +| TypeScript | `npx tsc --noEmit` | Compile-time TS contract checks. | +| Ruff | `ruff check utils/` | Python style and bug pattern checks. | +| Bandit | `bandit -r utils/ -ll` | Python security checks. | +| Trivy | Workflow + optional local run | FS/config vulnerability scans. | +| Gitleaks | Workflow + optional local run | Secret leak detection. | + +## Example Snippets +```json +{ + "scripts": { + "dev": "vite", + "build": "vite build", + "preview": "vite preview" + }, + "dependencies": { + "react": "^19.2.4", + "react-router-dom": "^7.13.1" + } +} +``` + +```toml +[tool.ruff] +target-version = "py310" +line-length = 120 + +[tool.bandit] +exclude_dirs = ["__pycache__", ".venv"] +skips = ["B101"] +``` + +## Compatibility Notes +- Local scripts account for macOS vs Linux differences in `date` and `stat` usage. +- Some workflows/scripts require OpenSSL features used with Ed25519 and `pkeyutl -rawin`. +- Windows support is strongest for Node-based tooling; POSIX shell paths may require WSL/Git Bash. +- Feed consumers include compatibility bypasses for migration phases, but signed mode is the intended steady state. + +## Versioning Notes +- Skill release tags follow `-v` and are parsed by CI/deploy automation. +- PR validation enforces version parity between `skill.json` and `SKILL.md` frontmatter for bumped skills. +- The public skills index keeps latest discovered version per skill for UI display. +- Signed artifact manifests (`checksums.json`) are versioned per release and include file hashes and URLs. + +## Source References +- package.json +- package-lock.json +- pyproject.toml +- eslint.config.js +- tsconfig.json +- scripts/prepare-to-push.sh +- scripts/populate-local-feed.sh +- scripts/populate-local-skills.sh +- .github/workflows/ci.yml +- .github/workflows/codeql.yml +- .github/workflows/scorecard.yml +- .github/workflows/poll-nvd-cves.yml +- .github/workflows/deploy-pages.yml +- .github/workflows/skill-release.yml diff --git a/wiki/glossary.md b/wiki/glossary.md new file mode 100644 index 0000000..c3dc6da --- /dev/null +++ b/wiki/glossary.md @@ -0,0 +1,57 @@ +# Glossary + +## Terms +| Term | Definition | +| --- | --- | +| Advisory Feed | JSON document (`feed.json`) containing security advisories for skills/platforms. | +| Affected Specifier | Skill selector such as `skill@1.2.3`, wildcard, or range used in matching logic. | +| Guarded Install | Two-step installer behavior that requires explicit confirmation when advisories match. | +| SBOM Files | Skill-declared artifact list in `skill.json` used for packaging and validation. | +| Detached Signature | Base64 signature file (`.sig`) stored separately from signed payload. | +| Checksum Manifest | File hash map (`checksums.json`) used to verify payload integrity. | + +## Skill Packaging Terms +| Term | Definition | +| --- | --- | +| Skill Tag | Git tag formatted as `-v` used by release automation. | +| Release Assets | Files attached to GitHub release (zip, `skill.json`, checksums, signatures). | +| Catalog Index | `public/skills/index.json`, generated list consumed by web catalog. | +| Embedded Components | Capability bundle from one skill included in another (for example feed embedded in suite). | + +## Advisory and Security Terms +| Term | Definition | +| --- | --- | +| Fail-Closed Verification | Reject payload if signature or checksum validation fails. | +| Unsigned Compatibility Mode | Temporary bypass path enabled via `CLAWSEC_ALLOW_UNSIGNED_FEED=1`. | +| Suppression Rule | Config entry matching `checkId` and `skill` to suppress known/accepted findings. | +| Key Fingerprint | SHA-256 digest of DER-encoded public key used for key consistency checks. | + +## Runtime and Platform Terms +| Term | Definition | +| --- | --- | +| OpenClaw Hook | Runtime event handler (`clawsec-advisory-guardian`) that checks advisories. | +| NanoClaw IPC | Host/container task exchange for advisory refresh, signature verification, integrity checks. | +| Integrity Baseline | Stored approved hashes/snapshots for protected files. | +| Hash-Chained Audit Log | Append-only audit log where each entry depends on prior hash. | + +## CI/CD Terms +| Term | Definition | +| --- | --- | +| Poll NVD CVEs Workflow | Scheduled workflow that fetches and transforms NVD CVEs into advisories. | +| Community Advisory Workflow | Issue-label-triggered workflow that publishes approved community advisories. | +| Skill Release Workflow | Tag-triggered packaging/signing/publishing pipeline for skills. | +| Deploy Pages Workflow | Workflow that builds site assets and mirrors release/advisory artifacts. | + +## Source References +- types.ts +- skills/clawsec-suite/skill.json +- skills/clawsec-nanoclaw/skill.json +- skills/clawsec-suite/scripts/guarded_skill_install.mjs +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/feed.mjs +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/suppression.mjs +- skills/clawsec-nanoclaw/guardian/integrity-monitor.ts +- scripts/populate-local-feed.sh +- .github/workflows/poll-nvd-cves.yml +- .github/workflows/community-advisory.yml +- .github/workflows/skill-release.yml +- .github/workflows/deploy-pages.yml diff --git a/wiki/modules/automation-release.md b/wiki/modules/automation-release.md new file mode 100644 index 0000000..0424dbc --- /dev/null +++ b/wiki/modules/automation-release.md @@ -0,0 +1,92 @@ +# Module: Automation and Release Pipelines + +## Responsibilities +- Enforce repository quality/security checks before merge and deployment. +- Generate and maintain advisory feed updates from automated and community sources. +- Package, sign, and publish skill release artifacts from tag events. +- Build and deploy static website outputs and mirrored release/advisory assets. + +## Key Files +- `.github/workflows/ci.yml`: lint/type/build/security/test matrix. +- `.github/workflows/poll-nvd-cves.yml`: daily NVD advisory ingestion. +- `.github/workflows/community-advisory.yml`: issue-label-driven advisory publishing. +- `.github/workflows/skill-release.yml`: release validation, packaging, signing, and publishing. +- `.github/workflows/deploy-pages.yml`: site build + asset mirroring to GitHub Pages. +- `.github/actions/sign-and-verify/action.yml`: shared Ed25519 sign/verify composite action. +- `scripts/prepare-to-push.sh`: local CI-like quality gate. +- `scripts/release-skill.sh`: manual helper for version bump + tag workflow. + +## Public Interfaces +| Interface | Trigger | Outcome | +| --- | --- | --- | +| CI workflow | Push/PR on `main` | Fails fast on lint/type/build/test/security regressions. | +| NVD poll workflow | Cron + dispatch | Updates advisory feed with deduped, normalized CVEs. | +| Community advisory workflow | Issue labeled `advisory-approved` | Opens PR adding signed advisory records. | +| Skill release workflow | Tag `-v*` | Creates GitHub release assets and signatures. | +| Deploy pages workflow | Successful CI/release run | Publishes site + mirrored artifacts to Pages. | + +## Inputs and Outputs +Inputs/outputs are summarized in the table below. + +| Type | Name | Location | Description | +| --- | --- | --- | --- | +| Input | Git refs/events | GitHub Actions event payloads | Determines which workflow path runs. | +| Input | Skill metadata/SBOM | `skills/*/skill.json` | Drives release asset assembly and validation. | +| Input | NVD API data | External API responses | Source CVEs for advisory feed generation. | +| Input | Signing secrets | GitHub Secrets | Private key material for signing artifacts. | +| Output | Signed advisories | `advisories/feed.json(.sig)` + mirrored public files | Consumable signed feed channel. | +| Output | Skill release assets | `release-assets/*` and GitHub release attachments | Installable and verifiable skill artifacts. | +| Output | Website build | `dist/` deployment artifact | Public web frontend and mirrors. | + +## Configuration +| Config Point | Location | Notes | +| --- | --- | --- | +| Workflow schedules | `poll-nvd-cves.yml`, `codeql.yml`, `scorecard.yml` | Daily/weekly security automation cadence. | +| Concurrency groups | Workflow `concurrency` blocks | Prevents destructive overlap in key pipelines. | +| Signing key checks | `scripts/ci/verify_signing_key_consistency.sh` | Ensures docs and canonical PEM files align. | +| Local pre-push gating | `scripts/prepare-to-push.sh` | Mirrors CI checks with optional auto-fix. | + +## Example Snippets +```yaml +# skill release trigger pattern +on: + push: + tags: + - '*-v[0-9]*.[0-9]*.[0-9]*' +``` + +```bash +# local all-in-one pre-push gate +./scripts/prepare-to-push.sh +# optional auto-fix +./scripts/prepare-to-push.sh --fix +``` + +## Edge Cases +- NVD API rate limiting (`403`/`429`) is handled with retry/backoff and can fail workflow on persistent errors. +- Release pipeline blocks on version mismatch between `skill.json` and `SKILL.md` frontmatter. +- Key fingerprint drift between canonical PEM files and docs hard-fails signing-related workflows. +- Deploy workflow intentionally allows unsigned legacy checksums for backward compatibility in some branches. +- Manual helper script has safety checks but includes destructive rollback logic in error branches; use carefully. + +## Tests +| Validation Layer | Location | +| --- | --- | +| Workflow execution tests | CI jobs in `.github/workflows/ci.yml` | +| Skill-level unit/property tests | `skills/*/test/*.test.mjs` invoked by CI | +| Local deterministic checks | `scripts/prepare-to-push.sh` | +| Release link checks | `scripts/validate-release-links.sh` | + +## Source References +- .github/workflows/ci.yml +- .github/workflows/poll-nvd-cves.yml +- .github/workflows/community-advisory.yml +- .github/workflows/skill-release.yml +- .github/workflows/deploy-pages.yml +- .github/workflows/codeql.yml +- .github/workflows/scorecard.yml +- .github/actions/sign-and-verify/action.yml +- scripts/prepare-to-push.sh +- scripts/release-skill.sh +- scripts/validate-release-links.sh +- scripts/ci/verify_signing_key_consistency.sh diff --git a/wiki/modules/clawsec-suite.md b/wiki/modules/clawsec-suite.md new file mode 100644 index 0000000..035e416 --- /dev/null +++ b/wiki/modules/clawsec-suite.md @@ -0,0 +1,96 @@ +# Module: ClawSec Suite Core + +## Responsibilities +- Act as the main skill-of-skills security bundle for OpenClaw-style agents. +- Verify advisory feed authenticity (Ed25519 signatures and optional checksum manifests). +- Detect advisory matches against installed skills and emit actionable runtime alerts. +- Enforce two-step confirmation for risky skill installations. + +## Key Files +- `skills/clawsec-suite/skill.json`: suite metadata, embedded components, catalog defaults. +- `skills/clawsec-suite/hooks/clawsec-advisory-guardian/handler.ts`: runtime event handler. +- `skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/feed.mjs`: signed feed loading/parsing. +- `skills/clawsec-suite/hooks/.../lib/matching.ts`: advisory-to-skill matching logic. +- `skills/clawsec-suite/hooks/.../lib/state.ts`: scan state persistence. +- `skills/clawsec-suite/hooks/.../lib/suppression.mjs`: allowlist-based suppression loader. +- `skills/clawsec-suite/scripts/guarded_skill_install.mjs`: advisory-gated installer wrapper. +- `skills/clawsec-suite/scripts/discover_skill_catalog.mjs`: remote/fallback catalog discovery. + +## Public Interfaces +| Interface | Consumer | Behavior | +| --- | --- | --- | +| Hook handler default export | OpenClaw hook runtime | Handles `agent:bootstrap` and `command:new` events. | +| `guarded_skill_install.mjs` CLI | Operators/automation | Blocks on advisory matches unless `--confirm-advisory`. | +| `discover_skill_catalog.mjs` CLI | Suite docs/automation | Lists installable skills with fallback metadata. | +| `feed.mjs` functions | Suite scripts and tests | Feed load, signature verification, checksum manifest checks. | +| Exit code contract | External wrappers | `42` indicates explicit second confirmation required. | + +## Inputs and Outputs +Inputs/outputs are summarized in the table below. + +| Type | Name | Location | Description | +| --- | --- | --- | --- | +| Input | Advisory feed + signatures | Remote URLs or local `advisories/` files | Risk intelligence source for hook and installer. | +| Input | Installed skill metadata | Skill directories under install root | Matcher compares installed versions to advisory affected specs. | +| Input | Suppression config | `OPENCLAW_AUDIT_CONFIG` or default config paths | Selective suppression by check and skill name. | +| Output | Runtime alert messages | Hook event `messages[]` | Advisories and recommended user actions. | +| Output | Persistent state | `~/.openclaw/clawsec-suite-feed-state.json` | De-dup alerts, track scan windows. | +| Output | CLI gating exit codes | Installer process status | Ensures deliberate user confirmation on risk. | + +## Configuration +| Variable | Default | Module Effect | +| --- | --- | --- | +| `CLAWSEC_FEED_URL` | Hosted advisory URL | Chooses primary remote feed endpoint. | +| `CLAWSEC_LOCAL_FEED*` vars | Suite-local advisories directory | Configures local signed fallback artifacts. | +| `CLAWSEC_FEED_PUBLIC_KEY` | `advisories/feed-signing-public.pem` | Verification key path. | +| `CLAWSEC_ALLOW_UNSIGNED_FEED` | `0` | Enables temporary migration bypass mode. | +| `CLAWSEC_VERIFY_CHECKSUM_MANIFEST` | `1` | Enables checksum manifest verification layer. | +| `CLAWSEC_HOOK_INTERVAL_SECONDS` | `300` | Controls event-driven scan throttling. | + +## Example Snippets +```ts +// hook only handles selected events +function shouldHandleEvent(event: HookEvent): boolean { + const eventName = toEventName(event); + return eventName === 'agent:bootstrap' || eventName === 'command:new'; +} +``` + +```js +// guarded installer confirmation contract +if (matches.length > 0 && !args.confirmAdvisory) { + process.stdout.write('Re-run with --confirm-advisory to proceed.\n'); + process.exit(EXIT_CONFIRM_REQUIRED); // 42 +} +``` + +## Edge Cases +- Missing/malformed feed signatures force remote rejection and local fallback attempts. +- Ambiguous checksum manifest basename collisions are treated as errors. +- Unknown skill versions are treated conservatively in version matching logic. +- Suppression is disabled unless config includes the pipeline sentinel (`enabledFor`). +- Invalid environment path tokens are rejected to avoid accidental literal path usage. + +## Tests +| Test File | Focus | +| --- | --- | +| `skills/clawsec-suite/test/feed_verification.test.mjs` | Signature/checksum verification and fail-closed behavior. | +| `skills/clawsec-suite/test/guarded_install.test.mjs` | Confirmation gating and match semantics. | +| `skills/clawsec-suite/test/path_resolution.test.mjs` | Home/path expansion and invalid token handling. | +| `skills/clawsec-suite/test/advisory_suppression.test.mjs` | Suppression config parsing and matching. | +| `skills/clawsec-suite/test/skill_catalog_discovery.test.mjs` | Remote index and fallback merge behavior. | + +## Source References +- skills/clawsec-suite/skill.json +- skills/clawsec-suite/SKILL.md +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/handler.ts +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/feed.mjs +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/matching.ts +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/state.ts +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/suppression.mjs +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/version.mjs +- skills/clawsec-suite/scripts/guarded_skill_install.mjs +- skills/clawsec-suite/scripts/discover_skill_catalog.mjs +- skills/clawsec-suite/test/feed_verification.test.mjs +- skills/clawsec-suite/test/guarded_install.test.mjs +- skills/clawsec-suite/test/path_resolution.test.mjs diff --git a/wiki/modules/frontend-web.md b/wiki/modules/frontend-web.md new file mode 100644 index 0000000..fa75a2c --- /dev/null +++ b/wiki/modules/frontend-web.md @@ -0,0 +1,98 @@ +# Module: Frontend Web App + +## Responsibilities +- Render the ClawSec website for home, skills catalog/detail, and advisory feed/detail experiences. +- Provide resilient JSON fetch behavior that handles SPA HTML fallback cases. +- Display install commands, checksums, and advisory metadata in a browser-focused UX. + +## Key Files +- `index.tsx`: React bootstrap and root mount. +- `App.tsx`: Router map and page entry wiring. +- `pages/Home.tsx`: Landing page, install card, animated platform/file labels. +- `pages/SkillsCatalog.tsx`: Catalog fetch/filter state machine and empty-state handling. +- `pages/SkillDetail.tsx`: Loads `skill.json`, checksums, README/SKILL docs with markdown renderer. +- `pages/FeedSetup.tsx`: Advisory listing UI with pagination. +- `pages/AdvisoryDetail.tsx`: Advisory deep-dive view and source links. +- `components/Layout.tsx` + `components/Header.tsx`: Shared shell and nav behavior. + +## Public Interfaces +- Browser routes: + - `/` + - `/skills` + - `/skills/:skillId` + - `/feed` + - `/feed/:advisoryId` +- Static fetch targets: + - `/skills/index.json` + - `/skills//skill.json` + - `/skills//checksums.json` + - `/advisories/feed.json` +- Display contracts: + - `SkillMetadata`, `SkillJson`, `SkillChecksums`, `AdvisoryFeed`, `Advisory` from `types.ts`. + +## Inputs and Outputs +Inputs/outputs are summarized in the table below. + +| Type | Name | Location | Description | +| --- | --- | --- | --- | +| Input | Skills index JSON | `/skills/index.json` | List of published skills and metadata. | +| Input | Skill payload files | `/skills//skill.json`, markdown docs, `checksums.json` | Detail-page content and integrity table. | +| Input | Advisory feed JSON | `/advisories/feed.json` or remote URL fallback | Advisory list/detail content. | +| Output | Route-specific UI states | Browser view state | Loading, empty, error, and populated experiences. | +| Output | Copy-to-clipboard commands | Clipboard API | Install and checksum snippets copied for users. | + +## Configuration +- Build/runtime config comes from: + - `vite.config.ts` (port, host, path alias) + - `index.html` Tailwind config + custom fonts/colors + - `constants.ts` (`ADVISORY_FEED_URL`, `LOCAL_FEED_PATH`) +- Runtime behavior assumptions: + - JSON responses may be empty or HTML fallback and must be validated. + - Advisory list pagination uses `ITEMS_PER_PAGE = 9`. + +## Example Snippets +```tsx +// Catalog fetch logic guards against HTML fallback responses +const contentType = response.headers.get('content-type') ?? ''; +const raw = await response.text(); +if (!raw.trim() || contentType.includes('text/html') || isProbablyHtmlDocument(raw)) { + setSkills([]); + setFilteredSkills([]); + return; +} +``` + +```tsx +// Route map defined in App.tsx +} /> +} /> +``` + +## Edge Cases +- Missing `skills/index.json` returns empty catalog instead of hard failure. +- Some environments return `index.html` for missing JSON paths with status `200`; code defends against this. +- Skill detail tolerates missing/malformed checksums and missing markdown docs. +- Advisory detail handles absent optional fields (`cvss_score`, `reporter`, `references`). + +## Tests +| Test Type | Location | Notes | +| --- | --- | --- | +| Type/lint/build checks | `scripts/prepare-to-push.sh` + CI | Frontend confidence comes from static checks and build success. | +| App-wide CI gates | `.github/workflows/ci.yml` | Multi-OS TypeScript/ESLint/build checks. | +| Manual smoke checks | `npm run dev` | Validate route rendering and fetch paths during development. | + +## Source References +- index.tsx +- App.tsx +- pages/Home.tsx +- pages/SkillsCatalog.tsx +- pages/SkillDetail.tsx +- pages/FeedSetup.tsx +- pages/AdvisoryDetail.tsx +- pages/Checksums.tsx +- components/Layout.tsx +- components/Header.tsx +- constants.ts +- types.ts +- vite.config.ts +- index.html diff --git a/wiki/modules/local-tooling.md b/wiki/modules/local-tooling.md new file mode 100644 index 0000000..b954496 --- /dev/null +++ b/wiki/modules/local-tooling.md @@ -0,0 +1,84 @@ +# Module: Local Validation and Packaging Tools + +## Responsibilities +- Validate skill directory metadata/schema and SBOM file presence before release. +- Generate per-skill checksums manifests for local testing or packaging. +- Provide local data bootstrap scripts that mirror CI behavior for advisories and skills. +- Offer release-link and signing-key consistency checks for maintainers. + +## Key Files +- `utils/validate_skill.py`: schema and file existence checks for a skill directory. +- `utils/package_skill.py`: checksum manifest generator with skill pre-validation. +- `scripts/populate-local-skills.sh`: generates local catalog and checksums under `public/skills/`. +- `scripts/populate-local-feed.sh`: pulls NVD data and updates feed copies. +- `scripts/validate-release-links.sh`: verifies docs reference releasable assets. +- `scripts/ci/verify_signing_key_consistency.sh`: verifies key fingerprints across docs/files. + +## Public Interfaces +| Tool | Interface | Primary Output | +| --- | --- | --- | +| `validate_skill.py` | `python utils/validate_skill.py ` | Exit code + validation summary with warnings/errors. | +| `package_skill.py` | `python utils/package_skill.py [out-dir]` | `checksums.json` artifact for skill files. | +| `populate-local-skills.sh` | shell CLI | `public/skills/index.json` and per-skill files/checksums. | +| `populate-local-feed.sh` | shell CLI flags `--days`, `--force` | Updated advisory feeds in repo/skill/public paths. | +| `validate-release-links.sh` | shell CLI optional skill arg | Release-link validation report. | + +## Inputs and Outputs +Inputs/outputs are summarized in the table below. + +| Type | Name | Location | Description | +| --- | --- | --- | --- | +| Input | Skill metadata and SBOM | `skills//skill.json` | Enumerates required files and release artifacts. | +| Input | Existing feed state | `advisories/feed.json` | Determines incremental NVD polling start date. | +| Input | Environment tools | `jq`, `curl`, `openssl`, Python runtime | Required execution dependencies. | +| Output | Validation diagnostics | stdout/stderr + exit code | Signals readiness for release/CI. | +| Output | Checksums manifests | `checksums.json` | Integrity data for skill artifacts. | +| Output | Local mirrors | `public/skills/*`, `public/advisories/feed.json` | Makes local web preview match CI outputs. | + +## Configuration +| Setting | Location | Purpose | +| --- | --- | --- | +| Ruff/Bandit policy | `pyproject.toml` | Python lint/security baseline. | +| CLI flags (`--days`, `--force`) | `populate-local-feed.sh` | Controls window and overwrite semantics. | +| `OPENCLAW_AUDIT_CONFIG` | suppression loaders in scripts | Chooses suppression config path. | +| `CLAWSEC_*` env vars | installer/hook scripts | Path and verification behavior tuning. | + +## Example Snippets +```bash +# validate and package a skill locally +python utils/validate_skill.py skills/clawsec-feed +python utils/package_skill.py skills/clawsec-feed ./dist +``` + +```bash +# refresh local UI data to mirror CI-generated artifacts +./scripts/populate-local-skills.sh +./scripts/populate-local-feed.sh --days 120 +``` + +## Edge Cases +- Validation allows warnings (for example missing optional files) while still returning success when required fields/files are present. +- NVD poll script handles macOS/Linux differences in `date` and `stat` utilities. +- Release-link validation can detect doc references to files missing from SBOM-derived release assets. +- Path expansion guards reject unexpanded home-token literals to avoid misdirected filesystem writes. + +## Tests +| Test/Check | Scope | +| --- | --- | +| `ruff check utils/` | Python style and correctness checks. | +| `bandit -r utils/ -ll` | Python security issue scan. | +| `scripts/prepare-to-push.sh` | Combined local gate across TS/Python/shell/security checks. | +| Skill-local tests | `skills/*/test/*.test.mjs` (targeted invocation) | + +## Source References +- utils/validate_skill.py +- utils/package_skill.py +- pyproject.toml +- scripts/populate-local-skills.sh +- scripts/populate-local-feed.sh +- scripts/prepare-to-push.sh +- scripts/validate-release-links.sh +- scripts/ci/verify_signing_key_consistency.sh +- skills/openclaw-audit-watchdog/scripts/load_suppression_config.mjs +- skills/clawsec-suite/scripts/guarded_skill_install.mjs +- skills/clawsec-suite/scripts/discover_skill_catalog.mjs diff --git a/wiki/modules/nanoclaw-integration.md b/wiki/modules/nanoclaw-integration.md new file mode 100644 index 0000000..b930ae8 --- /dev/null +++ b/wiki/modules/nanoclaw-integration.md @@ -0,0 +1,97 @@ +# Module: NanoClaw Integration + +## Responsibilities +- Port ClawSec advisory/signature logic into NanoClaw host+container architecture. +- Provide MCP tools that expose advisory checks, signature verification, and integrity monitoring. +- Maintain host-side cached advisory state with TLS/signature enforcement and IPC-triggered refresh. +- Protect critical NanoClaw files with baseline drift detection and hash-chained audit trails. + +## Key Files +- `skills/clawsec-nanoclaw/skill.json`: NanoClaw package contract and MCP tool registry. +- `skills/clawsec-nanoclaw/lib/signatures.ts`: secure fetch and Ed25519 verification primitives. +- `skills/clawsec-nanoclaw/lib/advisories.ts`: feed load and advisory matching helpers. +- `skills/clawsec-nanoclaw/host-services/advisory-cache.ts`: host cache manager. +- `skills/clawsec-nanoclaw/host-services/ipc-handlers.ts`: IPC request dispatch for advisory/signature tasks. +- `skills/clawsec-nanoclaw/host-services/skill-signature-handler.ts`: package signature verification service. +- `skills/clawsec-nanoclaw/guardian/integrity-monitor.ts`: baseline/diff/restore/audit engine. +- `skills/clawsec-nanoclaw/mcp-tools/*.ts`: container-side tool definitions. + +## Public Interfaces +| Interface | Context | Notes | +| --- | --- | --- | +| `clawsec_check_advisories` | MCP tool | Lists advisories affecting installed skills. | +| `clawsec_check_skill_safety` | MCP tool | Returns install recommendation for a specific skill. | +| `clawsec_verify_skill_package` | MCP tool | Verifies detached package signature through host IPC. | +| `clawsec_check_integrity` | MCP tool | Runs integrity check, optional auto-restore for critical targets. | +| IPC task `verify_skill_signature` | Host service | Returns structured verification response with error codes. | +| IPC task `refresh_advisory_cache` | Host service | Refreshes signed advisory cache on demand. | + +## Inputs and Outputs +Inputs/outputs are summarized in the table below. + +| Type | Name | Location | Description | +| --- | --- | --- | --- | +| Input | Signed advisory feed | `https://clawsec.prompt.security/advisories/feed.json(.sig)` | Threat intelligence source for cache refresh. | +| Input | Package + signature files | Host filesystem paths | Pre-install package authenticity checks. | +| Input | Integrity policy | `guardian/policy.json` | Per-path mode and priority controls. | +| Output | Advisory cache | `/workspace/project/data/clawsec-advisory-cache.json` | Host-managed verified advisory data. | +| Output | Verification results | `/workspace/ipc/clawsec_results/*.json` | IPC response payload for tool calls. | +| Output | Integrity state | `.../soul-guardian/` | Baselines, snapshots, patches, quarantine, audit logs. | + +## Configuration +| Setting | Default | Effect | +| --- | --- | --- | +| Feed URL | Hosted ClawSec advisory endpoint | Primary remote source for advisory cache manager. | +| Cache TTL | `5 minutes` | Controls staleness threshold before requiring refresh. | +| Fetch timeout | `10 seconds` | Limits host network wait time. | +| Allowed domains | `clawsec.prompt.security`, `prompt.security`, `raw.githubusercontent.com`, `github.com` | Restricts remote fetch targets. | +| Integrity policy modes | `restore`, `alert`, `ignore` | Controls automatic restoration and alert-only behavior. | + +## Example Snippets +```ts +// host-side signature verification dispatch +const result = await deps.signatureVerifier.verify({ + packagePath, + signaturePath, + publicKeyPem, + allowUnsigned: allowUnsigned || false, +}); +``` + +```ts +// integrity monitor drift handling +if (baseline.mode === 'restore' && autoRestore) { + // quarantine modified file, restore approved snapshot, append audit event +} +``` + +## Edge Cases +- Disallowed domains or non-HTTPS URLs are blocked by security policy wrappers. +- Missing signature files can be tolerated only when `allowUnsigned` is explicitly set. +- IPC result waits can timeout, causing conservative block recommendations. +- Integrity engine refuses symlink operations to reduce path-redirection attacks. +- Audit-chain validation can detect tampering or corruption in historical records. + +## Tests +| Test Scope | File/Path | Notes | +| --- | --- | --- | +| Type contracts | `skills/clawsec-nanoclaw/lib/types.ts` | Defines tool/IPC DB payload contracts. | +| Operational docs | `docs/SKILL_SIGNING.md`, `docs/INTEGRITY.md` | Describes verification/integrity usage patterns. | +| Cross-module behavior | Reuses suite verification patterns | Signature/checksum primitives ported from suite logic. | + +## Source References +- skills/clawsec-nanoclaw/skill.json +- skills/clawsec-nanoclaw/lib/types.ts +- skills/clawsec-nanoclaw/lib/signatures.ts +- skills/clawsec-nanoclaw/lib/advisories.ts +- skills/clawsec-nanoclaw/host-services/advisory-cache.ts +- skills/clawsec-nanoclaw/host-services/ipc-handlers.ts +- skills/clawsec-nanoclaw/host-services/skill-signature-handler.ts +- skills/clawsec-nanoclaw/host-services/integrity-handler.ts +- skills/clawsec-nanoclaw/guardian/integrity-monitor.ts +- skills/clawsec-nanoclaw/guardian/policy.json +- skills/clawsec-nanoclaw/mcp-tools/advisory-tools.ts +- skills/clawsec-nanoclaw/mcp-tools/signature-verification.ts +- skills/clawsec-nanoclaw/mcp-tools/integrity-tools.ts +- skills/clawsec-nanoclaw/docs/SKILL_SIGNING.md +- skills/clawsec-nanoclaw/docs/INTEGRITY.md diff --git a/wiki/overview.md b/wiki/overview.md new file mode 100644 index 0000000..72fbdf3 --- /dev/null +++ b/wiki/overview.md @@ -0,0 +1,102 @@ +# Overview + +## Purpose +- ClawSec is a security-focused repository that combines a public web catalog with installable security skills for OpenClaw and NanoClaw environments. +- The codebase supports three delivery paths at once: static website publishing, signed advisory distribution, and per-skill GitHub release packaging. +- Primary users are agent operators, skill developers, and maintainers running CI-based security automation. + +![Prompt Security Logo](assets/overview_img_01_prompt-security-logo.png) +![ClawSec Mascot](assets/overview_img_02_clawsec-mascot.png) + +## Repo Layout +| Path | Role | Notes | +| --- | --- | --- | +| `pages/`, `components/`, `App.tsx`, `index.tsx` | Vite + React UI | Skill catalog, advisory feed, and detail pages. | +| `skills/` | Security skill packages | Each skill has `skill.json`, `SKILL.md`, optional scripts/tests/docs. | +| `advisories/` | Repository advisory channel | Signed `feed.json` + `feed.json.sig` and key material. | +| `scripts/` | Local automation | Populate feed/skills, pre-push checks, release helpers. | +| `.github/workflows/` | CI/CD pipelines | CI, releases, NVD polling, community advisory ingestion, pages deploy. | +| `utils/` | Python utilities | Skill validation and checksum packaging helpers. | +| `public/` | Published static assets | Site media, mirrored advisories, and generated skill artifacts. | +| `docs/` | Operational docs | Signing runbooks, migration plans, compatibility and verification guides. | + +## Entry Points +| Entry | Type | Purpose | +| --- | --- | --- | +| `index.tsx` | Frontend bootstrap | Mounts React app into `#root`. | +| `App.tsx` | Frontend router | Defines route map for home, skills, and feed pages. | +| `scripts/prepare-to-push.sh` | Dev workflow | Runs lint/type/build/security checks before push. | +| `scripts/populate-local-feed.sh` | Data bootstrap | Pulls CVEs from NVD and updates local advisory feeds. | +| `scripts/populate-local-skills.sh` | Data bootstrap | Builds `public/skills/index.json` and per-skill checksums. | +| `.github/workflows/skill-release.yml` | Release entry | Handles tag-based skill packaging/signing/release. | +| `.github/workflows/poll-nvd-cves.yml` | Scheduled feed updates | Polls NVD and updates advisories. | + +## Key Artifacts +| Artifact | Produced By | Consumed By | +| --- | --- | --- | +| `advisories/feed.json` | NVD poll + community advisory workflows | Web UI, clawsec-suite hook, installers. | +| `advisories/feed.json.sig` | Signing workflow steps | Signature verification in suite/nanoclaw tooling. | +| `public/skills/index.json` | Deploy workflow / local populate script | `pages/SkillsCatalog.tsx` and `pages/SkillDetail.tsx`. | +| `public/checksums.json` + `public/checksums.sig` | Deploy workflow | Client-side integrity verification flows. | +| `release-assets/checksums.json` | Skill release workflow | Release consumers verifying zip integrity. | +| `skills/*/skill.json` | Skill authors | Site catalog generation, validators, and release pipelines. | + +## Key Workflows +- Local web development: `npm install` then `npm run dev`. +- Local security data preview: run `./scripts/populate-local-skills.sh` and `./scripts/populate-local-feed.sh` before loading `/skills` and `/feed` pages. +- Pre-push quality gate: run `./scripts/prepare-to-push.sh` (optionally `--fix`). +- Skill lifecycle: edit `skills//`, validate with `python utils/validate_skill.py`, then tag `-vX.Y.Z` to trigger release workflow. +- Advisory lifecycle: scheduled NVD poll and issue-label-based community ingestion both merge into the same signed feed. + +## Example Snippets +```bash +# local UI + locally populated data +npm install +./scripts/populate-local-skills.sh +./scripts/populate-local-feed.sh --days 120 +npm run dev +``` + +```bash +# canonical TypeScript quality checks used by CI +npx eslint . --ext .ts,.tsx,.js,.jsx,.mjs --max-warnings 0 +npx tsc --noEmit +npm run build +``` + +## Where to Start +- Read `README.md` for product positioning and install paths. +- Open `App.tsx` and `pages/` to understand user-facing behavior. +- Open `skills/clawsec-suite/skill.json` to understand the suite contract and embedded components. +- Review `.github/workflows/ci.yml`, `.github/workflows/skill-release.yml`, and `.github/workflows/deploy-pages.yml` for production behavior. + +## How to Navigate +- UI behavior is centered in `pages/`; visual wrappers sit in `components/`. +- Skill-specific logic is isolated by folder under `skills/`; each folder includes its own scripts/tests/docs. +- Feed handling appears in three layers: repository feed files, workflow updates, and runtime consumers (`clawsec-suite`/`clawsec-nanoclaw`). +- Operational quality gates live in `scripts/` and workflow YAML files. +- For generation traces and update baselines, start from `wiki/GENERATION.md` and then branch into module pages. + +## Common Pitfalls +- Using literal home tokens (for example `\$HOME`) in config path env vars can trigger path validation failures. +- Fetching JSON from SPA routes can return HTML with status 200; pages guard for this and treat it as empty-state. +- Unsigned feed bypass mode (`CLAWSEC_ALLOW_UNSIGNED_FEED=1`) exists for migration compatibility and should not be used in steady state. +- Skill release automation expects version parity between `skill.json` and `SKILL.md` frontmatter. +- Some scripts are POSIX shell oriented; Windows users should prefer PowerShell equivalents or WSL. + +## Source References +- README.md +- package.json +- App.tsx +- index.tsx +- pages/Home.tsx +- pages/SkillsCatalog.tsx +- pages/SkillDetail.tsx +- pages/FeedSetup.tsx +- scripts/prepare-to-push.sh +- scripts/populate-local-feed.sh +- scripts/populate-local-skills.sh +- skills/clawsec-suite/skill.json +- .github/workflows/ci.yml +- .github/workflows/skill-release.yml +- .github/workflows/deploy-pages.yml diff --git a/wiki/security.md b/wiki/security.md new file mode 100644 index 0000000..5d788ed --- /dev/null +++ b/wiki/security.md @@ -0,0 +1,74 @@ +# Security + +## Security Model Overview +- ClawSec secures both content distribution (signed artifacts) and runtime behavior (advisory gating, integrity monitoring). +- Trust anchors are pinned public keys committed in repo and verified against workflow-generated outputs. +- Runtime consumers default to verification-first behavior with explicit migration bypass flags. + +## Cryptographic Controls +| Control | Mechanism | Location | +| --- | --- | --- | +| Feed authenticity | Ed25519 detached signatures (`feed.json.sig`) | Advisory workflows + consumer verification libs. | +| Artifact integrity | SHA-256 checksum manifests (`checksums.json`) | Skill release and pages deploy workflows. | +| Key consistency | Fingerprint comparison across docs + canonical PEMs | `scripts/ci/verify_signing_key_consistency.sh`. | +| Signature verification action | Composite sign+verify action in CI | `.github/actions/sign-and-verify/action.yml`. | + +## Runtime Enforcement Controls +| Control | Component | Effect | +| --- | --- | --- | +| Advisory hook gating | `clawsec-advisory-guardian` | Alerts and cautious guidance based on matched advisories. | +| Double-confirmation installer | `guarded_skill_install.mjs` | Exit `42` until explicit confirmation for matched advisories. | +| Reputation extension | `clawsec-clawhub-checker` | Additional risk scoring before install. | +| NanoClaw signature gate | `skill-signature-handler.ts` + MCP tool | Blocks tampered/unsigned package installs by policy. | +| Integrity baseline monitor | `soul-guardian` + NanoClaw integrity monitor | Drift detection, quarantine, restore, auditable history. | + +## Supply-Chain and CI Controls +- CI runs Trivy, npm audit, gitleaks, CodeQL, and Scorecard workflows. +- Release workflows validate SBOM file existence before packaging. +- Deploy workflow verifies generated signing key fingerprint against canonical key material. +- Release docs include manual verification commands for downstream consumers. + +## Incident and Rotation Playbooks +- `docs/SECURITY-SIGNING.md` defines key generation, custody, rotation, and incident phases. +- `docs/MIGRATION-SIGNED-FEED.md` defines staged enforcement and rollback levels. +- Rollback paths prioritize preserving signed publishing where possible and time-boxing any bypass. + +## Example Snippets +```bash +# verify canonical public key fingerprint +openssl pkey -pubin -in clawsec-signing-public.pem -outform DER | shasum -a 256 +``` + +```bash +# run repo key-consistency guardrail used in CI +./scripts/ci/verify_signing_key_consistency.sh +``` + +## Known Security Tradeoffs +- Unsigned compatibility mode can reduce assurance and should be disabled once migration completes. +- Some deploy paths tolerate unsigned legacy checksum assets for backward compatibility. +- Reputation checks rely on external tooling output and may include heuristic false positives/negatives. +- Local scripts inherit environment trust; compromised local shells can still subvert operator workflows. + +## Hardening Opportunities +- Remove unsigned compatibility flags after migration stabilization. +- Expand deterministic checksum/signature verification for all mirrored release files. +- Add explicit tests for workflow-level signature failure scenarios. +- Increase runtime telemetry for advisory fetch/verification failures to simplify incident triage. + +## Source References +- SECURITY.md +- docs/SECURITY-SIGNING.md +- docs/MIGRATION-SIGNED-FEED.md +- scripts/ci/verify_signing_key_consistency.sh +- .github/actions/sign-and-verify/action.yml +- .github/workflows/poll-nvd-cves.yml +- .github/workflows/community-advisory.yml +- .github/workflows/skill-release.yml +- .github/workflows/deploy-pages.yml +- skills/clawsec-suite/hooks/clawsec-advisory-guardian/lib/feed.mjs +- skills/clawsec-suite/scripts/guarded_skill_install.mjs +- skills/clawsec-clawhub-checker/scripts/enhanced_guarded_install.mjs +- skills/soul-guardian/scripts/soul_guardian.py +- skills/clawsec-nanoclaw/host-services/skill-signature-handler.ts +- skills/clawsec-nanoclaw/guardian/integrity-monitor.ts diff --git a/wiki/testing.md b/wiki/testing.md new file mode 100644 index 0000000..a326463 --- /dev/null +++ b/wiki/testing.md @@ -0,0 +1,73 @@ +# Testing + +## Testing Strategy +- The repository uses layered verification rather than a single root `npm test` command. +- Core confidence comes from lint/type/build gates plus skill-local Node test suites. +- Python and shell tooling are validated through dedicated lint/security checks. +- Workflow pipelines run the same command classes used in local pre-push automation. + +## Verification Layers +| Layer | Commands | Scope | +| --- | --- | --- | +| Frontend/static checks | ESLint + `tsc --noEmit` + `npm run build` | TS/TSX correctness and build viability. | +| Skill unit tests | `node skills//test/*.test.mjs` | Signature, matching, suppression, installer contracts. | +| Python quality | `ruff check utils/`, `bandit -r utils/ -ll` | Utility correctness and security patterns. | +| Shell/script quality | ShellCheck + manual script smoke runs | Script hygiene and command robustness. | +| CI security scans | Trivy, npm audit, gitleaks, CodeQL, Scorecard | Dependency, config, and supply-chain security posture. | + +## Skill Test Matrix +| Skill | Test Files | Primary Focus | +| --- | --- | --- | +| `clawsec-suite` | `feed_verification`, `guarded_install`, `path_resolution`, fuzz tests | Signature checks, advisory gating, path safety, matching robustness. | +| `openclaw-audit-watchdog` | suppression config and render tests | Config parsing, suppression behavior, report formatting. | +| `clawsec-clawhub-checker` | `reputation_check.test.mjs` | Input validation and reputation gating behavior. | + +## CI Workflow Coverage +| Workflow | Trigger | Key Assertions | +| --- | --- | --- | +| `ci.yml` | PR/push to `main` | Lint/type/build, Python checks, security scans, skill tests. | +| `codeql.yml` | PR/push/schedule | JS/TS static security analysis. | +| `scorecard.yml` | schedule/push | Supply-chain posture reporting and SARIF upload. | +| `skill-release.yml` | tags + PRs | Version parity and release artifact verification. | + +## Local Testing Commands +```bash +# baseline frontend + config checks +npx eslint . --ext .ts,.tsx,.js,.jsx,.mjs --max-warnings 0 +npx tsc --noEmit +npm run build +``` + +```bash +# representative skill tests +node skills/clawsec-suite/test/feed_verification.test.mjs +node skills/clawsec-suite/test/guarded_install.test.mjs +node skills/openclaw-audit-watchdog/test/suppression_config.test.mjs +``` + +## Failure Patterns to Watch +- Signature/test fixtures can fail from key/payload mismatch when expected files are regenerated inconsistently. +- Path-resolution tests intentionally fail on escaped home tokens; this behavior is expected and security-relevant. +- Local scripts relying on `openclaw` or `clawhub` binaries may fail in environments where those CLIs are absent. +- Deploy/release logic can pass locally while failing in CI if signing secrets or workflow permissions differ. + +## Suggested Test Order +1. Run `./scripts/prepare-to-push.sh` for a full local gate. +2. Run directly impacted skill-local tests. +3. For feed/signing changes, run suite verification tests first (`feed_verification`, `guarded_install`). +4. For workflow or release changes, also run `scripts/validate-release-links.sh` and key consistency script. + +## Source References +- AGENTS.md +- scripts/prepare-to-push.sh +- scripts/validate-release-links.sh +- .github/workflows/ci.yml +- .github/workflows/codeql.yml +- .github/workflows/scorecard.yml +- .github/workflows/skill-release.yml +- skills/clawsec-suite/test/feed_verification.test.mjs +- skills/clawsec-suite/test/guarded_install.test.mjs +- skills/clawsec-suite/test/path_resolution.test.mjs +- skills/openclaw-audit-watchdog/test/suppression_config.test.mjs +- skills/clawsec-clawhub-checker/test/reputation_check.test.mjs +- docs/PLATFORM_VERIFICATION.md diff --git a/wiki/workflow.md b/wiki/workflow.md new file mode 100644 index 0000000..c61bc01 --- /dev/null +++ b/wiki/workflow.md @@ -0,0 +1,74 @@ +# Workflow + +## End-to-End Lifecycle +- Development starts with local coding + local data population for realistic UI preview. +- PR CI validates quality/security and skill test suites. +- Tag-driven release workflow packages and signs skill artifacts. +- Pages deploy workflow mirrors release/advisory artifacts and publishes the static site. +- Scheduled workflows continuously enrich advisory feed and supply-chain visibility. + +## Primary Workflow Map +| Workflow | Trigger | Main Steps | +| --- | --- | --- | +| CI | PR/push to `main` | Lint, typecheck, build, Python checks, security scans, skill tests. | +| Poll NVD CVEs | Daily cron + manual dispatch | Fetch CVEs, transform/dedupe, update feed, sign artifacts, PR changes. | +| Process Community Advisory | Issue label `advisory-approved` | Parse issue form, create advisory, sign feed, open PR, comment issue. | +| Skill Release | Tag `-v*` | Validate versions, package SBOM files, sign checksums, publish release. | +| Deploy Pages | Successful CI/Release or manual dispatch | Discover releases, mirror assets, sign public advisories/checksums, deploy site. | + +## Local Operator Workflow +| Step | Command | Outcome | +| --- | --- | --- | +| Install deps | `npm install` | Ready local environment. | +| Populate local catalog | `./scripts/populate-local-skills.sh` | `public/skills/index.json` and file checksums. | +| Populate local feed | `./scripts/populate-local-feed.sh --days 120` | Updated local advisory feed copy. | +| Run local gate | `./scripts/prepare-to-push.sh` | CI-like pass/fail signal. | +| Start dev UI | `npm run dev` | Browser preview at local Vite endpoint. | + +## Release Workflow Details +- Version bump and docs parity are enforced for PR/tag paths. +- Skill packaging includes SBOM-declared files and integrity manifests. +- `checksums.json` is signed and immediately verified in workflow execution. +- Optional publish-to-ClawHub job runs after successful GitHub release when configured. +- Older releases within same major line can be superseded/deleted by automation. + +## Advisory Workflow Details +- NVD workflow determines incremental window from previous feed `updated` timestamp. +- Transform phase maps CVE metrics to severity/type and normalizes affected targets. +- Community advisory workflow creates deterministic IDs (`CLAW-YYYY-NNNN`) from issue metadata. +- Both advisory workflows update skill feed copies and signature companions. + +## Example Snippets +```bash +# manual release prep for a skill +./scripts/release-skill.sh clawsec-feed 0.0.5 +# then push tag if running in release branch mode +``` + +```yaml +# pages deploy depends on successful upstream workflow run +on: + workflow_run: + workflows: ["CI", "Skill Release"] + types: [completed] +``` + +## Operational Risks +- Workflow permissions and secret scope misconfiguration can block signing/publishing. +- NVD/API transient failures may delay advisory freshness. +- Invalid tag naming or version mismatches halt release automation. +- Local scripts and CI can diverge if operator machine lacks expected binaries (`jq`, `openssl`, `clawhub`). + +## Source References +- scripts/release-skill.sh +- scripts/prepare-to-push.sh +- scripts/populate-local-feed.sh +- scripts/populate-local-skills.sh +- .github/workflows/ci.yml +- .github/workflows/poll-nvd-cves.yml +- .github/workflows/community-advisory.yml +- .github/workflows/skill-release.yml +- .github/workflows/deploy-pages.yml +- .github/workflows/codeql.yml +- .github/workflows/scorecard.yml +- .github/actions/sign-and-verify/action.yml