Files

T

0xallam 8ed5311b8e Surface the previously-undocumented sandbox tools and unbreak two of them

The image ships 15 tools (jwt_tool, interactsh-client, arjun, dirsearch,
gospider, wafw00f, retire, eslint, jshint, js-beautify, JS-Snooper,
jsniper.sh, vulnx, ncat, uv) that the always-loaded skills never name
with usage guidance — agents could discover them via the environment
catalog but had no when/how. Add concise mentions in the natural home
for each: jwt_tool in the JWT skill, interactsh-client in the OAST
sections of SSRF/XXE/RCE, arjun in IDOR recon, dirsearch as the broad
alternate in the ffuf skill, gospider + the JS scrapers in katana,
wafw00f next to httpx, retire/eslint/jshint/js-beautify as a new
JavaScript-Side Coverage block in the SAST playbook, uv in python,
vulnx in the deep scan-mode CVE bullet, ncat in a new RCE Tooling
block.

Audit also turned up three real breakages along the way:

- jwt_tool's shebang resolves to /usr/bin/python3 but its dependencies
  live in /app/.venv, so every invocation died with
  ModuleNotFoundError: ratelimit. Replace the bare symlink with a
  wrapper that execs /app/.venv/bin/python against the real script.
- dirsearch's pipx venv ended up with setuptools 82, which dropped
  pkg_resources — startup failed before parsing args. Pin the inject
  to setuptools<81.
- ESLint's --no-eslintrc flag was removed in v9; the surviving
  --no-config-lookup covers it. Drop the dead flag from the SAST
  command block.

Also corrected the JS-Snooper / jsniper.sh entry in katana.md — both
take a bare domain and run their own JS discovery internally, not the
JS URLs Katana already harvested.

2026-05-25 22:02:15 -07:00

4.0 KiB

Raw Blame History

name, description

name	description
katana	Katana crawler syntax, depth/js/known-files behavior, and stable concurrency controls.

Katana CLI Playbook

Official docs:

Canonical syntax: katana [flags]

High-signal flags:

-u, -list <url|file> target URL(s)
-d, -depth <n> crawl depth
-jc, -js-crawl parse JavaScript-discovered endpoints
-jsl, -jsluice deeper JS parsing (memory intensive)
-kf, -known-files <all|robotstxt|sitemapxml> known-file crawling mode
-proxy <http|socks5 proxy> explicit proxy setting
-c, -concurrency <n> concurrent fetchers
-p, -parallelism <n> concurrent input targets
-rl, -rate-limit <n> request rate limit
-timeout <seconds> request timeout
-retry <n> retry count
-ef, -extension-filter <list> extension exclusions
-tlsi, -tls-impersonate experimental JA3/TLS impersonation
-hl, -headless enable hybrid headless crawling
-sc, -system-chrome use local Chrome for headless mode
-ho, -headless-options <csv> extra Chrome options (for example proxy-server)
-nos, -no-sandbox run Chrome headless with no-sandbox
-noi, -no-incognito disable incognito in headless mode
-cdd, -chrome-data-dir <dir> persist browser profile/session
-xhr, -xhr-extraction include XHR endpoints in JSONL output
-silent, -j, -jsonl, -o <file> output controls

Agent-safe baseline for automation: mkdir -p crawl && katana -u https://target.tld -d 3 -jc -kf robotstxt -c 10 -p 10 -rl 50 -timeout 10 -retry 1 -ef png,jpg,jpeg,gif,svg,css,woff,woff2,ttf,eot,map -silent -j -o crawl/katana.jsonl

Common patterns:

Fast crawl baseline: katana -u https://target.tld -d 3 -jc -silent
Deeper JS-aware crawl: katana -u https://target.tld -d 5 -jc -jsl -kf all -c 10 -p 10 -rl 50 -o katana_urls.txt
Multi-target run with JSONL output: katana -list urls.txt -d 3 -jc -silent -j -o katana.jsonl
Headless crawl with local Chrome: katana -u https://target.tld -hl -sc -nos -xhr -j -o crawl/katana_headless.jsonl
Headless crawl through proxy: katana -u https://target.tld -hl -sc -ho proxy-server=http://127.0.0.1:48080 -j -o crawl/katana_proxy.jsonl

Critical correctness rules:

-kf must be followed by one of all, robotstxt, or sitemapxml.
Use documented -hl for headless mode.
-proxy expects a single proxy URL string (for example http://127.0.0.1:8080).
-ho expects comma-separated Chrome options (example: -ho --disable-gpu,proxy-server=http://127.0.0.1:8080).
For -kf, keep depth at least -d 3 so known files are fully covered.
If writing to a file, ensure parent directory exists before -o.

Usage rules:

Keep -d, -c, -p, and -rl explicit for reproducible runs.
Use -ef early to reduce static-file noise before fuzzing.
Prefer -proxy over environment proxy variables when proxying only Katana traffic.
Use -hc only for one-time diagnostics, not routine crawling loops.
Do not use -h/--help for routine runs unless absolutely necessary.

Failure recovery:

If crawl runs too long, lower -d and optionally add -ct.
If memory spikes, disable -jsl and lower -c/-p.
If headless fails with Chrome errors, drop -sc or install system Chrome.
If output is noisy, tighten scope and add -ef filters.

If uncertain, query web_search with: site:docs.projectdiscovery.io katana <flag> usage

Complementary crawlers / JS endpoint extractors in the sandbox:

gospider -s https://target.tld -d 3 -c 10 -t 20 — alternate crawler; picks up things Katana misses on weird sites; use it as a second pass when Katana output looks thin.
~/tools/JS-Snooper/js_snooper.sh <domain> and ~/tools/jsniper.sh/jsniper.sh <domain> — both take a bare domain and run their own JS-file discovery internally (jsniper drives httpx + katana + nuclei file templates). Reach for them when you want a quick "find endpoints/keys/secrets in any JS this domain serves" sweep without wiring it up yourself.

4.0 KiB Raw Blame History

Katana CLI Playbook

4.0 KiB

Raw Blame History