Commit Graph

3 Commits

Author SHA1 Message Date
Mads Hvelplund 962d4459d9 Add configurable token / cost usage limits (#576)
* fix: resolve pre-commit check failures

- Change RuntimeError to TypeError for type validation in report/writer.py
- Update pyupgrade to v3.21.2 for Python 3.14 compatibility

* feat(cli): add --max-budget-usd flag

Raises BudgetExceededError in ReportUsageHooks after each LLM call when
accumulated cost reaches the limit, with clean "stopped" status and
child-agent cancellation in non-interactive mode.

* test: add budget enforcement unit tests

7 tests covering no-budget, under-budget, at-limit, over-limit, error
message content, None report state, and exception hierarchy.
Also adds pytest/pytest-asyncio to dev deps and a mypy override for tests.

* fix(budget): validate positive budget and check the live cost ledger

Two hardening fixes for --max-budget-usd enforcement:

- Reject non-positive budgets. ReportUsageHooks now raises ValueError for
  max_budget_usd <= 0, and the CLI validates the flag via a custom argparse
  type so '--max-budget-usd 0' fails fast with a friendly message instead of
  silently killing the scan on the first model response.
- Read the live cost. The budget check now reads ReportState.get_total_llm_cost()
  (the live ledger) instead of the persisted run-record snapshot, so it stays
  accurate even when a usage save fails after a model call.

* fix(budget): stop the entire scan deterministically when the limit is hit

Previously a BudgetExceededError was handled per-agent: it was swallowed in
interactive mode (the loop kept waiting), a child's error escaped its detached
task as an unretrieved-exception warning, the parent was never released from
wait_for_message, and the stop was logged at ERROR with a traceback as if the
agent had failed.

Replace that with a single scan-wide signal on the coordinator:

- AgentCoordinator.trigger_budget_stop() sets a flag and wakes every parked
  agent; wait_for_message returns as soon as the flag is set.
- The run loops check coordinator.budget_stopped and raise to exit cleanly,
  marking themselves 'stopped'. The root's exception reaches run_strix_scan's
  handler, which cancels descendants and tears the scan down once; child
  exceptions are swallowed in their detached task.
- The budget stop is logged at INFO, not as a failure.

This is deterministic regardless of tree depth or which agent first sees the
limit, fixing the interactive/TUI hang where a deep agent's stop never reached
a parked root. Also re-raises BudgetExceededError explicitly in the stream
handler so it can't be mistaken for the LiteLLM 'after shutdown' race.

* fix(budget): treat a budget stop as a clean stop in the TUI

Add an explicit BudgetExceededError handler in the TUI scan thread so that, if
the error ever reaches it, the budget stop is logged as a graceful stop rather
than surfaced as a red scan error by the broad 'except Exception'. The runner
normally absorbs the error and returns cleanly, so this is defensive depth for
a money-spending feature.

* docs(cli): document --max-budget-usd behavior and limitations

Clarify that the budget is cumulative across all agents, checked after each
model response, that the scan stops cleanly (not as a failure), that the value
must be > 0, and that spend can slightly overshoot due to in-flight calls and
best-effort cost estimation.

* Apply suggestions from code review

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-06-22 11:17:08 -04:00
0xallam a6d578c4a8 chore: nuke tests/ and the entire test toolchain
The test suite was carrying migration scars and a long tail of
low-density assertions over SDK-derived behavior. Drop it wholesale.

- Delete ``tests/`` (42 files, ~4900 LoC).
- Drop ``pytest`` / ``pytest-asyncio`` / ``pytest-cov`` /
  ``pytest-mock`` from the dev dependency group; ``uv sync``
  uninstalls the matching wheels.
- Strip the pytest + coverage config blocks, the
  ``flake8-pytest-style`` ruff selector, the ``tests/**`` per-file
  ignores, the ``[tool.mypy.overrides] tests.*`` block, and the
  ``"tests"`` entry from bandit's ``exclude_dirs``.
- Drop the ``test`` / ``test-cov`` Makefile targets; ``dev`` no
  longer depends on tests.
- Strip the ``# Testing`` block from ``.gitignore`` (``.coverage``,
  ``.pytest_cache/``, ``htmlcov/``, ``coverage.xml``, ``nosetests.xml``,
  ``.tox/``, ``.hypothesis/``).

ruff (27) and mypy (82) baselines unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 13:01:20 -07:00
Jeong-Ryeol 65c3383ecc test: add initial unit tests for argument_parser module
Add comprehensive test suite for the argument_parser module including:
- Tests for _convert_to_bool with truthy/falsy values
- Tests for _convert_to_list with JSON and comma-separated inputs
- Tests for _convert_to_dict with valid/invalid JSON
- Tests for convert_string_to_type with various type annotations
- Tests for convert_arguments with typed functions
- Tests for ArgumentConversionError exception class

This establishes the foundation for the project's test infrastructure
with pytest configuration already in place.
2025-12-04 00:02:14 +04:00