fastmcp-feedback/CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Calendar Versioning (YYYY.MM.DD).

[2026.09.28] - 2026-09-28

Keep the tool-call table small: leave out or sample busy tools, and prune old rows.

Added

• InstrumentationMiddleware(..., exclude_tools=...): tools that are never recorded and never kept for feedback links. The calls themselves are untouched.
• InstrumentationMiddleware(..., sample_rates={"tool": 0.01}): record that fraction of a tool's ok calls. Failures are always recorded: calls with outcome error or soft_error are kept whatever the rate. Rates outside (0, 1] raise ValueError. Calls sampled out skip the hooks and redaction and are not kept for feedback links. random= takes a () -> float for tests.
• Records have a sample_rate: the configured rate on sampled-in ok rows, None on everything recorded unconditionally (including failures of sampled tools). SUM(1.0 / COALESCE(sample_rate, 1)) estimates the true call count.
• DatabaseSink(..., retention=timedelta(days=30), prune_interval=..., prune_batch=5000). With retention set, the background writer prunes after an insert at most once per prune_interval (one hour by default). A failing prune is logged and never stops the insert.
• DatabaseSink.prune(older_than=None) -> int deletes older rows in batches of prune_batch, one transaction each, on SQLite and Postgres. Calls linked to feedback are never pruned, and links are never deleted.

Migration

• New nullable sample_rate column (Float) on {prefix}tool_calls. Hosts that manage the schema with Alembic need a migration adding it. create_tables=True does not alter a table that already exists, so add it by hand there (ALTER TABLE ffb_tool_calls ADD COLUMN sample_rate FLOAT). Until the column exists, DatabaseSink writes fail and those records are dropped (logged). Hosts still on 2026.09.27.3 or earlier also need the outcome column from 2026.09.27.4.

[2026.09.27.4] - 2026-09-27

Count errors that tools return as data instead of raising.

Added

• Records have an outcome: ok, soft_error (the tool returned a result flagged as a failure) or error (the tool raised). Soft errors get error_type="SoftError" and an error_message built from the result, such as status=failed: disk full, redacted and truncated to max_error_chars.
• soft_error_classifier(statuses=..., status_keys=..., flag_keys=..., extra_statuses=...) builds a classifier that flags dict results where ok or success is False, where status or state is error, failed or failure, or that the tool marked isError without raising. DEFAULT_RESULT_CLASSIFIER is soft_error_classifier().
• InstrumentationMiddleware(..., result_classifier=...) takes any (tool, result) -> str | None; None turns classification off. A classifier that raises is logged and the call is recorded as ok.
• feedback_context() rows include outcome.

Changed

• ok is now outcome == "ok", so it is also false for soft errors. Existing WHERE NOT ok queries and error rates will rise; filter on outcome = 'error' for exceptions only. CallLink.ok follows.
• The package's own feedback tools answer {"success": False, ...} on failure, so those calls are now recorded as soft errors.

Migration

• New nullable, indexed outcome column (String(16)) on {prefix}tool_calls. Hosts that manage the schema with Alembic need a migration adding it. create_tables=True does not alter a table that already exists, so add the column by hand there (ALTER TABLE ffb_tool_calls ADD COLUMN outcome VARCHAR(16) plus an index). Until the column exists, DatabaseSink writes fail and those records are dropped (logged). Rows from earlier versions keep NULL.

[2026.09.27.3] - 2026-09-27

Link a feedback item to the tool calls that preceded it.

Added

• InstrumentationMiddleware.capture_recent_calls(ctx, n=20) returns the calls that led up to now. It matches the same client session first (session rule), then fills remaining slots with calls by the same user_sub inside window (user_window rule, 15 minutes by default), and each link records which rule matched.
• Calls are kept in an in-memory buffer as they happen, so calls still waiting in the sink queue count. With a DatabaseSink the lookup falls back to the database, which covers calls from other workers and from before a restart.
• link_feedback(ref, links) stores links against a feedback id of any format (bug-7Q2X, 42) in a new {prefix}feedback_call_links table. build_metadata() includes it; hosts that manage the schema with Alembic need a new migration.
• feedback_context(ref) returns the linked calls in order with tool, outcome, error type and matching rule, filling in calls not yet written from memory.
• add_feedback_tools(..., instrumentation=mw): submit_feedback links the preceding calls automatically and returns linked_calls in its response.
• session_key(ctx) gives a per-client key that stays stable across calls: the HTTP mcp-session-id header when present, then one key per process for stdio, then ctx.session_id.

Fixed

• With create_tables=True, the background writer and link_feedback could both run create_all at once and fail with "table already exists". Table creation is now serialized by a lock.

Note

• Clients on MCP protocol 2026-07-28 (Claude Code, FastMCP 4's Client) open no HTTP sessions, so their calls link by user only. HTTP servers should configure an identity_resolver. Stdio servers link by session with every client.

[2026.09.27.2] - 2026-09-27

Fixed

• DatabaseSink stored missing args/result (every meta mode row) as the JSON literal null, so WHERE args IS NULL matched nothing. They are now SQL NULL. No schema change; rows written by 2026.09.27.1 keep JSON null and can be backfilled with UPDATE ffb_tool_calls SET args = NULL WHERE args = 'null'::jsonb (same for result). Reported from production by blender-mcp.

[2026.09.27.1] - 2026-09-27

Per-call instrumentation for any FastMCP server.

Added

• fastmcp_feedback.instrumentation: InstrumentationMiddleware and instrument(app) record every tool call on the server (tool, timing, outcome, original error type, session and request ids, argument and result sizes). Modes off, meta (default) and full, set by argument or FEEDBACK_INSTRUMENTATION_MODE; meta_only_tools keeps chosen tools at meta.
• Recording never slows or breaks a call: records reach sinks from a background task through a bounded queue and are dropped (and counted) when it is full. Sink and hook errors are logged, never raised; async hooks are cut off after hook_timeout.
• Redactor: secrets removed from arguments, results, error text and hook output, by key at any depth and by value pattern (JWT, bearer, sk-, GitHub, Slack, AWS, PyPI and bmcp_ tokens, password=... in text).
• Sinks: JsonLinesSink (stderr), CallbackSink, MemorySink, and DatabaseSink on async SQLAlchemy. DatabaseSink accepts the host's AsyncEngine, writes to a prefixed ffb_tool_calls table, never creates tables unless asked, and build_metadata() plugs the table into a host's Alembic env.py.
• identity_resolver and enricher hooks fill user_sub, caller_kind, client_id, server_version and free-form identity/extra fields.
• instrumentation extra: sqlalchemy[asyncio] and aiosqlite.
• CI covers FastMCP 3.x as well as 2.12, 2.14 and 4.x.

[2026.09.27] - 2026-09-27

FastMCP 4 support, and a composition bug that affected every earlier release.

Fixed

• compose_feedback_services() composed nothing on FastMCP 2.14. It called import_server(), which became async in 2.14, without awaiting it, then logged success anyway. On FastMCP 4, where import_server() no longer exists, it raised AttributeError. It now uses mount(), which is synchronous and available on every supported version.
• The test suite could not start (a pytest-html hook with pytest-html not installed), so it had not run since the package was split out. It runs again: 407 tests.

Changed

• Dependency is now fastmcp>=2.12.2,<5. Earlier releases had no ceiling, so a fresh install picked up FastMCP 4 untested.
• examples/ moved into this repository and ships in the sdist. create_demo_server() accepts an optional database_url. Examples list and call tools through fastmcp.Client, which works on 2.x and 4.x.

Added

• CI (Gitea Actions) runs the suite against FastMCP 2.12.2, 2.14.3 and 4.x on every push and weekly, plus a non-blocking job against the newest FastMCP with no ceiling.
• A composition test using real servers instead of mocks.

Documentation

• README rewritten against the code and every Python example executed. Previous docs described things that do not exist: four tools (there are five, including delete_feedback), get_feedback_stats (it is get_feedback_statistics), InsightsMixin, an enable_insights parameter, feedback types bug_report/feature_request/general (they are bug/feature/improvement/question), and a new -> reviewing -> planned workflow (it is open -> in_progress -> resolved -> closed).
• The default database is in-memory SQLite (sqlite:///:memory:), not sqlite:///feedback.db as previously documented. Pass database_url to keep feedback across restarts.

[2026.01.12.1] - 2026-01-12

Fixed

• ImportError on install: mixins/base.py did not re-export the names the other mixin modules import from it.

[2026.01.12] - 2026-01-12

Added

• Initial public release
• Core feedback tools: submit_feedback, list_feedback, get_feedback_stats, update_feedback_status
• add_feedback_tools() one-line integration function
• Modular mixin architecture:
  • SubmissionMixin - feedback collection
  • RetrievalMixin - listing and statistics
  • ManagementMixin - status workflow management
  • InsightsMixin - privacy-compliant analytics
• Multi-database support: SQLite, PostgreSQL, MySQL
• create_feedback_server() for server composition
• Comprehensive test suite (97%+ coverage)
• Full type hints and Pydantic validation
• Documentation site at https://fastmcp-feedback.l.supported.systems

Technical Details

• Built on FastMCP 2.12.2+
• SQLAlchemy 2.0+ ORM
• Pydantic 2.11+ validation
• Python 3.11+ required