Compare commits

...
Author SHA1 Message Date
rsp2k 3abd5c6311 Merge feature/soft-errors: classify error results returned as data
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-09-27 19:34:30 -06:00
rsp2k 472071730e Release 2026.09.27.4: soft-error classification
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-09-27 19:34:30 -06:00
rsp2k 9ddd929a13 Classify error results that tools return instead of raising 2026-09-27 19:34:30 -06:00
rsp2k 052862f4fc Point README examples at the in-repo examples/ directory
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
The showcase repo is private and fastmcp-feedback-showcase was never
published, so both the link and the uvx command failed for everyone else.
Tested the new clone-and-run instructions against the now-public repo.
2026-09-27 12:31:18 -06:00
rsp2k 975c24c766 Merge feature/correlation: link feedback to preceding tool calls
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-09-27 12:26:00 -06:00
rsp2k a530634eb9 Release 2026.09.27.3: feedback-to-call correlation
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
Changelog entry and version bump for linking feedback items to the tool
calls that preceded them. README documents the instrumentation parameter
of add_feedback_tools and the new feedback_call_links table that Alembic
hosts need to migrate.
2026-09-27 12:26:00 -06:00
rsp2k 4b16764faa Attribute per-call session ids to the stateless MCP protocol
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
Tested against real clients: a FastMCP 4.0.10 server keeps ctx.session_id
stable for clients that open sessions (curl on 2025-06-18, the FastMCP 3.3
Client, Claude Code over stdio). Claude Code and FastMCP 4's Client speak
MCP 2026-07-28, which has no HTTP sessions, so FastMCP generates an id per
request. Not a FastMCP bug; correct the comments and README note.
2026-09-27 10:01:44 -06:00
rsp2k 58e531019d Add feedback-to-call correlation (unreleased)
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
capture_recent_calls() picks the calls that led up to a feedback item:
same client session first, then the same user within a window, each link
recording which rule matched. It reads an in-memory buffer (so calls still
queued count) and falls back to DatabaseSink for other workers and
restarts. link_feedback() stores links against any feedback id in a
prefixed feedback_call_links table; feedback_context() reads them back.
add_feedback_tools(instrumentation=mw) makes submit_feedback link
automatically and report linked_calls.

session_key() works around FastMCP 4.0 returning a new ctx.session_id on
every call: it prefers the HTTP mcp-session-id header, then one key per
process for stdio, then ctx.session_id.

Also fixes a race where the writer and link_feedback both ran create_all
with create_tables=True.

468 tests pass on FastMCP 2.12.2, 2.14.3, 3.3.1, 3.4.7; 467 + 1 skip on
4.0.10. Still to do before release: execute the new README examples,
CHANGELOG, version bump.
2026-09-27 09:12:10 -06:00
rsp2k 00fae0b46b Merge fix/json-sql-null: store absent payloads as SQL NULL
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-09-27 08:58:34 -06:00
rsp2k 10b1eea86c Version 2026.09.27.2
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-09-27 08:58:34 -06:00
rsp2k 39d775a0cd Store absent JSON payloads as SQL NULL
DatabaseSink wrote the JSON literal null for missing args/result, so
WHERE args IS NULL matched nothing. Use none_as_null on the JSON and JSONB
column types. DDL is unchanged; existing rows keep JSON null.

Reported by blender-mcp from production.
2026-09-27 08:57:06 -06:00
rsp2k a052ca3b4b Merge feature/instrumentation: per-call instrumentation middleware
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-09-27 08:37:09 -06:00
rsp2k 2e1278d81f Add per-call instrumentation middleware
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
InstrumentationMiddleware (and instrument(app)) records every tool call on
a FastMCP server: tool, timing, outcome and original error type, session
and request ids, argument and result sizes, plus identity and enrichment
hook output. Modes off/meta/full; full adds redacted args and results.

Recording can't slow or break a call. Records reach sinks through a
bounded queue drained by a background task and are dropped, counted, when
it fills. Sink and hook failures are logged, never raised, and async hooks
are cut off after hook_timeout.

Redactor strips secrets by key at any depth and by value pattern from
args, results, error text and hook output. DatabaseSink runs on async
SQLAlchemy, takes a host AsyncEngine, writes to a prefixed table and never
creates it unless asked; build_metadata() plugs into Alembic env.py.

Verified on FastMCP 2.12.2, 2.14.3, 3.3.1, 3.4.7 and 4.0.10. CI matrix
gains 3.x. Version 2026.09.27.1.
2026-09-27 08:37:09 -06:00
rsp2k ddf23d10e4 Support FastMCP 4 and fix silent composition failure
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
compose_feedback_services() called import_server() without awaiting it.
That method became async in FastMCP 2.14, so on the version this package
was built against it composed nothing while logging success. FastMCP 4
removed it outright. Use mount(), which is sync on 2.12 through 4.x.

The test suite could not start (pytest-html hook, plugin not installed),
so nothing caught this. Add pytest-html, bring examples/ over from the docs
repo since the integration tests import them, and fix the tests that had
drifted from the code. 407 tests pass on FastMCP 2.12.2, 2.14.3 and 4.0.10.

- Declare fastmcp>=2.12.2,<5; CI matrix covers each family plus a
  non-blocking canary on the newest FastMCP
- Add a composition test with real servers; the mocked ones could not
  see a missing await
- Rewrite the README against the actual API and execute its examples
- Ignore ruff UP042: StrEnum would change stored enum formatting
- Version 2026.09.27
2026-09-27 06:09:30 -06:00
37 changed files with 5671 additions and 572 deletions
+67
View File
@@ -0,0 +1,67 @@
name: Tests
# fastmcp-feedback declares fastmcp>=2.12.2,<5, so every version family in
# that range gets the full suite. A user on PyPI gets the newest FastMCP the
# range allows, not what uv.lock pins, which is how 2026.01.12.1 shipped with
# composition broken on 2.14 and 4.x without anyone noticing.
on:
push:
pull_request:
workflow_dispatch:
schedule:
# Weekly, so a new FastMCP release is caught even when nobody pushes
- cron: '17 9 * * 1'
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
fastmcp: ['==2.12.2', '==2.14.3', '>=3.3,<4', '>=4,<5']
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
python-version: '3.13'
- name: Install against fastmcp${{ matrix.fastmcp }}
run: |
uv venv
uv pip install -e ".[dev]" "fastmcp${{ matrix.fastmcp }}"
uv pip show fastmcp | grep -i '^version'
- name: Lint
run: uv run --no-sync ruff check src/ examples/ tests/_compat.py tests/unit/test_instrumentation.py
- name: Test
run: uv run --no-sync pytest -q --no-cov
# Newest FastMCP with no ceiling. Allowed to fail: it exists to say
# "a new major version is out and here is what breaks" before users do.
canary:
runs-on: ubuntu-latest
timeout-minutes: 15
continue-on-error: true
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
python-version: '3.13'
- name: Install newest FastMCP, ignoring our ceiling
run: |
uv venv
uv pip install fastmcp "sqlalchemy[asyncio]" aiosqlite pydantic \
pytest pytest-asyncio pytest-cov pytest-mock pytest-html
uv pip install --no-deps -e .
uv pip show fastmcp | grep -i '^version'
- name: Test
run: uv run --no-sync pytest -q --no-cov
+157
View File
@@ -5,6 +5,163 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Calendar Versioning](https://calver.org/) (YYYY.MM.DD).
## [2026.09.27.4] - 2026-09-27
Count errors that tools return as data instead of raising.
### Added
- Records have an `outcome`: `ok`, `soft_error` (the tool returned a result
flagged as a failure) or `error` (the tool raised). Soft errors get
`error_type="SoftError"` and an `error_message` built from the result, such
as `status=failed: disk full`, redacted and truncated to `max_error_chars`.
- `soft_error_classifier(statuses=..., status_keys=..., flag_keys=...,
extra_statuses=...)` builds a classifier that flags dict results where `ok`
or `success` is `False`, where `status` or `state` is `error`, `failed` or
`failure`, or that the tool marked `isError` without raising.
`DEFAULT_RESULT_CLASSIFIER` is `soft_error_classifier()`.
- `InstrumentationMiddleware(..., result_classifier=...)` takes any
`(tool, result) -> str | None`; `None` turns classification off. A
classifier that raises is logged and the call is recorded as `ok`.
- `feedback_context()` rows include `outcome`.
### Changed
- `ok` is now `outcome == "ok"`, so it is also false for soft errors. Existing
`WHERE NOT ok` queries and error rates will rise; filter on
`outcome = 'error'` for exceptions only. `CallLink.ok` follows.
- The package's own feedback tools answer `{"success": False, ...}` on failure,
so those calls are now recorded as soft errors.
### Migration
- New nullable, indexed `outcome` column (`String(16)`) on
`{prefix}tool_calls`. Hosts that manage the schema with Alembic need a
migration adding it. `create_tables=True` does not alter a table that
already exists, so add the column by hand there
(`ALTER TABLE ffb_tool_calls ADD COLUMN outcome VARCHAR(16)` plus an index).
Until the column exists, `DatabaseSink` writes fail and those records are
dropped (logged). Rows from earlier versions keep NULL.
## [2026.09.27.3] - 2026-09-27
Link a feedback item to the tool calls that preceded it.
### Added
- `InstrumentationMiddleware.capture_recent_calls(ctx, n=20)` returns the
calls that led up to now. It matches the same client session first
(`session` rule), then fills remaining slots with calls by the same
`user_sub` inside `window` (`user_window` rule, 15 minutes by default), and
each link records which rule matched.
- Calls are kept in an in-memory buffer as they happen, so calls still
waiting in the sink queue count. With a `DatabaseSink` the lookup falls back
to the database, which covers calls from other workers and from before a
restart.
- `link_feedback(ref, links)` stores links against a feedback id of any
format (`bug-7Q2X`, `42`) in a new `{prefix}feedback_call_links` table.
`build_metadata()` includes it; hosts that manage the schema with Alembic
need a new migration.
- `feedback_context(ref)` returns the linked calls in order with tool,
outcome, error type and matching rule, filling in calls not yet written
from memory.
- `add_feedback_tools(..., instrumentation=mw)`: `submit_feedback` links the
preceding calls automatically and returns `linked_calls` in its response.
- `session_key(ctx)` gives a per-client key that stays stable across calls:
the HTTP `mcp-session-id` header when present, then one key per process for
stdio, then `ctx.session_id`.
### Fixed
- With `create_tables=True`, the background writer and `link_feedback` could
both run `create_all` at once and fail with "table already exists". Table
creation is now serialized by a lock.
### Note
- Clients on MCP protocol 2026-07-28 (Claude Code, FastMCP 4's `Client`)
open no HTTP sessions, so their calls link by user only. HTTP servers should
configure an `identity_resolver`. Stdio servers link by session with every
client.
## [2026.09.27.2] - 2026-09-27
### Fixed
- `DatabaseSink` stored missing `args`/`result` (every `meta` mode row) as the
JSON literal `null`, so `WHERE args IS NULL` matched nothing. They are now
SQL `NULL`. No schema change; rows written by 2026.09.27.1 keep JSON `null`
and can be backfilled with
`UPDATE ffb_tool_calls SET args = NULL WHERE args = 'null'::jsonb` (same for
`result`). Reported from production by blender-mcp.
## [2026.09.27.1] - 2026-09-27
Per-call instrumentation for any FastMCP server.
### Added
- `fastmcp_feedback.instrumentation`: `InstrumentationMiddleware` and
`instrument(app)` record every tool call on the server (tool, timing,
outcome, original error type, session and request ids, argument and result
sizes). Modes `off`, `meta` (default) and `full`, set by argument or
`FEEDBACK_INSTRUMENTATION_MODE`; `meta_only_tools` keeps chosen tools at
`meta`.
- Recording never slows or breaks a call: records reach sinks from a
background task through a bounded queue and are dropped (and counted) when it
is full. Sink and hook errors are logged, never raised; async hooks are cut
off after `hook_timeout`.
- `Redactor`: secrets removed from arguments, results, error text and hook
output, by key at any depth and by value pattern (JWT, bearer, `sk-`,
GitHub, Slack, AWS, PyPI and `bmcp_` tokens, `password=...` in text).
- Sinks: `JsonLinesSink` (stderr), `CallbackSink`, `MemorySink`, and
`DatabaseSink` on async SQLAlchemy. `DatabaseSink` accepts the host's
`AsyncEngine`, writes to a prefixed `ffb_tool_calls` table, never creates
tables unless asked, and `build_metadata()` plugs the table into a host's
Alembic `env.py`.
- `identity_resolver` and `enricher` hooks fill `user_sub`, `caller_kind`,
`client_id`, `server_version` and free-form `identity`/`extra` fields.
- `instrumentation` extra: `sqlalchemy[asyncio]` and `aiosqlite`.
- CI covers FastMCP 3.x as well as 2.12, 2.14 and 4.x.
## [2026.09.27] - 2026-09-27
FastMCP 4 support, and a composition bug that affected every earlier release.
### Fixed
- `compose_feedback_services()` composed nothing on FastMCP 2.14. It called
`import_server()`, which became async in 2.14, without awaiting it, then
logged success anyway. On FastMCP 4, where `import_server()` no longer
exists, it raised `AttributeError`. It now uses `mount()`, which is
synchronous and available on every supported version.
- The test suite could not start (a pytest-html hook with pytest-html not
installed), so it had not run since the package was split out. It runs
again: 407 tests.
### Changed
- Dependency is now `fastmcp>=2.12.2,<5`. Earlier releases had no ceiling,
so a fresh install picked up FastMCP 4 untested.
- `examples/` moved into this repository and ships in the sdist.
`create_demo_server()` accepts an optional `database_url`. Examples list
and call tools through `fastmcp.Client`, which works on 2.x and 4.x.
### Added
- CI (Gitea Actions) runs the suite against FastMCP 2.12.2, 2.14.3 and 4.x
on every push and weekly, plus a non-blocking job against the newest
FastMCP with no ceiling.
- A composition test using real servers instead of mocks.
### Documentation
- README rewritten against the code and every Python example executed.
Previous docs described things that do not exist: four tools (there are
five, including `delete_feedback`), `get_feedback_stats` (it is
`get_feedback_statistics`), `InsightsMixin`, an `enable_insights`
parameter, feedback types `bug_report`/`feature_request`/`general` (they
are `bug`/`feature`/`improvement`/`question`), and a
`new -> reviewing -> planned` workflow (it is
`open -> in_progress -> resolved -> closed`).
- The default database is in-memory SQLite (`sqlite:///:memory:`), not
`sqlite:///feedback.db` as previously documented. Pass `database_url` to
keep feedback across restarts.
## [2026.01.12.1] - 2026-01-12
### Fixed
- `ImportError` on install: `mixins/base.py` did not re-export the names
the other mixin modules import from it.
## [2026.01.12] - 2026-01-12
### Added
+325 -58
View File
@@ -1,17 +1,19 @@
# FastMCP Feedback
Production-ready feedback collection system for [FastMCP](https://gofastmcp.com) servers.
Instrumentation and QA feedback for [FastMCP](https://gofastmcp.com) servers.
Add structured feedback collection (bug reports, feature requests, general feedback) to any FastMCP server with a single line of code.
Record every tool call your server handles, and let people and models report problems with it, each with a line or two of code.
## Features
- **Per-Call Instrumentation** - Middleware records every tool call (timing, outcome, identity, payload sizes) without slowing or breaking it
- **Secret Redaction** - Tokens and passwords removed from recorded arguments, results and errors by default
- **One-Line Integration** - Add feedback tools to any FastMCP server instantly
- **Modular Mixins** - Selective tool registration for fine-grained control
- **Multi-Database Support** - SQLite (dev), PostgreSQL, MySQL (production)
- **Privacy-Compliant Analytics** - Optional usage insights without sensitive data
- **Type-Safe** - Full Pydantic validation and type hints
- **Well-Tested** - 97%+ test coverage
- **Well-Tested** - 98% coverage, run against FastMCP 2.12, 2.14, 3.x and 4.x
## Installation
@@ -67,59 +69,304 @@ from fastmcp import FastMCP
from fastmcp_feedback import add_feedback_tools
app = FastMCP("My Server")
add_feedback_tools(app) # Adds 4 feedback tools instantly!
add_feedback_tools(app, database_url="sqlite:///feedback.db")
# That's it! Your server now has:
# - submit_feedback: Collect bug reports, feature requests, general feedback
# - list_feedback: Retrieve and filter feedback items
# - get_feedback_stats: Aggregate statistics
# - update_feedback_status: Workflow management (new → reviewing → resolved)
# Your server now has five tools:
# - submit_feedback: bug reports, feature requests, improvements, questions
# - list_feedback: page through feedback, filtered by type or status
# - get_feedback_statistics: counts by type and status, plus the last 7 days
# - update_feedback_status: move an item through open -> in_progress -> resolved -> closed
# - delete_feedback: remove an item
```
### With Custom Database
Pass a `database_url`. Without one the tools use an in-memory SQLite
database, which is handy for tests but loses everything when the process exits.
### PostgreSQL
```python
# SQLite with custom path
add_feedback_tools(app, database_url="sqlite:///./data/feedback.db")
# PostgreSQL for production
add_feedback_tools(app, database_url="postgresql://user:pass@localhost/mydb")
add_feedback_tools(app, database_url="postgresql://user:pass@db.example.com/feedback")
```
### With Analytics Enabled
Install the driver with `uv add "fastmcp-feedback[postgresql]"`.
### Prefixed Tool Names
```python
add_feedback_tools(app, enable_insights=True)
# Enables privacy-compliant usage tracking
add_feedback_tools(app, database_url="sqlite:///feedback.db", prefix="support")
# support_submit_feedback, support_list_feedback, ...
```
The prefix and tool name are joined with `separator` (default `_`), so leave
the trailing underscore off the prefix.
### Usage Analytics
```python
from fastmcp_feedback import FeedbackInsights
add_feedback_tools(
app,
database_url="sqlite:///feedback.db",
insights=FeedbackInsights(enabled=True),
)
```
Analytics are off unless you pass `FeedbackInsights(enabled=True)` or set
`FEEDBACK_INSIGHTS_ENABLED=true`. When enabled they record tool usage and
submission metadata (type, lengths, timing), never feedback text or contact details.
## Instrumentation
Record every tool call on your server:
```python
from fastmcp import FastMCP
from fastmcp_feedback.instrumentation import instrument
app = FastMCP("My Server")
instrument(app) # one JSON line per tool call on stderr
```
Each record has the tool name, start time, duration, outcome and error type,
session and request ids, argument and result sizes, and whatever your hooks
add. The recording happens in middleware, so it covers every tool on the server,
not just the feedback ones.
It never slows or breaks a tool call. Records go to the sinks from a background
task through a bounded queue, and when the queue is full they are dropped and
counted instead of waited on. Errors in a sink or a hook are logged and swallowed.
### Modes
| Mode | Records |
|------|---------|
| `off` | Nothing |
| `meta` (default) | Tool, timing, outcome, identity, payload sizes |
| `full` | Also the arguments and result, redacted |
Set it with `mode=` or the `FEEDBACK_INSTRUMENTATION_MODE` environment variable.
`meta_only_tools={"run_code", "create_token"}` keeps chosen tools at `meta`
even in `full` mode.
### Redaction
Secrets are removed from arguments, results, error messages and hook output
before anything reaches a sink. It matches keys at any depth (`password`,
`token`, `authorization`, `client_secret`, `*_api_key`, ...) and secret-shaped
values anywhere in text (JWTs, `Bearer ...`, `sk-...`, GitHub, Slack, AWS and
PyPI tokens, `password=...` in error text). Add your own:
```python
from fastmcp_feedback.instrumentation import Redactor, instrument
instrument(
app,
mode="full",
redactor=Redactor(extra_keys=["license"], extra_patterns=[r"acme_[A-Za-z0-9]{32}"]),
)
```
### Sinks
```python
from fastmcp_feedback.instrumentation import CallbackSink, DatabaseSink, JsonLinesSink, instrument
instrument(app, [
JsonLinesSink(), # stderr
DatabaseSink("sqlite+aiosqlite:///calls.db", create_tables=True),
CallbackSink(lambda records: print(len(records))), # sync or async
])
```
`DatabaseSink` needs the `instrumentation` extra
(`uv add "fastmcp-feedback[instrumentation]"`), plus `asyncpg` for Postgres.
### Using your own database and migrations
Pass the `AsyncEngine` you already have. Records go to a `ffb_tool_calls`
table (the prefix is configurable), and nothing is created at startup unless you
ask. To manage the table with Alembic, add it to your `env.py`:
```python
from fastmcp_feedback.instrumentation import build_metadata
target_metadata = [Base.metadata, build_metadata(prefix="ffb_")]
```
```python
from fastmcp_feedback.instrumentation import DatabaseSink, instrument
sink = DatabaseSink(engine, prefix="ffb_") # your engine is never disposed
instrument(app, [sink])
```
### Errors returned as data
Many tools report failure in their result instead of raising, for example
`{"status": "failed", "error": "disk full"}` or `{"success": False, ...}`. A
result classifier catches these, so error rates count them. Each record has an
`outcome`:
| `outcome` | `ok` | Meaning |
|-----------|------|---------|
| `ok` | true | The call succeeded |
| `soft_error` | false | The tool returned a result the classifier flagged |
| `error` | false | The tool raised |
Soft errors get `error_type="SoftError"` and an `error_message` such as
`status=failed: disk full`, redacted and truncated like exception text.
The default classifier flags a dict result when `ok` or `success` is exactly
`False`, when `status` or `state` is `error`, `failed` or `failure` (any case),
or when the tool marked the result as an MCP error (`isError`) without raising.
The message comes from the first of `error`, `message`, `detail` and `reason`.
Add your own status values, or pass any `(tool, result) -> str | None`:
```python
from fastmcp_feedback.instrumentation import instrument, soft_error_classifier
# blender-mcp answers {"status": "unknown_target", ...} for a missing object
instrument(app, result_classifier=soft_error_classifier(extra_statuses=("unknown_target",)))
```
```python
from fastmcp_feedback.instrumentation import instrument
def classify(tool, result):
"""None means ok; a string marks a soft error and is its message."""
if tool == "search" and isinstance(result, dict) and result.get("hits") == []:
return "no hits"
return None
instrument(app, result_classifier=classify)
```
`result_classifier=None` turns classification off. The classifier only sees
calls that did not raise, gets the same payload the record stores (structured
content when present), and if it raises, the error is logged and the call is
recorded as `ok`. It never changes what the client receives.
**Upgrading:** soft errors set `ok` to false, so existing `WHERE NOT ok` counts
include them from this release on. Filter on `outcome = 'error'` for exceptions
only. The `outcome` column is new in 2026.09.27.4; it is nullable, and rows
written by earlier versions have NULL there. Hosts that manage the schema with
Alembic need a migration adding it (with its index) to `ffb_tool_calls`, and a
table made earlier by `create_tables=True` needs
`ALTER TABLE ffb_tool_calls ADD COLUMN outcome VARCHAR(16)`, since existing
tables are not altered. Until the column exists, database writes fail and
those records are dropped (logged).
### Identity and enrichment hooks
```python
from fastmcp_feedback.instrumentation import instrument
def who(context):
"""Who made this call. May be async."""
user = context.fastmcp_context # read your auth state from here
return {"user_sub": "user-123", "caller_kind": "pat:7", "client_id": "cli"}
async def enrich(tool, args, result, context):
"""Extra indexed fields. May be async."""
return {"job_id": (result or {}).get("job_id"), "server_version": "1.4.0"}
instrument(app, identity_resolver=who, enricher=enrich)
```
`user_sub`, `caller_kind` and `client_id` become columns; other identity keys
go in `identity`. Enricher output goes in `extra`, except `server_version`,
which has its own column. Async hooks that take longer than `hook_timeout`
(0.25 s by default) are skipped for that call.
### Linking feedback to the calls behind it
When a model reports "the render tool hung", the report is far more useful with
the calls that led up to it attached. Pass the middleware to
`add_feedback_tools` and every `submit_feedback` does this for you:
```python
from fastmcp_feedback import add_feedback_tools
from fastmcp_feedback.instrumentation import DatabaseSink, instrument
mw = instrument(app, [DatabaseSink("sqlite+aiosqlite:///calls.db", create_tables=True)])
add_feedback_tools(app, database_url="sqlite:///feedback.db", instrumentation=mw)
# submit_feedback now answers {"feedback_id": ..., "linked_calls": 3, ...}
```
Read the linked calls back, in order, with their outcomes:
```python
for call in await mw.feedback_context(feedback_id):
print(call["position"], call["tool"], call["outcome"], call["error_type"], call["rule"])
```
If you have your own feedback tools, use the same pieces directly. Feedback ids
can be anything (`bug-7Q2X`, `42`):
```python
from fastmcp import Context
@app.tool
async def report_problem(summary: str, ctx: Context) -> dict:
ref = save_my_feedback(summary) # your storage, your id
links = await mw.capture_recent_calls(ctx, n=20)
await mw.link_feedback(ref, links)
return {"id": ref, "linked_calls": len(links)}
```
Calls are matched in two ways, and each link records which one (`rule`):
- `session`: the same MCP client session. Over HTTP this is the
`mcp-session-id`; over stdio it is the server process, which serves one client.
- `user_window`: the same `user_sub` within the last 15 minutes (`window=`),
for clients that open a new session per call (personal access tokens, CI).
This needs an `identity_resolver`.
Session matches come first, and user matches fill any remaining slots. Calls
still waiting in the queue are included, and with a `DatabaseSink`, so are
calls from other workers or from before a restart.
Links live in a `ffb_feedback_call_links` table next to `ffb_tool_calls`.
`build_metadata()` includes it, so hosts that manage the schema with Alembic
need one new migration before calling `link_feedback`.
**Stateless clients:** clients on MCP protocol 2026-07-28, including Claude
Code and FastMCP 4's own `Client`, do not open sessions over HTTP. Their calls
link by user only, so configure an `identity_resolver` for HTTP servers. Stdio
servers link by session with every client.
### Middleware order and shutdown
FastMCP runs the first middleware added as the outermost. Call `instrument()`
before adding other middleware to include their time and record calls they
reject; call it last to time only the tool. On shutdown, `await mw.aclose()`
flushes queued records and closes the sinks.
## Advanced Usage
### Mixin Architecture
For fine-grained control over which tools are exposed:
Pick only the tool groups you want to expose:
```python
from fastmcp_feedback import (
SubmissionMixin,
RetrievalMixin,
ManagementMixin,
RetrievalMixin,
SubmissionMixin,
get_database_session,
)
db_session = get_database_session("sqlite:///feedback.db")
db = get_database_session("sqlite:///feedback.db")
# Only expose submission (public-facing)
submission = SubmissionMixin(db_session)
submission.register_tools(app)
# Public server: submission only
SubmissionMixin(db).register_tools(app)
# Add retrieval with custom prefix
retrieval = RetrievalMixin(db_session)
retrieval.register_tools(app, prefix="analytics_")
# Reporting tools under a prefix: analytics_list_feedback, ...
RetrievalMixin(db).register_tools(app, prefix="analytics")
# Admin-only management tools
management = ManagementMixin(db_session)
management.register_tools(admin_app, prefix="admin_")
# Admin server gets the workflow tools
ManagementMixin(db).register_tools(admin_app, prefix="admin")
```
### Available Mixins
@@ -127,18 +374,16 @@ management.register_tools(admin_app, prefix="admin_")
| Mixin | Tools | Use Case |
|-------|-------|----------|
| `SubmissionMixin` | `submit_feedback` | Public feedback collection |
| `RetrievalMixin` | `list_feedback`, `get_feedback_stats` | Dashboards, reporting |
| `ManagementMixin` | `update_feedback_status` | Admin workflow |
| `InsightsMixin` | `record_insight`, `get_insights_summary` | Analytics |
| `RetrievalMixin` | `list_feedback`, `get_feedback_statistics` | Dashboards, reporting |
| `ManagementMixin` | `update_feedback_status`, `delete_feedback` | Admin workflow |
### Multi-Tenant Pattern
```python
# Each tenant gets isolated database
def get_tenant_tools(tenant_id: str):
session = get_database_session(f"sqlite:///data/{tenant_id}.db")
submission = SubmissionMixin(session)
submission.register_tools(app, prefix=f"tenant_{tenant_id}_")
# Each tenant gets its own database
def add_tenant_tools(tenant_id: str):
db = get_database_session(f"sqlite:///data/{tenant_id}.db")
SubmissionMixin(db).register_tools(app, prefix=f"tenant_{tenant_id}")
```
### Server Composition
@@ -146,49 +391,72 @@ def get_tenant_tools(tenant_id: str):
```python
from fastmcp_feedback import create_feedback_server
# Create dedicated feedback server
feedback_server = create_feedback_server("Feedback API")
feedback_server = create_feedback_server(
"Feedback API", database_url="sqlite:///feedback.db"
)
# Compose into main application
main_app.import_server(feedback_server, prefix="feedback_")
# Tools appear on main_app as feedback_submit_feedback, feedback_list_feedback, ...
main_app.mount(feedback_server, "feedback")
```
`mount()` works on every supported FastMCP version. `import_server()` was
removed in FastMCP 4.
## Compatibility
Tested against FastMCP 2.12, 2.14, 3.x and 4.x on every commit. The dependency is
declared as `fastmcp>=2.12.2,<5`, and the ceiling moves up once a new major
version passes the suite.
## API Reference
### `add_feedback_tools(app, database_url=None, enable_insights=False)`
### `add_feedback_tools(mcp, database_url=None, insights=None, prefix="", separator="_", instrumentation=None)`
Add all feedback tools to a FastMCP server.
Add all five feedback tools to a FastMCP server.
**Parameters:**
- `app`: FastMCP server instance
- `database_url`: Database connection string (default: `sqlite:///feedback.db`)
- `enable_insights`: Enable privacy-compliant analytics (default: `False`)
- `mcp`: FastMCP server instance
- `database_url`: SQLAlchemy URL. Defaults to in-memory SQLite (`sqlite:///:memory:`)
- `insights`: a `FeedbackInsights` instance. Defaults to one with analytics disabled
- `prefix`: prepended to every tool name
- `separator`: placed between `prefix` and the tool name
- `instrumentation`: the middleware returned by `instrument()`. When given,
`submit_feedback` links the calls that preceded it to the new item and
returns `linked_calls` (see [Linking feedback to the calls behind it](#linking-feedback-to-the-calls-behind-it))
### Feedback Types
- `bug_report` - Bug reports with severity levels
- `feature_request` - Feature suggestions with priority
- `general` - General feedback and comments
- `bug` - something is broken
- `feature` - a new capability
- `improvement` - make an existing capability better
- `question` - unclear behavior or usage
### Status Workflow
`new` → `reviewing` → `planned` → `in_progress` → `resolved` / `wont_fix`
`open` → `in_progress` → `resolved` → `closed`
## Documentation
Full documentation: https://fastmcp-feedback.l.supported.systems
- [Quick Start Guide](https://fastmcp-feedback.l.supported.systems/guides/quickstart/)
- [Architecture Overview](https://fastmcp-feedback.l.supported.systems/reference/architecture/)
- [Quick Start Tutorial](https://fastmcp-feedback.l.supported.systems/tutorials/quickstart/)
- [Architecture](https://fastmcp-feedback.l.supported.systems/explanation/architecture/)
- [API Reference](https://fastmcp-feedback.l.supported.systems/reference/api/)
- [Examples](https://fastmcp-feedback.l.supported.systems/examples/)
- [Integration Patterns](https://fastmcp-feedback.l.supported.systems/how-to/integration-patterns/)
## Examples
See the [fastmcp-feedback-showcase](https://git.supported.systems/fastmcp-feedback/fastmcp-example) repository for comprehensive examples demonstrating all capabilities:
The [`examples/`](https://git.supported.systems/fastmcp-feedback/fastmcp-feedback/src/branch/main/examples)
directory has runnable servers, each with a walkthrough:
- `simple_integration.py`: the one-line integration
- `mcp_server.py`: a production-style server configured from environment variables
- `demo_server.py`: every feature, including server composition and sample data
```bash
uvx fastmcp-feedback-showcase # Run full demo
git clone https://git.supported.systems/fastmcp-feedback/fastmcp-feedback.git
cd fastmcp-feedback
uv run python examples/simple_integration.py
```
## Contributing
@@ -236,5 +504,4 @@ MIT License - see [LICENSE](LICENSE) for details.
- **Documentation**: https://fastmcp-feedback.l.supported.systems
- **Repository**: https://git.supported.systems/fastmcp-feedback/fastmcp-feedback
- **Issues**: https://git.supported.systems/fastmcp-feedback/fastmcp-feedback/issues
- **Showcase**: https://git.supported.systems/fastmcp-feedback/fastmcp-example
- **FastMCP**: https://gofastmcp.com
+318
View File
@@ -0,0 +1,318 @@
# FastMCP Feedback Examples
This directory contains comprehensive examples demonstrating all features of the FastMCP Feedback system.
## 🚀 Demo Server
### `demo_server.py`
**Complete demonstration server showcasing all features**
**Features Demonstrated:**
- ✅ Complete feedback collection workflow
- ✅ Privacy-compliant analytics (GDPR-ready)
- ✅ Multi-database support (SQLite, PostgreSQL, MySQL)
- ✅ Thread-safe operations with 97% test coverage
- ✅ Server composition patterns
- ✅ Performance monitoring and health checks
- ✅ Sample data for testing
**Available Tools:**
- `submit_feedback` - Submit bug reports, feature requests, improvements, questions
- `list_feedback` - Query and filter feedback with pagination
- `get_feedback_statistics` - Analytics and aggregate metrics
- `update_feedback_status` - Change status (open → in_progress → resolved)
- `delete_feedback` - Remove feedback items
- `get_system_info` - Version and dependency information
- `setup_sample_data` - Populate with realistic demo data
**Usage:**
```bash
# Run the demo server
python examples/demo_server.py
# Or with uv
uv run examples/demo_server.py
```
**Claude Desktop MCP Configuration:**
```json
{
"mcpServers": {
"fastmcp-feedback-demo": {
"command": "uv",
"args": ["run", "examples/demo_server.py"],
"env": {
"LOG_LEVEL": "INFO"
}
}
}
}
```
## 🏗️ Architecture Examples
### `simple_integration.py`
**Minimal integration example - 3 lines of code**
```python
from fastmcp import FastMCP
from fastmcp_feedback import add_feedback_tools
app = FastMCP("My App")
add_feedback_tools(app, database_url="sqlite:///feedback.db")
```
### `advanced_configuration.py`
**Production-ready configuration with all features**
```python
from fastmcp_feedback import (
add_feedback_tools,
FeedbackInsights,
FeedbackDatabase
)
# Configure privacy-compliant analytics
insights = FeedbackInsights(
enabled=True,
retention_days=90,
privacy_compliant=True
)
# Setup database with connection pooling
database = FeedbackDatabase(
"postgresql://user:pass@localhost/feedback",
pool_size=20,
max_overflow=10
)
# Add tools with custom prefix
add_feedback_tools(
app,
database=database,
insights=insights,
prefix="support_"
)
```
### `server_composition.py`
**Multi-server composition patterns**
```python
from fastmcp_feedback.feedback import (
create_submission_server,
create_management_server,
create_analytics_server
)
# Create specialized servers
public_server = create_submission_server("Public Portal")
admin_server = create_management_server("Admin Panel")
analytics = create_analytics_server("Dashboard")
# Compose into main application
main_app = FastMCP("Main Application")
await main_app.import_server(public_server, prefix="public")
await main_app.import_server(admin_server, prefix="admin")
await main_app.import_server(analytics, prefix="stats")
```
## 🎯 Usage Patterns
### Basic Feedback Workflow
1. **Submit Feedback:**
```python
# Submit a bug report
result = await submit_feedback({
"request": {
"type": "bug",
"title": "Login page not responsive on mobile",
"description": "The login form overflows on screens < 768px width",
"submitter": "user_123",
"contact_info": "user@example.com"
}
})
```
2. **List and Filter:**
```python
# Get all open bug reports
bugs = await list_feedback({
"type_filter": "bug",
"status_filter": "open",
"limit": 20
})
```
3. **Update Status:**
```python
# Mark as in progress
await update_feedback_status({
"feedback_id": "123",
"new_status": "in_progress",
"note": "Engineering team investigating"
})
```
4. **Get Analytics:**
```python
# View statistics
stats = await get_feedback_statistics()
# Returns: {"total_count": 45, "by_type": {"BUG": 12, "FEATURE": 20}, ...}
```
## 🔒 Privacy & Security Examples
### Privacy-Compliant Configuration
```python
insights = FeedbackInsights(
enabled=True, # Enable analytics
retention_days=90, # Auto-cleanup after 90 days
privacy_compliant=True, # Enforce GDPR compliance
anonymize_identifiers=True, # Hash user identifiers
exclude_contact_info=True # Don't store in metrics
)
```
### Security Features
```python
# Input validation with Pydantic
from fastmcp_feedback.models import SubmitFeedbackRequest
# SQL injection prevention via SQLAlchemy ORM
# Rate limiting support (extend mixins)
# Authentication ready (add middleware)
```
## 📊 Performance Examples
### Connection Pooling
```python
database = FeedbackDatabase(
"postgresql://localhost/feedback",
pool_size=20, # Base connections
max_overflow=10, # Burst capacity
pool_timeout=30, # Connection timeout
pool_recycle=3600 # Refresh every hour
)
```
### Bulk Operations
```python
# Bulk status updates
result = await bulk_update_status(
feedback_ids=["1", "2", "3", "4", "5"],
new_status="resolved"
)
```
### Performance Monitoring
```python
# Built-in health checks
health = await database.health_check()
if not health:
# Trigger alerts
pass
# Performance metrics
stats = await get_database_stats()
# Returns connection pool status, query times, etc.
```
## 🧪 Testing Examples
### Unit Testing with FastMCP Feedback
```python
import pytest
from fastmcp_feedback import FeedbackDatabase
@pytest.mark.asyncio
async def test_feedback_submission():
# Use in-memory database for testing
db = FeedbackDatabase("sqlite:///:memory:")
await db.initialize()
# Test feedback submission
# ... test code here
await db.close()
```
### Integration Testing
```python
# Test complete workflows
async def test_complete_feedback_lifecycle():
# Submit → List → Update → Verify → Analytics
pass
```
## 🚀 Production Deployment Examples
### Docker Configuration
```dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY pyproject.toml .
RUN pip install -e .
COPY src/ src/
COPY examples/ examples/
# Health check
HEALTHCHECK --interval=30s --timeout=10s \
CMD python -c "import asyncio; from examples.health_check import check; asyncio.run(check())"
CMD ["python", "examples/demo_server.py"]
```
### Environment Configuration
```bash
# Database
DATABASE_URL=postgresql://user:pass@db:5432/feedback
# Analytics
INSIGHTS_ENABLED=true
INSIGHTS_RETENTION_DAYS=90
# Performance
CONNECTION_POOL_SIZE=20
MAX_OVERFLOW=10
# Security
RATE_LIMIT_REQUESTS=100
RATE_LIMIT_WINDOW=3600
```
## 📚 Documentation
Each example includes:
- **Comprehensive comments** explaining every feature
- **Error handling** patterns
- **Performance considerations**
- **Security best practices**
- **Testing approaches**
- **Production deployment guidance**
## 🎯 Getting Started
1. **Start with the demo server** - `python examples/demo_server.py`
2. **Connect to Claude Desktop** using the MCP configuration above
3. **Try the tools** - submit feedback, query results, update status
4. **Explore the code** - see how each feature is implemented
5. **Customize for your needs** - adapt patterns to your use case
## 🤝 Contributing Examples
We welcome additional examples! Please:
- Include comprehensive comments
- Add error handling
- Provide usage instructions
- Update this README
- Add tests if applicable
---
**These examples demonstrate production-ready FastMCP Feedback integration with 97% test coverage and enterprise-grade reliability.**
+247
View File
@@ -0,0 +1,247 @@
# FastMCP Feedback Demo Server
**File:** `demo_server.py`
**Purpose:** Comprehensive demonstration of all FastMCP Feedback features
**Lines:** 396 lines of production-ready code
**Test Coverage:** Validated through 396+ comprehensive tests
## 🎯 Overview
The demo server is a complete, production-ready FastMCP server that showcases every feature of the FastMCP Feedback system. It's designed to demonstrate real-world usage patterns and serve as a reference implementation for developers.
## ✨ Features Demonstrated
### Core Feedback System
- **Complete CRUD Operations** - Submit, retrieve, update, delete feedback
- **Type Support** - Bug reports, feature requests, improvements, questions
- **Status Management** - Open → In Progress → Resolved → Closed workflow
- **Rich Metadata** - Timestamps, submitter info, contact details
### Advanced Capabilities
- **Privacy-Compliant Analytics** - GDPR-ready insights collection
- **Multi-Database Support** - SQLite for development, PostgreSQL for production
- **Thread-Safe Operations** - Concurrent request handling
- **Performance Monitoring** - Health checks and dependency validation
- **Server Composition** - Multiple specialized server types
### Data Integrity
- **Input Validation** - Pydantic models with comprehensive validation
- **Unicode Support** - Emojis, special characters, international text
- **Timezone Handling** - UTC-aware timestamp management
- **Whitespace Normalization** - Consistent data cleaning
## 🛠️ Available Tools (7 Total)
| Tool | Description | Use Case |
|------|-------------|----------|
| `submit_feedback` | Submit new feedback items | Users reporting bugs/requesting features |
| `list_feedback` | Query and filter feedback | Viewing feedback with pagination/filtering |
| `get_feedback_statistics` | Analytics and metrics | Dashboard data and reporting |
| `update_feedback_status` | Change feedback status | Admin workflow management |
| `delete_feedback` | Remove feedback items | Content moderation |
| `get_system_info` | Version/dependency info | System diagnostics |
| `setup_sample_data` | Populate demo data | Testing and demonstration |
## 📊 Sample Data
The demo server includes realistic sample data:
```python
sample_feedback = [
{
"type": "bug",
"title": "Memory leak in analytics processing",
"description": "Analytics worker consumes increasing memory over time...",
"submitter": "alice@example.com",
"contact_info": "alice@engineering.example.com"
},
{
"type": "feature",
"title": "Add export functionality for feedback data",
"description": "Users need ability to export feedback data as CSV or JSON...",
"submitter": "product_manager",
"contact_info": "pm@example.com"
}
# ... 3 more realistic examples
]
```
## 🚀 Usage
### Direct Execution
```bash
# Run with Python
python examples/demo_server.py
# Run with uv (recommended)
uv run examples/demo_server.py
```
### Claude Desktop Integration
```json
{
"mcpServers": {
"fastmcp-feedback-demo": {
"command": "uv",
"args": ["run", "examples/demo_server.py"],
"env": {
"DATABASE_URL": "sqlite:///feedback_demo.db",
"LOG_LEVEL": "INFO"
}
}
}
}
```
### Docker Deployment
```dockerfile
FROM python:3.11-slim
COPY . /app
WORKDIR /app
RUN pip install uv && uv sync
CMD ["uv", "run", "examples/demo_server.py"]
```
## 🔧 Configuration Options
### Environment Variables
- `DATABASE_URL` - Database connection string (default: temporary SQLite)
- `LOG_LEVEL` - Logging verbosity (DEBUG, INFO, WARNING, ERROR)
- `INSIGHTS_ENABLED` - Enable/disable analytics collection
- `RETENTION_DAYS` - Analytics data retention period
### Database Support
- **SQLite** - `sqlite:///path/to/database.db` (development)
- **PostgreSQL** - `postgresql://user:pass@host:port/db` (production)
- **In-Memory** - `sqlite:///:memory:` (testing)
## 📈 Performance Characteristics
### Throughput
- **Concurrent Requests** - Handles 100+ concurrent submissions
- **Database Performance** - Optimized queries with indexing
- **Memory Usage** - Efficient async/await patterns
### Scalability
- **Horizontal Scaling** - Stateless design supports load balancing
- **Database Scaling** - Connection pooling and query optimization
- **Analytics Scaling** - Configurable retention and aggregation
## 🧪 Testing Integration
The demo server is thoroughly tested through:
### Unit Tests
- Tool registration and configuration
- Sample data generation
- Error handling scenarios
### Integration Tests
- End-to-end workflow validation
- Database operations
- Server composition patterns
### Performance Tests
- High-volume operations
- Concurrent request handling
- Memory usage monitoring
## 🎓 Learning Examples
### Basic Feedback Submission
```python
# Submit a bug report
result = await submit_feedback({
"type": "bug",
"title": "Application crashes on startup",
"description": "Detailed error description...",
"submitter": "developer@company.com"
})
```
### Advanced Filtering
```python
# Get all open bug reports from last week
feedback = await list_feedback({
"type_filter": "bug",
"status_filter": "open",
"limit": 50
})
```
### Analytics Query
```python
# Get comprehensive statistics
stats = await get_feedback_statistics()
# Returns: total_count, by_type, by_status, recent_count
```
## 🔍 Architecture Insights
### Server Composition Pattern
```python
# The demo server demonstrates server composition:
main_server = FastMCP("Demo Server")
feedback_server = create_feedback_server("Feedback Service")
main_server.import_server(feedback_server, prefix="feedback_")
```
### Database Abstraction
```python
# Flexible database backend:
database = FeedbackDatabase(database_url)
await database.initialize() # Auto-creates tables
```
### Analytics Integration
```python
# Privacy-compliant analytics:
insights = FeedbackInsights(enabled=True, retention_days=90)
insights.record_metric("feedback_submitted", metadata)
```
## 📚 Related Documentation
- **[Simple Integration](simple_integration.md)** - Minimal 3-line integration
- **[Production Server](mcp_server.md)** - Environment-configured deployment
- **[Testing Guide](../tests/README.md)** - Comprehensive test documentation
- **[Architecture Overview](../docs/architecture.md)** - System design details
## 🤝 Contributing
When modifying the demo server:
1. **Maintain Compatibility** - Ensure all existing tools continue working
2. **Update Documentation** - Keep this file synchronized with code changes
3. **Add Tests** - Include test coverage for new features
4. **Version Compatibility** - Support multiple FastMCP versions where possible
## 🐛 Troubleshooting
### Common Issues
**Database Errors**
```bash
# Check database permissions
ls -la feedback_demo.db
# Reset database
rm feedback_demo.db && python examples/demo_server.py
```
**Import Errors**
```bash
# Ensure development installation
uv sync
# Check Python path
python -c "import fastmcp_feedback; print('OK')"
```
**Performance Issues**
```bash
# Enable debug logging
LOG_LEVEL=DEBUG python examples/demo_server.py
# Monitor memory usage
ps aux | grep demo_server
```
This demo server represents the gold standard for FastMCP Feedback integration, demonstrating every feature with production-ready code patterns and comprehensive error handling.
+402
View File
@@ -0,0 +1,402 @@
#!/usr/bin/env python3
"""
FastMCP Feedback Demo Server
A comprehensive demonstration server that showcases all features of the
FastMCP Feedback system including:
- Complete feedback collection workflow
- Privacy-compliant analytics
- Multi-database support
- Performance monitoring
- Server composition patterns
Usage:
# Run the demo server
python examples/demo_server.py
# Or use uv
uv run examples/demo_server.py
# Connect from Claude Desktop with this MCP server configuration:
# {
# "feedback-demo": {
# "command": "uv",
# "args": ["run", "examples/demo_server.py"]
# }
# }
"""
import asyncio
import logging
import sys
import tempfile
from datetime import UTC, datetime
from pathlib import Path
# Add the src directory to the path for development
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from fastmcp import Client, FastMCP
from fastmcp_feedback import (
FeedbackDatabase,
FeedbackInsights,
add_feedback_tools,
check_dependencies,
get_version_info,
)
from fastmcp_feedback.feedback import (
create_analytics_server,
create_management_server,
create_submission_server,
)
# Configure logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
handlers=[logging.StreamHandler(), logging.FileHandler("feedback_demo.log")],
)
logger = logging.getLogger(__name__)
def create_temp_database() -> str:
"""Create a temporary SQLite database for the demo."""
temp_dir = tempfile.mkdtemp(prefix="fastmcp_feedback_demo_")
db_path = Path(temp_dir) / "demo_feedback.db"
return f"sqlite:///{db_path}"
async def setup_demo_data(database: FeedbackDatabase) -> None:
"""Populate the database with sample feedback for demonstration."""
logger.info("Setting up demo data...")
sample_feedback = [
{
"type": "bug",
"title": "Memory leak in analytics processing",
"description": "Analytics worker consumes increasing memory over time, eventually causing OOM errors.",
"submitter": "alice@example.com",
"contact_info": "alice@engineering.example.com",
},
{
"type": "feature",
"title": "Add export functionality for feedback data",
"description": "Users need ability to export feedback data as CSV or JSON for external analysis.",
"submitter": "product_manager",
"contact_info": "pm@example.com",
},
{
"type": "improvement",
"title": "Optimize database query performance",
"description": "List feedback queries are slow with large datasets. Need pagination and indexing improvements.",
"submitter": "performance_team",
},
{
"type": "question",
"title": "Best practices for rate limiting?",
"description": "What are recommended rate limits for feedback submission to prevent abuse?",
"submitter": "security_team",
"contact_info": "security@example.com",
},
{
"type": "bug",
"title": "Unicode handling in feedback titles",
"description": "Emojis and special characters cause encoding errors in feedback titles. 🐛",
"submitter": "qa_engineer",
},
]
try:
await database.initialize()
# Import the model here to avoid circular imports
from fastmcp_feedback.models import Feedback, FeedbackStatus, FeedbackType
async with database.session() as session:
for item in sample_feedback:
feedback = Feedback(
type=FeedbackType(item["type"]),
status=FeedbackStatus.OPEN,
title=item["title"],
description=item["description"],
submitter=item["submitter"],
contact_info=item.get("contact_info"),
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
)
session.add(feedback)
session.commit()
logger.info(f"Added {len(sample_feedback)} sample feedback items")
except Exception as e:
logger.error(f"Failed to setup demo data: {e}")
raise
def create_demo_server(database_url: str | None = None) -> FastMCP:
"""Create the main demo server with all feedback tools.
Uses a fresh temporary SQLite database unless database_url is given.
"""
logger.info("🚀 Creating FastMCP Feedback Demo Server")
# Create the main server
server = FastMCP("FastMCP Feedback Demo")
server.description = (
"Comprehensive demonstration of FastMCP Feedback system featuring "
"complete feedback lifecycle, analytics, and multi-database support."
)
# Create temporary database for demo
database_url = database_url or create_temp_database()
logger.info(f"Using demo database: {database_url}")
# Configure insights for demo (privacy-compliant by design)
insights = FeedbackInsights(
enabled=True,
retention_days=30, # Short retention for demo
)
# Add all feedback tools to the main server
add_feedback_tools(
server,
database_url=database_url,
insights=insights,
prefix="", # No prefix for main tools
)
# Add version info tool
@server.tool(description="Get FastMCP Feedback version and dependency information")
def get_system_info() -> dict:
"""Get comprehensive system information."""
version_info = get_version_info()
dependency_status = check_dependencies()
return {
"system": "FastMCP Feedback Demo Server",
"status": "running",
"started_at": datetime.now(UTC).isoformat(),
"versions": version_info,
"dependencies": dependency_status,
"demo_features": [
"Complete feedback lifecycle",
"Privacy-compliant analytics",
"Multi-database support",
"Performance monitoring",
"Sample data included",
],
}
# Add demo data setup tool
@server.tool(description="Populate database with sample feedback data")
async def setup_sample_data() -> dict:
"""Setup sample feedback data for demonstration."""
try:
database = FeedbackDatabase(database_url)
await setup_demo_data(database)
await database.close()
return {
"success": True,
"message": "Sample data created successfully",
"items_added": 5,
"data_includes": [
"Bug reports with technical details",
"Feature requests from product team",
"Performance improvement suggestions",
"Security questions",
"Unicode handling issues",
],
}
except Exception as e:
logger.error(f"Failed to setup sample data: {e}")
return {"success": False, "error": str(e)}
return server
async def create_composed_demo_server() -> FastMCP:
"""Create a demo server using server composition patterns."""
logger.info("🏗️ Creating Composed Demo Server")
# Create main application server
main_server = FastMCP("Composed Feedback System")
main_server.description = "Demonstration of FastMCP server composition patterns"
# Create temporary databases for different services
public_db = create_temp_database()
admin_db = create_temp_database()
analytics_db = create_temp_database()
# Create specialized servers
public_server = create_submission_server(
"Public Feedback Portal",
database_url=public_db,
insights=FeedbackInsights(enabled=True, retention_days=90),
)
admin_server = create_management_server(
"Admin Management Panel",
database_url=admin_db,
insights=FeedbackInsights(enabled=True, retention_days=365),
)
analytics_server = create_analytics_server(
"Analytics Dashboard",
database_url=analytics_db,
insights=FeedbackInsights(enabled=True, retention_days=30),
)
# Compose servers with prefixes
main_server.mount(public_server, "public")
main_server.mount(admin_server, "admin")
main_server.mount(analytics_server, "analytics")
# Add composition info tool
@main_server.tool(description="Get information about server composition")
def get_composition_info() -> dict:
"""Get information about the composed server architecture."""
return {
"architecture": "Multi-server composition",
"servers": {
"public": {
"purpose": "Feedback submission for end users",
"tools": ["submit_feedback"],
"database": "isolated",
},
"admin": {
"purpose": "Management and moderation",
"tools": [
"update_feedback_status",
"delete_feedback",
"bulk_operations",
],
"database": "isolated",
},
"analytics": {
"purpose": "Reporting and insights",
"tools": ["generate_reports", "export_data", "usage_statistics"],
"database": "isolated",
},
},
"benefits": [
"Separation of concerns",
"Independent scaling",
"Role-based access control",
"Database isolation",
"Service modularity",
],
}
return main_server
async def run_demo_tests(server: FastMCP) -> dict:
"""Run basic functionality tests on the demo server."""
logger.info("🧪 Running demo functionality tests...")
try:
# Go through a client so this works the same on every FastMCP version
async with Client(server) as client:
tool_names = [tool.name for tool in await client.list_tools()]
system_info = None
if "get_system_info" in tool_names:
result = await client.call_tool("get_system_info", {})
system_info = result.structured_content
setup_result = None
if "setup_sample_data" in tool_names:
result = await client.call_tool("setup_sample_data", {})
setup_result = result.structured_content
return {
"test_status": "passed",
"tools_available": len(tool_names),
"tool_names": tool_names,
"system_info": system_info,
"sample_data_setup": setup_result,
"test_timestamp": datetime.now(UTC).isoformat(),
}
except Exception as e:
logger.error(f"Demo tests failed: {e}")
return {
"test_status": "failed",
"error": str(e),
"test_timestamp": datetime.now(UTC).isoformat(),
}
def print_demo_banner():
"""Print an informative banner for the demo server."""
banner = """
╔══════════════════════════════════════════════════════════════════╗
║ 🚀 FastMCP Feedback Demo Server ║
║ ║
║ Features Demonstrated: ║
║ • Complete feedback collection system ║
║ • Privacy-compliant analytics (GDPR-ready) ║
║ • Multi-database support (SQLite, PostgreSQL) ║
║ • Thread-safe operations with 97% test coverage ║
║ • Server composition patterns ║
║ • Performance monitoring and health checks ║
║ ║
║ Available Tools: ║
║ 📝 submit_feedback - Submit bug reports & feature requests║
║ 📋 list_feedback - Query and filter feedback ║
║ 📊 get_feedback_statistics - Analytics and metrics ║
║ 🔧 update_feedback_status - Change status (admin) ║
║ 🗑️ delete_feedback - Remove feedback items ║
║ ℹ️ get_system_info - Version and dependency info ║
║ 🎯 setup_sample_data - Populate with demo data ║
║ ║
║ Usage: Connect this as an MCP server in Claude Desktop ║
║ Configuration: See comments in this file ║
╚══════════════════════════════════════════════════════════════════╝
"""
print(banner)
async def main():
"""Main entry point for the demo server."""
print_demo_banner()
try:
# Check dependencies first
deps = check_dependencies()
if not deps["all_available"]:
logger.error("Missing required dependencies!")
for dep, info in deps["details"].items():
if not info["available"]:
logger.error(f" Missing: {dep} {info['required']}")
sys.exit(1)
logger.info("All dependencies satisfied ✅")
# Create the main demo server
server = create_demo_server()
# Run functionality tests
test_results = await run_demo_tests(server)
logger.info(f"Demo tests: {test_results['test_status']}")
if test_results["test_status"] == "passed":
logger.info(f"Server ready with {test_results['tools_available']} tools")
# Start the FastMCP server
logger.info("🎬 Starting FastMCP Feedback Demo Server...")
await server.run()
except KeyboardInterrupt:
logger.info("Demo server stopped by user")
except Exception as e:
logger.error(f"Demo server failed: {e}")
sys.exit(1)
if __name__ == "__main__":
asyncio.run(main())
+396
View File
@@ -0,0 +1,396 @@
# FastMCP Feedback Production Server
**File:** `mcp_server.py`
**Purpose:** Production-ready MCP server with environment configuration
**Lines:** 89 lines of deployment-focused code
**Target:** Production and staging environments
## 🎯 Overview
This is a production-ready MCP server launcher designed for real-world deployments. It demonstrates environment-based configuration, proper logging, and integration patterns suitable for staging and production environments.
## ✨ Production Features
### Environment Configuration
- **Database URL** - Configurable via `DATABASE_URL` environment variable
- **Insights Control** - Enable/disable analytics via `INSIGHTS_ENABLED`
- **Logging Levels** - Configurable via `LOG_LEVEL` (DEBUG, INFO, WARNING, ERROR)
- **Graceful Defaults** - Sensible fallbacks for missing configuration
### Security Considerations
- **No Hardcoded Secrets** - All sensitive data via environment variables
- **Input Validation** - Comprehensive Pydantic validation
- **Error Handling** - Graceful failure without exposing internals
- **Audit Logging** - Configurable logging for compliance
### Deployment Ready
- **Docker Compatible** - Environment variable based configuration
- **Kubernetes Ready** - ConfigMap and Secret integration
- **Load Balancer Friendly** - Stateless design
- **Health Checks** - Built-in system info endpoint
## 🚀 Usage
### Local Development
```bash
# Run with default settings
python examples/mcp_server.py
# Run with uv (recommended)
uv run examples/mcp_server.py
# Run with custom database
DATABASE_URL="postgresql://localhost/feedback" uv run examples/mcp_server.py
```
### Environment Variables
```bash
# Required/Optional Configuration
export DATABASE_URL="postgresql://user:pass@host:5432/feedback_prod"
export INSIGHTS_ENABLED="true"
export LOG_LEVEL="INFO"
export RETENTION_DAYS="90"
# Run the server
uv run examples/mcp_server.py
```
## 🔧 Configuration Options
### Database Configuration
```bash
# SQLite (development/testing)
DATABASE_URL="sqlite:///feedback.db"
# PostgreSQL (production)
DATABASE_URL="postgresql://user:password@host:5432/database"
# PostgreSQL with SSL (production)
DATABASE_URL="postgresql://user:pass@host:5432/db?sslmode=require"
# Environment-specific databases
DATABASE_URL_DEV="sqlite:///feedback_dev.db"
DATABASE_URL_STAGING="postgresql://staging_user:pass@staging.db/feedback"
DATABASE_URL_PROD="postgresql://prod_user:pass@prod.db/feedback"
```
### Analytics Configuration
```bash
# Enable analytics with retention
INSIGHTS_ENABLED="true"
RETENTION_DAYS="90"
# Disable analytics (GDPR compliance)
INSIGHTS_ENABLED="false"
# Different retention by environment
RETENTION_DAYS_DEV="7"
RETENTION_DAYS_STAGING="30"
RETENTION_DAYS_PROD="365"
```
### Logging Configuration
```bash
# Production logging
LOG_LEVEL="INFO"
LOG_FORMAT="json" # For log aggregation
# Development logging
LOG_LEVEL="DEBUG"
LOG_FORMAT="standard"
# Disable logging (not recommended)
LOG_LEVEL="CRITICAL"
```
## 🐳 Docker Deployment
### Dockerfile
```dockerfile
FROM python:3.11-slim
# Set working directory
WORKDIR /app
# Install uv for faster dependency management
RUN pip install uv
# Copy dependency files
COPY pyproject.toml uv.lock ./
# Install dependencies
RUN uv sync --frozen
# Copy application code
COPY . .
# Set environment variables
ENV DATABASE_URL="postgresql://postgres:password@db:5432/feedback"
ENV INSIGHTS_ENABLED="true"
ENV LOG_LEVEL="INFO"
# Expose port (if needed)
EXPOSE 8000
# Run the MCP server
CMD ["uv", "run", "examples/mcp_server.py"]
```
### Docker Compose
```yaml
version: '3.8'
services:
feedback-server:
build: .
environment:
- DATABASE_URL=postgresql://postgres:password@db:5432/feedback
- INSIGHTS_ENABLED=true
- LOG_LEVEL=INFO
- RETENTION_DAYS=90
depends_on:
- db
restart: unless-stopped
db:
image: postgres:15
environment:
- POSTGRES_DB=feedback
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
volumes:
- postgres_data:/var/lib/postgresql/data
restart: unless-stopped
volumes:
postgres_data:
```
## ☸️ Kubernetes Deployment
### ConfigMap
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: feedback-config
data:
LOG_LEVEL: "INFO"
INSIGHTS_ENABLED: "true"
RETENTION_DAYS: "90"
```
### Secret
```yaml
apiVersion: v1
kind: Secret
metadata:
name: feedback-secrets
type: Opaque
stringData:
DATABASE_URL: "postgresql://user:password@postgres-service:5432/feedback"
```
### Deployment
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: feedback-server
spec:
replicas: 3
selector:
matchLabels:
app: feedback-server
template:
metadata:
labels:
app: feedback-server
spec:
containers:
- name: feedback-server
image: fastmcp-feedback:latest
ports:
- containerPort: 8000
envFrom:
- configMapRef:
name: feedback-config
- secretRef:
name: feedback-secrets
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
```
## 🎛️ Claude Desktop Configuration
### Development
```json
{
"mcpServers": {
"fastmcp-feedback-dev": {
"command": "uv",
"args": ["run", "examples/mcp_server.py"],
"env": {
"DATABASE_URL": "sqlite:///feedback_dev.db",
"INSIGHTS_ENABLED": "true",
"LOG_LEVEL": "DEBUG"
}
}
}
}
```
### Production
```json
{
"mcpServers": {
"fastmcp-feedback-prod": {
"command": "uv",
"args": ["run", "examples/mcp_server.py"],
"env": {
"DATABASE_URL": "postgresql://user:pass@prod.db/feedback",
"INSIGHTS_ENABLED": "true",
"LOG_LEVEL": "INFO",
"RETENTION_DAYS": "365"
}
}
}
}
```
## 📊 Monitoring & Observability
### Health Checks
```python
# Built-in health check endpoint
await get_system_info()
# Returns: version, dependencies, database_status, insights_status
```
### Logging Format
```json
{
"timestamp": "2024-01-15T10:30:00Z",
"level": "INFO",
"logger": "fastmcp_feedback.tools",
"message": "Feedback submitted successfully",
"feedback_id": "123",
"submitter_hash": "a1b2c3",
"type": "bug"
}
```
### Metrics Collection
- **Request Counts** - Total submissions, retrievals, updates
- **Error Rates** - Failed operations and reasons
- **Performance** - Response times and database query duration
- **Usage Patterns** - Popular feedback types, status distributions
## 🔍 Troubleshooting
### Common Issues
**Database Connection Errors**
```bash
# Check database connectivity
DATABASE_URL="postgresql://..." python -c "
import asyncio
from fastmcp_feedback import FeedbackDatabase
async def test():
db = FeedbackDatabase('$DATABASE_URL')
await db.initialize()
print('✅ Database connection successful')
asyncio.run(test())
"
```
**Environment Variable Issues**
```bash
# Verify environment variables
env | grep -E "(DATABASE_URL|INSIGHTS_ENABLED|LOG_LEVEL)"
# Test with explicit variables
DATABASE_URL="sqlite:///test.db" \
INSIGHTS_ENABLED="false" \
LOG_LEVEL="DEBUG" \
python examples/mcp_server.py
```
**Permission Issues**
```bash
# Check file permissions
ls -la feedback.db
# Check directory permissions
ls -ld $(dirname $(readlink -f feedback.db))
# Fix permissions
chmod 644 feedback.db
chmod 755 $(dirname $(readlink -f feedback.db))
```
### Performance Tuning
**Database Optimization**
```bash
# PostgreSQL connection pooling
DATABASE_URL="postgresql://user:pass@host/db?pool_size=20&max_overflow=30"
# SQLite optimization
DATABASE_URL="sqlite:///feedback.db?journal_mode=WAL&synchronous=NORMAL"
```
**Memory Optimization**
```bash
# Limit analytics retention
RETENTION_DAYS="30"
# Disable insights in memory-constrained environments
INSIGHTS_ENABLED="false"
```
## 🚀 Scaling Considerations
### Horizontal Scaling
- **Stateless Design** - No server-side session storage
- **Database Sharing** - Multiple instances can share database
- **Load Balancer Compatible** - Standard HTTP/JSON interfaces
### Database Scaling
- **Read Replicas** - Use read-only database URLs for list operations
- **Connection Pooling** - Configure appropriate pool sizes
- **Query Optimization** - Built-in indexing and pagination
### Caching Strategy
```python
# Redis caching for statistics (optional)
REDIS_URL="redis://localhost:6379/0"
CACHE_TTL="300" # 5 minutes
# Memcached alternative
MEMCACHED_SERVERS="localhost:11211"
```
## 📚 Related Documentation
- **[Demo Server](demo_server.md)** - Complete feature showcase
- **[Simple Integration](simple_integration.md)** - Minimal integration
- **[Architecture Guide](../docs/architecture.md)** - System design
- **[Security Guide](../docs/security.md)** - Security best practices
## 🤝 Contributing
When modifying the production server:
1. **Maintain Backward Compatibility** - Don't break existing environment variables
2. **Add Configuration Validation** - Validate new environment variables
3. **Update Documentation** - Keep this file synchronized with changes
4. **Test in Multiple Environments** - Docker, Kubernetes, bare metal
5. **Consider Security Implications** - Audit any configuration changes
This production server demonstrates enterprise-ready deployment patterns while maintaining the simplicity and power of the FastMCP Feedback system.
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env python3
"""
FastMCP Feedback - MCP Server Launcher
This script provides a clean MCP server launcher that can be easily
configured in Claude Desktop or other MCP clients.
Usage:
# Direct execution
python examples/mcp_server.py
# With uv
uv run examples/mcp_server.py
# Claude Desktop configuration:
{
"mcpServers": {
"fastmcp-feedback": {
"command": "uv",
"args": ["run", "examples/mcp_server.py"],
"env": {
"DATABASE_URL": "sqlite:///feedback.db",
"INSIGHTS_ENABLED": "true",
"LOG_LEVEL": "INFO"
}
}
}
}
"""
import asyncio
import logging
import os
import sys
from pathlib import Path
# Add the src directory to the path for development
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from fastmcp import Client, FastMCP
from fastmcp_feedback import FeedbackInsights, add_feedback_tools
# Configure logging based on environment
log_level = os.getenv("LOG_LEVEL", "INFO").upper()
logging.basicConfig(
level=getattr(logging, log_level),
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
)
logger = logging.getLogger(__name__)
def create_production_server() -> FastMCP:
"""Create a production-ready FastMCP Feedback server."""
# Server configuration
server_name = os.getenv("SERVER_NAME", "FastMCP Feedback Server")
database_url = os.getenv("DATABASE_URL", "sqlite:///feedback.db")
# Create server
server = FastMCP(server_name)
server.description = "Production FastMCP Feedback collection system"
# Configure insights
insights_enabled = os.getenv("INSIGHTS_ENABLED", "true").lower() == "true"
retention_days = int(os.getenv("INSIGHTS_RETENTION_DAYS", "90"))
insights = FeedbackInsights(enabled=insights_enabled, retention_days=retention_days)
# Add feedback tools
tool_prefix = os.getenv("TOOL_PREFIX", "")
add_feedback_tools(
server, database_url=database_url, insights=insights, prefix=tool_prefix
)
logger.info(f"Server '{server_name}' configured")
logger.info(f"Database: {database_url}")
logger.info(f"Insights: {'enabled' if insights_enabled else 'disabled'}")
logger.info(f"Tool prefix: '{tool_prefix}' (empty = no prefix)")
return server
async def main():
"""Main entry point for the MCP server."""
try:
logger.info("🚀 Starting FastMCP Feedback Server")
# Create and start server
server = create_production_server()
# Show available tools
async with Client(server) as client:
tools = await client.list_tools()
logger.info(f"Server ready with {len(tools)} tools:")
for tool in tools:
logger.info(f" • {tool.name}")
# Start the server
await server.run()
except KeyboardInterrupt:
logger.info("Server stopped by user")
except Exception as e:
logger.error(f"Server failed: {e}")
sys.exit(1)
if __name__ == "__main__":
asyncio.run(main())
+274
View File
@@ -0,0 +1,274 @@
# Simple FastMCP Feedback Integration
**File:** `simple_integration.py`
**Purpose:** Minimal 3-line integration example
**Lines:** 48 lines of clean, documented code
**Complexity:** Beginner-friendly
## 🎯 Overview
This example demonstrates the absolute minimal integration of FastMCP Feedback into any FastMCP server. With just 3 lines of code, you get a complete, production-ready feedback system.
## ✨ Key Features
### Minimal Code
```python
# This is all you need - 3 lines of code!
app = FastMCP("Simple Feedback App")
add_feedback_tools(app, database_url="sqlite:///simple_feedback.db")
```
### Automatic Tool Registration
The integration automatically adds 5 production-ready tools:
- ✅ `submit_feedback` - Submit bug reports & feature requests
- ✅ `list_feedback` - Query and filter feedback
- ✅ `get_feedback_statistics` - Analytics and metrics
- ✅ `delete_feedback` - Remove feedback items
- ✅ `update_feedback_status` - Change status (admin)
### Zero Configuration
- **Database** - Automatically creates SQLite database
- **Schema** - Auto-generates tables on first run
- **Validation** - Built-in Pydantic input validation
- **Error Handling** - Graceful failure scenarios
## 🚀 Usage
### Direct Execution
```bash
# Run the example
python examples/simple_integration.py
# With uv (recommended)
uv run examples/simple_integration.py
```
### Expected Output
```
🚀 Simple FastMCP Feedback Integration
==================================================
✅ Feedback system added!
📊 Available tools:
• submit_feedback - Submit feedback to the system
• get_feedback_statistics - Get feedback statistics and analytics
• list_feedback - List feedback items with optional filtering
• delete_feedback - Delete feedback item
• update_feedback_status - Update feedback status
🎯 Total tools: 5
🎬 Starting server...
Connect this as an MCP server in Claude Desktop
```
## 📋 Integration Steps
### Step 1: Install Dependencies
```bash
# Using uv (recommended)
uv add fastmcp-feedback
# Using pip
pip install fastmcp-feedback
```
### Step 2: Add to Your Server
```python
from fastmcp import FastMCP
from fastmcp_feedback import add_feedback_tools
# Your existing FastMCP server
app = FastMCP("My Awesome App")
# Add your existing tools here...
# app.add_tool(my_custom_tool)
# Add complete feedback system in one line
add_feedback_tools(app, database_url="sqlite:///feedback.db")
```
### Step 3: Configure Claude Desktop
```json
{
"mcpServers": {
"my-app-with-feedback": {
"command": "python",
"args": ["my_server.py"]
}
}
}
```
## 🔧 Customization Options
### Database Configuration
```python
# SQLite (development)
add_feedback_tools(app, database_url="sqlite:///feedback.db")
# PostgreSQL (production)
add_feedback_tools(app, database_url="postgresql://user:pass@host/db")
# In-memory (testing)
add_feedback_tools(app, database_url="sqlite:///:memory:")
```
### Analytics Configuration
```python
from fastmcp_feedback import add_feedback_tools, FeedbackInsights
# Enable analytics with custom retention
insights = FeedbackInsights(enabled=True, retention_days=30)
add_feedback_tools(app, insights=insights)
```
### Tool Prefixing
```python
# Add with prefix to avoid naming conflicts
add_feedback_tools(app, prefix="support_")
# Creates: support_submit_feedback, support_list_feedback, etc.
```
## 📊 What You Get
### Complete CRUD Operations
```python
# Users can submit feedback
await submit_feedback({
"type": "bug",
"title": "App crashes on login",
"description": "Detailed error description",
"submitter": "user@example.com"
})
# Admins can manage feedback
await update_feedback_status({
"feedback_id": "123",
"new_status": "resolved"
})
# Analytics for insights
stats = await get_feedback_statistics()
# Returns: total_count, by_type, by_status, recent_count
```
### Input Validation
- **Type Validation** - Must be: bug, feature, improvement, question
- **Required Fields** - title, description, submitter cannot be empty
- **Length Limits** - title ≤ 255 chars, description ≤ 10,000 chars
- **Format Validation** - email validation for contact_info (optional)
### Database Schema
Automatically creates:
```sql
CREATE TABLE feedback (
id INTEGER PRIMARY KEY AUTOINCREMENT,
type VARCHAR(50) NOT NULL,
title VARCHAR(255) NOT NULL,
description TEXT NOT NULL,
submitter VARCHAR(255) NOT NULL,
contact_info VARCHAR(255),
status VARCHAR(50) NOT NULL DEFAULT 'open',
created_at DATETIME NOT NULL,
updated_at DATETIME NOT NULL
);
```
## 🎓 Learning Examples
### Basic Usage Pattern
```python
#!/usr/bin/env python3
from fastmcp import FastMCP
from fastmcp_feedback import add_feedback_tools
app = FastMCP("My App")
# Your custom tools
@app.tool()
def my_custom_feature():
return "Custom functionality"
# Add feedback system
add_feedback_tools(app)
# Run server
app.run()
```
### Error Handling
```python
try:
add_feedback_tools(app, database_url="invalid://url")
except Exception as e:
print(f"Failed to add feedback tools: {e}")
# Graceful fallback or alternative configuration
```
## 🔍 Under the Hood
### What `add_feedback_tools()` Does
1. **Creates Database Instance** - Initializes FeedbackDatabase with provided URL
2. **Sets Up Insights** - Configures optional analytics collection
3. **Registers Tools** - Adds 5 tools to your FastMCP server
4. **Validates Configuration** - Ensures database connectivity
5. **Logs Success** - Confirms successful integration
### Tool Implementation Pattern
```python
# Each tool is implemented as an async function
@app.tool(description="Submit feedback to the system")
async def submit_feedback(request: dict) -> dict:
# Validate input with Pydantic
validated_request = SubmitFeedbackRequest(**request)
# Store in database
feedback = await database.create_feedback(validated_request)
# Record analytics (if enabled)
insights.record_metric("feedback_submitted", metadata)
return {"success": True, "feedback_id": feedback.id}
```
## 🚀 Next Steps
### For More Features
If you need advanced capabilities, see:
- **[Demo Server](demo_server.md)** - Complete feature showcase
- **[Production Server](mcp_server.md)** - Environment configuration
- **[Architecture Guide](../docs/architecture.md)** - System design
### For Customization
```python
# Use individual mixins for fine-grained control
from fastmcp_feedback.mixins import SubmissionMixin, RetrievalMixin
submission = SubmissionMixin(database, insights)
submission.register_tools(app, prefix="feedback_")
```
### For Testing
```python
# Test your integration
import pytest
from fastmcp_feedback import add_feedback_tools
@pytest.mark.asyncio
async def test_feedback_integration():
app = FastMCP("Test App")
add_feedback_tools(app, database_url="sqlite:///:memory:")
tools = await app.get_tools()
assert "submit_feedback" in tools
```
## 🤝 Contributing
When modifying this example:
1. **Keep It Simple** - This is the "Hello World" of FastMCP Feedback
2. **Document Changes** - Update this file with any modifications
3. **Test Integration** - Ensure the 3-line pattern still works
4. **Preserve Clarity** - Maintain beginner-friendly explanations
This simple integration example proves that adding a complete, production-ready feedback system to any FastMCP server requires just 3 lines of code while providing enterprise-grade functionality.
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env python3
"""
Simple FastMCP Feedback Integration Example
This demonstrates the absolute minimal integration - just 3 lines of code
to add a complete feedback system to any FastMCP server.
Usage:
python examples/simple_integration.py
"""
import asyncio
import sys
from pathlib import Path
# Add the src directory to the path for development
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from fastmcp import Client, FastMCP
from fastmcp_feedback import add_feedback_tools
async def main():
"""Minimal FastMCP Feedback integration example."""
print("🚀 Simple FastMCP Feedback Integration")
print("=" * 50)
# This is all you need - 3 lines of code!
app = FastMCP("Simple Feedback App")
add_feedback_tools(app, database_url="sqlite:///simple_feedback.db")
print("✅ Feedback system added!")
print("📊 Available tools:")
# Show what tools were added
async with Client(app) as client:
tools = await client.list_tools()
for tool in tools:
print(f" • {tool.name} - {tool.description}")
print(f"\n🎯 Total tools: {len(tools)}")
print("\n🎬 Starting server...")
print(" Connect this as an MCP server in Claude Desktop")
# Start the server
await app.run()
if __name__ == "__main__":
asyncio.run(main())
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env python3
"""
Test script to verify demo server functionality without running the full MCP server.
"""
import asyncio
import sys
from pathlib import Path
# Add the src directory to the path for development
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
sys.path.insert(0, str(Path(__file__).parent))
from demo_server import create_demo_server
from fastmcp import Client
async def test_demo_functionality():
"""Test the demo server tools without running the MCP server."""
print("🧪 Testing FastMCP Feedback Demo Functionality")
print("=" * 55)
try:
# Create the demo server
server = create_demo_server()
async with Client(server) as client:
# Get available tools
tools = {tool.name: tool for tool in await client.list_tools()}
print(f"✅ Server created with {len(tools)} tools")
# List available tools
print("\n📋 Available Tools:")
for name, tool in tools.items():
print(f" • {name}: {tool.description}")
# Test system info tool
if "get_system_info" in tools:
print("\n🔍 Testing system info...")
result = await client.call_tool("get_system_info", {})
info = result.structured_content
print(f" Version: {info['versions']['fastmcp_feedback']}")
print(f" Python: {info['versions']['python']}")
print(f" Features: {len(info['demo_features'])}")
# Test sample data setup
if "setup_sample_data" in tools:
print("\n📊 Testing sample data setup...")
result = await client.call_tool("setup_sample_data", {})
data_result = result.structured_content
if data_result["success"]:
print(f" ✅ Added {data_result['items_added']} sample items")
else:
print(f" ❌ Failed: {data_result['error']}")
# Test feedback listing
if "list_feedback" in tools:
print("\n📝 Testing feedback listing...")
result = await client.call_tool("list_feedback", {})
feedback_data = result.structured_content
print(f" Found {len(feedback_data['feedback'])} feedback items")
# Show first item details
if feedback_data["feedback"]:
first = feedback_data["feedback"][0]
print(f" First item: {first['type']} - {first['title']}")
# Test statistics
if "get_feedback_statistics" in tools:
print("\n📈 Testing statistics...")
result = await client.call_tool("get_feedback_statistics", {})
stats = result.structured_content
print(f" Total items: {stats['total_count']}")
print(f" By type: {stats['by_type']}")
print(f" By status: {stats['by_status']}")
print("\n🎉 All tests passed! Demo server is working correctly.")
return True
except Exception as e:
print(f"\n❌ Test failed: {e}")
import traceback
traceback.print_exc()
return False
if __name__ == "__main__":
success = asyncio.run(test_demo_functionality())
sys.exit(0 if success else 1)
+18 -3
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "fastmcp-feedback"
version = "2026.01.12.1" # YYYYMMDD.patch versioning - update for each release
version = "2026.09.27.4" # YYYYMMDD.patch versioning - update for each release
description = "Production-ready feedback collection system for FastMCP servers"
readme = "README.md"
license = "MIT"
@@ -43,27 +43,35 @@ classifiers = [
]
requires-python = ">=3.11"
dependencies = [
"fastmcp>=2.12.2",
"fastmcp>=2.12.2,<5", # CI tests 2.12, 2.14 and 4.0; raise the ceiling once 5.x passes
"sqlalchemy>=2.0.43",
"pydantic>=2.11.7"
]
[project.optional-dependencies]
postgresql = ["psycopg2-binary>=2.9.0"]
# Async database sink for per-call instrumentation. Postgres hosts add asyncpg.
instrumentation = ["sqlalchemy[asyncio]>=2.0.43", "aiosqlite>=0.22.1"]
mysql = ["pymysql>=1.1.0"]
dev = [
"sqlalchemy[asyncio]>=2.0.43",
"aiosqlite>=0.22.1",
"pytest>=8.2.0",
"pytest-asyncio>=0.23.0",
"pytest-cov>=5.0.0",
"pytest-mock>=3.12.0",
"pytest-html>=4.2.0",
"ruff>=0.1.0",
"mypy>=1.7.0",
]
test = [
"sqlalchemy[asyncio]>=2.0.43",
"aiosqlite>=0.22.1",
"pytest>=8.2.0",
"pytest-asyncio>=0.23.0",
"pytest-cov>=5.0.0",
"pytest-mock>=3.12.0",
"pytest-html>=4.2.0",
]
all = ["fastmcp-feedback[postgresql,mysql,dev]"]
@@ -81,6 +89,7 @@ packages = ["src/fastmcp_feedback"]
include = [
"/src",
"/tests",
"/examples",
"README.md",
"LICENSE",
"CHANGELOG.md",
@@ -142,7 +151,13 @@ src = ["src", "tests"]
[tool.ruff.lint]
select = ["E", "W", "F", "UP", "B", "SIM", "I", "N", "C4"]
ignore = ["E501", "B008"]
ignore = [
"E501",
"B008",
# StrEnum changes str()/format() output of FeedbackType/FeedbackStatus,
# which reach the database and API responses. Worth doing, not silently.
"UP042",
]
[tool.ruff.lint.per-file-ignores]
"tests/**/*.py" = ["ARG", "PLR2004"]
+3 -3
View File
@@ -44,7 +44,7 @@ Quick Start:
import logging
# Version information - CalVer YYYY.MM.DD
__version__ = "2026.01.12.1"
__version__ = "2026.09.27.4"
__author__ = "Ryan Malloy"
__email__ = "ryan@supported.systems"
@@ -259,8 +259,8 @@ from fastmcp_feedback import create_feedback_server
main_app = FastMCP("Main Application")
feedback_service = create_feedback_server("Feedback Service")
# Import feedback service with prefix
main_app.import_server(feedback_service, prefix="feedback")
# Mount feedback service; its tools become feedback_<name>
main_app.mount(feedback_service, "feedback")
""",
"multi_tenant": """
from fastmcp import FastMCP
+4 -2
View File
@@ -260,8 +260,10 @@ def compose_feedback_services(
f"{service_name} Service", database_url, shared_insights
)
# Import service with prefix
main_server.import_server(service_server, prefix=service_name)
# mount() is sync on every supported FastMCP; the second positional
# arg is `prefix` on 2.x and `namespace` on 4.x. import_server() is
# gone in 4.x and was async (so silently never ran here) on 2.14.
main_server.mount(service_server, service_name)
logger.info(f"Composed {len(services)} feedback services into main server")
@@ -0,0 +1,48 @@
"""Per-call instrumentation for any FastMCP server.
from fastmcp_feedback.instrumentation import instrument, DatabaseSink
mw = instrument(app) # every tool call as JSON lines on stderr
mw = instrument(app, [DatabaseSink(engine)], identity_resolver=who)
mw = instrument(app, result_classifier=soft_error_classifier(extra_statuses=("unknown_target",)))
"""
from .classify import (
DEFAULT_RESULT_CLASSIFIER,
ResultClassifier,
soft_error_classifier,
)
from .dispatch import RecordDispatcher
from .middleware import MODE_ENV_VAR, MODES, InstrumentationMiddleware, instrument
from .record import ToolCallRecord
from .redaction import (
DEFAULT_SECRET_KEYS,
DEFAULT_SECRET_PATTERNS,
REDACTED,
Redactor,
)
from .sinks import CallbackSink, JsonLinesSink, MemorySink, Sink
from .storage import DEFAULT_TABLE_PREFIX, DatabaseSink, build_metadata
__all__ = [
"CallbackSink",
"DatabaseSink",
"DEFAULT_RESULT_CLASSIFIER",
"DEFAULT_SECRET_KEYS",
"DEFAULT_SECRET_PATTERNS",
"DEFAULT_TABLE_PREFIX",
"InstrumentationMiddleware",
"JsonLinesSink",
"MemorySink",
"MODE_ENV_VAR",
"MODES",
"RecordDispatcher",
"REDACTED",
"Redactor",
"ResultClassifier",
"Sink",
"ToolCallRecord",
"build_metadata",
"instrument",
"soft_error_classifier",
]
@@ -0,0 +1,97 @@
"""Spotting tools that report failure as data instead of raising.
A result classifier is ``(tool, result) -> str | None``. ``result`` is the
recorded payload: the structured content when there is one, otherwise the
content items. None means the call succeeded; a string marks it as a soft
error and becomes the record's ``error_message`` (after redaction).
A result the tool marked as an MCP error (``isError``) without raising reaches
the classifier as a dict with ``"isError": True``, alongside the payload's own
keys or with the payload under ``"content"``.
"""
from collections.abc import Callable, Iterable
from typing import Any
ResultClassifier = Callable[[str, Any], str | None]
MESSAGE_KEYS = ("error", "message", "detail", "reason")
ERROR_FLAG_KEYS = ("isError", "is_error")
def _message(result: dict, depth: int = 0) -> str | None:
for key in MESSAGE_KEYS:
value = result.get(key)
if value is None or value == "":
continue
if isinstance(value, dict):
# {"error": {"code": 404, "message": "..."}}
if depth == 0 and (inner := _message(value, 1)):
return inner
continue
if isinstance(value, list | tuple):
continue
return str(value)
return None
def _content_text(content: Any) -> str | None:
if isinstance(content, str):
return content
if not isinstance(content, list | tuple):
return None
texts = [
item if isinstance(item, str) else item.get("text")
for item in content
if isinstance(item, str | dict)
]
return " ".join(t for t in texts if t) or None
def _describe(label: str, detail: str | None) -> str:
return f"{label}: {detail}" if detail else label
def soft_error_classifier(
*,
statuses: Iterable[str] = ("error", "failed", "failure"),
status_keys: Iterable[str] = ("status", "state"),
flag_keys: Iterable[str] = ("ok", "success"),
extra_statuses: Iterable[str] = (),
) -> ResultClassifier:
"""Build a classifier for dict results that describe their own failure.
A dict result is a soft error when any of ``flag_keys`` is exactly False,
when any of ``status_keys`` holds one of ``statuses`` or
``extra_statuses`` (case-insensitive), or when the tool marked it
``isError``. The message is the first of ``error``, ``message``,
``detail`` and ``reason`` present, e.g. ``status=failed: disk full``.
>>> classify = soft_error_classifier(extra_statuses=("unknown_target",))
>>> classify("render", {"status": "unknown_target", "error": "no such object"})
'status=unknown_target: no such object'
"""
bad = frozenset(s.lower() for s in (*statuses, *extra_statuses))
status_keys = tuple(status_keys)
flag_keys = tuple(flag_keys)
def classify(tool: str, result: Any) -> str | None:
if not isinstance(result, dict):
return None
for key in status_keys:
value = result.get(key)
if isinstance(value, str) and value.lower() in bad:
return _describe(f"{key}={value}", _message(result))
for key in flag_keys:
if key in result and result[key] is False:
return _describe(f"{key}=False", _message(result))
for key in ERROR_FLAG_KEYS:
if result.get(key) is True:
detail = _message(result) or _content_text(result.get("content"))
return _describe("isError", detail)
return None
return classify
DEFAULT_RESULT_CLASSIFIER = soft_error_classifier()
@@ -0,0 +1,106 @@
"""Linking feedback to the tool calls that came before it."""
from collections.abc import Iterable
from dataclasses import asdict, dataclass
from datetime import UTC, datetime
from typing import Any
RULE_SESSION = "session"
RULE_USER_WINDOW = "user_window"
@dataclass
class CallLink:
"""A tool call linked to a piece of feedback, and why it matched."""
call_id: str
rule: str
tool: str
started_at: datetime
ok: bool
position: int = 0
def to_dict(self) -> dict[str, Any]:
data = asdict(self)
data["started_at"] = self.started_at.isoformat()
return data
@dataclass
class Candidate:
"""The fields correlation needs, from a buffered record or a database row."""
id: str
tool: str
started_at: datetime
ok: bool
session_id: str | None
user_sub: str | None
class ResolverContext:
"""What an identity resolver receives when called during correlation.
Outside a tool call there is no MiddlewareContext, so this carries only the
``fastmcp_context`` attribute, which is what resolvers normally read.
"""
def __init__(self, fastmcp_context: Any):
self.fastmcp_context = fastmcp_context
self.message = None
def as_utc(value: datetime) -> datetime:
# SQLite drops the timezone from DateTime(timezone=True) columns.
return value.replace(tzinfo=UTC) if value.tzinfo is None else value
def select_calls(
candidates: Iterable[Candidate],
*,
session_id: str | None,
user_sub: str | None,
since: datetime,
limit: int,
) -> list[CallLink]:
"""Pick up to ``limit`` recent calls for a feedback item.
Calls from the same MCP session come first. If there are fewer than
``limit``, the rest are filled with the same user's calls since ``since``,
which covers clients that open a new session per call (PATs, CI). The
result is in chronological order, numbered from 0.
"""
unique: dict[str, Candidate] = {}
for c in candidates:
unique.setdefault(c.id, c)
newest_first = sorted(
unique.values(), key=lambda c: as_utc(c.started_at), reverse=True
)
chosen: list[tuple[Candidate, str]] = []
if session_id:
chosen += [
(c, RULE_SESSION) for c in newest_first if c.session_id == session_id
][:limit]
if user_sub and len(chosen) < limit:
taken = {c.id for c, _ in chosen}
chosen += [
(c, RULE_USER_WINDOW)
for c in newest_first
if c.id not in taken
and c.user_sub == user_sub
and as_utc(c.started_at) >= since
][: limit - len(chosen)]
chosen.sort(key=lambda pair: as_utc(pair[0].started_at))
return [
CallLink(
call_id=c.id,
rule=rule,
tool=c.tool,
started_at=as_utc(c.started_at),
ok=c.ok,
position=i,
)
for i, (c, rule) in enumerate(chosen)
]
@@ -0,0 +1,117 @@
"""Background delivery of records to sinks, off the tool-call path."""
import asyncio
import contextlib
import logging
from collections.abc import Sequence
from .record import ToolCallRecord
from .sinks import Sink
logger = logging.getLogger(__name__)
class RecordDispatcher:
"""Queue records and hand them to sinks from a background task.
``submit`` never blocks and never raises: when the queue is full the record
is dropped and counted, so a slow or broken sink cannot slow down or fail
a tool call. Sink errors are logged and counted, never propagated.
"""
def __init__(
self,
sinks: Sequence[Sink],
*,
max_queue: int = 1000,
batch_size: int = 100,
sink_timeout: float = 10.0,
):
self.sinks = list(sinks)
self.max_queue = max_queue
self.batch_size = batch_size
self.sink_timeout = sink_timeout
self.dropped = 0
self.sink_errors = 0
self._queue: asyncio.Queue[ToolCallRecord] | None = None
self._task: asyncio.Task | None = None
self._loop: asyncio.AbstractEventLoop | None = None
def submit(self, record: ToolCallRecord) -> bool:
"""Queue a record for delivery. Returns False if it was dropped."""
try:
loop = asyncio.get_running_loop()
except RuntimeError:
self._drop("no running event loop")
return False
# A queue belongs to one loop; tests and some hosts run several.
if self._queue is None or self._loop is not loop:
self._loop = loop
self._queue = asyncio.Queue(maxsize=self.max_queue)
self._task = loop.create_task(self._run(self._queue))
try:
self._queue.put_nowait(record)
except asyncio.QueueFull:
self._drop("queue full")
return False
return True
def _drop(self, reason: str) -> None:
self.dropped += 1
if self.dropped == 1 or self.dropped % 1000 == 0:
logger.warning(
"Dropped tool-call record (%s); %d dropped so far", reason, self.dropped
)
async def _run(self, queue: asyncio.Queue) -> None:
while True:
batch = [await queue.get()]
while len(batch) < self.batch_size:
try:
batch.append(queue.get_nowait())
except asyncio.QueueEmpty:
break
try:
await self._deliver(batch)
finally:
for _ in batch:
queue.task_done()
async def _deliver(self, batch: list[ToolCallRecord]) -> None:
for sink in self.sinks:
try:
await asyncio.wait_for(sink.write(batch), self.sink_timeout)
except Exception:
self.sink_errors += 1
logger.warning(
"Sink %s failed to write %d records",
type(sink).__name__,
len(batch),
exc_info=True,
)
async def flush(self) -> None:
"""Wait until every queued record has been handed to the sinks."""
if self._queue is not None and self._loop is asyncio.get_running_loop():
await self._queue.join()
async def aclose(self) -> None:
"""Flush, stop the background task and close the sinks."""
await self.flush()
if self._task is not None:
self._task.cancel()
with contextlib.suppress(asyncio.CancelledError, RuntimeError):
await self._task
self._task = None
self._queue = None
for sink in self.sinks:
close = getattr(sink, "aclose", None)
if close is not None:
try:
await close()
except Exception:
logger.warning(
"Sink %s failed to close", type(sink).__name__, exc_info=True
)
@@ -0,0 +1,463 @@
"""FastMCP middleware that records every tool call."""
import asyncio
import inspect
import json
import logging
import os
import time
import uuid
from collections import deque
from collections.abc import Awaitable, Callable, Iterable, Sequence
from datetime import UTC, datetime, timedelta
from typing import Any
from fastmcp import FastMCP
from fastmcp.server.middleware import Middleware, MiddlewareContext
from .classify import DEFAULT_RESULT_CLASSIFIER, ResultClassifier
from .correlation import (
CallLink,
Candidate,
ResolverContext,
as_utc,
select_calls,
)
from .dispatch import RecordDispatcher
from .record import ToolCallRecord
from .redaction import Redactor
from .sinks import JsonLinesSink, Sink
from .storage import DatabaseSink
logger = logging.getLogger(__name__)
MODES = ("off", "meta", "full")
MODE_ENV_VAR = "FEEDBACK_INSTRUMENTATION_MODE"
IdentityResolver = Callable[
[MiddlewareContext], dict[str, Any] | Awaitable[dict[str, Any]] | None
]
Enricher = Callable[
[str, Any, Any, MiddlewareContext],
dict[str, Any] | Awaitable[dict[str, Any]] | None,
]
# Keys an identity resolver can return that get their own columns.
_IDENTITY_COLUMNS = ("user_sub", "caller_kind", "client_id")
def _attr(obj: Any, name: str) -> Any:
try:
value = getattr(obj, name, None)
except Exception:
return None
return str(value) if value is not None else None
# stdio serves exactly one client per process, so the process is the session.
_STDIO_SESSION = f"stdio:{os.getpid()}:{uuid.uuid4().hex[:8]}"
def session_key(ctx: Any) -> str | None:
"""A per-client key that stays the same across that client's calls.
Clients on MCP protocol 2026-07-28 (Claude Code, FastMCP 4's Client) do
not open sessions, and FastMCP then generates a new ``ctx.session_id`` for
every request. So stable signals come first: the HTTP ``mcp-session-id``
header (older protocol clients), then one key per process for stdio, then
``ctx.session_id``. With a stateless HTTP client there is no session, and
correlation relies on the same-user rule.
"""
if ctx is None:
return None
try:
from fastmcp.server.dependencies import get_http_headers
header = (get_http_headers(include_all=True) or {}).get("mcp-session-id")
if header:
return header
except Exception:
pass
if _attr(ctx, "transport") == "stdio":
return _STDIO_SESSION
return _attr(ctx, "session_id")
def _result_payload(result: Any) -> Any:
structured = getattr(result, "structured_content", None)
if structured is not None:
return structured
content = getattr(result, "content", None)
if content is not None:
return [
getattr(item, "text", None) or {"type": getattr(item, "type", None)}
for item in content
]
return result
def _json_size(value: Any) -> int | None:
try:
return len(json.dumps(value, default=str).encode())
except Exception:
return None
class InstrumentationMiddleware(Middleware):
"""Record every tool call without ever slowing or breaking it.
Modes:
``off``: pass calls straight through.
``meta`` (default): tool, timing, outcome, identity and payload sizes.
``full``: also the arguments and result, redacted.
The mode defaults to the ``FEEDBACK_INSTRUMENTATION_MODE`` environment
variable, then ``meta``. Records go to the sinks from a background task
through a bounded queue; when it fills, records are dropped, not waited on.
Args:
sinks: where records go. Defaults to JSON lines on stderr.
mode: see above.
redactor: secret redaction for args, results, error text and extras.
meta_only_tools: tools that are never recorded above ``meta``.
identity_resolver: ``(context) -> dict`` returning ``user_sub``,
``caller_kind`` and ``client_id`` (stored as columns) plus any
other keys (stored in ``identity``). May be async.
enricher: ``(tool, args, result, context) -> dict`` of extra fields,
stored in ``extra``. A ``server_version`` key fills that column.
May be async.
server_version: default for the ``server_version`` column.
hook_timeout: seconds an async hook may take before it is skipped.
recent_calls: how many recent records to keep in memory for
correlation, including ones still waiting to reach the sinks.
correlation_timeout: seconds a database lookup during correlation may
take before it is skipped.
result_classifier: ``(tool, result) -> str | None`` run on results
of calls that did not raise. A string marks the call as a soft
error (``outcome="soft_error"``, ``ok=False``) and becomes the
error message. Defaults to ``DEFAULT_RESULT_CLASSIFIER``; None
turns it off. If it raises, the call is recorded as ok.
"""
def __init__(
self,
sinks: Sequence[Sink] | None = None,
*,
mode: str | None = None,
redactor: Redactor | None = None,
meta_only_tools: Iterable[str] = (),
identity_resolver: IdentityResolver | None = None,
enricher: Enricher | None = None,
server_version: str | None = None,
hook_timeout: float = 0.25,
max_queue: int = 1000,
batch_size: int = 100,
sink_timeout: float = 10.0,
max_error_chars: int = 500,
recent_calls: int = 2000,
correlation_timeout: float = 1.0,
result_classifier: ResultClassifier | None = DEFAULT_RESULT_CLASSIFIER,
):
mode = (mode or os.getenv(MODE_ENV_VAR) or "meta").lower()
if mode not in MODES:
raise ValueError(f"mode must be one of {MODES}, got {mode!r}")
self.mode = mode
self.redactor = redactor or Redactor()
self.meta_only_tools = frozenset(meta_only_tools)
self.identity_resolver = identity_resolver
self.enricher = enricher
self.server_version = server_version
self.hook_timeout = hook_timeout
self.max_error_chars = max_error_chars
self.correlation_timeout = correlation_timeout
self.result_classifier = result_classifier
self.recent: deque[ToolCallRecord] = deque(maxlen=recent_calls)
self.dispatcher = RecordDispatcher(
sinks if sinks is not None else [JsonLinesSink()],
max_queue=max_queue,
batch_size=batch_size,
sink_timeout=sink_timeout,
)
async def on_call_tool(self, context: MiddlewareContext, call_next):
if self.mode == "off":
return await call_next(context)
started_at = datetime.now(UTC)
start = time.perf_counter()
result = None
error: BaseException | None = None
try:
result = await call_next(context)
return result
except BaseException as exc:
error = exc
raise
finally:
duration_ms = (time.perf_counter() - start) * 1000
try:
record = await self._build_record(
context, started_at, duration_ms, result, error
)
self.recent.append(record)
self.dispatcher.submit(record)
except Exception:
logger.warning("Failed to record tool call", exc_info=True)
async def _call_hook(self, hook: Callable | None, *args: Any) -> dict[str, Any]:
if hook is None:
return {}
try:
out = hook(*args)
if inspect.isawaitable(out):
out = await asyncio.wait_for(out, self.hook_timeout)
return dict(out) if out else {}
except Exception:
logger.warning("Instrumentation hook %r failed", hook, exc_info=True)
return {}
def _classify(self, tool: str, result: Any, payload: Any) -> str | None:
if getattr(result, "is_error", False) is True:
# FastMCP 3.4+ keeps isError on the ToolResult, not in the payload.
if isinstance(payload, dict):
payload = {**payload, "isError": True}
else:
payload = {"isError": True, "content": payload}
try:
message = self.result_classifier(tool, payload)
except Exception:
logger.warning(
"Result classifier %r failed", self.result_classifier, exc_info=True
)
return None
return None if message is None else str(message)
async def _build_record(
self,
context: MiddlewareContext,
started_at: datetime,
duration_ms: float,
result: Any,
error: BaseException | None,
) -> ToolCallRecord:
message = context.message
tool = getattr(message, "name", None) or "unknown"
args = getattr(message, "arguments", None) or {}
ctx = context.fastmcp_context
payload = _result_payload(result) if result is not None else None
identity = await self._call_hook(self.identity_resolver, context)
extra = await self._call_hook(self.enricher, tool, args, payload, context)
identity = self.redactor.redact(identity)
extra = self.redactor.redact(extra)
columns = {k: identity.pop(k) for k in _IDENTITY_COLUMNS if k in identity}
server_version = extra.pop("server_version", None) or self.server_version
mode = "meta" if tool in self.meta_only_tools else self.mode
full = mode == "full"
outcome = "ok"
error_type = error_message = None
if error is not None:
# FastMCP wraps tool exceptions in ToolError; report the original.
cause = error.__cause__ or error
outcome = "error"
error_type = type(cause).__name__
error_message = self.redactor.redact_text(str(cause))[
: self.max_error_chars
]
elif self.result_classifier is not None and payload is not None:
soft = self._classify(tool, result, payload)
if soft is not None:
outcome = "soft_error"
error_type = "SoftError"
error_message = (
self.redactor.redact_text(soft)[: self.max_error_chars] or None
)
return ToolCallRecord(
tool=tool,
started_at=started_at,
duration_ms=duration_ms,
ok=outcome == "ok",
outcome=outcome,
mode=mode,
error_type=error_type,
error_message=error_message,
session_id=session_key(ctx),
request_id=_attr(ctx, "request_id"),
client_id=columns.get("client_id") or _attr(ctx, "client_id"),
user_sub=columns.get("user_sub"),
caller_kind=columns.get("caller_kind"),
identity=identity,
server_version=server_version,
args_size=_json_size(args),
result_size=_json_size(payload) if payload is not None else None,
args=self.redactor.redact(args) if full else None,
result=self.redactor.redact(payload)
if full and payload is not None
else None,
extra=extra,
)
@property
def database_sink(self) -> DatabaseSink | None:
"""The first DatabaseSink among the sinks, if any."""
return next(
(s for s in self.dispatcher.sinks if isinstance(s, DatabaseSink)), None
)
async def capture_recent_calls(
self,
ctx: Any = None,
n: int = 20,
*,
user_sub: str | None = None,
window: timedelta = timedelta(minutes=15),
) -> list[CallLink]:
"""The calls that led up to now, for attaching to a feedback item.
Call from inside a tool. Returns up to ``n`` calls in chronological
order: first from the current MCP session, then, if that yields fewer
than ``n``, the same user's calls within ``window``. Each link says
which rule matched. Never raises; returns [] on any failure.
Args:
ctx: the FastMCP Context. Looked up automatically when omitted.
n: maximum number of calls.
user_sub: the caller's user id. When omitted, the identity
resolver is asked, given an object whose ``fastmcp_context``
is ``ctx``.
window: how far back the same-user fallback reaches.
"""
try:
if ctx is None:
ctx = _current_context()
session_id = session_key(ctx)
if user_sub is None and self.identity_resolver is not None:
identity = await self._call_hook(
self.identity_resolver, ResolverContext(ctx)
)
user_sub = identity.get("user_sub")
since = datetime.now(UTC) - window
candidates = [
Candidate(r.id, r.tool, r.started_at, r.ok, r.session_id, r.user_sub)
for r in self.recent
]
links = select_calls(
candidates,
session_id=session_id,
user_sub=user_sub,
since=since,
limit=n,
)
sink = self.database_sink
if sink is not None and len(links) < n:
# Other workers, or calls from before a restart.
rows = await asyncio.wait_for(
sink.recent_calls(
session_id=session_id, user_sub=user_sub, since=since, limit=n
),
self.correlation_timeout,
)
candidates += [
Candidate(
r["id"],
r["tool"],
as_utc(r["started_at"]),
r["ok"],
r["session_id"],
r["user_sub"],
)
for r in rows
]
links = select_calls(
candidates,
session_id=session_id,
user_sub=user_sub,
since=since,
limit=n,
)
return links
except Exception:
logger.warning("Failed to capture recent calls", exc_info=True)
return []
async def link_feedback(self, feedback_ref: str, links: Sequence[CallLink]) -> bool:
"""Store ``links`` against a feedback id of any format (``bug-XXXX``, 42).
Needs a DatabaseSink. Replaces earlier links for the same id. Returns
False, without raising, if they could not be stored.
"""
sink = self.database_sink
if sink is None:
logger.warning("link_feedback needs a DatabaseSink; links not stored")
return False
linked_at = datetime.now(UTC)
try:
await sink.replace_links(
str(feedback_ref),
[
{
"feedback_ref": str(feedback_ref),
"call_id": link.call_id,
"position": link.position,
"rule": link.rule,
"linked_at": linked_at,
}
for link in links
],
)
return True
except Exception:
logger.warning("Failed to link feedback %s", feedback_ref, exc_info=True)
return False
async def feedback_context(self, feedback_ref: str) -> list[dict[str, Any]]:
"""The calls linked to a feedback id, in order, with their details.
Calls not yet written to the database are filled in from memory.
"""
sink = self.database_sink
if sink is None:
return []
rows = await sink.linked_calls(str(feedback_ref))
in_memory = {r.id: r for r in self.recent}
for row in rows:
if row["tool"] is None and row["call_id"] in in_memory:
record = in_memory[row["call_id"]]
for key in row:
if key not in ("call_id", "position", "rule"):
row[key] = getattr(record, key, None)
if row["started_at"] is not None:
row["started_at"] = as_utc(row["started_at"])
return rows
async def flush(self) -> None:
"""Wait until queued records have reached the sinks."""
await self.dispatcher.flush()
async def aclose(self) -> None:
"""Flush and close the sinks. Call on server shutdown."""
await self.dispatcher.aclose()
def _current_context() -> Any:
try:
from fastmcp.server.dependencies import get_context
return get_context()
except Exception:
return None
def instrument(app: FastMCP, sinks: Sequence[Sink] | None = None, **kwargs: Any):
"""Add an InstrumentationMiddleware to ``app`` and return it.
>>> mw = instrument(app) # JSON lines to stderr
>>> mw = instrument(app, [DatabaseSink(engine)], mode="full")
"""
middleware = InstrumentationMiddleware(sinks, **kwargs)
app.add_middleware(middleware)
return middleware
@@ -0,0 +1,55 @@
"""The per-call record produced by InstrumentationMiddleware."""
import uuid
from dataclasses import asdict, dataclass, field
from datetime import datetime
from typing import Any
@dataclass
class ToolCallRecord:
"""One MCP tool call, as seen by the instrumentation middleware.
``args`` and ``result`` are only populated in ``full`` mode, and are
already redacted when they are. Sizes are measured before redaction.
``outcome`` is ``ok``, ``error`` (the tool raised) or ``soft_error`` (it
returned a result the classifier flagged). ``ok`` is ``outcome == "ok"``;
when ``outcome`` is not given it is derived from ``ok``.
"""
tool: str
started_at: datetime
duration_ms: float
ok: bool
mode: str
outcome: str | None = None
id: str = field(default_factory=lambda: str(uuid.uuid4()))
error_type: str | None = None
error_message: str | None = None
session_id: str | None = None
request_id: str | None = None
client_id: str | None = None
user_sub: str | None = None
caller_kind: str | None = None
identity: dict[str, Any] = field(default_factory=dict)
server_version: str | None = None
args_size: int | None = None
result_size: int | None = None
args: Any = None
result: Any = None
extra: dict[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
if self.outcome is None:
self.outcome = "ok" if self.ok else "error"
def to_row(self) -> dict[str, Any]:
"""Column values for the tool_calls table."""
return asdict(self)
def to_dict(self) -> dict[str, Any]:
"""JSON-friendly form, with the timestamp as ISO 8601."""
data = asdict(self)
data["started_at"] = self.started_at.isoformat()
return data
@@ -0,0 +1,125 @@
"""Secret redaction for recorded tool arguments, results and error text."""
import re
from collections.abc import Iterable
from typing import Any
REDACTED = "[REDACTED]"
TRUNCATED = "[TRUNCATED]"
# Matched after lowercasing and turning "-" into "_", so "Authorization",
# "X-Api-Key" style variants are covered too.
DEFAULT_SECRET_KEYS = frozenset(
{
"token",
"access_token",
"refresh_token",
"id_token",
"auth_token",
"session_token",
"csrf_token",
"bearer",
"authorization",
"proxy_authorization",
"cookie",
"set_cookie",
"password",
"passwd",
"passphrase",
"secret",
"client_secret",
"api_key",
"apikey",
"x_api_key",
"private_key",
}
)
# github_token, db_password, stripe_api_key, ...
DEFAULT_SECRET_SUFFIXES = (
"_token",
"_secret",
"_password",
"_api_key",
"_apikey",
"_private_key",
)
# Values that are secrets wherever they appear, including inside free text.
DEFAULT_SECRET_PATTERNS = (
r"\bbmcp_[A-Za-z0-9_-]{8,}", # blender-mcp personal access tokens
r"\beyJ[A-Za-z0-9_-]{5,}\.[A-Za-z0-9_-]{5,}\.[A-Za-z0-9_-]{5,}", # JWT
r"(?i)\bbearer\s+[A-Za-z0-9._~+/=-]{8,}",
r"\bsk-[A-Za-z0-9_-]{16,}", # OpenAI/Anthropic-style API keys
r"\bgh[pousr]_[A-Za-z0-9]{20,}", # GitHub tokens
r"\bgithub_pat_[A-Za-z0-9_]{20,}",
r"\bxox[abposr]-[A-Za-z0-9-]{10,}", # Slack
r"\bAKIA[0-9A-Z]{16}\b", # AWS access key id
r"\bpypi-[A-Za-z0-9_-]{16,}",
)
# "password=hunter2", "api_key: abc" in error messages and logs. The key is
# kept so the text still says what was removed.
_KEY_VALUE_TEXT = re.compile(
r"(?i)\b(password|passwd|secret|token|api[_-]?key|authorization)(\s*[=:]\s*)[^\s,;&'\"]+"
)
def _normalize_key(key: Any) -> str:
return str(key).lower().replace("-", "_")
class Redactor:
"""Replace secrets in nested JSON-like data with a placeholder.
Dict values are redacted when their key looks like a secret, at any depth.
String values anywhere are scanned for secret-shaped substrings. The input
is never modified; a redacted copy is returned.
"""
def __init__(
self,
keys: Iterable[str] = DEFAULT_SECRET_KEYS,
patterns: Iterable[str] = DEFAULT_SECRET_PATTERNS,
*,
extra_keys: Iterable[str] = (),
extra_patterns: Iterable[str] = (),
suffixes: Iterable[str] = DEFAULT_SECRET_SUFFIXES,
replacement: str = REDACTED,
max_depth: int = 32,
):
self.keys = frozenset(_normalize_key(k) for k in (*keys, *extra_keys))
self.suffixes = tuple(_normalize_key(s) for s in suffixes)
self.patterns = tuple(re.compile(p) for p in (*patterns, *extra_patterns))
self.replacement = replacement
self.max_depth = max_depth
def is_secret_key(self, key: Any) -> bool:
normalized = _normalize_key(key)
return normalized in self.keys or normalized.endswith(self.suffixes)
def redact_text(self, text: str) -> str:
for pattern in self.patterns:
text = pattern.sub(self.replacement, text)
return _KEY_VALUE_TEXT.sub(
lambda m: f"{m.group(1)}{m.group(2)}{self.replacement}", text
)
def redact(self, value: Any) -> Any:
return self._redact(value, 0)
def _redact(self, value: Any, depth: int) -> Any:
if depth > self.max_depth:
return TRUNCATED
if isinstance(value, str):
return self.redact_text(value)
if isinstance(value, dict):
return {
k: self.replacement
if self.is_secret_key(k) and v is not None
else self._redact(v, depth + 1)
for k, v in value.items()
}
if isinstance(value, list | tuple | set | frozenset):
return [self._redact(v, depth + 1) for v in value]
return value
@@ -0,0 +1,55 @@
"""Destinations for tool-call records."""
import inspect
import json
import sys
from collections.abc import Callable, Sequence
from typing import IO, Any, Protocol, runtime_checkable
from .record import ToolCallRecord
@runtime_checkable
class Sink(Protocol):
"""Anything with ``async write(records)``. ``async aclose()`` is optional."""
async def write(self, records: Sequence[ToolCallRecord]) -> None: ...
class JsonLinesSink:
"""One JSON object per record, to stderr by default."""
def __init__(self, stream: IO[str] | None = None):
self._stream = stream
async def write(self, records: Sequence[ToolCallRecord]) -> None:
# Resolved per write so a stderr swapped by the host or a test is honored.
stream = self._stream or sys.stderr
for record in records:
stream.write(json.dumps(record.to_dict(), default=str) + "\n")
stream.flush()
class CallbackSink:
"""Pass each batch to a function, sync or async.
A sync callback runs on the event loop, so keep it quick.
"""
def __init__(self, callback: Callable[[Sequence[ToolCallRecord]], Any]):
self.callback = callback
async def write(self, records: Sequence[ToolCallRecord]) -> None:
result = self.callback(records)
if inspect.isawaitable(result):
await result
class MemorySink:
"""Keep records in a list. Meant for tests."""
def __init__(self):
self.records: list[ToolCallRecord] = []
async def write(self, records: Sequence[ToolCallRecord]) -> None:
self.records.extend(records)
@@ -0,0 +1,217 @@
"""Async SQLAlchemy storage for tool-call records and feedback links.
Hosts that manage their own schema include ``build_metadata(prefix)`` in their
Alembic ``env.py`` (``target_metadata``) and pass their existing AsyncEngine to
``DatabaseSink``. Nothing is created unless ``create_tables=True``.
"""
import asyncio
from collections.abc import Sequence
from datetime import datetime
from typing import Any
from sqlalchemy import (
JSON,
Boolean,
Column,
DateTime,
Float,
Integer,
MetaData,
String,
Table,
Text,
and_,
delete,
or_,
select,
)
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
from .record import ToolCallRecord
DEFAULT_TABLE_PREFIX = "ffb_"
# none_as_null: a missing payload is SQL NULL, so "WHERE args IS NULL" works.
_JSON = JSON(none_as_null=True).with_variant(JSONB(none_as_null=True), "postgresql")
def build_metadata(
prefix: str = DEFAULT_TABLE_PREFIX, metadata: MetaData | None = None
) -> MetaData:
"""Return MetaData holding ``{prefix}tool_calls`` and ``{prefix}feedback_call_links``.
Pass an existing MetaData to add the tables to it instead.
"""
metadata = metadata if metadata is not None else MetaData()
calls = f"{prefix}tool_calls"
links = f"{prefix}feedback_call_links"
if calls not in metadata.tables:
Table(
calls,
metadata,
Column("id", String(36), primary_key=True),
Column("started_at", DateTime(timezone=True), nullable=False, index=True),
Column("duration_ms", Float, nullable=False),
Column("tool", String(255), nullable=False, index=True),
Column("mode", String(8), nullable=False),
Column("ok", Boolean, nullable=False, index=True),
# ok, soft_error or error. NULL on rows from before 2026.09.27.4.
Column("outcome", String(16), index=True),
Column("error_type", String(255)),
Column("error_message", Text),
Column("session_id", String(255), index=True),
Column("request_id", String(255)),
Column("client_id", String(255)),
Column("user_sub", String(255), index=True),
Column("caller_kind", String(64)),
Column("identity", _JSON),
Column("server_version", String(64), index=True),
Column("args_size", Integer),
Column("result_size", Integer),
Column("args", _JSON),
Column("result", _JSON),
Column("extra", _JSON),
)
if links not in metadata.tables:
# No foreign key to tool_calls: a link can be written before its call
# record has left the queue, and hosts may keep calls elsewhere.
Table(
links,
metadata,
Column("feedback_ref", String(255), primary_key=True),
Column("call_id", String(36), primary_key=True, index=True),
Column("position", Integer, nullable=False),
Column("rule", String(32), nullable=False),
Column("linked_at", DateTime(timezone=True), nullable=False),
)
return metadata
class DatabaseSink:
"""Write records to ``{prefix}tool_calls`` through an async engine.
Args:
engine: an AsyncEngine the host already owns, or a URL such as
``sqlite+aiosqlite:///calls.db`` or ``postgresql+asyncpg://...``.
An engine built from a URL is disposed on ``aclose()``; a passed-in
engine is left alone.
prefix: table name prefix, so the tables cannot collide with the host's.
create_tables: create the tables on first use. Off by default; hosts
with Alembic should generate a migration from ``build_metadata``.
"""
def __init__(
self,
engine: AsyncEngine | str,
*,
prefix: str = DEFAULT_TABLE_PREFIX,
create_tables: bool = False,
):
self._owns_engine = isinstance(engine, str)
self.engine = create_async_engine(engine) if isinstance(engine, str) else engine
self.metadata = build_metadata(prefix)
self.table = self.metadata.tables[f"{prefix}tool_calls"]
self.links = self.metadata.tables[f"{prefix}feedback_call_links"]
self.create_tables = create_tables
self._tables_ready = False
# The background writer and link_feedback can both reach _ready() at
# once; without the lock both run create_all and one fails.
self._tables_lock = asyncio.Lock()
async def ensure_tables(self) -> None:
"""Create the tables if missing. Only called when create_tables=True."""
async with self.engine.begin() as conn:
await conn.run_sync(self.metadata.create_all)
self._tables_ready = True
async def _ready(self) -> None:
if self.create_tables and not self._tables_ready:
async with self._tables_lock:
if not self._tables_ready:
await self.ensure_tables()
async def write(self, records: Sequence[ToolCallRecord]) -> None:
if not records:
return
await self._ready()
async with self.engine.begin() as conn:
await conn.execute(self.table.insert(), [r.to_row() for r in records])
async def recent_calls(
self,
*,
session_id: str | None,
user_sub: str | None,
since: datetime,
limit: int,
) -> list[dict[str, Any]]:
"""Calls from this session, or from this user since ``since``, newest first."""
t = self.table
conditions = []
if session_id:
conditions.append(t.c.session_id == session_id)
if user_sub:
conditions.append(and_(t.c.user_sub == user_sub, t.c.started_at >= since))
if not conditions:
return []
await self._ready()
query = (
select(
t.c.id, t.c.tool, t.c.started_at, t.c.ok, t.c.session_id, t.c.user_sub
)
.where(or_(*conditions))
.order_by(t.c.started_at.desc())
.limit(limit)
)
async with self.engine.connect() as conn:
return [dict(row) for row in (await conn.execute(query)).mappings()]
async def replace_links(
self, feedback_ref: str, links: Sequence[dict[str, Any]]
) -> None:
"""Set the calls linked to ``feedback_ref``, replacing any earlier set."""
await self._ready()
async with self.engine.begin() as conn:
await conn.execute(
delete(self.links).where(self.links.c.feedback_ref == feedback_ref)
)
if links:
await conn.execute(self.links.insert(), list(links))
async def linked_calls(self, feedback_ref: str) -> list[dict[str, Any]]:
"""Links for ``feedback_ref`` in order, joined to their call records.
Call columns are None for calls not (yet) in this database.
"""
await self._ready()
link, call = self.links, self.table
query = (
select(
link.c.call_id,
link.c.position,
link.c.rule,
call.c.tool,
call.c.started_at,
call.c.duration_ms,
call.c.ok,
call.c.outcome,
call.c.error_type,
call.c.error_message,
call.c.session_id,
call.c.user_sub,
call.c.caller_kind,
call.c.server_version,
call.c.extra,
)
.select_from(link.outerjoin(call, link.c.call_id == call.c.id))
.where(link.c.feedback_ref == feedback_ref)
.order_by(link.c.position)
)
async with self.engine.connect() as conn:
return [dict(row) for row in (await conn.execute(query)).mappings()]
async def aclose(self) -> None:
if self._owns_engine:
await self.engine.dispose()
+35 -4
View File
@@ -1,15 +1,37 @@
"""Submission mixin for feedback collection."""
from datetime import UTC, datetime
from typing import Any
from typing import TYPE_CHECKING, Any
from ..insights import record_feedback_submission
from ..models import SubmitFeedbackRequest, create_feedback_from_request
from .base import BaseFeedbackMixin, logger, mcp_tool
from .base import (
BaseFeedbackMixin,
FeedbackDatabase,
FeedbackInsights,
logger,
mcp_tool,
)
if TYPE_CHECKING:
from ..instrumentation import InstrumentationMiddleware
class SubmissionMixin(BaseFeedbackMixin):
"""Mixin for feedback submission functionality."""
"""Mixin for feedback submission functionality.
Pass an InstrumentationMiddleware to link each submission to the tool
calls that led up to it.
"""
def __init__(
self,
database: FeedbackDatabase,
insights: FeedbackInsights | None = None,
instrumentation: "InstrumentationMiddleware | None" = None,
):
super().__init__(database, insights)
self.instrumentation = instrumentation
@mcp_tool(
description="Submit feedback about your MCP experience - Use this tool when you encounter issues, have suggestions for improvement, or want to report positive experiences. AI models should proactively use this to help improve future MCP sessions. PRIVACY: Only include information you've already shared via previous MCP calls. Ask human permission before including any new sensitive data."
@@ -62,14 +84,23 @@ class SubmissionMixin(BaseFeedbackMixin):
source="mcp_tool",
)
linked_calls = 0
if self.instrumentation is not None:
links = await self.instrumentation.capture_recent_calls()
if await self.instrumentation.link_feedback(feedback_id, links):
linked_calls = len(links)
self._record_tool_usage("submit_feedback", start_time, True)
logger.info(f"Feedback submitted successfully: {feedback_id}")
return {
response = {
"success": True,
"feedback_id": feedback_id,
"message": "Feedback submitted successfully",
}
if self.instrumentation is not None:
response["linked_calls"] = linked_calls
return response
except Exception as e:
logger.error(f"Failed to submit feedback: {e}")
+5 -1
View File
@@ -17,6 +17,7 @@ def add_feedback_tools(
insights: FeedbackInsights | None = None,
prefix: str = "",
separator: str = "_",
instrumentation=None,
) -> None:
"""Add all feedback tools to a FastMCP server.
@@ -29,6 +30,9 @@ def add_feedback_tools(
insights: Optional insights instance. Creates default if not provided.
prefix: Optional prefix for tool names.
separator: Separator between prefix and tool name.
instrumentation: an InstrumentationMiddleware (see ``instrument``).
When given, each submission is linked to the tool calls that led
up to it, and the response reports ``linked_calls``.
"""
try:
# Initialize database
@@ -39,7 +43,7 @@ def add_feedback_tools(
insights = FeedbackInsights()
# Create mixin instances
submission_mixin = SubmissionMixin(database, insights)
submission_mixin = SubmissionMixin(database, insights, instrumentation)
retrieval_mixin = RetrievalMixin(database, insights)
management_mixin = ManagementMixin(database, insights)
+10
View File
@@ -0,0 +1,10 @@
"""Helpers that paper over FastMCP API differences between 2.x and 4.x."""
from fastmcp import FastMCP
async def tool_map(server: FastMCP) -> dict:
"""Return {name: Tool}. 2.x has get_tools() -> dict, 4.x has list_tools() -> list."""
if hasattr(server, "get_tools"):
return await server.get_tools()
return {tool.name: tool for tool in await server.list_tools()}
+22
View File
@@ -0,0 +1,22 @@
"""A stdio MCP server used by test_correlation to check session linking end to end."""
from fastmcp import Context, FastMCP
from fastmcp_feedback.instrumentation import MemorySink, instrument
app = FastMCP("stdio-correlation")
mw = instrument(app, [MemorySink()])
@app.tool
def echo(text: str) -> str:
return text
@app.tool
async def report(ctx: Context) -> list:
return [[link.tool, link.rule] for link in await mw.capture_recent_calls(ctx)]
if __name__ == "__main__":
app.run(show_banner=False)
+19 -18
View File
@@ -11,11 +11,14 @@ import sys
from pathlib import Path
from unittest.mock import patch
from sqlalchemy import text
# Add the examples directory to the path for imports
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "examples"))
from fastmcp import FastMCP
from fastmcp_feedback import FeedbackDatabase, FeedbackInsights
from tests._compat import tool_map
@pytest.mark.asyncio
@@ -35,7 +38,7 @@ class TestDemoServerIntegration:
assert "demonstration" in server.description.lower()
# Get all tools
tools = await server.get_tools()
tools = await tool_map(server)
# Verify all expected tools are present
expected_tools = {
@@ -72,12 +75,12 @@ class TestDemoServerIntegration:
from fastmcp_feedback.models import Feedback
async with database.session() as session:
# Count feedback items
result = session.execute("SELECT COUNT(*) FROM feedback")
result = session.execute(text("SELECT COUNT(*) FROM feedback"))
count = result.scalar()
assert count >= 5, f"Expected at least 5 sample items, got {count}"
# Verify types are present
result = session.execute("SELECT DISTINCT type FROM feedback")
result = session.execute(text("SELECT DISTINCT type FROM feedback"))
types = [row[0] for row in result.fetchall()]
expected_types = {"BUG", "FEATURE", "IMPROVEMENT", "QUESTION"}
assert expected_types.issubset(set(types)), f"Missing feedback types: {expected_types - set(types)}"
@@ -89,21 +92,19 @@ class TestDemoServerIntegration:
"""Test actual execution of demo server tools."""
from demo_server import create_demo_server, create_temp_database, setup_demo_data
# Create server with temporary database
server = create_demo_server()
# Override database URL for testing
# Seed a temporary database, then point the server at it
db_url = create_temp_database()
database = FeedbackDatabase(db_url)
await setup_demo_data(database)
server = create_demo_server(database_url=db_url)
tools = await server.get_tools()
tools = await tool_map(server)
# Test system info tool
system_info_tool = tools["get_system_info"]
info_result = await system_info_tool.run({})
assert info_result.structured_content["success"] is True
assert "version" in info_result.structured_content
assert info_result.structured_content["status"] == "running"
assert "versions" in info_result.structured_content
assert "dependencies" in info_result.structured_content
# Test statistics tool (should work with sample data)
@@ -126,7 +127,7 @@ class TestDemoServerIntegration:
add_feedback_tools(app, database_url="sqlite:///:memory:")
# Verify tools were added
tools = await app.get_tools()
tools = await tool_map(app)
expected_tools = {
"submit_feedback",
"list_feedback",
@@ -176,7 +177,7 @@ class TestDemoServerIntegration:
from demo_server import create_demo_server
server = create_demo_server()
tools = await server.get_tools()
tools = await tool_map(server)
# Test submit_feedback with invalid data
submit_tool = tools["submit_feedback"]
@@ -199,7 +200,7 @@ class TestDemoServerIntegration:
from demo_server import create_demo_server, create_temp_database
server = create_demo_server()
tools = await server.get_tools()
tools = await tool_map(server)
# Submit feedback
submit_tool = tools["submit_feedback"]
@@ -225,7 +226,7 @@ class TestDemoServerIntegration:
assert our_feedback is not None, "Submitted feedback not found in list"
assert our_feedback["title"] == test_feedback["title"]
assert our_feedback["type"] == test_feedback["type"]
assert our_feedback["status"] == "OPEN"
assert our_feedback["status"] == "open"
# Update feedback status
update_tool = tools["update_feedback_status"]
@@ -239,7 +240,7 @@ class TestDemoServerIntegration:
list_result = await list_tool.run({})
feedback_list = list_result.structured_content["feedback"]
updated_feedback = next((f for f in feedback_list if f["id"] == feedback_id), None)
assert updated_feedback["status"] == "IN_PROGRESS"
assert updated_feedback["status"] == "in_progress"
# Get statistics (should include our feedback)
stats_tool = tools["get_feedback_statistics"]
@@ -272,7 +273,7 @@ class TestDemoServerDocumentation:
from demo_server import create_demo_server
server = create_demo_server()
tools = await server.get_tools()
tools = await tool_map(server)
# According to demo_server.md, should have exactly 7 tools
assert len(tools) == 7, f"Expected 7 tools as documented, got {len(tools)}"
@@ -296,7 +297,7 @@ class TestDemoServerDocumentation:
app = FastMCP("Test App")
add_feedback_tools(app, database_url="sqlite:///:memory:")
tools = await app.get_tools()
tools = await tool_map(app)
assert len(tools) == 5, f"Expected 5 tools as documented, got {len(tools)}"
# Verify documented tool names
@@ -318,7 +319,7 @@ class TestDemoServerPerformance:
from demo_server import create_demo_server
server = create_demo_server()
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
# Submit multiple feedback items concurrently
+14 -13
View File
@@ -10,6 +10,7 @@ from fastmcp_feedback.feedback import create_feedback_server
from fastmcp_feedback.models import SubmitFeedbackRequest, FeedbackStatus
from fastmcp_feedback.database import FeedbackDatabase
from fastmcp_feedback.insights import FeedbackInsights
from tests._compat import tool_map
@pytest.mark.integration
@@ -25,7 +26,7 @@ class TestCompleteWorkflow:
add_feedback_tools(server, database_url=memory_db_url, insights=insights)
# Get tools
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
list_tool = tools["list_feedback"]
update_tool = tools["update_feedback_status"]
@@ -106,7 +107,7 @@ class TestCompleteWorkflow:
server = FastMCP("Multi-Type Test Server")
add_feedback_tools(server, database_url=memory_db_url)
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
list_tool = tools["list_feedback"]
@@ -179,7 +180,7 @@ class TestCompleteWorkflow:
server = FastMCP("Error Test Server")
add_feedback_tools(server, database_url=memory_db_url)
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
update_tool = tools["update_feedback_status"]
delete_tool = tools["delete_feedback"]
@@ -233,7 +234,7 @@ class TestCompleteWorkflow:
insights = FeedbackInsights(enabled=True)
add_feedback_tools(server, database_url=memory_db_url, insights=insights)
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
list_tool = tools["list_feedback"]
@@ -285,7 +286,7 @@ class TestCompleteWorkflow:
server = FastMCP("Bulk Test Server")
add_feedback_tools(server, database_url=memory_db_url)
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
list_tool = tools["list_feedback"]
update_tool = tools["update_feedback_status"]
@@ -355,7 +356,7 @@ class TestServerCompositionWorkflow:
)
# Get tools from dedicated server
tools = await feedback_server.get_tools()
tools = await tool_map(feedback_server)
submit_tool = tools["submit_feedback"]
list_tool = tools["list_feedback"]
@@ -384,11 +385,11 @@ class TestServerCompositionWorkflow:
admin_feedback = create_feedback_server("Admin Feedback", database_url="sqlite:///:memory:")
# Import servers with different prefixes
await main_server.import_server(user_feedback, prefix="user")
await main_server.import_server(admin_feedback, prefix="admin")
main_server.mount(user_feedback, "user")
main_server.mount(admin_feedback, "admin")
# Get tools
tools = await main_server.get_tools()
tools = await tool_map(main_server)
user_submit = tools["user_submit_feedback"]
admin_submit = tools["admin_submit_feedback"]
@@ -439,7 +440,7 @@ class TestServerCompositionWorkflow:
add_feedback_tools(server, database_url=memory_db_url, prefix="public")
add_feedback_tools(server, database_url=memory_db_url, prefix="internal")
tools = await server.get_tools()
tools = await tool_map(server)
public_submit = tools["public_submit_feedback"]
internal_submit = tools["internal_submit_feedback"]
@@ -490,7 +491,7 @@ class TestPerformanceWorkflow:
insights = FeedbackInsights(enabled=True)
add_feedback_tools(server, database_url=memory_db_url, insights=insights)
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
list_tool = tools["list_feedback"]
stats_tool = tools["get_feedback_statistics"]
@@ -537,7 +538,7 @@ class TestPerformanceWorkflow:
server = FastMCP("Filter Performance Server")
add_feedback_tools(server, database_url=memory_db_url)
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
list_tool = tools["list_feedback"]
update_tool = tools["update_feedback_status"]
@@ -611,7 +612,7 @@ class TestPerformanceWorkflow:
server = FastMCP("Concurrent Mixed Server")
add_feedback_tools(server, database_url=memory_db_url)
tools = await server.get_tools()
tools = await tool_map(server)
submit_tool = tools["submit_feedback"]
list_tool = tools["list_feedback"]
update_tool = tools["update_feedback_status"]
+416
View File
@@ -0,0 +1,416 @@
"""Tests for linking feedback to the tool calls that preceded it."""
import asyncio
import contextlib
import json
import sys
from datetime import UTC, datetime, timedelta
from pathlib import Path
import fastmcp
import pytest
from fastmcp import Client, Context, FastMCP
from fastmcp.client.transports import PythonStdioTransport
from fastmcp.exceptions import ToolError
from fastmcp_feedback import add_feedback_tools
from fastmcp_feedback.instrumentation import (
DatabaseSink,
MemorySink,
instrument,
)
from fastmcp_feedback.instrumentation import middleware as mw_module
from fastmcp_feedback.instrumentation.correlation import (
RULE_SESSION,
RULE_USER_WINDOW,
Candidate,
select_calls,
)
NOW = datetime(2026, 9, 27, 12, 0, tzinfo=UTC)
# FastMCP 4's Client speaks MCP 2026-07-28, which has no sessions, so each
# in-memory call gets a new ctx.session_id and there is no header or stdio
# signal to fall back on. Calls from one test client link by user instead.
FASTMCP_4 = int(fastmcp.__version__.split(".")[0]) >= 4
SAME_CLIENT_RULE = RULE_USER_WINDOW if FASTMCP_4 else RULE_SESSION
@pytest.fixture
async def db(tmp_path):
"""Make DatabaseSinks whose engines are always disposed, even on failure.
An undisposed aiosqlite engine keeps a worker thread alive and the test
process never exits.
"""
made = []
def factory(name="calls", cls=DatabaseSink):
sink = cls(f"sqlite+aiosqlite:///{tmp_path}/{name}.db", create_tables=True)
made.append(sink)
return sink
yield factory
for sink in made:
await sink.engine.dispose()
def cand(id, minutes_ago, session="s1", user="u1", tool="t", ok=True):
return Candidate(id, tool, NOW - timedelta(minutes=minutes_ago), ok, session, user)
class TestSelectCalls:
def test_session_first_then_user_window_chronological(self):
links = select_calls(
[cand("a", 1), cand("b", 3, session="s2"), cand("c", 2)],
session_id="s1",
user_sub="u1",
since=NOW - timedelta(minutes=15),
limit=10,
)
assert [(link.call_id, link.rule, link.position) for link in links] == [
("b", RULE_USER_WINDOW, 0),
("c", RULE_SESSION, 1),
("a", RULE_SESSION, 2),
]
def test_session_matches_take_priority_over_newer_user_calls(self):
links = select_calls(
[cand("old", 10), cand("new", 1, session="s2")],
session_id="s1",
user_sub="u1",
since=NOW - timedelta(minutes=15),
limit=1,
)
assert [(link.call_id, link.rule) for link in links] == [("old", RULE_SESSION)]
def test_user_window_excludes_old_calls_and_other_users(self):
links = select_calls(
[
cand("recent", 5, session="s2"),
cand("stale", 30, session="s2"),
cand("other", 1, session="s3", user="u2"),
],
session_id="s9",
user_sub="u1",
since=NOW - timedelta(minutes=15),
limit=10,
)
assert [link.call_id for link in links] == ["recent"]
def test_limit_keeps_the_most_recent(self):
links = select_calls(
[cand(str(i), i) for i in range(10)],
session_id="s1",
user_sub=None,
since=NOW,
limit=3,
)
assert [link.call_id for link in links] == ["2", "1", "0"]
def test_duplicates_counted_once_and_naive_times_treated_as_utc(self):
naive = Candidate(
"a",
"t",
(NOW - timedelta(minutes=1)).replace(tzinfo=None),
True,
"s1",
"u1",
)
links = select_calls(
[naive, cand("a", 1)], session_id="s1", user_sub=None, since=NOW, limit=5
)
assert len(links) == 1 and links[0].started_at.tzinfo is not None
def test_no_session_or_user_means_no_links(self):
assert (
select_calls(
[cand("a", 1)], session_id=None, user_sub=None, since=NOW, limit=5
)
== []
)
def build(sinks=None, user="u1", **kwargs):
app = FastMCP("correlated")
@app.tool
def echo(text: str) -> str:
return text
@app.tool
def render(scene: str) -> str:
raise RuntimeError(f"render of {scene} hung")
captured = {}
@app.tool
async def report(ctx: Context) -> list:
links = await mw.capture_recent_calls(ctx)
captured["links"] = links
return [link.call_id for link in links]
identity = {"user": user}
mw = instrument(
app,
sinks if sinks is not None else [MemorySink()],
identity_resolver=lambda c: {"user_sub": identity["user"]},
**kwargs,
)
return app, mw, captured, identity
class TestCaptureRecentCalls:
async def test_same_session_calls_including_failures(self):
app, mw, captured, _ = build()
async with Client(app) as client:
await client.call_tool("echo", {"text": "a"})
with contextlib.suppress(ToolError):
await client.call_tool("render", {"scene": "castle"})
await client.call_tool("report", {})
links = captured["links"]
assert [(link.tool, link.ok, link.rule) for link in links] == [
("echo", True, SAME_CLIENT_RULE),
("render", False, SAME_CLIENT_RULE),
]
async def test_new_session_per_call_falls_back_to_user_window(self):
"""PAT/CI clients open a new MCP session for every call."""
app, mw, captured, _ = build()
for text in ("one", "two"):
async with Client(app) as client:
await client.call_tool("echo", {"text": text})
async with Client(app) as client:
await client.call_tool("report", {})
assert [link.rule for link in captured["links"]] == [RULE_USER_WINDOW] * 2
async def test_other_users_calls_are_not_linked(self):
app, mw, captured, identity = build()
identity["user"] = "someone-else"
async with Client(app) as client:
await client.call_tool("echo", {"text": "not mine"})
identity["user"] = "u1"
async with Client(app) as client:
await client.call_tool("report", {})
assert captured["links"] == []
async def test_calls_still_queued_are_found(self):
release = asyncio.Event()
class Stalled:
async def write(self, records):
await release.wait()
app, mw, captured, _ = build(sinks=[Stalled()])
async with Client(app) as client:
await client.call_tool("echo", {"text": "queued"})
await client.call_tool("report", {})
release.set()
assert [link.tool for link in captured["links"]] == ["echo"]
async def test_database_fallback_after_restart(self, db):
app1, mw1, _, _ = build(sinks=[db("restart")])
async with Client(app1) as client:
await client.call_tool("echo", {"text": "before restart"})
await mw1.aclose()
# A new process: empty in-memory buffer, same database.
app2, mw2, captured, _ = build(sinks=[db("restart")])
async with Client(app2) as client:
await client.call_tool("report", {})
await mw2.aclose()
[link] = captured["links"]
assert link.tool == "echo" and link.rule == RULE_USER_WINDOW
@pytest.mark.skipif(
FASTMCP_4, reason="FastMCP 4 Client is sessionless (MCP 2026-07-28)"
)
async def test_failing_resolver_still_links_by_session(self):
app, mw, captured, _ = build()
mw.identity_resolver = lambda c: 1 / 0
async with Client(app) as client:
await client.call_tool("echo", {"text": "a"})
await client.call_tool("report", {})
assert [link.rule for link in captured["links"]] == [RULE_SESSION]
async def test_slow_database_is_skipped(self, db):
class SlowDb(DatabaseSink):
async def recent_calls(self, **kwargs):
await asyncio.sleep(5)
return []
sink = db("slow", SlowDb)
app, mw, captured, _ = build(sinks=[sink], correlation_timeout=0.05)
async with Client(app) as client:
await client.call_tool("echo", {"text": "a"})
await client.call_tool("report", {})
await mw.aclose()
# The lookup timed out, so capture returned [] rather than hanging.
assert captured["links"] == []
async def test_outside_a_tool_call_returns_empty(self):
_, mw, _, _ = build()
mw.identity_resolver = None
assert await mw.capture_recent_calls() == []
class TestLinksAndContext:
async def test_link_and_read_back_with_external_ids(self, db):
sink = db("links")
app, mw, captured, _ = build(sinks=[sink])
async with Client(app) as client:
await client.call_tool("echo", {"text": "a"})
with contextlib.suppress(ToolError):
await client.call_tool("render", {"scene": "castle"})
await client.call_tool("report", {})
assert await mw.link_feedback("bug-7Q2X", captured["links"])
await mw.flush()
context = await mw.feedback_context("bug-7Q2X")
await mw.aclose()
assert [(c["position"], c["tool"], c["ok"], c["rule"]) for c in context] == [
(0, "echo", True, SAME_CLIENT_RULE),
(1, "render", False, SAME_CLIENT_RULE),
]
assert context[1]["error_type"] == "RuntimeError"
async def test_relinking_replaces_links(self, db):
sink = db("relink")
app, mw, captured, _ = build(sinks=[sink])
async with Client(app) as client:
await client.call_tool("echo", {"text": "a"})
await client.call_tool("report", {})
links = captured["links"]
await mw.link_feedback("fb-1", links)
await mw.link_feedback("fb-1", links)
assert len(await mw.feedback_context("fb-1")) == 1
await mw.aclose()
async def test_context_fills_unflushed_calls_from_memory(self, db):
release = asyncio.Event()
class StalledDb(DatabaseSink):
async def write(self, records):
await release.wait()
sink = db("unflushed", StalledDb)
app, mw, captured, _ = build(sinks=[sink])
async with Client(app) as client:
await client.call_tool("echo", {"text": "a"})
await client.call_tool("report", {})
await mw.link_feedback("fb-2", captured["links"])
[row] = await mw.feedback_context("fb-2")
release.set()
await mw.aclose()
assert row["tool"] == "echo" and row["ok"] is True
async def test_link_without_database_sink_is_false(self):
_, mw, _, _ = build()
assert await mw.link_feedback("x", []) is False
assert await mw.feedback_context("x") == []
class TestSubmitFeedbackAutoLinks:
async def test_submission_links_preceding_calls(self, db):
app = FastMCP("app")
@app.tool
def render(scene: str) -> str:
raise RuntimeError("hung")
sink = db("submit")
mw = instrument(app, [sink], identity_resolver=lambda c: {"user_sub": "u1"})
add_feedback_tools(app, database_url="sqlite:///:memory:", instrumentation=mw)
async with Client(app) as client:
with contextlib.suppress(ToolError):
await client.call_tool("render", {"scene": "castle"})
result = await client.call_tool(
"submit_feedback",
{
"request": {
"type": "bug",
"title": "render hangs",
"description": "the render tool hung",
"submitter": "model",
}
},
)
assert result.data["linked_calls"] == 1
await mw.flush()
[row] = await mw.feedback_context(result.data["feedback_id"])
await mw.aclose()
assert (row["tool"], row["ok"], row["error_type"]) == (
"render",
False,
"RuntimeError",
)
async def test_without_instrumentation_response_unchanged(self):
app = FastMCP("plain")
add_feedback_tools(app, database_url="sqlite:///:memory:")
async with Client(app) as client:
result = await client.call_tool(
"submit_feedback",
{
"request": {
"type": "bug",
"title": "t",
"description": "no instrumentation",
"submitter": "m",
}
},
)
assert "linked_calls" not in result.data
class FakeContext:
def __init__(self, transport=None, session_id="ctx-session"):
self.transport = transport
self.session_id = session_id
class TestSessionKey:
def test_http_session_header_wins(self, monkeypatch):
import fastmcp.server.dependencies as deps
monkeypatch.setattr(
deps,
"get_http_headers",
lambda include_all=False: {"mcp-session-id": "hdr-1"},
)
assert (
mw_module.session_key(FakeContext(transport="streamable-http")) == "hdr-1"
)
def test_stdio_uses_one_key_per_process(self):
a = mw_module.session_key(FakeContext(transport="stdio", session_id="x"))
b = mw_module.session_key(FakeContext(transport="stdio", session_id="y"))
assert a == b and a.startswith("stdio:")
def test_falls_back_to_ctx_session_id(self):
assert mw_module.session_key(FakeContext()) == "ctx-session"
def test_no_context(self):
assert mw_module.session_key(None) is None
class TestStdioEndToEnd:
async def test_same_stdio_client_links_by_session(self):
"""On a real stdio transport, one client's calls link by session on every FastMCP version."""
server = (
Path(__file__).parent.parent / "fixtures" / "stdio_correlation_server.py"
)
transport = PythonStdioTransport(str(server), python_cmd=sys.executable)
async with Client(transport) as client:
await client.call_tool("echo", {"text": "a"})
await client.call_tool("echo", {"text": "b"})
result = await client.call_tool("report", {})
pairs = json.loads(result.content[0].text)
assert pairs == [["echo", RULE_SESSION], ["echo", RULE_SESSION]]
+28 -3
View File
@@ -345,7 +345,7 @@ class TestServerComposition:
async def test_compose_feedback_services_success(self):
"""Test successful service composition."""
main_server = Mock()
main_server.import_server = Mock()
main_server.mount = Mock()
services = {
"service1": {
@@ -376,13 +376,38 @@ class TestServerComposition:
mock_submission.assert_called_once()
mock_management.assert_called_once()
mock_feedback.assert_called_once()
assert main_server.import_server.call_count == 3
assert main_server.mount.call_count == 3
@pytest.mark.asyncio
async def test_compose_feedback_services_real_servers(self, tmp_path):
"""Composed tools must actually appear under their prefix.
The mocked tests above can't catch a missing await: a sync Mock
records the call either way. This one uses real FastMCP servers.
"""
from fastmcp import FastMCP
from tests._compat import tool_map
main_server = FastMCP("Main")
compose_feedback_services(
main_server,
{
"public": {"database_url": f"sqlite:///{tmp_path}/p.db", "type": "submission"},
"admin": {"database_url": f"sqlite:///{tmp_path}/a.db", "type": "full"},
},
)
names = set(await tool_map(main_server))
assert "public_submit_feedback" in names
assert "admin_list_feedback" in names
assert "admin_update_feedback_status" in names
@pytest.mark.asyncio
async def test_compose_feedback_services_with_shared_insights(self):
"""Test service composition with shared insights."""
main_server = Mock()
main_server.import_server = Mock()
main_server.mount = Mock()
shared_insights = Mock()
services = {
+484
View File
@@ -0,0 +1,484 @@
"""Tests for per-call instrumentation: redaction, middleware, dispatch, sinks.
These run against real FastMCP servers through fastmcp.Client, so they
exercise the same middleware path on every supported FastMCP version.
"""
import asyncio
import io
import json
import time
import pytest
from fastmcp import Client, FastMCP
from fastmcp.exceptions import ToolError
from sqlalchemy import MetaData, inspect, select
from sqlalchemy.ext.asyncio import create_async_engine
from fastmcp_feedback import add_feedback_tools
from fastmcp_feedback.instrumentation import (
REDACTED,
CallbackSink,
DatabaseSink,
InstrumentationMiddleware,
JsonLinesSink,
MemorySink,
RecordDispatcher,
Redactor,
ToolCallRecord,
build_metadata,
instrument,
)
BMCP_TOKEN = "bmcp_" + "A1b2C3d4E5f6G7h8I9j0"
JWT = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTYifQ.c2lnbmF0dXJlLXZhbHVl"
def make_app(sinks=None, **kwargs):
sink = MemorySink()
app = FastMCP("instrumented")
@app.tool
def echo(text: str) -> dict:
return {"echo": text}
@app.tool
def create_token(label: str) -> dict:
return {"label": label, "access_token": BMCP_TOKEN, "note": f"use {BMCP_TOKEN}"}
@app.tool
def login(user: str, password: str) -> str:
return f"welcome {user}"
@app.tool
def fail(secret: str) -> str:
raise ValueError(f"bad input secret={secret}")
mw = instrument(app, sinks if sinks is not None else [sink], **kwargs)
return app, sink, mw
async def call(app, tool, args=None):
async with Client(app) as client:
return await client.call_tool(tool, args or {})
class TestRedactor:
def test_secret_keys_at_any_depth(self):
data = {"a": {"b": [{"password": "hunter2", "keep": 1}]}, "Authorization": "x"}
out = Redactor().redact(data)
assert out["a"]["b"][0] == {"password": REDACTED, "keep": 1}
assert out["Authorization"] == REDACTED
def test_suffix_and_dash_variants(self):
out = Redactor().redact(
{
"github_token": "t",
"DB-Password": "p",
"X-Api-Key": "k",
"stripe_api_key": "s",
}
)
assert set(out.values()) == {REDACTED}
def test_non_secret_keys_untouched(self):
data = {"max_tokens": 100, "token_count": 3, "tool": "echo", "tokens": [1, 2]}
assert Redactor().redact(data) == data
def test_secret_values_inside_text(self):
text = f"pat {BMCP_TOKEN}, jwt {JWT}, header Bearer abcdefghijklmnop, key sk-{'x' * 20}"
out = Redactor().redact_text(text)
for secret in (BMCP_TOKEN, JWT, "abcdefghijklmnop", "sk-"):
assert secret not in out
assert out.count(REDACTED) == 4
def test_key_value_text_keeps_the_key(self):
out = Redactor().redact_text("failed: password=hunter2 api_key: abc123, ok")
assert out == f"failed: password={REDACTED} api_key: {REDACTED}, ok"
def test_input_not_mutated(self):
data = {"password": "hunter2", "nested": {"token": "t"}}
Redactor().redact(data)
assert data == {"password": "hunter2", "nested": {"token": "t"}}
def test_none_secret_value_left_as_none(self):
assert Redactor().redact({"password": None}) == {"password": None}
def test_extra_keys_and_patterns(self):
r = Redactor(extra_keys=["license"], extra_patterns=[r"ACME-\d{6}"])
assert r.redact({"license": "L", "msg": "code ACME-123456"}) == {
"license": REDACTED,
"msg": f"code {REDACTED}",
}
def test_depth_limit(self):
deep = current = {}
for _ in range(50):
current["n"] = {}
current = current["n"]
assert "[TRUNCATED]" in json.dumps(Redactor(max_depth=5).redact(deep))
class TestMiddlewareRecording:
async def test_success_in_meta_mode(self):
app, sink, mw = make_app()
await call(app, "echo", {"text": "hello"})
await mw.flush()
[rec] = sink.records
assert rec.tool == "echo" and rec.ok and rec.mode == "meta"
assert rec.duration_ms > 0
assert rec.args_size == len(json.dumps({"text": "hello"}))
assert rec.result_size and rec.result_size > 0
assert rec.args is None and rec.result is None
assert rec.session_id and rec.request_id
async def test_full_mode_redacts_args_and_results(self):
app, sink, mw = make_app(mode="full")
await call(app, "create_token", {"label": "ci"})
await call(app, "login", {"user": "ryan", "password": "hunter2"})
await mw.flush()
token_rec, login_rec = sink.records
dumped = json.dumps([token_rec.to_dict(), login_rec.to_dict()])
assert BMCP_TOKEN not in dumped and "hunter2" not in dumped
assert token_rec.result["access_token"] == REDACTED
assert token_rec.result["note"] == f"use {REDACTED}"
assert token_rec.result["label"] == "ci"
assert login_rec.args == {"user": "ryan", "password": REDACTED}
async def test_failed_call_recorded_and_error_unchanged(self):
app, sink, mw = make_app()
with pytest.raises(ToolError, match="bad input"):
await call(app, "fail", {"secret": "hunter2"})
await mw.flush()
[rec] = sink.records
assert not rec.ok
assert rec.error_type == "ValueError"
assert "hunter2" not in rec.error_message
assert f"secret={REDACTED}" in rec.error_message
async def test_meta_only_tools_override_full_mode(self):
app, sink, mw = make_app(mode="full", meta_only_tools={"create_token"})
await call(app, "create_token", {"label": "ci"})
await call(app, "echo", {"text": "hi"})
await mw.flush()
token_rec, echo_rec = sink.records
assert token_rec.mode == "meta" and token_rec.result is None
assert token_rec.result_size > 0
assert echo_rec.mode == "full" and echo_rec.result == {"echo": "hi"}
async def test_off_mode_records_nothing(self):
app, sink, mw = make_app(mode="off")
await call(app, "echo", {"text": "hi"})
await mw.flush()
assert sink.records == []
async def test_mode_from_environment(self, monkeypatch):
monkeypatch.setenv("FEEDBACK_INSTRUMENTATION_MODE", "full")
assert InstrumentationMiddleware([]).mode == "full"
def test_invalid_mode_rejected(self):
with pytest.raises(ValueError, match="mode must be one of"):
InstrumentationMiddleware([], mode="everything")
class TestHooks:
async def test_identity_resolver_fills_columns(self):
def who(context):
assert context.fastmcp_context is not None
return {
"user_sub": "user-1",
"caller_kind": "pat:7",
"client_id": "blender-addon",
"org": "acme",
"token": BMCP_TOKEN,
}
app, sink, mw = make_app(identity_resolver=who)
await call(app, "echo", {"text": "hi"})
await mw.flush()
[rec] = sink.records
assert (rec.user_sub, rec.caller_kind, rec.client_id) == (
"user-1",
"pat:7",
"blender-addon",
)
assert rec.identity == {"org": "acme", "token": REDACTED}
async def test_async_enricher_gets_result_and_sets_extra(self):
seen = {}
async def enrich(tool, args, result, context):
seen.update(tool=tool, args=args, result=result)
return {"job_id": "job-42", "server_version": "2026.9.27", "api_key": "k"}
app, sink, mw = make_app(enricher=enrich, server_version="default")
await call(app, "echo", {"text": "hi"})
await mw.flush()
[rec] = sink.records
assert seen == {
"tool": "echo",
"args": {"text": "hi"},
"result": {"echo": "hi"},
}
assert rec.extra == {"job_id": "job-42", "api_key": REDACTED}
assert rec.server_version == "2026.9.27"
async def test_server_version_default(self):
app, sink, mw = make_app(server_version="1.0")
await call(app, "echo", {"text": "hi"})
await mw.flush()
assert sink.records[0].server_version == "1.0"
async def test_hook_exceptions_do_not_break_calls(self):
def boom(*args):
raise RuntimeError("hook bug")
app, sink, mw = make_app(identity_resolver=boom, enricher=boom)
result = await call(app, "echo", {"text": "hi"})
await mw.flush()
assert result.data == {"echo": "hi"}
[rec] = sink.records
assert rec.ok and rec.identity == {} and rec.extra == {}
async def test_slow_async_hook_is_cut_off(self):
async def slow(*args):
await asyncio.sleep(5)
return {"never": True}
app, sink, mw = make_app(enricher=slow, hook_timeout=0.05)
start = time.perf_counter()
await call(app, "echo", {"text": "hi"})
assert time.perf_counter() - start < 2
await mw.flush()
assert sink.records[0].extra == {}
class TestNeverBlocksOrBreaks:
async def test_failing_sink_does_not_break_calls_or_other_sinks(self):
class Broken:
async def write(self, records):
raise ConnectionError("database down")
good = MemorySink()
app, _, mw = make_app(sinks=[Broken(), good])
result = await call(app, "echo", {"text": "hi"})
await mw.flush()
assert result.data == {"echo": "hi"}
assert mw.dispatcher.sink_errors == 1
assert len(good.records) == 1
async def test_stalled_sink_drops_records_instead_of_waiting(self):
release = asyncio.Event()
class Stalled:
def __init__(self):
self.records = []
async def write(self, records):
await release.wait()
self.records.extend(records)
stalled = Stalled()
app, _, mw = make_app(sinks=[stalled], max_queue=2, batch_size=1)
start = time.perf_counter()
async with Client(app) as client:
for i in range(6):
await client.call_tool("echo", {"text": str(i)})
assert time.perf_counter() - start < 3
# One record is in the stalled write, two fit the queue, the rest drop.
assert mw.dispatcher.dropped == 3
release.set()
await mw.flush()
assert len(stalled.records) == 3
async def test_record_building_failure_does_not_break_calls(self, monkeypatch):
app, sink, mw = make_app()
async def explode(*args, **kwargs):
raise RuntimeError("bug in record building")
monkeypatch.setattr(mw, "_build_record", explode)
result = await call(app, "echo", {"text": "hi"})
assert result.data == {"echo": "hi"}
async def test_tool_error_passes_through_even_if_recording_fails(self, monkeypatch):
app, sink, mw = make_app()
async def explode(*args, **kwargs):
raise RuntimeError("bug in record building")
monkeypatch.setattr(mw, "_build_record", explode)
with pytest.raises(ToolError, match="bad input"):
await call(app, "fail", {"secret": "x"})
def test_submit_without_event_loop_drops(self):
dispatcher = RecordDispatcher([MemorySink()])
record = ToolCallRecord(
tool="t",
started_at=__import__("datetime").datetime.now(),
duration_ms=1,
ok=True,
mode="meta",
)
assert dispatcher.submit(record) is False
assert dispatcher.dropped == 1
class TestSinks:
async def test_json_lines_sink(self):
stream = io.StringIO()
app, _, mw = make_app(sinks=[JsonLinesSink(stream)])
await call(app, "echo", {"text": "hi"})
await call(app, "echo", {"text": "again"})
await mw.flush()
lines = [json.loads(line) for line in stream.getvalue().splitlines()]
assert [line["tool"] for line in lines] == ["echo", "echo"]
assert lines[0]["started_at"].endswith("+00:00")
async def test_callback_sink_sync_and_async(self):
got_sync, got_async = [], []
async def async_cb(records):
got_async.extend(records)
app, _, mw = make_app(
sinks=[CallbackSink(got_sync.extend), CallbackSink(async_cb)]
)
await call(app, "echo", {"text": "hi"})
await mw.flush()
assert len(got_sync) == len(got_async) == 1
async def test_aclose_closes_sinks(self):
closed = []
class Closable(MemorySink):
async def aclose(self):
closed.append(True)
app, _, mw = make_app(sinks=[Closable()])
await call(app, "echo", {"text": "hi"})
await mw.aclose()
assert closed == [True]
class TestDatabaseSink:
async def test_end_to_end_with_created_tables(self, tmp_path):
url = f"sqlite+aiosqlite:///{tmp_path}/calls.db"
db = DatabaseSink(url, create_tables=True)
app, _, mw = make_app(sinks=[db], mode="full", server_version="9.9")
await call(app, "create_token", {"label": "ci"})
with pytest.raises(ToolError):
await call(app, "fail", {"secret": "hunter2"})
await mw.flush()
async with db.engine.connect() as conn:
rows = (
(await conn.execute(select(db.table).order_by(db.table.c.started_at)))
.mappings()
.all()
)
await mw.aclose()
assert [r["tool"] for r in rows] == ["create_token", "fail"]
assert rows[0]["ok"] is True and rows[1]["ok"] is False
assert rows[0]["result"]["access_token"] == REDACTED
assert rows[1]["error_type"] == "ValueError"
assert rows[0]["server_version"] == "9.9"
assert "hunter2" not in json.dumps([dict(r) for r in rows], default=str)
async def test_prefix_and_host_owned_engine(self, tmp_path):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path}/host.db")
db = DatabaseSink(engine, prefix="qa_", create_tables=True)
app, _, mw = make_app(sinks=[db])
await call(app, "echo", {"text": "hi"})
await mw.aclose()
# The host's engine is still usable: the sink did not dispose it.
async with engine.connect() as conn:
tables = await conn.run_sync(lambda c: inspect(c).get_table_names())
await engine.dispose()
assert sorted(tables) == ["qa_feedback_call_links", "qa_tool_calls"]
async def test_no_tables_created_by_default(self, tmp_path):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path}/host.db")
app, _, mw = make_app(sinks=[DatabaseSink(engine)])
result = await call(app, "echo", {"text": "hi"})
await mw.flush()
async with engine.connect() as conn:
tables = await conn.run_sync(lambda c: inspect(c).get_table_names())
await engine.dispose()
# The insert fails (no table), is counted, and the call still worked.
assert result.data == {"echo": "hi"}
assert tables == []
assert mw.dispatcher.sink_errors == 1
async def test_absent_payloads_are_sql_null(self, tmp_path):
"""Meta mode stores no payloads; they must be SQL NULL, not JSON null."""
db = DatabaseSink(f"sqlite+aiosqlite:///{tmp_path}/n.db", create_tables=True)
app, _, mw = make_app(sinks=[db])
await call(app, "echo", {"text": "hi"})
await mw.flush()
t = db.table
async with db.engine.connect() as conn:
nulls = (
await conn.execute(select(t.c.args.is_(None), t.c.result.is_(None)))
).one()
await mw.aclose()
assert tuple(nulls) == (True, True)
def test_build_metadata_joins_host_metadata(self):
host = MetaData()
assert build_metadata("ffb_", host) is host
build_metadata("ffb_", host)
assert sorted(host.tables) == ["ffb_feedback_call_links", "ffb_tool_calls"]
class TestIntegration:
async def test_records_mounted_server_tools_by_prefixed_name(self):
main = FastMCP("main")
sub = FastMCP("sub")
@sub.tool
def ping() -> str:
return "pong"
main.mount(sub, "svc")
sink = MemorySink()
mw = instrument(main, [sink])
await call(main, "svc_ping")
await mw.flush()
assert [r.tool for r in sink.records] == ["svc_ping"]
async def test_records_feedback_tools(self):
app = FastMCP("with-feedback")
add_feedback_tools(app, database_url="sqlite:///:memory:")
sink = MemorySink()
mw = instrument(app, [sink])
await call(
app,
"submit_feedback",
{
"request": {
"type": "bug",
"title": "t",
"description": "instrumented",
"submitter": "me",
}
},
)
await mw.flush()
assert [r.tool for r in sink.records] == ["submit_feedback"]
+9 -8
View File
@@ -7,6 +7,7 @@ from fastmcp import FastMCP
from fastmcp_feedback.mixins import SubmissionMixin, RetrievalMixin, ManagementMixin
from fastmcp_feedback.models import SubmitFeedbackRequest, FeedbackStatus
from fastmcp_feedback.database import FeedbackDatabase
from tests._compat import tool_map
@pytest.mark.unit
@@ -95,7 +96,7 @@ class TestSubmissionMixin:
mixin.register_tools(mock_fastmcp_server)
# Verify tool was registered
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "submit_feedback" in tool_names
@@ -107,7 +108,7 @@ class TestSubmissionMixin:
mixin.register_tools(mock_fastmcp_server, prefix="support", separator="_")
# Verify tool was registered with prefix
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "support_submit_feedback" in tool_names
@@ -267,7 +268,7 @@ class TestRetrievalMixin:
mixin.register_tools(mock_fastmcp_server)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
expected_tools = ["list_feedback", "get_feedback_statistics"]
@@ -437,7 +438,7 @@ class TestManagementMixin:
mixin.register_tools(mock_fastmcp_server)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
expected_tools = ["update_feedback_status", "delete_feedback"]
@@ -532,7 +533,7 @@ class TestMixinIntegration:
management.register_tools(mock_fastmcp_server)
# Verify all tools are registered
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
expected_tools = [
@@ -555,7 +556,7 @@ class TestMixinIntegration:
submission.register_tools(mock_fastmcp_server, prefix="admin")
retrieval.register_tools(mock_fastmcp_server, prefix="public")
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "admin_submit_feedback" in tool_names
@@ -663,7 +664,7 @@ class TestMixinConfiguration:
# Register with dash separator
submission.register_tools(mock_fastmcp_server, prefix="api", separator="-")
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "api-submit_feedback" in tool_names
@@ -676,7 +677,7 @@ class TestMixinConfiguration:
# Test the standard registration instead
submission.register_tools(mock_fastmcp_server)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# SubmissionMixin should only register its tools
+397
View File
@@ -0,0 +1,397 @@
"""Tests for classifying error results that tools return instead of raising.
Real FastMCP servers through fastmcp.Client, as in test_instrumentation.py.
"""
import contextlib
import logging
import pytest
from fastmcp import Client, FastMCP
from fastmcp.exceptions import ToolError
from mcp.types import CallToolResult, TextContent
from sqlalchemy import select
from fastmcp_feedback import add_feedback_tools
from fastmcp_feedback.instrumentation import (
DEFAULT_RESULT_CLASSIFIER,
REDACTED,
DatabaseSink,
MemorySink,
ToolCallRecord,
instrument,
soft_error_classifier,
)
from fastmcp_feedback.mixins import submission
BMCP_TOKEN = "bmcp_" + "A1b2C3d4E5f6G7h8I9j0"
try:
from fastmcp.tools.tool import ToolResult
except ImportError: # FastMCP 4
from fastmcp.tools import ToolResult
try:
ToolResult(content=[TextContent(type="text", text="x")], is_error=True)
TOOL_RESULT_IS_ERROR = True
except TypeError: # added in FastMCP 3.4
TOOL_RESULT_IS_ERROR = False
@pytest.fixture
async def db(tmp_path):
"""DatabaseSinks whose engines are always disposed, or the process hangs."""
made = []
def factory(name="calls"):
sink = DatabaseSink(
f"sqlite+aiosqlite:///{tmp_path}/{name}.db", create_tables=True
)
made.append(sink)
return sink
yield factory
for sink in made:
await sink.engine.dispose()
def make_app(sinks=None, **kwargs):
sink = MemorySink()
app = FastMCP("soft")
@app.tool
def render(status: str) -> dict:
return {"status": status, "error": f"render said {status}"}
@app.tool
def respond(payload: dict) -> dict:
return payload
@app.tool
def text(value: str) -> str:
return value
@app.tool
def items() -> list:
return [{"status": "failed"}]
@app.tool
def crash() -> dict:
raise RuntimeError("exploded")
@app.tool
def mcp_error() -> CallToolResult:
return CallToolResult(
content=[TextContent(type="text", text="target missing")], isError=True
)
mw = instrument(app, sinks if sinks is not None else [sink], **kwargs)
return app, sink, mw
async def run(app, mw, tool, args=None):
async with Client(app) as client:
with contextlib.suppress(ToolError):
await client.call_tool(tool, args or {})
await mw.flush()
async def record_for(tool, args=None, **kwargs) -> ToolCallRecord:
app, sink, mw = make_app(**kwargs)
await run(app, mw, tool, args)
[record] = sink.records
return record
class TestClassifier:
@pytest.mark.parametrize(
"result, expected",
[
({"status": "failed", "error": "disk full"}, "status=failed: disk full"),
({"state": "ERROR"}, "state=ERROR"),
({"ok": False, "message": "no scene"}, "ok=False: no scene"),
({"success": False, "error": "nope"}, "success=False: nope"),
({"success": False, "detail": "d", "reason": "r"}, "success=False: d"),
(
{"status": "failure", "error": {"code": 7, "message": "inner"}},
"status=failure: inner",
),
({"isError": True, "content": ["boom"]}, "isError: boom"),
(
{"is_error": True, "content": [{"type": "text", "text": "a"}]},
"isError: a",
),
],
)
def test_flagged(self, result, expected):
assert DEFAULT_RESULT_CLASSIFIER("t", result) == expected
@pytest.mark.parametrize(
"result",
[
{"status": "done"},
{"status": "unknown_target"},
{"ok": True},
{"ok": 0},
{"success": None},
{"result": False},
{"state": 3},
{"isError": False},
[{"status": "failed"}],
"failed",
None,
],
)
def test_not_flagged(self, result):
assert DEFAULT_RESULT_CLASSIFIER("t", result) is None
def test_extra_statuses(self):
classify = soft_error_classifier(extra_statuses=("unknown_target",))
assert (
classify("t", {"status": "unknown_target", "error": "no cube"})
== "status=unknown_target: no cube"
)
assert classify("t", {"status": "failed"}) == "status=failed"
def test_custom_keys_replace_defaults(self):
classify = soft_error_classifier(
statuses=("bad",), status_keys=("phase",), flag_keys=("passed",)
)
assert classify("t", {"phase": "bad"}) == "phase=bad"
assert classify("t", {"passed": False}) == "passed=False"
assert classify("t", {"status": "failed", "ok": False}) is None
class TestMiddleware:
async def test_status_failed_is_soft_error(self):
record = await record_for("render", {"status": "failed"})
assert (record.outcome, record.ok, record.error_type) == (
"soft_error",
False,
"SoftError",
)
assert record.error_message == "status=failed: render said failed"
@pytest.mark.parametrize("key", ["ok", "success"])
async def test_false_flags_are_soft_errors(self, key):
record = await record_for("respond", {"payload": {key: False, "error": "x"}})
assert (record.outcome, record.ok) == ("soft_error", False)
assert record.error_message == f"{key}=False: x"
async def test_message_is_redacted_and_truncated(self):
record = await record_for(
"respond",
{"payload": {"status": "error", "error": f"bad key {BMCP_TOKEN} " * 50}},
max_error_chars=60,
)
assert BMCP_TOKEN not in record.error_message
assert REDACTED in record.error_message
assert len(record.error_message) == 60
async def test_extra_statuses(self):
record = await record_for(
"render",
{"status": "unknown_target"},
result_classifier=soft_error_classifier(extra_statuses=("unknown_target",)),
)
assert record.outcome == "soft_error"
assert (
record.error_message == "status=unknown_target: render said unknown_target"
)
async def test_unknown_status_is_ok_by_default(self):
record = await record_for("render", {"status": "unknown_target"})
assert (record.outcome, record.ok, record.error_type) == ("ok", True, None)
@pytest.mark.parametrize(
"tool, args",
[
("text", {"value": "failed"}),
("items", {}),
("respond", {"payload": {"status": "done", "count": 3}}),
],
)
async def test_normal_results_stay_ok(self, tool, args):
record = await record_for(tool, args)
assert (record.outcome, record.ok, record.error_message) == ("ok", True, None)
async def test_none_disables(self):
record = await record_for(
"render", {"status": "failed"}, result_classifier=None
)
assert (record.outcome, record.ok, record.error_type) == ("ok", True, None)
async def test_custom_classifier(self):
seen = []
def classify(tool, result):
seen.append((tool, result))
return "empty" if result.get("count") == 0 else None
record = await record_for(
"respond", {"payload": {"count": 0}}, result_classifier=classify
)
assert seen == [("respond", {"count": 0})]
assert (record.outcome, record.error_message) == ("soft_error", "empty")
async def test_raising_classifier_records_ok(self, caplog):
def classify(tool, result):
raise KeyError("classifier bug")
app, sink, mw = make_app(result_classifier=classify)
with caplog.at_level(logging.WARNING):
async with Client(app) as client:
result = await client.call_tool("render", {"status": "failed"})
await mw.flush()
assert result.data["status"] == "failed"
[record] = sink.records
assert (record.outcome, record.ok, record.error_type) == ("ok", True, None)
assert "Result classifier" in caplog.text
async def test_classifier_not_run_on_exceptions(self):
calls = []
record = await record_for(
"crash", result_classifier=lambda t, r: calls.append(t) or "x"
)
assert calls == []
assert (record.outcome, record.ok, record.error_type) == (
"error",
False,
"RuntimeError",
)
assert record.error_message == "exploded"
async def test_mcp_error_result_without_raising(self):
"""A CallToolResult with isError, on every FastMCP version."""
record = await record_for("mcp_error")
assert (record.outcome, record.ok) == ("soft_error", False)
assert record.error_message == "isError: target missing"
@pytest.mark.skipif(not TOOL_RESULT_IS_ERROR, reason="ToolResult.is_error is 3.4+")
async def test_tool_result_is_error(self):
app = FastMCP("is_error")
@app.tool
def lookup():
return ToolResult(
content=[TextContent(type="text", text="not found")], is_error=True
)
sink = MemorySink()
mw = instrument(app, [sink])
async with Client(app) as client:
await client.call_tool("lookup", {}, raise_on_error=False)
await mw.flush()
[record] = sink.records
assert (record.outcome, record.error_message) == (
"soft_error",
"isError: not found",
)
def test_record_outcome_defaults_from_ok(self):
from datetime import UTC, datetime
now = datetime.now(UTC)
assert ToolCallRecord("t", now, 1.0, True, "meta").outcome == "ok"
assert ToolCallRecord("t", now, 1.0, False, "meta").outcome == "error"
class TestStorageAndCorrelation:
async def test_database_sink_writes_outcome(self, db):
sink = db()
app, _, mw = make_app(sinks=[sink])
async with Client(app) as client:
await client.call_tool("render", {"status": "done"})
await client.call_tool("render", {"status": "failed"})
with contextlib.suppress(ToolError):
await client.call_tool("crash", {})
await mw.flush()
t = sink.table
async with sink.engine.connect() as conn:
rows = (
await conn.execute(
select(t.c.outcome, t.c.ok, t.c.error_type).order_by(t.c.started_at)
)
).all()
assert [tuple(r) for r in rows] == [
("ok", True, None),
("soft_error", False, "SoftError"),
("error", False, "RuntimeError"),
]
async with sink.engine.connect() as conn:
[(failing,)] = (
await conn.execute(
select(t.c.tool).where(~t.c.ok, t.c.outcome != "error")
)
).all()
assert failing == "render"
async def test_links_and_feedback_context_show_soft_errors(self, db):
app, _, mw = make_app(
sinks=[db()], identity_resolver=lambda c: {"user_sub": "u1"}
)
captured = {}
@app.tool
async def report() -> int:
captured["links"] = await mw.capture_recent_calls()
await mw.link_feedback("bug-1", captured["links"])
return len(captured["links"])
async with Client(app) as client:
await client.call_tool("render", {"status": "failed"})
await client.call_tool("report", {})
links = captured["links"]
assert [(link.tool, link.ok) for link in links] == [("render", False)]
# Filled from memory or read from the database, depending on timing.
[row] = await mw.feedback_context("bug-1")
assert (row["outcome"], row["ok"], row["error_type"]) == (
"soft_error",
False,
"SoftError",
)
await mw.flush()
[row] = await mw.feedback_context("bug-1")
assert (row["tool"], row["outcome"], row["ok"]) == (
"render",
"soft_error",
False,
)
await mw.aclose()
async def test_submit_feedback_failure_is_soft_error(self, monkeypatch):
def broken(request):
raise RuntimeError("database down, password=hunter2")
monkeypatch.setattr(submission, "create_feedback_from_request", broken)
app = FastMCP("fb")
sink = MemorySink()
mw = instrument(app, [sink])
add_feedback_tools(app, database_url="sqlite:///:memory:", instrumentation=mw)
async with Client(app) as client:
result = await client.call_tool(
"submit_feedback",
{
"request": {
"type": "bug",
"title": "t",
"description": "d",
"submitter": "m",
}
},
)
await mw.flush()
assert result.data["success"] is False
[record] = sink.records
assert (record.tool, record.outcome, record.ok, record.error_type) == (
"submit_feedback",
"soft_error",
False,
"SoftError",
)
assert record.error_message == (
f"success=False: database down, password={REDACTED}"
)
+37 -36
View File
@@ -7,6 +7,7 @@ from fastmcp import FastMCP
from fastmcp_feedback.tools import add_feedback_tools
from fastmcp_feedback.feedback import create_feedback_server
from fastmcp_feedback.models import SubmitFeedbackRequest
from tests._compat import tool_map
@pytest.mark.unit
@@ -18,7 +19,7 @@ class TestAddFeedbackTools:
"""Test adding feedback tools with basic configuration."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
expected_tools = [
@@ -40,7 +41,7 @@ class TestAddFeedbackTools:
prefix="support"
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
expected_tools = [
@@ -62,7 +63,7 @@ class TestAddFeedbackTools:
insights=test_insights
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
# Should have all standard tools - check tool names in the keys
assert len([name for name in tools.keys() if "feedback" in name]) >= 5
@@ -76,7 +77,7 @@ class TestAddFeedbackTools:
# Should use default SQLite database
add_feedback_tools(mock_fastmcp_server)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "submit_feedback" in tool_names
@@ -91,7 +92,7 @@ class TestAddFeedbackTools:
separator="-"
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "api-submit_feedback" in tool_names
@@ -105,7 +106,7 @@ class TestAddFeedbackTools:
# Second call with different prefix
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url, prefix="user")
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should have tools with both prefixes
@@ -128,7 +129,7 @@ class TestCreateFeedbackServer:
assert isinstance(server, FastMCP)
assert server.name == "Test Feedback Server"
tools = await server.get_tools()
tools = await tool_map(server)
tool_names = list(tools.keys())
expected_tools = [
@@ -150,7 +151,7 @@ class TestCreateFeedbackServer:
)
assert server.name == "Custom DB Server"
tools = await server.get_tools()
tools = await tool_map(server)
assert len(tools) >= 5
async def test_create_feedback_server_with_insights(self, test_insights):
@@ -162,7 +163,7 @@ class TestCreateFeedbackServer:
assert server.name == "Analytics Server"
# Verify insights are integrated
tools = await server.get_tools()
tools = await tool_map(server)
assert len(tools) >= 5
async def test_create_feedback_server_with_description(self):
@@ -184,8 +185,8 @@ class TestCreateFeedbackServer:
assert server1.name != server2.name
# Each should have independent tool sets
tools1 = await server1.get_tools()
tools2 = await server2.get_tools()
tools1 = await tool_map(server1)
tools2 = await tool_map(server2)
assert len(tools1) >= 5
assert len(tools2) >= 5
@@ -201,7 +202,7 @@ class TestToolIntegration:
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
# Get the submit_feedback tool
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
submit_tool = tools["submit_feedback"]
assert submit_tool is not None
@@ -218,7 +219,7 @@ class TestToolIntegration:
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
# Get the list_feedback tool
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
list_tool = tools["list_feedback"]
assert list_tool is not None
@@ -234,7 +235,7 @@ class TestToolIntegration:
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
# Get the statistics tool
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
stats_tool = tools["get_feedback_statistics"]
assert stats_tool is not None
@@ -250,7 +251,7 @@ class TestToolIntegration:
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
# Get the update status tool
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
update_tool = tools["update_feedback_status"]
assert update_tool is not None
@@ -269,7 +270,7 @@ class TestToolIntegration:
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
# Get the delete tool
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
delete_tool = tools["delete_feedback"]
assert delete_tool is not None
@@ -290,7 +291,7 @@ class TestToolValidation:
"""Test submit_feedback tool parameter validation."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
submit_tool = tools["submit_feedback"]
# Test with invalid parameters
@@ -301,7 +302,7 @@ class TestToolValidation:
"""Test list_feedback tool parameter validation."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
list_tool = tools["list_feedback"]
# Test with valid optional parameters
@@ -318,7 +319,7 @@ class TestToolValidation:
"""Test update_feedback_status parameter validation."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
update_tool = tools["update_feedback_status"]
# Test with missing required parameters
@@ -329,7 +330,7 @@ class TestToolValidation:
"""Test delete_feedback parameter validation."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
delete_tool = tools["delete_feedback"]
# Test with missing required parameters
@@ -348,9 +349,9 @@ class TestServerComposition:
feedback_server = create_feedback_server("Feedback Service")
# Import feedback server with prefix
await main_server.import_server(feedback_server, prefix="feedback")
main_server.mount(feedback_server, "feedback")
tools = await main_server.get_tools()
tools = await tool_map(main_server)
tool_names = list(tools.keys())
# Should have feedback tools with prefix
@@ -364,10 +365,10 @@ class TestServerComposition:
user_feedback = create_feedback_server("User Feedback")
admin_feedback = create_feedback_server("Admin Feedback")
await main_server.import_server(user_feedback, prefix="user")
await main_server.import_server(admin_feedback, prefix="admin")
main_server.mount(user_feedback, "user")
main_server.mount(admin_feedback, "admin")
tools = await main_server.get_tools()
tools = await tool_map(main_server)
tool_names = list(tools.keys())
# Should have tools from both servers
@@ -386,9 +387,9 @@ class TestServerComposition:
# Implementation would need to support this feature
# FastMCP import_server doesn't support selective tool import
# but we can test the full import with prefix
await main_server.import_server(feedback_server, prefix="public")
main_server.mount(feedback_server, "public")
tools = await main_server.get_tools()
tools = await tool_map(main_server)
tool_names = list(tools.keys())
# All tools should be imported with prefix
@@ -416,7 +417,7 @@ class TestToolPerformance:
assert performance_timer.duration_ms < 1000 # 1 second
# Verify all tools were registered
tools = await server.get_tools()
tools = await tool_map(server)
assert len(tools) >= 5
async def test_multiple_server_creation_performance(self, performance_timer):
@@ -436,14 +437,14 @@ class TestToolPerformance:
# Each server should be functional
for server in servers:
tools = await server.get_tools()
tools = await tool_map(server)
assert len(tools) >= 5
async def test_tool_execution_performance(self, mock_fastmcp_server, memory_db_url, performance_timer):
"""Test performance of tool execution."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
list_tool = tools["list_feedback"]
performance_timer.start()
@@ -467,7 +468,7 @@ class TestToolConfiguration:
"""Test that tools have proper descriptions."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
for tool_name, tool_obj in tools.items():
# Each tool should have a description
@@ -480,7 +481,7 @@ class TestToolConfiguration:
"""Test tool metadata and categorization."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
feedback_tools = [name for name in tools.keys() if "feedback" in name]
# Should have categorized the tools appropriately
@@ -490,7 +491,7 @@ class TestToolConfiguration:
"""Test tool parameter schemas."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
submit_tool = tools["submit_feedback"]
# Tool should have parameter schema defined
@@ -510,7 +511,7 @@ class TestToolErrorHandling:
# Use invalid database URL
add_feedback_tools(server, database_url="sqlite:///nonexistent/path/db.sqlite")
tools = await server.get_tools()
tools = await tool_map(server)
list_tool = tools["list_feedback"]
# Tool execution should handle database errors gracefully
@@ -523,7 +524,7 @@ class TestToolErrorHandling:
"""Test tool handling of invalid parameters."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
submit_tool = tools["submit_feedback"]
# Test with malformed request
@@ -534,7 +535,7 @@ class TestToolErrorHandling:
"""Test concurrent execution of tools."""
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
list_tool = tools["list_feedback"]
import asyncio
+29 -28
View File
@@ -23,6 +23,7 @@ from fastmcp_feedback.tools import (
)
from fastmcp_feedback.database import FeedbackDatabase
from fastmcp_feedback.insights import FeedbackInsights
from tests._compat import tool_map
@pytest.mark.unit
@@ -34,7 +35,7 @@ class TestSubmissionTools:
"""Test adding submission tools with basic configuration."""
add_submission_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should have submission-related tools
@@ -51,7 +52,7 @@ class TestSubmissionTools:
prefix="user"
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "user_submit_feedback" in tool_names
@@ -65,14 +66,14 @@ class TestSubmissionTools:
insights=test_insights
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert len(tools) > 0
async def test_add_submission_tools_without_database_url(self, mock_fastmcp_server):
"""Test adding submission tools without database URL (uses default)."""
add_submission_tools(mock_fastmcp_server)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert "submit_feedback" in tools
@patch('fastmcp_feedback.tools.FeedbackDatabase')
@@ -105,7 +106,7 @@ class TestSubmissionTools:
prefix="api_" # With trailing underscore
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should strip trailing underscore
@@ -121,7 +122,7 @@ class TestRetrievalTools:
"""Test adding retrieval tools with basic configuration."""
add_retrieval_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should have retrieval-related tools
@@ -139,7 +140,7 @@ class TestRetrievalTools:
prefix="public"
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "public_list_feedback" in tool_names
@@ -154,14 +155,14 @@ class TestRetrievalTools:
insights=test_insights
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert len(tools) > 0
async def test_add_retrieval_tools_without_database_url(self, mock_fastmcp_server):
"""Test adding retrieval tools without database URL (uses default)."""
add_retrieval_tools(mock_fastmcp_server)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert "list_feedback" in tools
assert "get_feedback_statistics" in tools
@@ -197,7 +198,7 @@ class TestManagementTools:
"""Test adding management tools with basic configuration."""
add_management_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should have management-related tools
@@ -215,7 +216,7 @@ class TestManagementTools:
prefix="admin"
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "admin_update_feedback_status" in tool_names
@@ -230,14 +231,14 @@ class TestManagementTools:
insights=test_insights
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert len(tools) > 0
async def test_add_management_tools_without_database_url(self, mock_fastmcp_server):
"""Test adding management tools without database URL (uses default)."""
add_management_tools(mock_fastmcp_server)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert "update_feedback_status" in tools
assert "delete_feedback" in tools
@@ -347,7 +348,7 @@ class TestAdvancedIntegrationPatterns:
create_multi_tenant_tools(mock_fastmcp_server, tenants)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should have tools for each tenant with prefixes
@@ -365,7 +366,7 @@ class TestAdvancedIntegrationPatterns:
create_multi_tenant_tools(mock_fastmcp_server, tenants, insights=test_insights)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert len(tools) >= 10 # At least 5 tools per tenant
async def test_create_multi_tenant_tools_empty_tenants(self, mock_fastmcp_server):
@@ -374,7 +375,7 @@ class TestAdvancedIntegrationPatterns:
create_multi_tenant_tools(mock_fastmcp_server, tenants)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
# Should complete without error but not add any tools
assert len(tools) == 0
@@ -394,7 +395,7 @@ class TestAdvancedIntegrationPatterns:
"""Test creating role-based tools."""
create_role_based_tools(mock_fastmcp_server, database_url=memory_db_url)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should have public tools (submission and retrieval)
@@ -416,14 +417,14 @@ class TestAdvancedIntegrationPatterns:
insights=test_insights
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert len(tools) >= 7 # Public + admin tools
async def test_create_role_based_tools_without_database(self, mock_fastmcp_server):
"""Test creating role-based tools without database URL."""
create_role_based_tools(mock_fastmcp_server)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "public_submit_feedback" in tool_names
@@ -465,7 +466,7 @@ class TestMigrationHelpers:
mock_fastmcp_server.remove_tool.assert_any_call("old_list_tool")
# Should have new tools added
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
assert "submit_feedback" in tool_names
assert "list_feedback" in tool_names
@@ -501,7 +502,7 @@ class TestMigrationHelpers:
insights=test_insights
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert len(tools) >= 5
async def test_migrate_tools_to_mixins_empty_list(self, mock_fastmcp_server, memory_db_url):
@@ -515,7 +516,7 @@ class TestMigrationHelpers:
)
# Should still add new tools
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
assert len(tools) >= 5
@patch('fastmcp_feedback.tools.add_feedback_tools')
@@ -757,7 +758,7 @@ class TestErrorHandlingAndEdgeCases:
# Verify all servers have tools
for server in servers:
tools = await server.get_tools()
tools = await tool_map(server)
assert "submit_feedback" in tools
assert "public_list_feedback" in tools
assert "admin_update_feedback_status" in tools
@@ -774,7 +775,7 @@ class TestErrorHandlingAndEdgeCases:
# Multiple underscores
add_management_tools(mock_fastmcp_server, database_url=memory_db_url, prefix="admin___")
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should handle edge cases gracefully
@@ -812,7 +813,7 @@ class TestIntegrationWithExistingTests:
# Add all tools
add_feedback_tools(mock_fastmcp_server, database_url=memory_db_url, prefix="complete")
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
tool_names = list(tools.keys())
# Should have both individual and complete sets
@@ -830,7 +831,7 @@ class TestIntegrationWithExistingTests:
insights=test_insights
)
tools = await mock_fastmcp_server.get_tools()
tools = await tool_map(mock_fastmcp_server)
submit_tool = tools["submit_feedback"]
# Should work with existing sample data
@@ -848,9 +849,9 @@ class TestIntegrationWithExistingTests:
add_retrieval_tools(feedback_server, prefix="readonly")
# Import with prefix
await main_server.import_server(feedback_server, prefix="external")
main_server.mount(feedback_server, "external")
tools = await main_server.get_tools()
tools = await tool_map(main_server)
tool_names = list(tools.keys())
# Should have properly composed tool names
Generated
+420 -395
View File
File diff suppressed because it is too large Load Diff