Compare commits

..
Author SHA1 Message Date
rsp2k 84b7591858 Merge fix/closed-dispatcher: the dispatcher stays closed after aclose()
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-05 01:19:37 -06:00
rsp2k 8fa36d6838 Release 2026.10.05: the dispatcher stays closed after aclose()
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-05 01:17:06 -06:00
rsp2k b28482fffc Keep the dispatcher closed after aclose()
A record submitted after aclose(), from the loop or a thread, rebound the
dispatcher: a new queue and task that wrote to the sinks just closed. Now
aclose() closes it for good. Later records are dropped and counted under
"dispatcher closed" (new drop_reasons), with one warning; it never binds a
loop or starts a task again, and aclose() twice is a no-op. Calls made
after close are not kept in recent for linking.

flush() also waits for a record put just after queue.join() woke, which a
single join missed: a record a thread handed over just before aclose()
is delivered if it lands during the flush, else dropped and counted.
2026-10-05 01:16:32 -06:00
rsp2k 15dfe17095 Merge feature/payload-cap-dotted-keys: payload size limit, dotted classifier keys
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-04 14:45:18 -06:00
rsp2k 4206e15293 Release 2026.10.04: payload size limit, dotted classifier keys
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-04 14:43:32 -06:00
rsp2k 8bf0801cf6 Test a preview that ends inside a secret 2026-10-04 14:42:16 -06:00
rsp2k 72d2845502 Cap stored payloads and event attrs; dotted paths in the classifier
max_payload_bytes (default 64 KiB, None for no limit) bounds the arguments
and results stored in mode full and every event's attrs. A larger value is
stored as {"_truncated": true, "size": ..., "preview": ...}, the preview
taken from the redacted value. Call payloads reuse the size already
measured for args_size and result_size; attrs keep their keys, with only
the largest values replaced.

soft_error_classifier takes message_keys, and message_keys and error_keys
accept dotted paths into nested dicts ("snapshot.error"). An exact key is
matched before splitting; a missing or non-dict step is absent.
2026-10-04 14:41:13 -06:00
rsp2k 69a09f4400 Merge fix/narrow-late-identity: second identity run only for an empty answer or a missing late key
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-02 00:11:38 -06:00
rsp2k b2e36f630a Release 2026.10.02.1: second identity run only for an empty answer or a missing late key
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-02 00:09:25 -06:00
rsp2k 6c16fdda03 Rerun the identity resolver only for an empty answer or a missing late key
A missing user_sub, caller_kind or client_id no longer triggers the second
run on its own, so a resolver that never returns one of them (mcspeak's has
no client_id) runs once per call again. The second run happens when the
first answer is empty (nothing, only None values, or a failed run) or lacks
a key in late_identity_keys, which stays empty by default.
2026-10-02 00:08:32 -06:00
rsp2k 2760854db9 Merge fix/late-identity: second identity run for tools that create the caller's identity
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-02 00:01:45 -06:00
rsp2k d0d2ee1b00 Release 2026.10.02: second identity run for tools that create the caller's identity
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-02 00:01:16 -06:00
rsp2k 6987d947d3 Run the identity resolver again after the tool when its first answer is incomplete
Since 2026.10.01.4 the resolver runs before the tool, so a tool that creates
the caller's identity (blender-mcp's register_client, which makes the bus
client the resolver looks up) recorded its own row without it. When the
first answer lacks user_sub, caller_kind or client_id, or any of the new
late_identity_keys, the resolver runs once more after the tool, alongside
the enricher, for calls that are kept. Its values fill the gaps; the first
answer wins on conflict, matching what events during the call saw.
2026-10-01 23:57:07 -06:00
rsp2k 74780fef02 Merge fix/consistency: statistics labels, event identity, quiet logging
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-01 22:49:12 -06:00
rsp2k 22f2c007e9 Drop the private dev docs hostname from the 2026.01.12 changelog entry
Tests / canary (push) Canceled after 0s
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
2026-10-01 22:49:12 -06:00
rsp2k 7892a5d79f Release 2026.10.01.4: statistics labels, event identity, quiet logging
README and CONTRIBUTING rewritten where they described the old behavior
or code that does not exist: the example test and mixin recipe now run,
MySQL is marked untested, the two Postgres driver paths are spelled out,
and the coverage figure is the measured 95%.
2026-10-01 22:48:46 -06:00
rsp2k c6c928d528 Test statistics labels on SQLite and Postgres, event identity, quiet import 2026-10-01 22:38:21 -06:00
rsp2k 1213b1587e Statistics keyed by enum value, identity resolved before the tool, no logging config on import
get_feedback_statistics and get_database_stats grouped the raw columns,
which hold enum names (BUG, OPEN) on SQLite and Postgres, while
list_feedback returns values (bug, open). Both now count through
count_feedback_by and return values.

The identity resolver ran after the call, so record_event inside a tool
never had user_sub, caller_kind or client_id. It now runs once before
the tool; the call scope carries the result to record_event and the
call's own record reuses it.

Importing the package called logging.basicConfig at INFO. It now adds a
NullHandler to the fastmcp_feedback logger; setup_logging() stays as an
opt-in. PACKAGE_INFO is read from the installed metadata.

psycopg2-binary joins the dev and test extras so the feedback tools can
be tested on Postgres.
2026-10-01 22:36:30 -06:00
rsp2k 433a003d09 Point the docs links at the public site
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
fastmcp-feedback.supported.systems replaces the private dev address in the
README and the PyPI Documentation URL. Version 2026.10.01.3.
2026-10-01 22:13:34 -06:00
rsp2k e4da62d0df Merge fix/lazy-storage-import: instrumentation imports without the database extras
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-01 20:56:05 -06:00
rsp2k 5e9a9ca037 Release 2026.10.01.2: instrumentation imports without the database extras
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-01 20:56:02 -06:00
rsp2k 76999c942c Model SQLAlchemy 2.0 and 2.1 in the base-install test 2026-10-01 20:53:34 -06:00
rsp2k 4995143f8f Import instrumentation without the database extras
storage, embeddings and backfill need sqlalchemy[asyncio] (greenlet). The
package __init__ now loads their names on first access and raises an
ImportError naming the extra when they are missing. The middleware finds a
DatabaseSink or EmbeddingSink through sys.modules instead of importing them.
Table prefix, embedding dim, sources and event text keys move to constants.py
so they import on a base install.
2026-10-01 20:52:33 -06:00
rsp2k 28816477ff Merge fix/explicit-success-wins: an explicit success flag beats error keys
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-01 18:05:52 -06:00
rsp2k c81dc3bf8c Release 2026.10.01.1: an explicit success flag beats error keys
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
2026-10-01 18:05:50 -06:00
rsp2k 4922eaecbd Let an explicit success flag win over error keys
A result with ok or success exactly True skips the error-key check, so
{"ok": True, "error": "deprecated arg"} is ok and the string reads as a
warning. A failing status, a False flag and isError still flag it, and
truthy non-bools are not an explicit success.
2026-10-01 18:04:15 -06:00
rsp2k 36fed0267c Move the dev Postgres port to 55447
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
55432 and 55433 are taken by other projects' containers on the dev host,
so make pg-up failed. The test database URL carries the port too.
2026-10-01 18:02:49 -06:00
rsp2k ae52e7a0f9 Merge feature/error-key-classifier: treat error-string results as soft errors
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-01 16:44:37 -06:00
rsp2k a69bf989fd Release 2026.10.01: treat error-string results as soft errors
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-10-01 16:44:34 -06:00
rsp2k f4aef293a5 Document error_keys, unique event keys, and the call/event join
README: the error-string check, its default and the behavior change on
upgrade; event keys used for joining must be unique across restarts; the
SQLite and Postgres join from calls to events; ids are UUIDs, so order
by occurred_at.
2026-10-01 16:41:54 -06:00
rsp2k 146bbbff25 Treat results with a non-empty error string as soft errors
soft_error_classifier gains error_keys, default ("error",). A dict result
whose value there is a non-empty string is a soft error, after the
status, flag and isError checks. None, blank strings, numbers, booleans,
dicts and lists do not count. error_keys=() turns it off.
2026-10-01 16:40:24 -06:00
rsp2k 34d5130bd3 Merge feature/backfill: backfill embeddings, flatten list and dict text
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-09-30 11:24:06 -06:00
rsp2k 7378afae8e Release 2026.09.30.1: backfill embeddings, flatten list and dict text
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
2026-09-30 11:24:04 -06:00
rsp2k 9f3edc59d3 Backfill CLI: keep request logs quiet unless -v 2026-09-30 11:20:01 -06:00
rsp2k ac1d751045 Document backfilling embeddings and list/dict flattening 2026-09-30 11:17:21 -06:00
rsp2k 6dacdafd6e Backfill embeddings for stored records; flatten list and dict text
backfill_embeddings reads stored call errors (including rows from before the
outcome column) and events, rebuilds the records and sends them through the
live sink's extraction, redaction and truncation, so later records dedupe
against the backfilled rows. backfill_feedback does the same for feedback
kept in a host's own tables. A CLI wraps it:
python -m fastmcp_feedback.instrumentation.backfill.

List and tuple attrs are embedded one item per line instead of as a Python
repr; dicts as key: value lines sorted by key, since JSONB drops key order.
2026-09-30 11:13:12 -06:00
rsp2k 2bd3d9476c Add a tested Postgres Alpine + pgvector Dockerfile
Tests / test (==2.12.2) (push) Canceled after 0s
Tests / test (==2.14.3) (push) Canceled after 0s
Tests / test (>=3.3,<4) (push) Canceled after 0s
Tests / test (>=4,<5) (push) Canceled after 0s
Tests / test-postgres (==2.12.2) (push) Canceled after 0s
Tests / test-postgres (>=4,<5) (push) Canceled after 0s
Tests / canary (push) Canceled after 0s
Consumers on postgres:NN-alpine can't switch to the Debian pgvector image
without reindexing every text index (musl vs glibc collation). This builds
pgvector into the same Alpine image instead. Verified on PG 16 and 17.
2026-09-30 00:36:41 -06:00
39 changed files with 4388 additions and 354 deletions
+2 -2
View File
@@ -6,8 +6,8 @@ FFB_PG_USER=ffb
FFB_PG_PASSWORD=change-me
FFB_PG_DB=ffb_test
# Published on 127.0.0.1 only. 55432 is commonly taken by other local stacks.
FFB_PG_PORT=55433
FFB_PG_PORT=55447
# Read by the test suite. Unset, the Postgres test cases skip.
# Each test gets its own database created on this server and dropped after.
FFB_TEST_DATABASE_URL=postgresql+asyncpg://ffb:change-me@127.0.0.1:55433/ffb_test
FFB_TEST_DATABASE_URL=postgresql+asyncpg://ffb:change-me@127.0.0.1:55447/ffb_test
+279 -1
View File
@@ -5,6 +5,284 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Calendar Versioning](https://calver.org/) (YYYY.MM.DD).
## [2026.10.05] - 2026-10-05
A closed middleware stays closed.
### Fixed
- After `aclose()`, a later `record_event` or tool call record rebuilt the
dispatcher's queue and background task and wrote to the sinks that had just
been closed. The dispatcher now stays closed: every later record, from the
loop or another thread, is dropped and counted in `dispatcher.dropped` and
the new `dispatcher.drop_reasons` under `"dispatcher closed"`, with a single
warning; it never binds a loop or starts a task again. `record_event`
returns False. Tool calls still run normally, and their records are not
kept in memory for feedback linking.
- `flush()` (and so `aclose()`) could return while a record put just after
`asyncio.Queue.join()` woke was still queued. A record a thread hands over
just before `aclose()` is now delivered if it lands while the flush runs,
or dropped and counted if it lands after; never written after close.
### Changed
- `aclose()` is idempotent: a second call does nothing, and sinks are closed
once.
## [2026.10.04] - 2026-10-04
Bounded payloads, and classifier keys that reach into nested results.
### Added
- `max_payload_bytes` on `InstrumentationMiddleware` and `instrument()`,
64 KiB by default, `None` for no limit. Arguments and results stored in
mode `full`, and every event's `attrs` (including `record_llm_call`'s),
are bounded by it. A larger value is stored as
`{"_truncated": true, "size": <bytes>, "preview": "<start of its JSON>"}`;
the preview is 1000 characters (a quarter of the limit when smaller) and
is taken after redaction. Call payloads reuse the size already measured
for `args_size` and `result_size`, so the limit adds no encoding; the
preview encodes only the start of the value. In `attrs`, only the largest
values are replaced, so the dict keeps its keys. `EmbeddingSink` embeds a
marker's preview. `fastmcp_feedback.instrumentation.payload` holds the
marker helpers (`TRUNCATED`, `is_truncated`).
- `message_keys` on `soft_error_classifier` (default `error`, `message`,
`detail`, `reason`, as before).
- Dotted paths in `message_keys` and `error_keys`: `"snapshot.error"` reads
`result["snapshot"]["error"]`. A key that literally contains a dot is
matched first; a missing step or a non-dict step counts as absent and
never raises. Default outputs are unchanged.
### Changed
- Stored payloads and event attrs over 64 KiB become markers. Earlier
releases stored them whole; `max_payload_bytes=None` keeps that.
## [2026.10.02.1] - 2026-10-02
The identity resolver's second run is opt-in again for any resolver that
answers.
### Changed
- The second resolver run after the tool now happens only when the first
answer was empty (no keys, only None values, or a run that failed or timed
out) or lacks a key in `late_identity_keys`. In 2026.10.02 a missing
`user_sub`, `caller_kind` or `client_id` also triggered it, so a resolver
that never returns one of them (mcspeak's returns no `client_id`) ran twice
on every kept call. `late_identity_keys` stays empty by default: a resolver
that answers anything runs once per call unless the server opts in.
blender-mcp's registration fix still needs
`late_identity_keys=["bus_client_uuid"]`, as in 2026.10.02.
## [2026.10.02] - 2026-10-02
A tool that creates its caller's identity records it on its own row again.
### Fixed
- Regression in 2026.10.01.4: with the identity resolver moved before the
tool, a call whose tool creates the identity the resolver looks up (the
call that registers a client, or logs in) recorded its own row without
it. blender-mcp's `register_client` lost `bus_client_uuid`, `bus_role` and
`parent_uuid` from its row's `identity` this way. When the first answer is
incomplete, missing `user_sub`, `caller_kind` or `client_id`, or any key in
the new `late_identity_keys`, the resolver now runs a second time after the
tool, concurrently with the enricher, for calls that are kept (not for
sampled-out ones). Its values fill the row's gaps and replace `None`; on
conflict the first answer wins, so the row matches the events recorded
during the call, which are not rewritten. Both runs keep `hook_timeout`,
redaction and never raising. A complete first answer still means one run.
### Added
- `late_identity_keys` on `InstrumentationMiddleware` and `instrument()`:
identity keys that may only exist once the tool has run. A resolver that
already returns all three columns before such a tool (blender-mcp's does)
needs it to get the second run, e.g.
`late_identity_keys=["bus_client_uuid"]`.
## [2026.10.01.4] - 2026-10-01
Statistics speak the same enum values as the rest of the API, events carry
the caller's identity, and importing the package no longer configures
logging.
### Changed
- `get_feedback_statistics` and `get_database_stats` key `by_type` and
`by_status` by enum value (`bug`, `in_progress`), the spelling
`list_feedback` returns and accepts as a filter. They grouped the raw
columns, which hold the enum NAMES on both SQLite (VARCHAR) and Postgres
(native enum types `feedbacktype`, `feedbackstatus`), so they returned
`BUG`, `IN_PROGRESS`. **Code reading the uppercase keys must switch to
lowercase.** Nothing stored changes. A row holding a value instead of a
name is counted under the same key; an unknown label is kept as-is.
- The identity resolver runs once per call **before** the tool, instead of
after it. The call's record reuses the result (still redacted, still
limited by `hook_timeout`, still never raising). An async resolver can
therefore delay the tool by up to `hook_timeout`, and it now also runs for
ok calls that sampling drops afterwards. It does not run in mode `off` or
for `exclude_tools`. The enricher still runs after the call.
- `PACKAGE_INFO` is read from the installed distribution's metadata. It
listed only the `postgresql` and `dev` extras and a classifier pyproject
did not have.
- psycopg2-binary is in the `dev` and `test` extras, so `make test-pg`
covers the feedback tools on Postgres too.
### Fixed
- `record_event` (and `record_llm_call`) inside a tool call never set
`user_sub`, `caller_kind` or `client_id`, because identity was resolved
only after the call. Inside a call they now come from the call's resolved
identity, matching its row; `client_id` falls back to the FastMCP context's
as for calls. Values passed explicitly win, and `record_event` takes
`caller_kind=` and `client_id=` as well as `user_sub=`. Outside a call,
including a task the call started that records after it returned, nothing
is filled in.
- Importing `fastmcp_feedback` called `logging.basicConfig` at INFO when the
root logger had no handlers, so every host process (the backfill CLI
included) printed an "initialized" line and had its root logger set up by
a library. The package now only adds a `NullHandler` to the
`fastmcp_feedback` logger and logs the version at DEBUG. **Servers that
relied on the package's INFO or WARNING lines must configure logging
themselves**; `fastmcp_feedback.setup_logging()` restores the old
behavior in one line.
- The backfill CLI logs a summary of its counts at INFO, so `-v` shows
something on a dry run.
### Documentation
- README: statistics keys, identity resolver timing, a Logging section, the
two Postgres driver paths (`[postgresql]` for the feedback tools,
`[instrumentation]` plus asyncpg for the sinks), MySQL marked untested,
and the measured coverage (95% lines and branches with the Postgres cases
enabled) in place of "98%".
- CONTRIBUTING: the example test, the mixin recipe and the commands now
match the package and were run; `make test`, `make test-pg` and
`make lint` replace commands that did not pass on this repository.
## [2026.10.01.3] - 2026-10-01
### Documentation
- The documentation site is public at https://fastmcp-feedback.supported.systems,
rewritten against this package with every example executed. The README
links and the PyPI "Documentation" URL point there instead of the private
dev address.
## [2026.10.01.2] - 2026-10-01
The instrumentation imports on a base install, without the database extras.
### Fixed
- `import fastmcp_feedback.instrumentation` failed on `pip install
fastmcp-feedback` without the `instrumentation` extra: the package imported
`storage.py` eagerly, which imports `sqlalchemy.ext.asyncio`, and SQLAlchemy
2.1 raises `ImportError` there when `greenlet` is missing. `instrument(app)`
with `JsonLinesSink`, `CallbackSink` or `MemorySink`, events, redaction and
result classification now need only the base dependencies.
- `DatabaseSink`, `EmbeddingVector`, `build_metadata`, `Embedder`,
`EmbeddingSink`, `EmbeddingText`, `OpenAIEmbedder`, `extract_texts`,
`backfill_embeddings` and `backfill_feedback` load on first access. Without
the extra, accessing one raises `ImportError` naming the missing package and
the extra to install (`[instrumentation]`, or `[instrumentation,embeddings]`
for the embedding names), chained to the original error. Every access
raises, not only the first, although SQLAlchemy itself reports a missing
`greenlet` only once. Import paths are unchanged.
- The middleware finds a `DatabaseSink` or `EmbeddingSink` among its sinks
without importing their modules, so `database_sink`, `embedding_sink`,
`capture_recent_calls`, `link_feedback`, `feedback_context`, `events_for`,
`similar` and `similar_feedback` return their usual empty results on a base
install.
- `DEFAULT_TABLE_PREFIX`, `DEFAULT_EMBEDDING_DIM`, `EMBEDDING_SOURCES` and
`DEFAULT_EVENT_TEXT_KEYS` moved to `instrumentation/constants.py` and import
without the extras; `storage` and `embeddings` still export them.
## [2026.10.01.1] - 2026-10-01
A result that says it succeeded is no longer flagged for its own error string.
### Changed
- Explicit success flag wins over error keys: when any of the classifier's
`flag_keys` (`ok`, `success`) is exactly `True`, the `error_keys` check is
skipped and the string is treated as a warning, so
`{"ok": True, "error": "deprecated arg"}` is recorded as `ok` again. Only
that check is skipped: a failing status (`{"success": True, "status":
"failed"}`), any flag key exactly `False` and `isError` still flag the
result. Truthy values that are not `True`, such as `1` or `"yes"`, are not
an explicit success. Results with no flag key are classified as in
2026.10.01.
## [2026.10.01] - 2026-10-01
Results that report failure only as an `error` string now count as soft
errors, and the docs cover joining events to calls.
### Changed
- `soft_error_classifier(..., error_keys=("error",))`: a dict result whose
value at any of `error_keys` is a non-empty string (after stripping) is a
soft error, with message `<key>: <string>`, redacted and truncated as
before. Checked after the status, flag and `isError` checks, so results
those already flag keep their message and are counted once. None, empty or
blank strings, numbers, booleans, dicts and lists do not count, so
`{"error": None}` and `{"error": 0.02}` stay ok. `error_keys=()` turns it
off. `DEFAULT_RESULT_CLASSIFIER` has it on.
- **Behavior change:** results such as `{"played": False, "error": "..."}`,
`{"error": "..."}` and `{"status": "unknown_target", "error": "..."}` were
recorded as `ok` and are now `soft_error`, so `NOT ok` and soft-error
counts can rise after upgrading. This includes the package's own
`list_feedback`, `get_feedback_statistics` and analytics tools, whose
failure results carry an `error` string and no flag key. Pass
`result_classifier=soft_error_classifier(error_keys=())` for the old
behavior.
### Docs
- `record_event` / `events_for`: a `key` used for joining must be unique for
at least the retention period and across restarts; a per-boot counter
collides. Use a UUID or a startup token plus a counter.
- The join from a call to its events through an enricher-written `extra`
field, for SQLite (`json_extract(c.extra, '$.speech_id') = e.key`) and
Postgres (`c.extra->>'speech_id' = e.key`).
- Schema notes: `ffb_tool_calls` has primary key `id` and no `call_id`
column; `ffb_events.call_id` holds a call's `id`. Ids are UUIDs, so order
events by `occurred_at`.
- The soft-error section documents `error_keys`, its default and the
upgrade note.
## [2026.09.30.1] - 2026-09-30
Embed what was stored before embedding was switched on, and embed list and
dict attrs as text rather than as Python reprs.
### Added
- `await backfill_embeddings(embedding_sink, *, sources=None, since=None,
until=None, limit=None, batch_size=500, dry_run=False)`. Reads the
`DatabaseSink`'s tables for calls with outcome `error` or `soft_error`
(and, for rows from before the `outcome` column, outcome NULL with an
`error_message`), `feedback.submitted` events, `llm.call` events with
prompt or completion, and other events with any of the sink's
`event_text_keys`. Each record goes through the live sink's extraction,
redactor and `max_chars`, so text, `text_hash` and `source_id` match and
later records dedupe against backfilled rows. Idempotent: a second run
embeds nothing. Keyset-paged by timestamp and id, `batch_size` rows per
query; embeds in chunks of the sink's `batch_size`. `limit` embeds at most
that many texts, oldest first. Returns counts per source type plus
`skipped` and `errors`; embed failures are counted, never raised. Raises
only for bad arguments or a missing table. `sources` defaults to the
sink's.
- `await backfill_feedback(embedding_sink, items, *, batch_size=500,
dry_run=False)` for feedback in the host's own tables, as
`(ref, title, description)` tuples; the same text a `feedback.submitted`
event with that ref gives.
- `python -m fastmcp_feedback.instrumentation.backfill --database-url ...
--embed-base-url ... [--model --dim --prefix --sources --since --until
--limit --batch-size --max-chars --event-text-keys --create-tables
--dry-run]`. The API key comes from `FFB_EMBED_API_KEY` or the variable
named by `--api-key-env`. Prints counts as JSON; exits 1 when any text
failed, 2 on a setup error.
### Changed
- **Embedded text of list, tuple and dict values.** Lists and tuples are
embedded one cleaned, non-empty item per line instead of as their repr
(`"['a', 'b']"`); dicts as `key: value` lines, sorted by key because
PostgreSQL JSONB does not keep key order and a backfill must rebuild the
same text. Nested values are flattened the same way, to a depth of 32 and
about 64k characters before `max_chars` applies. This applies to event
text keys, feedback title and description, and LLM prompt and completion.
Existing rows embedded from such values keep their old text; a backfill
run replaces them, since their text changed.
## [2026.09.30] - 2026-09-30
Embed feedback, tool errors, event text and (opt-in) LLM text, and search
@@ -356,7 +634,7 @@ FastMCP 4 support, and a composition bug that affected every earlier release.
- `create_feedback_server()` for server composition
- Comprehensive test suite (97%+ coverage)
- Full type hints and Pydantic validation
- Documentation site at https://fastmcp-feedback.l.supported.systems
- Documentation site (on a private development address at the time)
### Technical Details
- Built on FastMCP 2.12.2+
+120 -74
View File
@@ -1,6 +1,6 @@
# Contributing to FastMCP Feedback
Thank you for your interest in contributing! This guide will help you get started.
How to set up, test and submit changes.
## Development Setup
@@ -13,17 +13,16 @@ Thank you for your interest in contributing! This guide will help you get starte
### Getting Started
```bash
# 1. Fork the repository on git.supported.systems
# 1. Fork fastmcp-feedback/fastmcp-feedback on git.supported.systems
# 2. Clone your fork
git clone git@git.supported.systems:YOUR_USERNAME/fastmcp-feedback.git
cd fastmcp-feedback
# 3. Create virtual environment and install dependencies
uv venv
uv pip install -e ".[dev]"
# 3. Create .venv from uv.lock with the dev extra
uv sync --extra dev
# 4. Verify setup
uv run pytest
# 4. Verify setup (SQLite only; Postgres cases skip)
make test
```
### Development Workflow
@@ -35,17 +34,14 @@ git checkout -b feature/your-feature-name
# Make your changes...
# Run tests
uv run pytest
make test
# Run linting
uv run ruff check .
uv run ruff format .
# Run type checking (optional but appreciated)
uv run mypy src/fastmcp_feedback
make lint
uv run ruff format --check src/
# Commit with clear message
git commit -m "feat: Add support for custom validators"
git commit -m "Count feedback by submitter in get_feedback_statistics"
# Push and create PR
git push origin feature/your-feature-name
@@ -56,19 +52,21 @@ git push origin feature/your-feature-name
We use [Ruff](https://docs.astral.sh/ruff/) for linting and formatting:
```bash
# Check for issues
uv run ruff check .
# Lint src/, examples/ and the test files that are kept ruff-clean
make lint
# Auto-fix issues
uv run ruff check --fix .
# Format code
uv run ruff format .
# Format what you changed
uv run ruff format src/ path/to/your_test.py
```
Many of the older test files predate the ruff configuration and do not pass
it, so `ruff check .` over the whole repository reports errors that are not
yours. `src/` must stay clean, and new test files should be added to the
`lint` target in the Makefile.
### Style Guidelines
- **Line length**: 88 characters (Black default)
- **Line length**: 88 characters
- **Imports**: Sorted with isort (via Ruff)
- **Type hints**: Required for public APIs
- **Docstrings**: Google style for public functions
@@ -78,11 +76,11 @@ uv run ruff format .
### Running Tests
```bash
# Run all tests
# Run all tests (coverage is on by default, see pyproject.toml)
uv run pytest
# Run with coverage
uv run pytest --cov
# Without the coverage report, as the Makefile does
uv run pytest -q --no-cov
# Run specific test file
uv run pytest tests/unit/test_tools.py
@@ -90,11 +88,8 @@ uv run pytest tests/unit/test_tools.py
# Run specific test
uv run pytest -k "test_submit_feedback"
# Run only unit tests
uv run pytest -m unit
# Run only integration tests
uv run pytest -m integration
# Run one directory
uv run pytest tests/integration
```
### Running the tests against Postgres
@@ -113,12 +108,15 @@ make pg-down # stop it; the data volume is kept (pg-reset deletes it)
```
The server listens on `127.0.0.1` only, on `FFB_PG_PORT` from `.env`
(55433 by default; change it if that port is taken). `make pg-psql` opens a
(55447 by default; change it if that port is taken). `make pg-psql` opens a
shell on it. Every Postgres test gets its own freshly created database with
the `vector` extension, dropped when the test ends, so tests never see each
other's rows. To write a test that runs on both backends, take the `db_url`
fixture from `tests/conftest.py`: `db_url("name")` returns an async URL for
an empty database, the same one for the same name within a test.
an empty database, the same one for the same name within a test. The
feedback tools use a synchronous engine, so convert that URL's driver
(`sqlite`, `postgresql+psycopg2`) as the `feedback_db` fixture in
`tests/unit/test_consistency.py` does; psycopg2 comes with the dev extra.
To point the suite at another server, set the variable yourself; the user
needs permission to create and drop databases:
@@ -148,44 +146,60 @@ FFB_TEST_EMBED_URL=https://api.example.com/v1 FFB_TEST_EMBED_KEY=... \
- Use descriptive test names: `test_submit_feedback_with_invalid_type_raises_error`
- Use fixtures from `tests/conftest.py`
Example test:
Tests are async without a marker (`asyncio_mode = "auto"`). Call tools the
way a client does, through `fastmcp.Client`. `submit_feedback` takes one
argument, `request`, holding the fields of `SubmitFeedbackRequest`; `type`
is one of `bug`, `feature`, `improvement` or `question`. Tools report
failure in the result (`success: False` and an `error`) rather than raising.
```python
import pytest
from fastmcp import Client, FastMCP
from fastmcp_feedback import add_feedback_tools
@pytest.mark.unit
async def test_submit_feedback_creates_record(app_with_feedback, db_session):
"""Submitting feedback should create a database record."""
result = await app_with_feedback.call_tool(
"submit_feedback",
type="bug_report",
title="Test Bug",
description="Description"
)
assert result["status"] == "success"
assert result["id"] is not None
async def test_submit_feedback_creates_record(tmp_path):
app = FastMCP("test")
add_feedback_tools(app, database_url=f"sqlite:///{tmp_path}/feedback.db")
async with Client(app) as client:
result = await client.call_tool(
"submit_feedback",
{
"request": {
"type": "bug",
"title": "Export fails",
"description": "Exporting a scene with no objects raises",
"submitter": "test",
}
},
)
listed = await client.call_tool("list_feedback", {})
stats = await client.call_tool("get_feedback_statistics", {})
assert result.structured_content["success"] is True
[item] = listed.structured_content["feedback"]
assert item["id"] == result.structured_content["feedback_id"]
assert (item["type"], item["status"]) == ("bug", "open")
assert stats.structured_content["by_type"] == {"bug": 1}
```
## Pull Request Guidelines
### Before Submitting
1. **Tests pass**: `uv run pytest`
2. **Linting passes**: `uv run ruff check .`
1. **Tests pass**: `make test`, and `make test-pg` when you touched database code
2. **Linting passes**: `make lint`
3. **New features have tests**
4. **Documentation updated** (if applicable)
### PR Title Format
Use [Conventional Commits](https://www.conventionalcommits.org/):
Say what changes, in the imperative, as the commit log does:
- `feat: Add custom validation support`
- `fix: Handle empty feedback list correctly`
- `docs: Update installation instructions`
- `test: Add edge case tests for insights`
- `refactor: Simplify database connection logic`
- `Key feedback statistics by enum value`
- `Resolve caller identity before the tool runs`
- `Import instrumentation without the database extras`
### PR Description Template
@@ -215,16 +229,20 @@ fastmcp-feedback/
│ ├── models.py # Pydantic models and SQLAlchemy ORM
│ ├── database.py # Database connection management
│ ├── feedback.py # Standalone server factory
│ ├── insights.py # Privacy-compliant analytics
│ └── mixins/ # Modular tool mixins
│ ├── base.py
│ ├── submission.py
│ ├── retrieval.py
│ └── management.py
│ ├── insights.py # In-process usage metrics
│ ├── mixins/ # The feedback tools, grouped
│ │ ├── base.py # BaseFeedbackMixin
│ │ ├── submission.py
│ │ ├── retrieval.py
│ │ └── management.py
│ └── instrumentation/ # Middleware, sinks, events, embeddings, backfill
├── tests/
│ ├── conftest.py # Shared fixtures
│ ├── conftest.py # Shared fixtures, including db_backend and db_url
│ ├── _databases.py # Per-test Postgres databases
│ ├── unit/ # Unit tests
│ └── integration/ # Integration tests
├── docker-compose.yml # Postgres + pgvector for make test-pg
├── Makefile
├── pyproject.toml # Package configuration
├── README.md
├── CONTRIBUTING.md # This file
@@ -236,28 +254,56 @@ fastmcp-feedback/
### Adding a New Mixin
1. Create `src/fastmcp_feedback/mixins/your_mixin.py`
2. Inherit from `BaseMixin`
3. Implement `register_tools(app, prefix="")`
4. Export from `src/fastmcp_feedback/mixins/__init__.py`
5. Add to main `__init__.py` exports
6. Write tests in `tests/unit/test_your_mixin.py`
7. Update documentation
A mixin subclasses `BaseFeedbackMixin`, which holds the `FeedbackDatabase`
(`self.database`) and the `FeedbackInsights` (`self.insights`). Its tools are
methods decorated with `mcp_tool` from `fastmcp.contrib.mcp_mixin`;
`register_tools(app, prefix=...)`, inherited from FastMCP's `MCPMixin`, adds
them to a server.
```python
from fastmcp import FastMCP
from fastmcp_feedback import FeedbackDatabase
from fastmcp_feedback.mixins.base import BaseFeedbackMixin, Feedback, mcp_tool
class CountMixin(BaseFeedbackMixin):
@mcp_tool(description="Count feedback items")
async def count_feedback(self) -> dict:
async with self.database.session() as session:
return {"count": session.query(Feedback).count()}
app = FastMCP("demo")
CountMixin(FeedbackDatabase("sqlite:///:memory:")).register_tools(app, prefix="admin")
# The tool is registered as admin_count_feedback.
```
Then:
1. Put it in `src/fastmcp_feedback/mixins/your_mixin.py`
2. Export it from `src/fastmcp_feedback/mixins/__init__.py` and the package `__init__.py`
3. Write tests in `tests/unit/`, calling the tool through `fastmcp.Client`
4. Document it in README.md and on the docs site
### Adding a New Tool
1. Add to appropriate mixin or create new one
2. Use `@app.tool()` decorator with clear description
3. Add Pydantic models for request/response if complex
4. Write unit tests
5. Add integration test
6. Document in README
1. Add a method to the mixin it belongs to, decorated with `mcp_tool` and a
description written for the model that will call it
2. Return a dict; report failures as `{"success": False, "error": ...}`
instead of raising
3. Return enum fields as their values (`bug`, `in_progress`), as
`feedback_to_dict` does; the database stores the names
4. Use a Pydantic model for structured input
5. Test it on both backends where it touches the database (`make test-pg`)
## Versioning
We use **calendar versioning** (CalVer): `YYYY.MM.DD`
Version bumps happen automatically on release. Contributors don't need to update version numbers.
A release bumps the version in `pyproject.toml` and
`src/fastmcp_feedback/__init__.py` and adds a CHANGELOG entry. Contributors
don't need to update version numbers.
## Getting Help
+4 -1
View File
@@ -60,4 +60,7 @@ test-all: test test-pg
lint:
uv run --no-sync ruff check src/ examples/ tests/_compat.py tests/_databases.py \
tests/unit/test_test_databases.py tests/unit/test_instrumentation.py \
tests/unit/test_embeddings.py
tests/unit/test_embeddings.py tests/unit/test_backfill.py \
tests/unit/test_consistency.py tests/unit/test_late_identity.py \
tests/unit/test_payload_cap.py tests/unit/test_classifier_paths.py \
tests/unit/test_closed_dispatcher.py
+343 -28
View File
@@ -10,10 +10,10 @@ Record every tool call your server handles, and let people and models report pro
- **Secret Redaction** - Tokens and passwords removed from recorded arguments, results and errors by default
- **One-Line Integration** - Add feedback tools to any FastMCP server instantly
- **Modular Mixins** - Selective tool registration for fine-grained control
- **Multi-Database Support** - SQLite (dev), PostgreSQL, MySQL (production)
- **SQLite and PostgreSQL** - Both covered by the test suite. A `mysql` extra exists but MySQL is untested
- **Privacy-Compliant Analytics** - Optional usage insights without sensitive data
- **Type-Safe** - Full Pydantic validation and type hints
- **Well-Tested** - 98% coverage, run against FastMCP 2.12, 2.14, 3.x and 4.x
- **Tested** - 95% line and branch coverage with the Postgres cases enabled (measured for 2026.10.01.4), run against FastMCP 2.12, 2.14, 3.x and 4.x
## Installation
@@ -26,10 +26,31 @@ uv add fastmcp-feedback
# Using pip
pip install fastmcp-feedback
# With PostgreSQL support
# Feedback tools on PostgreSQL (psycopg2, synchronous)
uv add "fastmcp-feedback[postgresql]"
# Instrumentation's DatabaseSink on PostgreSQL (async)
uv add "fastmcp-feedback[instrumentation]" asyncpg
```
What each install covers:
| Install | Adds | Covers |
|---------|------|--------|
| `fastmcp-feedback` | FastMCP, SQLAlchemy, Pydantic | The feedback tools (sync SQLAlchemy, no greenlet), and `instrument(app)` with `JsonLinesSink`, `CallbackSink` and `MemorySink`, events, redaction and result classification |
| `fastmcp-feedback[instrumentation]` | `sqlalchemy[asyncio]` (greenlet), `aiosqlite` | `DatabaseSink`, `build_metadata`, call history for linking feedback (`capture_recent_calls` across workers, `link_feedback`, `feedback_context`) |
| `fastmcp-feedback[instrumentation,embeddings]` | `pgvector` | `EmbeddingSink`, `OpenAIEmbedder`, `similar()`, the backfill helpers; pgvector is only needed on PostgreSQL |
The two database paths use different drivers. The feedback tools run a
synchronous engine, so on Postgres they need `[postgresql]` (psycopg2) and a
`postgresql://` URL. `DatabaseSink` and `EmbeddingSink` run an async engine,
so on Postgres they need `[instrumentation]` plus `asyncpg` and a
`postgresql+asyncpg://` URL. A server that does both installs both. Without the
`instrumentation` extra, `from fastmcp_feedback.instrumentation import
DatabaseSink` raises an `ImportError` that names the missing package and the
extra to install, and the middleware's database helpers return their empty
defaults (`[]`, `False`).
### From Git (Latest Development)
```bash
@@ -82,13 +103,37 @@ add_feedback_tools(app, database_url="sqlite:///feedback.db")
Pass a `database_url`. Without one the tools use an in-memory SQLite
database, which is handy for tests but loses everything when the process exits.
`get_feedback_statistics` keys its counts with the same lowercase values
`list_feedback` returns, so a key can be passed straight back as a filter:
```python
import asyncio
from fastmcp_feedback import FeedbackDatabase, RetrievalMixin, SubmissionMixin
from fastmcp_feedback import SubmitFeedbackRequest
async def main():
db = FeedbackDatabase("sqlite:///:memory:")
await db.initialize()
await SubmissionMixin(db).submit_feedback(SubmitFeedbackRequest(
type="bug", title="Export fails", description="...", submitter="cli"))
stats = await RetrievalMixin(db).get_feedback_statistics()
print(stats["by_type"], stats["by_status"]) # {'bug': 1} {'open': 1}
asyncio.run(main())
```
Before 2026.10.01.4 these keys were the uppercase names the database stores
(`BUG`, `OPEN`). Code that reads them needs the lowercase keys now.
### PostgreSQL
```python
add_feedback_tools(app, database_url="postgresql://user:pass@db.example.com/feedback")
```
Install the driver with `uv add "fastmcp-feedback[postgresql]"`.
Install the driver with `uv add "fastmcp-feedback[postgresql]"`. This is
psycopg2 for the feedback tools' synchronous engine; the instrumentation
sinks use asyncpg instead (see [Installation](#installation)).
### Prefixed Tool Names
@@ -149,6 +194,42 @@ Set it with `mode=` or the `FEEDBACK_INSTRUMENTATION_MODE` environment variable.
`meta_only_tools={"run_code", "create_token"}` keeps chosen tools at `meta`
even in `full` mode.
### Payload size limit
`max_payload_bytes` (64 KiB by default, `None` for no limit) bounds what a
row stores: the arguments and result in `full` mode, and every event's
`attrs`. A larger value is stored as a marker instead:
```json
{"_truncated": true, "size": 812345, "preview": "{\"frames\": [[0.1, 0.2], ..."}
```
`size` is the value's JSON size in bytes, the same measurement as
`args_size` and `result_size`, which the limit reuses, so it costs no second
encoding. `preview` is the start of the JSON text, 1000 characters (a quarter
of the limit when that is smaller), taken after redaction, so a secret cannot
survive in it. For event `attrs` only the largest values are replaced, until
the whole fits, so small fields such as `ok` keep their place:
```python
import asyncio
from fastmcp_feedback.instrumentation import InstrumentationMiddleware, MemorySink
async def main():
sink = MemorySink()
mw = InstrumentationMiddleware([sink])
mw.record_event("job.finished", key="j1", attrs={"ok": False, "error": "E" * 100_000})
await mw.aclose()
attrs = sink.events[0].attrs
print(attrs["ok"], attrs["error"]["_truncated"], attrs["error"]["size"]) # False True 100002
asyncio.run(main())
```
**Upgrading to 2026.10.04:** before this release payloads and attrs were
stored whole. A row that used to hold more than 64 KiB there now holds the
marker; pass `max_payload_bytes=None` to keep storing everything.
### Redaction
Secrets are removed from arguments, results, error messages and hook output
@@ -181,8 +262,7 @@ instrument(app, [
`DatabaseSink` needs the `instrumentation` extra
(`uv add "fastmcp-feedback[instrumentation]"`), plus `asyncpg` for Postgres.
Install the extra even if you already have SQLAlchemy: SQLAlchemy 2.1 no longer
pulls in `greenlet` by itself, and its async engine cannot run without it.
The other sinks work on the base install (see [Installation](#installation)).
### Using your own database and migrations
@@ -224,6 +304,68 @@ The default classifier flags a dict result when `ok` or `success` is exactly
`False`, when `status` or `state` is `error`, `failed` or `failure` (any case),
or when the tool marked the result as an MCP error (`isError`) without raising.
The message comes from the first of `error`, `message`, `detail` and `reason`.
Failing those, a result whose `error` holds a non-empty string is a soft
error too, with that string as the message: `{"played": False, "error":
"audio output stalled"}` is recorded as `error: audio output stalled`. Only a
string with something other than whitespace counts, so `{"error": None}`,
`{"error": ""}` and a statistic such as `{"error": 0.02}` stay ok. A dict or
list there (`{"error": {"code": 1}}`) is not flagged either; catch that shape
with a flag key, a status or your own classifier. `error_keys` names the keys
to check, and `error_keys=()` turns the check off.
A result that says it succeeded wins over its own error string: when `ok` or
`success` (the `flag_keys`) is exactly `True`, the `error` check is skipped and
the string is read as a warning, so `{"ok": True, "error": "deprecated arg"}`
is recorded as `ok`. Only that check is skipped. In order of precedence:
1. A failing `status` or `state` flags the result, even next to `ok: True`,
since `{"success": True, "status": "failed"}` contradicts itself and the
status is the more specific of the two.
2. Any flag key exactly `False` flags it, even if another is `True`.
3. `isError` flags it.
4. A flag key exactly `True` makes it ok. Truthy values that are not `True`,
such as `1` or `"yes"`, do not count.
5. A non-empty string at an `error_keys` key flags it.
```python
from fastmcp_feedback.instrumentation import DEFAULT_RESULT_CLASSIFIER as classify
assert classify("t", {"ok": True, "error": "deprecated arg"}) is None
assert classify("t", {"ok": 1, "error": "x"}) == "error: x"
assert classify("t", {"success": True, "status": "failed"}) == "status=failed"
assert classify("t", {"played": False, "error": "stalled"}) == "error: stalled"
```
```python
from fastmcp_feedback.instrumentation import instrument, soft_error_classifier
# Tools that report {"failure_reason": "..."} instead
instrument(app, result_classifier=soft_error_classifier(error_keys=("failure_reason",)))
# As before 2026.10.01: an error string alone is not a soft error
instrument(app, result_classifier=soft_error_classifier(error_keys=()))
```
`message_keys` sets where the message comes from (`error`, `message`,
`detail`, `reason` by default), and both `message_keys` and `error_keys`
take dotted paths into nested dicts. A key that literally contains a dot is
tried first; a missing step, or one that is not a dict, counts as absent:
```python
from fastmcp_feedback.instrumentation import soft_error_classifier
# mcardupilot: a wait that returned because the simulator died, cause nested
classify = soft_error_classifier(
status_keys=("status", "state", "reason"),
extra_statuses=("died",),
message_keys=("snapshot.error", "error", "message", "detail", "reason"),
)
died = {"reason": "died", "snapshot": {"error": "SITL exited: segfault"}}
assert classify("wait_until", died) == "reason=died: SITL exited: segfault"
assert classify("wait_until", {"reason": "timeout", "snapshot": {}}) is None
```
Add your own status values, or pass any `(tool, result) -> str | None`:
```python
@@ -260,6 +402,19 @@ table made earlier by `create_tables=True` needs
tables are not altered. Until the column exists, database writes fail and
those records are dropped (logged).
**Upgrading to 2026.10.01:** the `error` key check is on by default, so a
result that carries a non-empty `error` string with no failing status or flag
(such as `{"status": "unknown_target", "error": "no object named Cube"}`, or
`{"error": "..."}` alone) is now a soft error where earlier releases recorded
it as `ok`. Counts of `NOT ok` and `outcome = 'soft_error'` can rise after
upgrading. Results that also have a failing status or flag keep their old
message, such as `success=False: ...`, and are counted once. Pass
`soft_error_classifier(error_keys=())` to keep the old behavior.
**Upgrading to 2026.10.01.1:** a result with `ok` or `success` exactly `True`
and an `error` string, which 2026.10.01 recorded as `soft_error`, is `ok` again.
Nothing else changes.
### Identity and enrichment hooks
```python
@@ -282,6 +437,44 @@ go in `identity`. Enricher output goes in `extra`, except `server_version`,
which has its own column. Async hooks that take longer than `hook_timeout`
(0.25 s by default) are skipped for that call.
The identity resolver runs **before** the tool. Events the tool records get
the same `user_sub`, `caller_kind` and `client_id` as the call's row. So keep
it cheap: an async resolver can hold the tool back by up to `hook_timeout`,
and it also runs for calls that sampling later drops (sampling cannot know
beforehand whether a call will fail). It does not run in mode `off` or for
`exclude_tools`. The enricher still runs after the call, since it receives
the result.
Some tools create the identity they run under: a registration that makes the
client record the resolver looks up, or a login. The resolver runs a second
time after the tool, alongside the enricher and only for calls that are kept,
when its first answer was empty (nothing, only None values, or a run that
failed or timed out) or lacks a key named in `late_identity_keys`. A missing
`user_sub`, `caller_kind` or `client_id` alone does not trigger it, and
`late_identity_keys` is empty by default, so a resolver that answers anything
runs once per call unless you opt in.
The call's own row takes the second answer's values where the first had
none; where both have a value, the first wins, so the row agrees with the
events recorded during the call. Those events are not rewritten. A complete
first answer means one run.
```python
registered: dict[str, str] = {} # session -> bus client, filled by a tool
def who(context):
out = {"user_sub": "user-1", "caller_kind": "addon", "client_id": "dcr-9"}
session = getattr(context.fastmcp_context, "session_id", None)
if session in registered:
out["bus_client_uuid"] = registered[session]
return out
instrument(app, identity_resolver=who, late_identity_keys=["bus_client_uuid"])
```
Without `late_identity_keys` this resolver's first answer is not empty, so
there is no second run and the call that registers the client is recorded
without `bus_client_uuid`.
### Linking feedback to the calls behind it
When a model reports "the render tool hung", the report is far more useful with
@@ -435,23 +628,31 @@ async def submit(case: str) -> dict:
mw.record_event("job.finished", key="job-wing", attrs={"duration_s": 3512, "converged": True})
```
`record_event(kind, *, key=None, attrs=None, user_sub=None, call_id=None)`
returns True when the event was queued and False when it was not. It never
`record_event(kind, *, key=None, attrs=None, user_sub=None, call_id=None,
caller_kind=None, client_id=None)` returns True when the event was queued and False when it was not. It never
raises and never blocks, and it takes the same path as call records: the same
background queue, the same drop-and-count policy when the queue is full.
- `kind` is a dotted name (`job.submitted`, `llm.call`), a non-empty string of
at most 128 characters.
- `key` is what you join on, such as a job id. It is stored as a string, at
most 255 characters.
most 255 characters. A key used to join events to calls or to each other
must be unique for at least the retention period, **including across
restarts**. A per-boot counter (`speech-1`, `speech-2`, ...) starts over
on every restart, and the second boot's `speech-1` joins to the first
boot's events. Use a UUID, or a token chosen at startup plus a counter
(`speech-<boot token>-<n>`).
- `attrs` is any JSON-like dict. It is redacted like call arguments, then made
JSON-safe: `NaN` and infinities become the strings `"nan"`/`"inf"` (Postgres
JSONB rejects them), datetimes become ISO strings, anything else unknown
becomes `str(value)`.
- **Inside a tool call**, the event gets that call's id as `call_id` and its
session, so it joins to the call's row in `ffb_tool_calls`. Outside a call,
`call_id` is None. A task started by a tool that records after the tool has
returned counts as outside. Pass `call_id=` to set it yourself.
- **Inside a tool call**, the event gets that call's id as `call_id`, its
session, and the `user_sub`, `caller_kind` and `client_id` the identity
resolver returned for the call, so it joins to the call's row in
`ffb_tool_calls`. Outside a call, those are None unless you pass them. A
task started by a tool that records after the tool has returned counts as
outside. Any of `call_id=`, `user_sub=`, `caller_kind=` and `client_id=`
passed explicitly wins over the call's value.
- **From any thread.** Called off the server's event loop (a watcher thread, a
sync tool running in a worker thread), the event is handed to the loop with
`call_soon_threadsafe`. The loop is known once the server has handled a tool
@@ -468,6 +669,23 @@ for ev in await mw.events_for("job-wing"):
steps = await mw.events_for("job-wing", kinds=["step.started", "step.finished"], limit=100)
```
To join a call to its events in SQL, have the enricher write the key into
`extra` and match on it. This is how mcspeak reads the phases of one `speak`
call, with SQLite:
```sql
SELECT e.occurred_at, e.attrs FROM ffb_events e
JOIN ffb_tool_calls c ON json_extract(c.extra, '$.speech_id') = e.key
WHERE e.kind = 'speech.phase' AND c.id = ?
ORDER BY e.occurred_at;
```
On Postgres the join condition is `c.extra->>'speech_id' = e.key`. Events
recorded inside the call also carry its id, so `e.call_id = c.id` joins those
without a key. `ffb_tool_calls` has no `call_id` column; its primary key is
`id`, and `ffb_events.call_id` holds that value. Order events by
`occurred_at`: ids are random UUIDs and do not sort in recording order.
`events_for` reads the `DatabaseSink` and adds events still waiting in the
queue, so an event is visible as soon as `record_event` returns. With more
than `limit` (1000) matches it returns the newest `limit`. Without a
@@ -533,8 +751,10 @@ With the default `capture_llm_text=False` both are discarded, and so are
Events go to a `ffb_events` table: `id`, `occurred_at`, `kind`, `key`,
`call_id`, `session_id`, `user_sub`, `caller_kind`, `client_id`,
`server_version`, `attrs` (JSON; JSONB on Postgres), indexed on
`occurred_at`, `kind`, `key`, `call_id` and `user_sub`. It has no foreign key
to `ffb_tool_calls`, since calls are pruned and may be written later.
`occurred_at`, `kind`, `key`, `call_id` and `user_sub`. `id` is a UUID
string in both tables, so sort by `occurred_at` (events) or `started_at`
(calls), never by `id`. `call_id` refers to `ffb_tool_calls.id` but has no
foreign key, since calls are pruned and may be written later.
Sinks get calls and events in the same batches. `DatabaseSink` writes each to
its table, calls first in their own transaction, so a database without
@@ -570,7 +790,9 @@ you can search by meaning. It embeds four kinds of text:
| `llm` | `"prompt: ...\ncompletion: ..."` of an `llm.call` event, when `capture_llm_text=True` | the event id |
An `llm.call` without prompt or completion counts as a plain event, so a
failed LLM call's `error` is still embedded.
failed LLM call's `error` is still embedded. A list or tuple value is
embedded one item per line, a dict as `key: value` lines sorted by key (at
any depth), rather than as its Python repr.
It needs a `DatabaseSink` to store into and an embedder. `OpenAIEmbedder`
talks to any OpenAI-compatible `/v1/embeddings` endpoint (OpenAI, Ollama,
@@ -707,12 +929,105 @@ Wrap it in `op.execute(...)` in the migration's `upgrade()`, and
renders the column as `fastmcp_feedback.instrumentation.storage.EmbeddingVector(dim=1024)`,
which ties the migration to this package.
#### Backfilling existing data
The sink embeds records as they arrive. Calls, events and feedback stored
before it was running can be embedded afterwards with `backfill_embeddings`,
which reads them back from the `DatabaseSink`'s tables:
```python
from fastmcp_feedback.instrumentation import backfill_embeddings
counts = await backfill_embeddings(mw.embedding_sink)
# {"feedback": 4, "call_error": 90, "event": 12, "llm": 0, "skipped": 0, "errors": 0}
```
It picks up the same four kinds of text as the live sink: calls with outcome
`error` or `soft_error`, `feedback.submitted` events, `llm.call` events with
a prompt or completion, and other events with any of the sink's
`event_text_keys`. Calls recorded before the `outcome` column existed
(2026.09.27.4) have it NULL; those with an `error_message` count as errors.
Each text goes through the sink's own extraction, redactor and `max_chars`,
so it comes out with the same text, `text_hash` and `source_id` the live sink
would give that record. That makes it idempotent: a second run embeds nothing
and reports the rows as `skipped`, and records arriving later dedupe against
what was backfilled. When the middleware has a custom `Redactor`, build the
sink with it (`EmbeddingSink(..., redactor=mw.redactor)`), or use
`mw.embedding_sink`, which already has it.
`sources=` narrows it (default: the sink's sources), `since=` and `until=`
select rows by timestamp (naive means UTC), and `limit=` embeds at most that
many texts, oldest first; running it again carries on where it stopped. Rows
are read `batch_size` (500) at a time and embedded in chunks of the sink's
`batch_size`, so a large table never sits in memory. A failed chunk is
logged and counted in `errors`, never raised; only a missing table or bad
arguments raise. `dry_run=True` counts what would be embedded without calling
the embedder.
Feedback kept in your own tables rather than as events goes through
`backfill_feedback`, which makes the same text a `feedback.submitted` event
with that ref would:
```python
from fastmcp_feedback.instrumentation import backfill_feedback
rows = [("bug-7Q2X", "Render hangs", "Stuck at 99%"), ("bug-8K1D", "Crash", None)]
await backfill_feedback(mw.embedding_sink, rows) # (ref, title, description)
```
From a shell, the same over a database URL. The API key comes from
`FFB_EMBED_API_KEY` (or the variable `--api-key-env` names), never a flag:
```bash
export FFB_EMBED_API_KEY=...
python -m fastmcp_feedback.instrumentation.backfill \
--database-url postgresql+asyncpg://user:pass@db/app \
--embed-base-url https://api.example.com/v1 \
--model mxbai-embed-large --dim 1024 --prefix ffb_ \
--since 2026-09-01 --sources call_error,feedback --dry-run
```
It prints the counts as JSON and exits 1 if any text failed to embed, 2 on
a setup error. It logs warnings to stderr; `-v` adds INFO lines, including
a summary of the counts. `--create-tables` creates `ffb_embeddings` (and on PostgreSQL
the vector extension) when it is missing; without it a missing table is an
error. `--help` lists the rest (`--until`, `--limit`, `--batch-size`,
`--max-chars`, `--event-text-keys`).
### Middleware order and shutdown
FastMCP runs the first middleware added as the outermost. Call `instrument()`
before adding other middleware to include their time and record calls they
reject; call it last to time only the tool. On shutdown, `await mw.aclose()`
flushes queued records and closes the sinks.
flushes queued records and closes the sinks. After that the middleware
records nothing: tool calls still run, and a late `record_event` (from a
worker thread finishing up, say) returns False. Those records are dropped and
counted in `mw.dispatcher.drop_reasons["dispatcher closed"]`, never written,
so wait for your own threads before closing. Calling `aclose()` again does
nothing.
## Logging
The package configures no logging. Its loggers sit under `fastmcp_feedback`
with a `NullHandler`, so importing it prints nothing. The middleware never
raises into your tools; when a sink, hook or redactor fails it logs a
WARNING instead, and those warnings only appear once your server configures
logging:
```python
import logging
logging.basicConfig(level=logging.WARNING)
logging.getLogger("fastmcp_feedback").setLevel(logging.INFO) # optional: more detail
```
At INFO the package also reports pruning, table-creation retries and the
servers and tools it sets up. `fastmcp_feedback.setup_logging()` is a
one-line alternative: `basicConfig` at INFO with a timestamped format, then
the package version. Before
2026.10.01.4, importing the package called it automatically whenever the root
logger had no handlers, which also changed the logging of the host process.
## Advanced Usage
@@ -808,12 +1123,13 @@ Add all five feedback tools to a FastMCP server.
## Documentation
Full documentation: https://fastmcp-feedback.l.supported.systems
Full documentation: https://fastmcp-feedback.supported.systems
- [Quick Start Tutorial](https://fastmcp-feedback.l.supported.systems/tutorials/quickstart/)
- [Architecture](https://fastmcp-feedback.l.supported.systems/explanation/architecture/)
- [API Reference](https://fastmcp-feedback.l.supported.systems/reference/api/)
- [Integration Patterns](https://fastmcp-feedback.l.supported.systems/how-to/integration-patterns/)
- [Instrument a server in five minutes](https://fastmcp-feedback.supported.systems/tutorials/instrument-a-server/)
- [Architecture](https://fastmcp-feedback.supported.systems/explanation/architecture/)
- [Middleware reference](https://fastmcp-feedback.supported.systems/reference/middleware/)
- [Feedback linked to the calls behind it](https://fastmcp-feedback.supported.systems/tutorials/feedback-with-context/)
- [Embeddings and similarity search](https://fastmcp-feedback.supported.systems/how-to/embeddings/)
## Examples
@@ -832,7 +1148,7 @@ uv run python examples/simple_integration.py
## Contributing
We welcome contributions! FastMCP Feedback is designed to be contributor-friendly.
Contributions are welcome.
### Quick Contribution Setup
@@ -842,15 +1158,14 @@ git clone git@git.supported.systems:YOUR_USERNAME/fastmcp-feedback.git
cd fastmcp-feedback
# Install with dev dependencies
uv venv
uv pip install -e ".[dev]"
uv sync --extra dev
# Create feature branch
git checkout -b feature/your-feature
# Make changes, run tests
uv run pytest
uv run ruff check .
make test
make lint
# Submit PR
```
@@ -872,7 +1187,7 @@ MIT License - see [LICENSE](LICENSE) for details.
## Links
- **Documentation**: https://fastmcp-feedback.l.supported.systems
- **Documentation**: https://fastmcp-feedback.supported.systems
- **Repository**: https://git.supported.systems/fastmcp-feedback/fastmcp-feedback
- **Issues**: https://git.supported.systems/fastmcp-feedback/fastmcp-feedback/issues
- **FastMCP**: https://gofastmcp.com
+1 -1
View File
@@ -11,7 +11,7 @@ services:
POSTGRES_PASSWORD: ${FFB_PG_PASSWORD}
POSTGRES_DB: ${FFB_PG_DB}
ports:
- "127.0.0.1:${FFB_PG_PORT:-55433}:5432"
- "127.0.0.1:${FFB_PG_PORT:-55447}:5432"
volumes:
- ffb-postgres-data:/var/lib/postgresql/data
- ./docker/postgres-init:/docker-entrypoint-initdb.d:ro
+13
View File
@@ -0,0 +1,13 @@
# Postgres on Alpine with pgvector built in. Keeps the musl C library, so an
# existing data directory's collation and text indexes are unaffected.
# with_llvm=no: the base image's Postgres was built with a specific clang for
# JIT bitcode that Alpine may no longer ship; pgvector itself builds with gcc.
ARG PG_MAJOR=16
FROM postgres:${PG_MAJOR}-alpine
ARG PGVECTOR_VERSION=v0.8.6
RUN apk add --no-cache --virtual .build-deps git build-base \
&& git clone --depth 1 --branch ${PGVECTOR_VERSION} https://github.com/pgvector/pgvector.git /tmp/pgvector \
&& make -C /tmp/pgvector OPTFLAGS="" with_llvm=no \
&& make -C /tmp/pgvector install with_llvm=no \
&& rm -rf /tmp/pgvector \
&& apk del .build-deps
+28
View File
@@ -0,0 +1,28 @@
# Postgres on Alpine with pgvector
For stacks that already run `postgres:NN-alpine` and want the `vector`
extension for fastmcp-feedback embeddings.
Why not switch to the `pgvector/pgvector` image: it's Debian (glibc), and the
Alpine images use musl. The two C libraries sort text differently under
locales such as `en_US.utf8`, so reusing an Alpine data directory under a
Debian image changes the order of every text index. The indexes need a full
`REINDEX` or they silently return wrong results. Building pgvector into the
same Alpine image keeps the data directory's collation unchanged.
```bash
docker build --build-arg PG_MAJOR=16 -t postgres-pgvector:16-alpine .
docker build --build-arg PG_MAJOR=17 -t postgres-pgvector:17-alpine .
```
Then point the stack's Postgres service at the new image (same major version
as the existing data directory) and run `CREATE EXTENSION IF NOT EXISTS
vector;` once, or let a migration do it.
Tested 2026-09-30 with pgvector v0.8.6 on postgres:16-alpine and
postgres:17-alpine: the extension loads, and an HNSW `vector_cosine_ops`
index is used for `ORDER BY embedding <=> ...` queries.
`with_llvm=no` skips the LLVM bitcode used for JIT inlining. The base images'
Postgres was built with a clang version Alpine no longer ships, so the bitcode
step fails; pgvector itself still builds with gcc and works normally.
+5 -2
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "fastmcp-feedback"
version = "2026.09.30" # YYYYMMDD.patch versioning - update for each release
version = "2026.10.05" # YYYYMMDD.patch versioning - update for each release
description = "Production-ready feedback collection system for FastMCP servers"
readme = "README.md"
license = "MIT"
@@ -59,6 +59,8 @@ dev = [
"sqlalchemy[asyncio]>=2.0.43",
"aiosqlite>=0.22.1",
"asyncpg>=0.31.0",
# The feedback tools' synchronous engine on Postgres, for test-pg.
"psycopg2-binary>=2.9.0",
"pgvector>=0.5.0",
"pytest>=8.2.0",
"pytest-asyncio>=0.23.0",
@@ -72,6 +74,7 @@ test = [
"sqlalchemy[asyncio]>=2.0.43",
"aiosqlite>=0.22.1",
"asyncpg>=0.31.0",
"psycopg2-binary>=2.9.0",
"pgvector>=0.5.0",
"pytest>=8.2.0",
"pytest-asyncio>=0.23.0",
@@ -83,7 +86,7 @@ all = ["fastmcp-feedback[postgresql,mysql,dev]"]
[project.urls]
Homepage = "https://git.supported.systems/fastmcp-feedback/fastmcp-feedback"
Documentation = "https://fastmcp-feedback.l.supported.systems"
Documentation = "https://fastmcp-feedback.supported.systems"
Repository = "https://git.supported.systems/fastmcp-feedback/fastmcp-feedback.git"
Issues = "https://git.supported.systems/fastmcp-feedback/fastmcp-feedback/issues"
Changelog = "https://git.supported.systems/fastmcp-feedback/fastmcp-feedback/blob/main/CHANGELOG.md"
+64 -56
View File
@@ -1,20 +1,18 @@
"""FastMCP Feedback - Comprehensive feedback collection system for FastMCP servers.
"""FastMCP Feedback: feedback tools and call instrumentation for FastMCP servers.
This package provides a complete feedback collection and management system
that integrates seamlessly with FastMCP servers. It supports multiple
integration patterns, privacy-compliant analytics, and production-ready
database management.
Adds tools for submitting, listing and managing feedback (bug reports,
feature requests, improvements, questions) to a FastMCP server, stored in
SQLite or PostgreSQL through SQLAlchemy, with optional in-process usage
metrics. ``fastmcp_feedback.instrumentation`` records every tool call and
the events you report, so a feedback item can be traced back to the calls
that led to it.
Key Features:
- Modular mixin architecture for selective tool integration
- Privacy-compliant analytics with configurable data retention
- Support for SQLite (development) and PostgreSQL (production)
- Comprehensive error handling and logging
- Multiple server composition patterns
- Production-ready with connection pooling and health checks
The package does not configure logging. Its loggers live under
``fastmcp_feedback``; configure them in your server, or call
``setup_logging()`` for a basic stderr handler.
Quick Start:
Basic integration with existing FastMCP server:
Add the feedback tools to an existing server:
>>> from fastmcp import FastMCP
>>> from fastmcp_feedback import add_feedback_tools
@@ -22,19 +20,18 @@ Quick Start:
>>> app = FastMCP("My App")
>>> add_feedback_tools(app, database_url="sqlite:///feedback.db")
Create dedicated feedback server:
A dedicated feedback server:
>>> from fastmcp_feedback import create_feedback_server
>>>
>>> feedback_server = create_feedback_server("Feedback Service")
>>> # feedback_server now has all feedback tools ready
Selective tool integration using mixins:
Only some of the tools, through the mixins:
>>> from fastmcp_feedback import SubmissionMixin, RetrievalMixin
>>> from fastmcp_feedback import SubmissionMixin
>>> from fastmcp_feedback import FeedbackDatabase, FeedbackInsights
>>>
>>> db = FeedbackDatabase("postgresql://user:pass@host:5432/db")
>>> db = FeedbackDatabase("sqlite:///feedback.db")
>>> insights = FeedbackInsights(enabled=True, retention_days=90)
>>>
>>> submission = SubmissionMixin(db, insights)
@@ -44,7 +41,7 @@ Quick Start:
import logging
# Version information - CalVer YYYY.MM.DD
__version__ = "2026.09.30"
__version__ = "2026.10.05"
__author__ = "Ryan Malloy"
__email__ = "ryan@supported.systems"
@@ -142,38 +139,42 @@ from .tools import (
validate_tool_configuration,
)
# Package metadata
PACKAGE_INFO = {
"name": "fastmcp-feedback",
"version": __version__,
"description": "Comprehensive feedback collection system for FastMCP servers",
"author": __author__,
"email": __email__,
"keywords": ["fastmcp", "feedback", "mcp", "analytics", "database"],
"classifiers": [
"Development Status :: 5 - Production/Ready",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: Database :: Database Engines/Servers",
"Topic :: Internet :: WWW/HTTP :: Dynamic Content",
],
"dependencies": ["fastmcp>=2.12.2", "sqlalchemy>=2.0.43", "pydantic>=2.11.7"],
"optional_dependencies": {
"postgresql": ["psycopg2-binary>=2.9.0"],
"dev": [
"pytest>=7.4.0",
"pytest-asyncio>=0.21.0",
"pytest-html>=3.2.0",
"pytest-cov>=4.1.0",
"ruff>=0.1.0",
],
},
}
def _package_info() -> dict:
"""Name, version, dependencies and extras from the installed metadata.
Read from the distribution rather than written out here, so it cannot
drift from pyproject.toml.
"""
import re
from importlib.metadata import PackageNotFoundError, metadata
info = {
"name": "fastmcp-feedback",
"version": __version__,
"author": __author__,
"email": __email__,
}
try:
meta = metadata("fastmcp-feedback")
except PackageNotFoundError:
return info
requires = meta.get_all("Requires-Dist") or []
extras = {}
for extra in meta.get_all("Provides-Extra") or []:
marker = re.compile(rf"""extra\s*==\s*['"]{re.escape(extra)}['"]""")
extras[extra] = [r.split(";")[0].strip() for r in requires if marker.search(r)]
info.update(
description=meta["Summary"],
keywords=[k.strip() for k in (meta["Keywords"] or "").split(",") if k],
classifiers=meta.get_all("Classifier") or [],
dependencies=[r for r in requires if "extra ==" not in r],
optional_dependencies=extras,
)
return info
PACKAGE_INFO = _package_info()
# Public API - these are the main exports users should use
__all__ = [
@@ -275,11 +276,18 @@ create_multi_tenant_tools(app, tenants)
""",
}
# Logging configuration
# Logging. A library leaves configuration to the application; the
# NullHandler keeps "No handlers could be found" noise away without
# printing anything.
logging.getLogger("fastmcp_feedback").addHandler(logging.NullHandler())
def setup_logging(level: str = "INFO") -> None:
"""Set up logging for FastMCP Feedback.
"""Send log records to stderr through ``logging.basicConfig``.
Opt-in: importing the package configures nothing. Call this when your
server has no logging set up and you want the package's messages.
``basicConfig`` does nothing if the root logger already has handlers.
Args:
level: Logging level (DEBUG, INFO, WARNING, ERROR).
@@ -293,9 +301,9 @@ def setup_logging(level: str = "INFO") -> None:
logger.info(f"FastMCP Feedback v{__version__} initialized")
# Auto-setup logging if not already configured
if not logging.getLogger().handlers:
setup_logging()
logging.getLogger("fastmcp_feedback").debug(
"FastMCP Feedback v%s initialized", __version__
)
def get_version_info() -> dict:
+37 -13
View File
@@ -3,14 +3,15 @@
import logging
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from enum import Enum
from typing import Any
from sqlalchemy import create_engine, text
from sqlalchemy import String, create_engine, func, select, text, type_coerce
from sqlalchemy.exc import OperationalError, SQLAlchemyError
from sqlalchemy.orm import Session, sessionmaker
from sqlalchemy.pool import QueuePool, StaticPool
from .models import Base
from .models import Base, Feedback, FeedbackStatus, FeedbackType
logger = logging.getLogger(__name__)
@@ -171,6 +172,38 @@ class FeedbackDatabase:
# Database utility functions
def enum_label(enum_cls: type[Enum], stored: Any) -> str:
"""The API spelling (``bug``, ``in_progress``) of a stored enum value.
SQLAlchemy stores enum members by NAME (``BUG``, ``IN_PROGRESS``): a
VARCHAR on SQLite, a native enum type on Postgres. The tools return
values. Anything that is neither a member, a name nor a value is
returned as a string rather than dropped.
"""
if isinstance(stored, enum_cls):
return str(stored.value)
stored = str(stored)
if stored in enum_cls.__members__:
return str(enum_cls[stored].value)
return stored
def count_feedback_by(session: Session, field: str) -> dict[str, int]:
"""Feedback counts per ``type`` or ``status``, keyed by enum value."""
column, enum_cls = {
"type": (Feedback.type, FeedbackType),
"status": (Feedback.status, FeedbackStatus),
}[field]
# type_coerce: read the raw stored label, so a row the enum cannot load
# is still counted instead of failing the whole query.
raw = type_coerce(column, String)
counts: dict[str, int] = {}
for stored, n in session.execute(select(raw, func.count()).group_by(raw)):
label = enum_label(enum_cls, stored)
counts[label] = counts.get(label, 0) + n
return counts
async def create_feedback_database(database_url: str | None = None) -> FeedbackDatabase:
"""Create and initialize a feedback database.
@@ -200,17 +233,8 @@ async def get_database_stats(db: FeedbackDatabase) -> dict[str, Any]:
total_result = session.execute(text("SELECT COUNT(*) FROM feedback"))
total_count = total_result.scalar()
# Get counts by type
type_result = session.execute(
text("SELECT type, COUNT(*) FROM feedback GROUP BY type")
)
by_type = dict(type_result.fetchall())
# Get counts by status
status_result = session.execute(
text("SELECT status, COUNT(*) FROM feedback GROUP BY status")
)
by_status = dict(status_result.fetchall())
by_type = count_feedback_by(session, "type")
by_status = count_feedback_by(session, "status")
return {
"total_count": total_count,
@@ -10,22 +10,24 @@ mw.record_event("job.finished", key=job_id, attrs={"duration_s": 3600})
db = DatabaseSink(engine)
mw = instrument(app, [db, EmbeddingSink(OpenAIEmbedder(url, key), db)])
await mw.similar("render timed out")
await backfill_embeddings(mw.embedding_sink) # rows stored before embedding
"""
import importlib
from typing import TYPE_CHECKING, Any
from .classify import (
DEFAULT_RESULT_CLASSIFIER,
ResultClassifier,
soft_error_classifier,
)
from .dispatch import RecordDispatcher
from .embeddings import (
from .constants import (
DEFAULT_EMBEDDING_DIM,
DEFAULT_EVENT_TEXT_KEYS,
Embedder,
EmbeddingSink,
EmbeddingText,
OpenAIEmbedder,
extract_texts,
DEFAULT_TABLE_PREFIX,
EMBEDDING_SOURCES,
)
from .dispatch import RecordDispatcher
from .events import EventRecord
from .middleware import MODE_ENV_VAR, MODES, InstrumentationMiddleware, instrument
from .record import ToolCallRecord
@@ -36,14 +38,75 @@ from .redaction import (
Redactor,
)
from .sinks import CallbackSink, JsonLinesSink, MemorySink, Sink
from .storage import (
DEFAULT_EMBEDDING_DIM,
DEFAULT_TABLE_PREFIX,
EMBEDDING_SOURCES,
DatabaseSink,
EmbeddingVector,
build_metadata,
)
if TYPE_CHECKING:
from .backfill import backfill_embeddings, backfill_feedback
from .embeddings import (
Embedder,
EmbeddingSink,
EmbeddingText,
OpenAIEmbedder,
extract_texts,
)
from .storage import DatabaseSink, EmbeddingVector, build_metadata
# Names whose modules need the async database stack (sqlalchemy[asyncio],
# which brings greenlet). Imported on first access, so instrument() and the
# plain sinks work on a base install.
_LAZY = {
"DatabaseSink": "storage",
"EmbeddingVector": "storage",
"build_metadata": "storage",
"Embedder": "embeddings",
"EmbeddingSink": "embeddings",
"EmbeddingText": "embeddings",
"OpenAIEmbedder": "embeddings",
"extract_texts": "embeddings",
"backfill_embeddings": "backfill",
"backfill_feedback": "backfill",
}
_EXTRA = {
"storage": "instrumentation",
"embeddings": "instrumentation,embeddings",
"backfill": "instrumentation,embeddings",
}
def _missing_module(exc: BaseException) -> str:
seen: BaseException | None = exc
while seen is not None:
if isinstance(seen, ImportError) and seen.name:
return seen.name
seen = seen.__cause__ or seen.__context__
return "a dependency"
def __getattr__(name: str) -> Any:
module = _LAZY.get(name)
if module is None:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
try:
# Checked first because SQLAlchemy reports a missing greenlet only on
# the first import of its asyncio module; a retry would then succeed
# and leave an engine that fails at runtime.
importlib.import_module("greenlet")
value = getattr(importlib.import_module(f".{module}", __name__), name)
except ImportError as exc:
extra = _EXTRA[module]
missing = _missing_module(exc)
raise ImportError(
f"fastmcp_feedback.instrumentation.{name} needs the async database "
f"stack, and {missing} is not installed. Install it with: "
f'uv add "fastmcp-feedback[{extra}]" '
f'(or pip install "fastmcp-feedback[{extra}]")'
) from exc
globals()[name] = value
return value
def __dir__() -> list[str]:
return sorted(set(globals()) | set(_LAZY))
__all__ = [
"CallbackSink",
@@ -72,6 +135,8 @@ __all__ = [
"ResultClassifier",
"Sink",
"ToolCallRecord",
"backfill_embeddings",
"backfill_feedback",
"build_metadata",
"extract_texts",
"instrument",
@@ -0,0 +1,340 @@
"""Embed records that were stored before an EmbeddingSink was running.
``backfill_embeddings`` reads tool-call errors and events back out of the
DatabaseSink's tables, rebuilds the records, and sends them through the same
extraction, redaction and truncation as ``EmbeddingSink.write``. The texts,
hashes and source ids therefore match what the live sink would produce, so
records arriving later dedupe against backfilled rows, and a second run
embeds nothing new.
Command line: ``python -m fastmcp_feedback.instrumentation.backfill --help``.
"""
import asyncio
import logging
from collections.abc import AsyncIterator, Iterable
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import and_, inspect, or_, select, tuple_
from ..embeddings import EmbeddingSink, extract_texts
from ..events import EventRecord
from ..record import ToolCallRecord
from ..sinks import Record
from ..storage import EMBEDDING_SOURCES
logger = logging.getLogger(__name__)
FEEDBACK_KIND = "feedback.submitted"
LLM_KIND = "llm.call"
_CALL_COLUMNS = (
"id",
"tool",
"started_at",
"duration_ms",
"mode",
"ok",
"error_type",
"error_message",
)
def _utc(value: datetime | None) -> datetime | None:
if value is None or value.tzinfo is None:
return value if value is None else value.replace(tzinfo=UTC)
return value.astimezone(UTC)
class _Run:
"""Counts and limits for one backfill call."""
def __init__(
self,
sink: EmbeddingSink,
sources: Iterable[str],
limit: int | None,
dry_run: bool,
):
self.sink = sink
self.sources = tuple(sources)
self.limit = limit
self.dry_run = dry_run
self.counts = dict.fromkeys(self.sources, 0)
self.skipped = 0
self.errors = 0
self.sent = 0
self.have_table = True
@property
def done(self) -> bool:
return self.limit is not None and self.sent >= self.limit
def result(self) -> dict[str, int]:
return {**self.counts, "skipped": self.skipped, "errors": self.errors}
async def feed(self, records: list[Record]) -> None:
sink = self.sink
items = [
sink._prepare(i)
for i in extract_texts(
records, sources=self.sources, event_text_keys=sink.event_text_keys
)
]
# The last text for a source wins, as in the live sink.
latest = list({(i.source_type, i.source_id): i for i in items}.values())
if self.have_table:
todo, stored = await sink._unembedded(latest)
else:
todo, stored = latest, set()
self.skipped += len(latest) - len(todo)
if self.limit is not None:
todo = todo[: max(0, self.limit - self.sent)]
self.sent += len(todo)
if self.dry_run:
for item in todo:
self.counts[item.source_type] += 1
return
for start in range(0, len(todo), sink.batch_size):
chunk = todo[start : start + sink.batch_size]
try:
vectors = await asyncio.wait_for(
sink.embedder.embed([i.text for i in chunk]), sink.timeout
)
await sink._store(chunk, vectors, stored)
except Exception:
self.errors += len(chunk)
logger.warning(
"Backfill failed to embed %d texts with %s",
len(chunk),
sink.embedder.model,
exc_info=True,
)
continue
for item in chunk:
self.counts[item.source_type] += 1
async def _has_table(engine: Any, name: str) -> bool:
async with engine.connect() as conn:
return await conn.run_sync(lambda sync: inspect(sync).has_table(name))
async def _columns(engine: Any, name: str) -> set[str]:
async with engine.connect() as conn:
return await conn.run_sync(
lambda sync: {c["name"] for c in inspect(sync).get_columns(name)}
)
async def _pages(
engine: Any,
table: Any,
columns: list[Any],
ts: Any,
where: Any,
since: datetime | None,
until: datetime | None,
page_size: int,
) -> AsyncIterator[list[dict[str, Any]]]:
"""Rows matching ``where``, oldest first, ``page_size`` at a time.
Keyset paging on (timestamp, id), so rows written during the run neither
shift the pages nor get read twice.
"""
conditions = [where]
if since is not None:
conditions.append(ts >= since)
if until is not None:
conditions.append(ts < until)
last: tuple[Any, Any] | None = None
while True:
query = select(*columns).where(*conditions)
if last is not None:
query = query.where(tuple_(ts, table.c.id) > tuple_(*last))
query = query.order_by(ts, table.c.id).limit(page_size)
async with engine.connect() as conn:
rows = [dict(r) for r in (await conn.execute(query)).mappings()]
if not rows:
return
yield rows
if len(rows) < page_size:
return
last = (rows[-1][ts.name], rows[-1]["id"])
async def _backfill_calls(run: _Run, since, until, page_size) -> None:
db = run.sink.database_sink
t = db.table
have = await _columns(db.engine, t.name)
if "outcome" in have:
# Rows from before the outcome column was added have it NULL and
# were errors exactly when they carry an error message.
where = or_(
t.c.outcome.in_(("error", "soft_error")),
and_(t.c.outcome.is_(None), t.c.error_message.is_not(None)),
)
columns = [t.c[c] for c in (*_CALL_COLUMNS, "outcome")]
else:
where = t.c.error_message.is_not(None)
columns = [t.c[c] for c in _CALL_COLUMNS]
async for rows in _pages(
db.engine, t, columns, t.c.started_at, where, since, until, page_size
):
records: list[Record] = []
for row in rows:
row["outcome"] = row.get("outcome") or "error"
records.append(ToolCallRecord(**row))
await run.feed(records)
if run.done:
return
async def _backfill_events(run: _Run, since, until, page_size) -> None:
db = run.sink.database_sink
e = db.events
kinds = []
if "feedback" in run.sources:
kinds.append(e.c.kind == FEEDBACK_KIND)
if "llm" in run.sources:
kinds.append(e.c.kind == LLM_KIND)
if "event" in run.sources:
# Includes llm.call: one without prompt or completion is a plain event.
kinds.append(e.c.kind != FEEDBACK_KIND)
columns = [e.c.id, e.c.kind, e.c.key, e.c.occurred_at, e.c.attrs]
async for rows in _pages(
db.engine, e, columns, e.c.occurred_at, or_(*kinds), since, until, page_size
):
records: list[Record] = [
EventRecord(**{**row, "attrs": row["attrs"] or {}}) for row in rows
]
await run.feed(records)
if run.done:
return
async def _start(
sink: EmbeddingSink,
sources: Iterable[str] | None,
limit: int | None,
batch_size: int,
dry_run: bool,
) -> _Run:
chosen = tuple(sink.sources if sources is None else sources)
unknown = set(chosen) - set(EMBEDDING_SOURCES)
if unknown:
raise ValueError(
f"unknown sources {sorted(unknown)}; choose from {EMBEDDING_SOURCES}"
)
if batch_size < 1:
raise ValueError(f"batch_size must be at least 1, got {batch_size!r}")
if limit is not None and limit < 0:
raise ValueError(f"limit must not be negative, got {limit!r}")
run = _Run(sink, sorted(chosen, key=EMBEDDING_SOURCES.index), limit, dry_run)
db = sink.database_sink
if not dry_run:
await db.ensure_embeddings()
run.have_table = await _has_table(db.engine, db.embeddings.name)
if not run.have_table and not dry_run:
raise RuntimeError(
f"{db.embeddings.name} does not exist; create it with a migration "
"or pass create_tables=True to the DatabaseSink"
)
return run
async def backfill_embeddings(
embedding_sink: EmbeddingSink,
*,
sources: Iterable[str] | None = None,
since: datetime | None = None,
until: datetime | None = None,
limit: int | None = None,
batch_size: int = 500,
dry_run: bool = False,
) -> dict[str, int]:
"""Embed stored tool-call errors and events that have no embedding yet.
Reads the tables of ``embedding_sink.database_sink``:
- ``call_error``: calls with outcome ``error`` or ``soft_error``, and
calls with outcome NULL and an error message (rows written before the
outcome column existed).
- ``feedback``: ``feedback.submitted`` events.
- ``llm``: ``llm.call`` events with a prompt or completion.
- ``event``: other events with any of the sink's ``event_text_keys``.
Each text goes through the sink's extraction, redactor and ``max_chars``,
so it matches what the live sink makes of the same record. Texts already
stored with the same hash are skipped; a changed text replaces its row.
Args:
sources: which of the above; defaults to the sink's ``sources``.
since, until: only rows timestamped in [since, until). Naive
datetimes are taken as UTC.
limit: embed at most this many texts, oldest first. A later run
carries on, since what was embedded is then skipped.
batch_size: rows read per query. Texts are embedded in chunks of
the sink's ``batch_size``, each limited to its ``timeout``.
dry_run: count what would be embedded without calling the embedder
or writing anything.
Returns counts of texts embedded per source type, plus ``skipped``
(already embedded) and ``errors`` (texts whose embed or store failed;
logged, never raised). Raises ValueError for bad arguments and
RuntimeError when a table it needs does not exist.
"""
run = await _start(embedding_sink, sources, limit, batch_size, dry_run)
db = embedding_sink.database_sink
since, until = _utc(since), _utc(until)
if "call_error" in run.sources:
if not await _has_table(db.engine, db.table.name):
raise RuntimeError(f"{db.table.name} does not exist")
if not run.done:
await _backfill_calls(run, since, until, batch_size)
if set(run.sources) & {"feedback", "event", "llm"}:
if not await _has_table(db.engine, db.events.name):
raise RuntimeError(f"{db.events.name} does not exist")
if not run.done:
await _backfill_events(run, since, until, batch_size)
result = run.result()
logger.info("Backfill %s: %s", "dry run" if dry_run else "done", result)
return result
async def backfill_feedback(
embedding_sink: EmbeddingSink,
items: Iterable[tuple[str, str, str | None]],
*,
batch_size: int = 500,
dry_run: bool = False,
) -> dict[str, int]:
"""Embed feedback kept in the host's own tables, as (ref, title, description).
The text and source id are those of a ``feedback.submitted`` event with
key ``ref`` and those attrs, so feedback recorded later as an event
dedupes against these rows. Returns ``{"feedback", "skipped", "errors"}``
like ``backfill_embeddings``.
"""
run = await _start(embedding_sink, ("feedback",), None, batch_size, dry_run)
now = datetime.now(UTC)
page: list[Record] = []
for ref, title, description in items:
page.append(
EventRecord(
kind=FEEDBACK_KIND,
occurred_at=now,
key=str(ref),
attrs={"title": title, "description": description},
)
)
if len(page) >= batch_size:
await run.feed(page)
page = []
if page:
await run.feed(page)
return run.result()
__all__ = ["backfill_embeddings", "backfill_feedback"]
@@ -0,0 +1,5 @@
import sys
from .cli import main
sys.exit(main())
@@ -0,0 +1,143 @@
"""python -m fastmcp_feedback.instrumentation.backfill
Prints the counts as JSON. Exits 1 when any text failed to embed, 2 on a
setup error (bad arguments, missing table, unreachable database).
"""
import argparse
import asyncio
import json
import logging
import os
import sys
from datetime import datetime
from ..embeddings import DEFAULT_EVENT_TEXT_KEYS, EmbeddingSink, OpenAIEmbedder
from ..storage import DEFAULT_TABLE_PREFIX, EMBEDDING_SOURCES, DatabaseSink
from . import backfill_embeddings
API_KEY_ENV = "FFB_EMBED_API_KEY"
def _sources(value: str) -> tuple[str, ...]:
chosen = tuple(s.strip() for s in value.split(",") if s.strip())
unknown = set(chosen) - set(EMBEDDING_SOURCES)
if not chosen or unknown:
raise argparse.ArgumentTypeError(
f"choose from {','.join(EMBEDDING_SOURCES)}, comma separated"
)
return chosen
def _when(value: str) -> datetime:
try:
return datetime.fromisoformat(value)
except ValueError as exc:
raise argparse.ArgumentTypeError(f"not an ISO 8601 time: {value!r}") from exc
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="python -m fastmcp_feedback.instrumentation.backfill",
description=(
"Embed stored tool-call errors and events that have no embedding "
f"yet. The embedder's API key is read from ${API_KEY_ENV} (or the "
"variable named by --api-key-env)."
),
)
parser.add_argument(
"--database-url",
required=True,
help="async SQLAlchemy URL, e.g. postgresql+asyncpg://... or sqlite+aiosqlite:///calls.db",
)
parser.add_argument(
"--embed-base-url",
help="OpenAI-compatible base URL up to the version, e.g. https://api.example.com/v1 "
"(required unless --dry-run)",
)
parser.add_argument("--api-key-env", default=API_KEY_ENV, metavar="NAME")
parser.add_argument("--model", default="mxbai-embed-large")
parser.add_argument("--dim", type=int, default=1024)
parser.add_argument("--prefix", default=DEFAULT_TABLE_PREFIX)
parser.add_argument(
"--sources",
type=_sources,
default=EMBEDDING_SOURCES,
help=f"comma separated, default {','.join(EMBEDDING_SOURCES)}",
)
parser.add_argument(
"--event-text-keys",
default=",".join(DEFAULT_EVENT_TEXT_KEYS),
help="attrs making up an event text, comma separated",
)
parser.add_argument("--since", type=_when, help="ISO 8601; naive means UTC")
parser.add_argument("--until", type=_when, help="ISO 8601, exclusive")
parser.add_argument("--limit", type=int, help="embed at most this many texts")
parser.add_argument("--batch-size", type=int, default=500, help="rows per query")
parser.add_argument("--max-chars", type=int, default=4000)
parser.add_argument(
"--create-tables",
action="store_true",
help="create the embeddings table (and vector extension) if missing",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="count what would be embedded; call no embedder, write nothing",
)
parser.add_argument("-v", "--verbose", action="store_true")
return parser
async def _run(args: argparse.Namespace) -> dict[str, int]:
embedder = OpenAIEmbedder(
args.embed_base_url or "http://embedder.invalid/v1",
api_key=os.environ.get(args.api_key_env) or None,
model=args.model,
dim=args.dim,
)
db = DatabaseSink(
args.database_url,
prefix=args.prefix,
create_tables=args.create_tables and not args.dry_run,
embedding_dim=args.dim,
)
try:
sink = EmbeddingSink(
embedder,
db,
sources=args.sources,
event_text_keys=[k for k in args.event_text_keys.split(",") if k],
max_chars=args.max_chars,
)
return await backfill_embeddings(
sink,
since=args.since,
until=args.until,
limit=args.limit,
batch_size=args.batch_size,
dry_run=args.dry_run,
)
finally:
await db.aclose()
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
if not args.dry_run and not args.embed_base_url:
parser.error("--embed-base-url is required unless --dry-run")
# The CLI owns logging; the package configures none. force: replace
# anything a host process (or a test runner) already set up.
logging.basicConfig(
level=logging.INFO if args.verbose else logging.WARNING,
format="%(levelname)s %(name)s: %(message)s",
force=True,
)
try:
counts = asyncio.run(_run(args))
except Exception as exc:
print(f"backfill failed: {type(exc).__name__}: {exc}", file=sys.stderr)
return 2
print(json.dumps({"dry_run": args.dry_run, **counts}))
return 1 if counts["errors"] else 0
@@ -19,14 +19,36 @@ MESSAGE_KEYS = ("error", "message", "detail", "reason")
ERROR_FLAG_KEYS = ("isError", "is_error")
def _message(result: dict, depth: int = 0) -> str | None:
for key in MESSAGE_KEYS:
value = result.get(key)
def lookup(result: dict, key: str) -> Any:
"""``result[key]``, where ``key`` may be a dotted path into nested dicts.
A key present exactly as written wins, so a literal ``"a.b"`` key is
found before ``result["a"]["b"]``. Otherwise ``"snapshot.error"`` walks
``result["snapshot"]["error"]``. A missing step, or a step that is not a
dict, gives None: the same as the key not being there. Never raises.
"""
if key in result:
return result[key]
if "." not in key:
return None
value: Any = result
for part in key.split("."):
if not isinstance(value, dict) or part not in value:
return None
value = value[part]
return value
def _message(
result: dict, depth: int = 0, keys: tuple[str, ...] = MESSAGE_KEYS
) -> str | None:
for key in keys:
value = lookup(result, key)
if value is None or value == "":
continue
if isinstance(value, dict):
# {"error": {"code": 404, "message": "..."}}
if depth == 0 and (inner := _message(value, 1)):
if depth == 0 and (inner := _message(value, 1, keys)):
return inner
continue
if isinstance(value, list | tuple):
@@ -58,22 +80,62 @@ def soft_error_classifier(
status_keys: Iterable[str] = ("status", "state"),
flag_keys: Iterable[str] = ("ok", "success"),
extra_statuses: Iterable[str] = (),
error_keys: Iterable[str] = ("error",),
message_keys: Iterable[str] = MESSAGE_KEYS,
) -> ResultClassifier:
"""Build a classifier for dict results that describe their own failure.
A dict result is a soft error when any of ``flag_keys`` is exactly False,
when any of ``status_keys`` holds one of ``statuses`` or
``extra_statuses`` (case-insensitive), or when the tool marked it
``isError``. The message is the first of ``error``, ``message``,
``detail`` and ``reason`` present, e.g. ``status=failed: disk full``.
``isError``. The message is the first of ``message_keys`` present
(``error``, ``message``, ``detail``, ``reason`` by default), e.g.
``status=failed: disk full``. A dict found there is searched one level
down with the same keys, as in ``{"error": {"message": "..."}}``.
Failing those, it is a soft error when any of ``error_keys`` holds a
non-empty string, as in ``{"played": False, "error": "output stalled"}``,
with that string as the message (``error: output stalled``). None, empty
or blank strings, numbers, booleans, dicts and lists there do not count,
so ``{"error": None}`` and ``{"error": 0.02}`` stay ok. ``error_keys=()``
turns this off.
A result that says it succeeded, with any of ``flag_keys`` exactly True,
skips only that last check: ``{"ok": True, "error": "deprecated arg"}``
is ok, the string being a warning. A failing status, a False flag and
``isError`` still flag it. Truthy non-bools such as ``1`` or ``"yes"``
are not an explicit success.
``message_keys`` and ``error_keys`` accept dotted paths into nested
dicts: ``"snapshot.error"`` reads ``result["snapshot"]["error"]``. A key
that literally contains a dot is matched first; a missing step or one
that is not a dict counts as the key not being there. The message label
for an ``error_keys`` hit is the path as written. ``status_keys`` and
``flag_keys`` are plain keys.
>>> classify = soft_error_classifier(extra_statuses=("unknown_target",))
>>> classify("render", {"status": "unknown_target", "error": "no such object"})
'status=unknown_target: no such object'
>>> classify("speak", {"played": False, "error": "output stalled"})
'error: output stalled'
>>> print(classify("speak", {"ok": True, "error": "deprecated arg"}))
None
>>> sitl = soft_error_classifier(
... status_keys=("status", "state", "reason"),
... extra_statuses=("died",),
... message_keys=("snapshot.error", "error", "message", "detail", "reason"),
... )
>>> sitl("wait_until", {"reason": "died", "snapshot": {"error": "SITL exited"}})
'reason=died: SITL exited'
"""
bad = frozenset(s.lower() for s in (*statuses, *extra_statuses))
status_keys = tuple(status_keys)
flag_keys = tuple(flag_keys)
error_keys = tuple(error_keys)
message_keys = tuple(message_keys)
def message(result: dict) -> str | None:
return _message(result, 0, message_keys)
def classify(tool: str, result: Any) -> str | None:
if not isinstance(result, dict):
@@ -81,14 +143,20 @@ def soft_error_classifier(
for key in status_keys:
value = result.get(key)
if isinstance(value, str) and value.lower() in bad:
return _describe(f"{key}={value}", _message(result))
return _describe(f"{key}={value}", message(result))
for key in flag_keys:
if key in result and result[key] is False:
return _describe(f"{key}=False", _message(result))
return _describe(f"{key}=False", message(result))
for key in ERROR_FLAG_KEYS:
if result.get(key) is True:
detail = _message(result) or _content_text(result.get("content"))
detail = message(result) or _content_text(result.get("content"))
return _describe("isError", detail)
if any(result.get(key) is True for key in flag_keys):
return None
for key in error_keys:
value = lookup(result, key)
if isinstance(value, str) and value.strip():
return _describe(key, value.strip())
return None
return classify
@@ -0,0 +1,12 @@
"""Defaults shared by the storage and embedding modules.
Kept apart from them so these import without the async database stack.
"""
DEFAULT_TABLE_PREFIX = "ffb_"
DEFAULT_EMBEDDING_DIM = 1024
# source_type values in {prefix}embeddings. Feedback rows survive pruning.
EMBEDDING_SOURCES = ("feedback", "call_error", "event", "llm")
FEEDBACK_SOURCE = "feedback"
# Attribute keys whose text an EmbeddingSink embeds from events.
DEFAULT_EVENT_TEXT_KEYS = ("error_excerpt", "error", "message", "reason")
@@ -10,6 +10,8 @@ from .sinks import Record, Sink
logger = logging.getLogger(__name__)
CLOSED = "dispatcher closed"
class RecordDispatcher:
"""Queue records and hand them to sinks from a background task.
@@ -17,6 +19,10 @@ class RecordDispatcher:
``submit`` never blocks and never raises: when the queue is full the record
is dropped and counted, so a slow or broken sink cannot slow down or fail
a tool call. Sink errors are logged and counted, never propagated.
After ``aclose()`` the dispatcher stays closed: every later record, from
the loop or another thread, is dropped and counted under ``CLOSED``, and
it never binds a loop or starts a task again.
"""
def __init__(
@@ -32,15 +38,26 @@ class RecordDispatcher:
self.batch_size = batch_size
self.sink_timeout = sink_timeout
self.dropped = 0
# Drops per reason, e.g. {"queue full": 3, "dispatcher closed": 1}.
self.drop_reasons: dict[str, int] = {}
self.sink_errors = 0
self._closed = False
self._queue: asyncio.Queue[Record] | None = None
self._task: asyncio.Task | None = None
self._loop: asyncio.AbstractEventLoop | None = None
# submit_threadsafe can drop from any thread.
self._drop_lock = threading.Lock()
@property
def closed(self) -> bool:
"""True once ``aclose()`` has flushed; later records are dropped."""
return self._closed
def submit(self, record: Record) -> bool:
"""Queue a record for delivery. Returns False if it was dropped."""
if self._closed:
self._drop(CLOSED)
return False
try:
loop = asyncio.get_running_loop()
except RuntimeError:
@@ -60,6 +77,8 @@ class RecordDispatcher:
Called by the middleware as each tool call starts. Never raises.
"""
if self._closed:
return
with contextlib.suppress(Exception):
self._bind(asyncio.get_running_loop())
@@ -83,7 +102,13 @@ class RecordDispatcher:
it was handed over, and a queue that fills before it lands still drops
and counts it. With no running loop anywhere the record is dropped.
``on_queued(record)`` runs on the loop thread once the record is queued.
A record handed over before ``aclose()`` whose turn on the loop comes
after the dispatcher closed is dropped and counted there, not written.
"""
if self._closed:
self._drop(CLOSED)
return False
try:
try:
running = asyncio.get_running_loop()
@@ -123,6 +148,16 @@ class RecordDispatcher:
with self._drop_lock:
self.dropped += 1
dropped = self.dropped
for_reason = self.drop_reasons.get(reason, 0) + 1
self.drop_reasons[reason] = for_reason
if reason == CLOSED:
# Late records at shutdown are expected; say so once.
if for_reason == 1:
logger.warning(
"Dropped instrumentation record recorded after aclose(); "
"later ones are dropped silently (see drop_reasons)"
)
return
if dropped == 1 or dropped % 1000 == 0:
logger.warning(
"Dropped instrumentation record (%s); %d dropped so far",
@@ -158,13 +193,33 @@ class RecordDispatcher:
)
async def flush(self) -> None:
"""Wait until every queued record has been handed to the sinks."""
if self._queue is not None and self._loop is asyncio.get_running_loop():
await self._queue.join()
"""Wait until every queued record has been handed to the sinks.
Records queued while it waits, from this loop or handed over from a
thread, are waited for too.
"""
queue = self._queue
if queue is None or self._loop is not asyncio.get_running_loop():
return
await queue.join()
# join() can return while a record put just after its wake-up is
# still queued, so check again rather than trusting one join.
while _unfinished(queue):
await queue.join()
async def aclose(self) -> None:
"""Flush, stop the background task and close the sinks."""
"""Flush, stop the background task and close the sinks. Idempotent.
Records queued before this returns its flush are delivered; from then
on the dispatcher is closed and drops everything (``CLOSED``).
"""
if self._closed:
return
await self.flush()
# No await between the flush ending and this: a record handed over
# from a thread that lands later is dropped by submit, never queued
# for a task that is about to stop.
self._closed = True
if self._task is not None:
self._task.cancel()
with contextlib.suppress(asyncio.CancelledError, RuntimeError):
@@ -180,3 +235,12 @@ class RecordDispatcher:
logger.warning(
"Sink %s failed to close", type(sink).__name__, exc_info=True
)
def _unfinished(queue: asyncio.Queue) -> bool:
"""Whether ``queue`` holds records not yet marked done.
``_unfinished_tasks`` is the counter ``join()`` itself waits on (CPython
3.11 to 3.14). Without it, report False: flush then does a single join.
"""
return getattr(queue, "_unfinished_tasks", 0) > 0
@@ -20,16 +20,16 @@ from typing import Any, Protocol, runtime_checkable
from sqlalchemy import Float, and_, literal, not_, select, tuple_
from .constants import DEFAULT_EVENT_TEXT_KEYS, EMBEDDING_SOURCES
from .events import EventRecord
from .payload import is_truncated
from .record import ToolCallRecord
from .redaction import Redactor
from .sinks import Record
from .storage import EMBEDDING_SOURCES, EmbeddingVector
from .storage import EmbeddingVector
logger = logging.getLogger(__name__)
DEFAULT_EVENT_TEXT_KEYS = ("error_excerpt", "error", "message", "reason")
@runtime_checkable
class Embedder(Protocol):
@@ -143,10 +143,48 @@ class EmbeddingText:
return hashlib.sha256(self.text.encode()).hexdigest()
def _clean(value: Any) -> str | None:
# Flattening stops adding items past this many characters; max_chars cuts
# the result again later. Keeps a huge list from being joined in full.
_FLAT_CHARS = 65536
_FLAT_DEPTH = 32
def _clean(value: Any, depth: int = 0) -> str | None:
"""``value`` as stripped text, or None when there is none.
Lists and tuples become their items' texts, one per line. Dicts become
``key: value`` lines sorted by key: PostgreSQL JSONB does not keep key
order, and the text has to come out the same when a backfill rebuilds it
from a stored row. Other values go through ``str()``.
"""
if value is None:
return None
text = value if isinstance(value, str) else str(value)
if is_truncated(value):
# A value over the middleware's max_payload_bytes: its start is all
# that was kept.
value = value.get("preview")
if isinstance(value, str):
text = value
elif isinstance(value, list | tuple | dict):
if depth >= _FLAT_DEPTH:
return None
if isinstance(value, dict):
pairs = sorted(value.items(), key=lambda kv: str(kv[0]))
parts = ((f"{k}: ", _clean(v, depth + 1)) for k, v in pairs)
else:
parts = (("", _clean(v, depth + 1)) for v in value)
lines: list[str] = []
size = 0
for label, part in parts:
if not part:
continue
lines.append(label + part)
size += len(lines[-1]) + 1
if size > _FLAT_CHARS:
break
text = "\n".join(lines)
else:
text = str(value)
text = text.strip()
return text or None
@@ -168,6 +206,9 @@ def extract_texts(
those values joined by newlines, keyed by the event id. An ``llm.call``
without prompt or completion counts as a plain event, so a failed LLM
call's ``error`` is still embedded.
List and tuple values become one line per item, dicts ``key: value``
lines sorted by key, at any depth.
"""
wanted = frozenset(sources)
out: list[EmbeddingText] = []
@@ -379,18 +420,35 @@ class EmbeddingSink:
async def _process(self, batch: list[EmbeddingText]) -> None:
# The last text for a source wins within a batch.
latest = {(i.source_type, i.source_id): i for i in batch}
items = list(latest.values())
db = self.database_sink
await db._ready()
await db.ensure_embeddings()
todo, stored = await self._unembedded(list(latest.values()))
self.skipped += len(batch) - len(todo)
if not todo:
return
vectors = await asyncio.wait_for(
self.embedder.embed([i.text for i in todo]), self.timeout
)
await self._store(todo, vectors, stored)
self.embedded += len(todo)
async def _unembedded(
self, items: list[EmbeddingText]
) -> tuple[list[EmbeddingText], set[tuple[str, str]]]:
"""The items whose stored row is missing or holds another text, and
the keys that have a row (to be replaced). Items must be unique by
(source_type, source_id)."""
if not items:
return [], set()
t = self.table
model = self.embedder.model
async with db.engine.connect() as conn:
keys = [(i.source_type, i.source_id) for i in items]
async with self.database_sink.engine.connect() as conn:
rows = (
await conn.execute(
select(t.c.source_type, t.c.source_id, t.c.text_hash).where(
t.c.model == model,
tuple_(t.c.source_type, t.c.source_id).in_(list(latest)),
t.c.model == self.embedder.model,
tuple_(t.c.source_type, t.c.source_id).in_(keys),
)
)
).all()
@@ -398,12 +456,15 @@ class EmbeddingSink:
todo = [
i for i in items if stored.get((i.source_type, i.source_id)) != i.text_hash
]
self.skipped += len(batch) - len(todo)
if not todo:
return
vectors = await asyncio.wait_for(
self.embedder.embed([i.text for i in todo]), self.timeout
)
return todo, set(stored)
async def _store(
self,
todo: list[EmbeddingText],
vectors: list[list[float]],
replace: set[tuple[str, str]],
) -> None:
db = self.database_sink
if len(vectors) != len(todo):
raise ValueError(
f"embedder returned {len(vectors)} vectors for {len(todo)}"
@@ -420,7 +481,7 @@ class EmbeddingSink:
"id": str(uuid.uuid4()),
"source_type": i.source_type,
"source_id": i.source_id,
"model": model,
"model": self.embedder.model,
"dim": len(v),
"text_hash": i.text_hash,
"text": i.text,
@@ -429,8 +490,7 @@ class EmbeddingSink:
}
for i, v in zip(todo, vectors, strict=True)
]
await self._upsert(values, replace=set(stored))
self.embedded += len(values)
await self._upsert(values, replace=replace)
async def _upsert(self, values: list[dict[str, Any]], replace: set) -> None:
t = self.table
@@ -59,12 +59,18 @@ class CallScope:
Tasks started inside a tool inherit a copy of the context and outlive the
call, so the scope is switched off when the call ends instead of relying
on the ContextVar reset alone.
``user_sub``, ``caller_kind`` and ``client_id`` are the call's identity,
resolved before the tool runs, so events recorded inside it carry them.
"""
id: str
session_id: str | None
active: bool = True
events: int = 0
user_sub: str | None = None
caller_kind: str | None = None
client_id: str | None = None
def jsonable(value: Any, depth: int = 0) -> Any:
@@ -4,16 +4,16 @@ import asyncio
import contextlib
import contextvars
import inspect
import json
import logging
import os
import random as _random
import sys
import time
import uuid
from collections import deque
from collections.abc import Awaitable, Callable, Iterable, Mapping, Sequence
from datetime import UTC, datetime, timedelta
from typing import Any
from typing import TYPE_CHECKING, Any, NamedTuple
from fastmcp import FastMCP
from fastmcp.server.middleware import Middleware, MiddlewareContext
@@ -27,7 +27,6 @@ from .correlation import (
select_calls,
)
from .dispatch import RecordDispatcher
from .embeddings import EmbeddingSink
from .events import (
MAX_KEY_CHARS,
MAX_KIND_CHARS,
@@ -35,10 +34,14 @@ from .events import (
EventRecord,
jsonable,
)
from .payload import DEFAULT_MAX_PAYLOAD_BYTES, cap, cap_attrs, json_size
from .record import ToolCallRecord
from .redaction import Redactor
from .sinks import JsonLinesSink, Sink
from .storage import DatabaseSink
if TYPE_CHECKING:
from .embeddings import EmbeddingSink
from .storage import DatabaseSink
logger = logging.getLogger(__name__)
@@ -111,13 +114,6 @@ def _result_payload(result: Any) -> Any:
return result
def _json_size(value: Any) -> int | None:
try:
return len(json.dumps(value, default=str).encode())
except Exception:
return None
class InstrumentationMiddleware(Middleware):
"""Record every tool call without ever slowing or breaking it.
@@ -137,7 +133,20 @@ class InstrumentationMiddleware(Middleware):
meta_only_tools: tools that are never recorded above ``meta``.
identity_resolver: ``(context) -> dict`` returning ``user_sub``,
``caller_kind`` and ``client_id`` (stored as columns) plus any
other keys (stored in ``identity``). May be async.
other keys (stored in ``identity``). May be async. Runs before
the tool, so events recorded during the call get that identity;
it also runs for calls that sampling later drops, and an async
one can delay the tool by up to ``hook_timeout``; keep it cheap.
When that first answer is empty (nothing, or only None values,
including a run that failed or timed out) or lacks any of
``late_identity_keys``, it runs again after the tool, alongside
the enricher, and the call's own record takes the second
answer's values where the first had none. Any other answer means
one run, even without all three columns.
late_identity_keys: identity keys that may only exist once the tool
has run, such as an id a registration or login tool creates. A
first answer without one of them gets the second run. Empty by
default.
enricher: ``(tool, args, result, context) -> dict`` of extra fields,
stored in ``extra``. A ``server_version`` key fills that column.
May be async.
@@ -168,6 +177,11 @@ class InstrumentationMiddleware(Middleware):
``prompt``/``completion`` keys passed in its ``attrs``.
max_llm_text_chars: longest prompt or completion kept. Defaults to
``8 * max_error_chars`` (4000).
max_payload_bytes: largest stored arguments, result (mode ``full``)
or event ``attrs``, in bytes of JSON; None for no limit. Larger
ones are stored as ``{"_truncated": true, "size": ..., "preview":
...}`` markers, after redaction; for attrs, only the largest
values are replaced. See ``fastmcp_feedback.instrumentation.payload``.
Exclusion and sampling apply to tool calls only; events are always
recorded. A sampled tool's call is kept whenever an event was recorded
@@ -182,6 +196,7 @@ class InstrumentationMiddleware(Middleware):
redactor: Redactor | None = None,
meta_only_tools: Iterable[str] = (),
identity_resolver: IdentityResolver | None = None,
late_identity_keys: Iterable[str] = (),
enricher: Enricher | None = None,
server_version: str | None = None,
hook_timeout: float = 0.25,
@@ -198,6 +213,7 @@ class InstrumentationMiddleware(Middleware):
recent_events: int = 2000,
capture_llm_text: bool = False,
max_llm_text_chars: int | None = None,
max_payload_bytes: int | None = DEFAULT_MAX_PAYLOAD_BYTES,
):
mode = (mode or os.getenv(MODE_ENV_VAR) or "meta").lower()
if mode not in MODES:
@@ -206,6 +222,7 @@ class InstrumentationMiddleware(Middleware):
self.redactor = redactor or Redactor()
self.meta_only_tools = frozenset(meta_only_tools)
self.identity_resolver = identity_resolver
self.late_identity_keys = tuple(late_identity_keys)
self.enricher = enricher
self.server_version = server_version
self.hook_timeout = hook_timeout
@@ -229,6 +246,16 @@ class InstrumentationMiddleware(Middleware):
contextvars.ContextVar(f"ffb_call_scope_{id(self):x}", default=None)
)
self.capture_llm_text = capture_llm_text
if max_payload_bytes is not None and (
isinstance(max_payload_bytes, bool)
or not isinstance(max_payload_bytes, int)
or max_payload_bytes <= 0
):
raise ValueError(
f"max_payload_bytes must be a positive int or None, "
f"got {max_payload_bytes!r}"
)
self.max_payload_bytes = max_payload_bytes
self.max_llm_text_chars = (
max_llm_text_chars
if max_llm_text_chars is not None
@@ -237,7 +264,7 @@ class InstrumentationMiddleware(Middleware):
sinks = list(sinks) if sinks is not None else [JsonLinesSink()]
for sink in sinks:
# Embedded text gets this middleware's redaction rules too.
if isinstance(sink, EmbeddingSink) and sink.redactor is None:
if _is_sink(sink, "embeddings", "EmbeddingSink") and sink.redactor is None:
sink.redactor = self.redactor
self.dispatcher = RecordDispatcher(
sinks,
@@ -254,8 +281,9 @@ class InstrumentationMiddleware(Middleware):
if _tool_name(context) in self.exclude_tools:
return await call_next(context)
started_at = datetime.now(UTC)
start = time.perf_counter()
# Resolved before the tool runs, so events recorded inside it carry
# the caller; the call's own record reuses it.
who = await self._resolve_identity(context)
result = None
error: BaseException | None = None
scope = token = None
@@ -263,11 +291,17 @@ class InstrumentationMiddleware(Middleware):
# The record id exists before the tool runs, so events recorded
# inside it carry the id the stored call row will have.
scope = CallScope(
id=str(uuid.uuid4()), session_id=session_key(context.fastmcp_context)
id=str(uuid.uuid4()),
session_id=session_key(context.fastmcp_context),
user_sub=who.column("user_sub"),
caller_kind=who.column("caller_kind"),
client_id=who.column("client_id"),
)
token = self._call_scope.set(scope)
except Exception:
logger.warning("Failed to open call scope", exc_info=True)
started_at = datetime.now(UTC)
start = time.perf_counter()
try:
result = await call_next(context)
return result
@@ -284,7 +318,7 @@ class InstrumentationMiddleware(Middleware):
self._call_scope.reset(token)
try:
await self._record(
context, started_at, duration_ms, result, error, scope
context, started_at, duration_ms, result, error, scope, who
)
except Exception:
logger.warning("Failed to record tool call", exc_info=True)
@@ -296,7 +330,8 @@ class InstrumentationMiddleware(Middleware):
duration_ms: float,
result: Any,
error: BaseException | None,
scope: CallScope | None = None,
scope: CallScope | None,
who: "_Identity",
) -> None:
tool = _tool_name(context)
payload = _result_payload(result) if result is not None else None
@@ -313,13 +348,45 @@ class InstrumentationMiddleware(Middleware):
elif rate is not None and not self.random() < rate:
return
record = await self._build_record(
context, started_at, duration_ms, payload, error, soft, rate
context, started_at, duration_ms, payload, error, soft, rate, who
)
if scope is not None:
record.id = scope.id
self.recent.append(record)
if not self.dispatcher.closed:
# After aclose() calls are neither stored nor kept for linking.
self.recent.append(record)
self.dispatcher.submit(record)
async def _resolve_identity(self, context: MiddlewareContext) -> "_Identity":
"""Run the identity resolver once. Never raises.
Returns the redacted result split into the three columns and the
rest, plus the FastMCP context's ``client_id`` as the fallback for a
resolver that gives none.
"""
fallback = None
try:
fallback = _attr(context.fastmcp_context, "client_id")
identity = await self._call_hook(self.identity_resolver, context)
identity = self.redactor.redact(identity)
columns = {k: identity.pop(k) for k in _IDENTITY_COLUMNS if k in identity}
return _Identity(columns, identity, fallback)
except Exception:
logger.warning("Failed to resolve identity", exc_info=True)
return _Identity({}, {}, fallback)
def _identity_incomplete(self, who: "_Identity") -> bool:
"""Whether the resolver's first answer lacks something it may know later."""
if self.identity_resolver is None:
return False
answered = any(
v is not None for v in (*who.columns.values(), *who.rest.values())
)
return not answered or any(
who.rest.get(k) is None and who.columns.get(k) is None
for k in self.late_identity_keys
)
async def _call_hook(self, hook: Callable | None, *args: Any) -> dict[str, Any]:
if hook is None:
return {}
@@ -357,16 +424,24 @@ class InstrumentationMiddleware(Middleware):
error: BaseException | None,
soft: str | None,
sample_rate: float | None,
who: "_Identity",
) -> ToolCallRecord:
tool = _tool_name(context)
args = getattr(context.message, "arguments", None) or {}
ctx = context.fastmcp_context
identity = await self._call_hook(self.identity_resolver, context)
extra = await self._call_hook(self.enricher, tool, args, payload, context)
identity = self.redactor.redact(identity)
enrich = self._call_hook(self.enricher, tool, args, payload, context)
if self._identity_incomplete(who):
# A tool can create the identity it runs under (a registration,
# a login). Ask again, concurrently with the enricher so the
# record is not built any later than the slower of the two.
later, extra = await asyncio.gather(self._resolve_identity(context), enrich)
who = who.filled_from(later)
else:
extra = await enrich
identity = dict(who.rest)
columns = {k: who.column(k) for k in _IDENTITY_COLUMNS}
extra = self.redactor.redact(extra)
columns = {k: identity.pop(k) for k in _IDENTITY_COLUMNS if k in identity}
server_version = extra.pop("server_version", None) or self.server_version
mode = "meta" if tool in self.meta_only_tools else self.mode
@@ -389,6 +464,9 @@ class InstrumentationMiddleware(Middleware):
self.redactor.redact_text(soft)[: self.max_error_chars] or None
)
# Measured once: the size columns and the payload limit share it.
args_size = json_size(args)
result_size = json_size(payload) if payload is not None else None
return ToolCallRecord(
tool=tool,
started_at=started_at,
@@ -400,15 +478,19 @@ class InstrumentationMiddleware(Middleware):
error_message=error_message,
session_id=session_key(ctx),
request_id=_attr(ctx, "request_id"),
client_id=columns.get("client_id") or _attr(ctx, "client_id"),
client_id=columns.get("client_id"),
user_sub=columns.get("user_sub"),
caller_kind=columns.get("caller_kind"),
identity=identity,
server_version=server_version,
args_size=_json_size(args),
result_size=_json_size(payload) if payload is not None else None,
args=self.redactor.redact(args) if full else None,
result=self.redactor.redact(payload)
args_size=args_size,
result_size=result_size,
args=cap(self.redactor.redact(args), args_size, self.max_payload_bytes)
if full
else None,
result=cap(
self.redactor.redact(payload), result_size, self.max_payload_bytes
)
if full and payload is not None
else None,
extra=extra,
@@ -423,18 +505,28 @@ class InstrumentationMiddleware(Middleware):
attrs: Mapping[str, Any] | None = None,
user_sub: str | None = None,
call_id: str | None = None,
caller_kind: str | None = None,
client_id: str | None = None,
) -> bool:
"""Record something that is not a tool call. Never raises or blocks.
Safe from any thread. Inside a tool call the event gets that call's id
and session; ``call_id`` given here wins. ``attrs`` is redacted like
Safe from any thread. Inside a tool call the event gets that call's
id, session, and the ``user_sub``, ``caller_kind`` and ``client_id``
the identity resolver gave the call; each of those given here wins.
Outside a call (including a task the call started that is still
running after it returned) only what is passed is stored. ``attrs`` is redacted like
call arguments and made JSON-safe. ``kind`` must be a non-empty string
of at most 128 characters and ``key`` at most 255 once made a string.
A ``key`` used for joining must stay unique for the retention period
and across restarts, so not a per-boot counter: use a UUID or a
startup token plus a counter.
Returns False when the event was not recorded: mode ``off``, an invalid
``kind``, ``key`` or ``attrs``, a full queue, or no event loop to hand
it to (called from a thread before anything was recorded on the
server's loop). From another thread, True means handed to the loop.
server's loop), or after ``aclose()``. From another thread, True means
handed to the loop; if the middleware closes before the event lands,
it is dropped and counted, never written.
"""
try:
if self.mode == "off":
@@ -462,6 +554,12 @@ class InstrumentationMiddleware(Middleware):
session_id = scope.session_id
if call_id is None:
call_id = scope.id
if user_sub is None:
user_sub = scope.user_sub
if caller_kind is None:
caller_kind = scope.caller_kind
if client_id is None:
client_id = scope.client_id
record = EventRecord(
kind=kind,
occurred_at=datetime.now(UTC),
@@ -469,8 +567,13 @@ class InstrumentationMiddleware(Middleware):
call_id=None if call_id is None else str(call_id),
session_id=session_id,
user_sub=None if user_sub is None else str(user_sub),
caller_kind=None if caller_kind is None else str(caller_kind),
client_id=None if client_id is None else str(client_id),
server_version=self.server_version,
attrs=jsonable(self.redactor.redact(dict(attrs or {}))),
attrs=cap_attrs(
jsonable(self.redactor.redact(dict(attrs or {}))),
self.max_payload_bytes,
),
)
return self.dispatcher.submit_threadsafe(record, self.recent_events.append)
except Exception:
@@ -548,6 +651,7 @@ class InstrumentationMiddleware(Middleware):
memory, so an event is visible as soon as ``record_event`` returns (on
the loop thread). Without a DatabaseSink, memory only. ``occurred_at``
is a UTC datetime. A database failure is logged and memory is used.
Events are ordered by ``occurred_at``; their ids are random UUIDs.
"""
key = str(key)
kinds = None if kinds is None else frozenset(kinds)
@@ -569,11 +673,9 @@ class InstrumentationMiddleware(Middleware):
return rows[-limit:] if limit > 0 else []
@property
def embedding_sink(self) -> EmbeddingSink | None:
def embedding_sink(self) -> "EmbeddingSink | None":
"""The first EmbeddingSink among the sinks, if any."""
return next(
(s for s in self.dispatcher.sinks if isinstance(s, EmbeddingSink)), None
)
return _first_sink(self.dispatcher.sinks, "embeddings", "EmbeddingSink")
async def similar(
self,
@@ -640,11 +742,9 @@ class InstrumentationMiddleware(Middleware):
return []
@property
def database_sink(self) -> DatabaseSink | None:
def database_sink(self) -> "DatabaseSink | None":
"""The first DatabaseSink among the sinks, if any."""
return next(
(s for s in self.dispatcher.sinks if isinstance(s, DatabaseSink)), None
)
return _first_sink(self.dispatcher.sinks, "storage", "DatabaseSink")
async def capture_recent_calls(
self,
@@ -778,10 +878,70 @@ class InstrumentationMiddleware(Middleware):
await self.dispatcher.flush()
async def aclose(self) -> None:
"""Flush and close the sinks. Call on server shutdown."""
"""Flush and close the sinks. Call on server shutdown. Idempotent.
Afterwards the middleware records nothing: tool calls still run, but
their records, and any ``record_event`` from the loop or a thread, are
dropped and counted in ``dispatcher.drop_reasons["dispatcher closed"]``,
and they are not kept in memory for linking or ``events_for``.
"""
await self.dispatcher.aclose()
class _Identity(NamedTuple):
"""One answer from the identity resolver, redacted.
``columns`` holds whichever of user_sub, caller_kind and client_id it
returned, ``rest`` everything else, and ``fallback_client_id`` the
FastMCP context's client_id for when it returned none.
"""
columns: dict[str, Any]
rest: dict[str, Any]
fallback_client_id: str | None = None
def column(self, key: str) -> Any:
value = self.columns.get(key)
if key == "client_id" and not value:
return self.fallback_client_id
return value
def filled_from(self, later: "_Identity") -> "_Identity":
"""This answer, with gaps and None values filled from ``later``.
Values already here win, so the call's record agrees with the events
recorded during the call, which saw only this answer.
"""
def fill(first: dict[str, Any], second: dict[str, Any]) -> dict[str, Any]:
merged = dict(first)
for key, value in second.items():
if merged.get(key) is None:
merged[key] = value
return merged
return _Identity(
fill(self.columns, later.columns),
fill(self.rest, later.rest),
self.fallback_client_id or later.fallback_client_id,
)
def _is_sink(sink: Any, module: str, name: str) -> bool:
"""isinstance against a sink class without importing its module.
storage and embeddings need the async database stack. When the module was
never imported, no instance of its class can exist, so the answer is no.
"""
loaded = sys.modules.get(f"{__package__}.{module}")
cls = getattr(loaded, name, None) if loaded is not None else None
return cls is not None and isinstance(sink, cls)
def _first_sink(sinks: Iterable[Any], module: str, name: str) -> Any:
return next((s for s in sinks if _is_sink(s, module, name)), None)
def _current_context() -> Any:
try:
from fastmcp.server.dependencies import get_context
@@ -0,0 +1,108 @@
"""Size limits for stored payloads: call arguments and results, event attrs.
A value whose JSON encoding is larger than the limit is stored as a marker:
{"_truncated": true, "size": 812345, "preview": "{\"frames\": [[0.1, ..."}
``size`` is the value's encoded size in bytes, measured as for ``args_size``
(``json.dumps(value, default=str)``, non-ASCII escaped). ``preview`` is the
start of its JSON text, ``preview_chars(limit)`` characters: ``PREVIEW_CHARS``
(1000) under the default limit, a quarter of the limit under a smaller one,
so the marker itself fits. It is taken from the value after redaction, so a
secret cannot reach it. The keys and their meaning are stable.
"""
import json
from typing import Any
TRUNCATED = "_truncated"
PREVIEW_CHARS = 1000
DEFAULT_MAX_PAYLOAD_BYTES = 64 * 1024
# Incremental, so a preview of a large value encodes only its start.
_ENCODER = json.JSONEncoder(default=str, ensure_ascii=False)
def json_size(value: Any) -> int | None:
"""Bytes of ``value`` as JSON, or None when it cannot be encoded."""
try:
return len(json.dumps(value, default=str).encode())
except Exception:
return None
def is_truncated(value: Any) -> bool:
"""Whether ``value`` is a marker left by the size limit."""
return isinstance(value, dict) and value.get(TRUNCATED) is True
def preview(value: Any, chars: int = PREVIEW_CHARS) -> str:
"""The first ``chars`` characters of ``value``'s JSON text."""
if chars <= 0:
return ""
parts: list[str] = []
length = 0
try:
for chunk in _ENCODER.iterencode(value):
parts.append(chunk)
length += len(chunk)
if length >= chars:
break
except Exception:
return ""
return "".join(parts)[:chars]
def preview_chars(limit: int) -> int:
"""How much of a value's JSON text a marker keeps under ``limit``."""
return min(PREVIEW_CHARS, limit // 4)
def marker(value: Any, size: int | None, limit: int) -> dict[str, Any]:
"""The stand-in stored for a ``value`` over ``limit`` bytes."""
return {
TRUNCATED: True,
"size": size,
"preview": preview(value, preview_chars(limit)),
}
def cap(value: Any, size: int | None, limit: int | None) -> Any:
"""``value``, or its marker when ``size`` is over ``limit``.
``size`` is passed in because the caller has usually measured it already
(``args_size``, ``result_size``), so this costs no second encoding.
"""
if limit is None or size is None or size <= limit:
return value
return marker(value, size, limit)
def cap_attrs(attrs: dict[str, Any], limit: int | None) -> dict[str, Any]:
"""Event attrs within ``limit`` bytes, keeping the dict's keys.
The largest values are replaced by markers, one at a time, until the
whole fits, so small fields (``ok``, ``phase``, ``model``) stay readable.
If markers alone cannot get it under the limit (very many keys, or a
limit of a few dozen bytes), the whole dict becomes one marker, which
under such a tiny limit can itself be larger than the limit.
"""
if limit is None or not attrs:
return attrs
total = json_size(attrs)
if total is None or total <= limit:
return attrs
sizes = {key: json_size(value) or 0 for key, value in attrs.items()}
out = dict(attrs)
for key in sorted(sizes, key=sizes.__getitem__, reverse=True):
if total <= limit:
break
stand_in = marker(attrs[key], sizes[key], limit)
smaller = sizes[key] - (json_size(stand_in) or 0)
if smaller <= 0:
continue
out[key] = stand_in
total -= smaller
if total > limit:
return marker(attrs, json_size(attrs), limit)
return out
@@ -40,22 +40,21 @@ from sqlalchemy.exc import DBAPIError
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
from sqlalchemy.types import TypeDecorator
from .constants import (
DEFAULT_EMBEDDING_DIM,
DEFAULT_TABLE_PREFIX,
FEEDBACK_SOURCE,
)
from .constants import EMBEDDING_SOURCES as EMBEDDING_SOURCES # re-export
from .events import MAX_KEY_CHARS, MAX_KIND_CHARS, EventRecord
from .record import ToolCallRecord
from .sinks import Record
logger = logging.getLogger(__name__)
DEFAULT_TABLE_PREFIX = "ffb_"
# none_as_null: a missing payload is SQL NULL, so "WHERE args IS NULL" works.
_JSON = JSON(none_as_null=True).with_variant(JSONB(none_as_null=True), "postgresql")
DEFAULT_EMBEDDING_DIM = 1024
# source_type values in {prefix}embeddings. Feedback rows survive pruning.
EMBEDDING_SOURCES = ("feedback", "call_error", "event", "llm")
FEEDBACK_SOURCE = "feedback"
class EmbeddingVector(TypeDecorator):
"""pgvector ``VECTOR(dim)`` on PostgreSQL, a JSON list of floats elsewhere.
+9 -12
View File
@@ -5,6 +5,7 @@ from typing import Any
from sqlalchemy import text
from ..database import count_feedback_by
from ..insights import record_feedback_retrieval, record_statistics_view
from .base import BaseFeedbackMixin, Feedback, feedback_to_dict, logger, mcp_tool
@@ -93,7 +94,10 @@ class RetrievalMixin(BaseFeedbackMixin):
@mcp_tool(description="Get feedback statistics and analytics")
async def get_feedback_statistics(self) -> dict[str, Any]:
"""Get comprehensive feedback statistics.
"""Feedback counts: total, per type, per status, and the last 7 days.
``by_type`` and ``by_status`` are keyed by the same lowercase values
``list_feedback`` returns (``bug``, ``in_progress``).
Returns:
Dictionary with statistics.
@@ -106,17 +110,10 @@ class RetrievalMixin(BaseFeedbackMixin):
total_result = session.execute(text("SELECT COUNT(*) FROM feedback"))
total_count = total_result.scalar() or 0
# Count by type
type_result = session.execute(
text("SELECT type, COUNT(*) FROM feedback GROUP BY type")
)
by_type = dict(type_result.fetchall()) if total_count > 0 else {}
# Count by status
status_result = session.execute(
text("SELECT status, COUNT(*) FROM feedback GROUP BY status")
)
by_status = dict(status_result.fetchall()) if total_count > 0 else {}
# Keyed by enum value ("bug", "open"), as list_feedback
# returns them; the database stores the names.
by_type = count_feedback_by(session, "type")
by_status = count_feedback_by(session, "status")
# Recent count (last 7 days)
week_ago = datetime.now(UTC) - timedelta(days=7)
+105
View File
@@ -0,0 +1,105 @@
"""Exercise the instrumentation as a base install would, then print JSON.
Run in a fresh interpreter. greenlet and aiosqlite are made unimportable
before anything else loads, as on ``pip install fastmcp-feedback`` without
the ``instrumentation`` extra. Argument: a directory for the feedback database.
"""
import sys
from importlib.metadata import version
# SQLAlchemy 2.1 refuses to import sqlalchemy.ext.asyncio without greenlet,
# once: a second import succeeds. 2.0 imports it and fails on first use.
# On 2.0 the module is blocked too, so both versions fail at import.
_blocked = ["greenlet", "aiosqlite"]
if tuple(int(p) for p in version("sqlalchemy").split(".")[:2]) < (2, 1):
_blocked.append("sqlalchemy.ext.asyncio")
for _name in _blocked:
sys.modules[_name] = None # import raises ModuleNotFoundError
import asyncio # noqa: E402
import io # noqa: E402
import json # noqa: E402
from pathlib import Path # noqa: E402
STORAGE = "fastmcp_feedback.instrumentation.storage"
EMBEDDINGS = "fastmcp_feedback.instrumentation.embeddings"
async def run(db_dir: Path) -> dict:
from fastmcp import Client, FastMCP
import fastmcp_feedback.instrumentation as inst
from fastmcp_feedback import add_feedback_tools
from fastmcp_feedback.instrumentation import (
DEFAULT_TABLE_PREFIX,
JsonLinesSink,
MemorySink,
instrument,
)
out: dict = {"prefix": DEFAULT_TABLE_PREFIX}
app = FastMCP("base-install")
@app.tool
def add(a: int, b: int) -> int:
return a + b
memory = MemorySink()
lines = io.StringIO()
mw = instrument(app, [memory, JsonLinesSink(lines)], mode="full")
add_feedback_tools(
app, database_url=f"sqlite:///{db_dir / 'fb.db'}", instrumentation=mw
)
async with Client(app) as client:
result = await client.call_tool("add", {"a": 2, "b": 3})
out["result"] = result.content[0].text
fb = await client.call_tool(
"submit_feedback",
{
"request": {
"type": "bug",
"title": "base install",
"description": "submitted without the extra",
"submitter": "worker",
}
},
)
out["feedback"] = json.loads(fb.content[0].text)
mw.record_event("job.done", key="j1", attrs={"n": 1})
await mw.flush()
out["tools"] = [r.tool for r in memory.records if hasattr(r, "tool")]
out["event_kinds"] = [r.kind for r in memory.records if hasattr(r, "kind")]
out["jsonl_lines"] = len(lines.getvalue().splitlines())
out["capture"] = await mw.capture_recent_calls()
out["similar"] = await mw.similar("anything")
out["similar_feedback"] = await mw.similar_feedback("bug-1")
out["feedback_context"] = await mw.feedback_context("bug-1")
out["link_feedback"] = await mw.link_feedback("bug-1", [])
out["events_for"] = len(await mw.events_for("j1"))
out["database_sink"] = mw.database_sink is None
out["embedding_sink"] = mw.embedding_sink is None
await mw.aclose()
errors = {}
for name in (
"DatabaseSink",
"EmbeddingSink",
"build_metadata",
"backfill_feedback",
):
for _attempt in range(2): # SQLAlchemy only complains the first time
try:
getattr(inst, name)
errors.setdefault(name, []).append(None)
except ImportError as exc:
errors.setdefault(name, []).append(str(exc))
out["errors"] = errors
out["loaded"] = sorted(m for m in (STORAGE, EMBEDDINGS) if m in sys.modules)
return out
if __name__ == "__main__":
print(json.dumps(asyncio.run(run(Path(sys.argv[1]))), default=str))
+4 -3
View File
@@ -248,9 +248,10 @@ class TestDemoServerIntegration:
stats = stats_result.structured_content
assert stats["total_count"] >= 1
assert "FEATURE" in stats["by_type"]
assert stats["by_type"]["FEATURE"] >= 1
assert "IN_PROGRESS" in stats["by_status"]
# Keyed by the values list_feedback returns, not the stored names.
assert "feature" in stats["by_type"]
assert stats["by_type"]["feature"] >= 1
assert "in_progress" in stats["by_status"]
# Delete feedback
delete_tool = tools["delete_feedback"]
+8 -8
View File
@@ -61,10 +61,10 @@ class TestCompleteWorkflow:
stats = stats_result.structured_content
assert stats["total_count"] >= 1
assert "BUG" in stats["by_type"]
assert stats["by_type"]["BUG"] >= 1
assert "OPEN" in stats["by_status"]
assert stats["by_status"]["OPEN"] >= 1
assert "bug" in stats["by_type"]
assert stats["by_type"]["bug"] >= 1
assert "open" in stats["by_status"]
assert stats["by_status"]["open"] >= 1
# 4. Update feedback status
update_result = await update_tool.run({
@@ -94,8 +94,8 @@ class TestCompleteWorkflow:
final_stats_result = await stats_tool.run({})
final_stats = final_stats_result.structured_content
assert "RESOLVED" in final_stats["by_status"]
assert final_stats["by_status"]["RESOLVED"] >= 1
assert "resolved" in final_stats["by_status"]
assert final_stats["by_status"]["resolved"] >= 1
# 8. Verify insights were recorded
assert len(insights.get_metrics("feedback_submitted")) >= 1
@@ -526,8 +526,8 @@ class TestPerformanceWorkflow:
stats_result = await stats_tool.run({})
stats = stats_result.structured_content
assert stats["total_count"] >= 100
assert "IMPROVEMENT" in stats["by_type"]
assert stats["by_type"]["IMPROVEMENT"] >= 100
assert "improvement" in stats["by_type"]
assert stats["by_type"]["improvement"] >= 100
# Insights should have recorded all submissions
submit_metrics = insights.get_metrics("feedback_submitted")
+565
View File
@@ -0,0 +1,565 @@
"""Tests for backfill_embeddings, backfill_feedback, its CLI, and list/dict
text flattening.
DB tests take the ``db_url`` fixture, so they run on SQLite and, with
FFB_TEST_DATABASE_URL set, on Postgres with pgvector.
"""
import json
import os
import subprocess
import sys
from datetime import UTC, datetime, timedelta
import pytest
from sqlalchemy import func, select, update
from fastmcp_feedback.instrumentation import (
REDACTED,
DatabaseSink,
EmbeddingSink,
EventRecord,
Redactor,
backfill_embeddings,
backfill_feedback,
extract_texts,
)
from fastmcp_feedback.instrumentation.backfill.cli import main as cli_main
from .test_embeddings import DIM, SECRET, FakeEmbedder, call_record, event, rows
T0 = datetime(2026, 9, 1, 12, 0, tzinfo=UTC)
@pytest.fixture
async def db(db_url):
made = []
def factory(name="calls", **kwargs):
kwargs.setdefault("create_tables", True)
kwargs.setdefault("embedding_dim", DIM)
sink = DatabaseSink(db_url(name), **kwargs)
made.append(sink)
return sink
yield factory
for sink in made:
await sink.engine.dispose()
def at(minutes: float) -> datetime:
return T0 + timedelta(minutes=minutes)
def error_call(minute, tool="render", message="render timed out", **kw):
record = call_record(tool=tool, message=message, **kw)
record.started_at = at(minute)
return record
def ev(minute, kind, key=None, **attrs):
record = event(kind, key, **attrs)
record.occurred_at = at(minute)
return record
def mixed_records():
"""One of everything backfill should pick up, and some it should not."""
return [
error_call(1, message="render timed out after 30s"),
error_call(2, tool="find", outcome="soft_error", message="no object Cube"),
call_record(outcome="ok"),
ev(
3, "feedback.submitted", "bug-1", title="Render hangs", description="at 99%"
),
ev(
4, "llm.call", model="m", prompt="summarize the log", completion="it failed"
),
ev(5, "llm.call", model="m", ok=False, error="rate limited"),
ev(6, "llm.call", model="m", ok=True),
ev(7, "job.failed", "job-1", error_excerpt=["line one", "line two"]),
ev(8, "job.progress", "job-1", percent=50),
]
def keyed(stored):
return {(r["source_type"], r["source_id"]): r for r in stored}
async def count_rows(sink):
t = sink.embeddings
async with sink.engine.connect() as conn:
return (await conn.execute(select(func.count()).select_from(t))).scalar_one()
class TestFlattening:
def test_list_and_tuple_join_lines(self):
records = [
event("job.failed", error_excerpt=[" a ", "", None, "b"]),
event("job.failed", error=("x", 1, 2.5)),
]
texts = [t.text for t in extract_texts(records)]
assert texts == ["a\nb", "x\n1\n2.5"]
def test_dict_becomes_sorted_key_lines(self):
records = [event("job.failed", error={"zeta": "last", "alpha": 1, "mid": None})]
[item] = extract_texts(records)
assert item.text == "alpha: 1\nzeta: last"
def test_nested(self):
records = [
event(
"job.failed",
error_excerpt=[["a", "b"], {"k": ["c", "d"]}, ("e",)],
)
]
[item] = extract_texts(records)
assert item.text == "a\nb\nk: c\nd\ne"
assert "[" not in item.text and "{" not in item.text
def test_flattening_applies_to_feedback_and_llm(self):
records = [
event("feedback.submitted", "f1", title=["Two", "parts"], description=None),
event("llm.call", prompt=["p1", "p2"], completion={"text": "done"}),
]
by_type = {t.source_type: t.text for t in extract_texts(records)}
assert by_type["feedback"] == "Two\nparts"
assert by_type["llm"] == "prompt: p1\np2\ncompletion: text: done"
def test_empty_containers_are_no_text(self):
records = [event("job.failed", error_excerpt=[], error={}, message=[[""]])]
assert extract_texts(records) == []
def test_bounded(self):
deep: list = ["bottom"]
for _ in range(100):
deep = [deep]
huge = ["x" * 1000] * 10_000
records = [event("job.failed", error=deep), event("job.failed", message=huge)]
texts = extract_texts(records)
assert len(texts) == 1 # the deep one is cut off before any text
assert len(texts[0].text) < 70_000
def test_other_values_use_str(self):
[item] = extract_texts([event("job.failed", error=42)])
assert item.text == "42"
class TestBackfill:
async def test_everything_once(self, db):
store = db()
await store.write(mixed_records())
emb = EmbeddingSink(FakeEmbedder(), store)
counts = await backfill_embeddings(emb)
assert counts == {
"feedback": 1,
"call_error": 2,
"event": 2,
"llm": 1,
"skipped": 0,
"errors": 0,
}
stored = keyed(await rows(store))
texts = {k[0]: [] for k in stored}
for (source_type, _), row in stored.items():
texts[source_type].append(row["text"])
assert sorted(texts["call_error"]) == [
"find: TimeoutError: no object Cube",
"render: TimeoutError: render timed out after 30s",
]
assert texts["feedback"] == ["Render hangs\nat 99%"]
assert texts["llm"] == ["prompt: summarize the log\ncompletion: it failed"]
assert sorted(texts["event"]) == ["line one\nline two", "rate limited"]
assert ("feedback", "bug-1") in stored
async def test_pre_outcome_error_rows(self, db):
store = db()
old = error_call(1, message="boom from before outcome existed")
old_ok = call_record(outcome="ok")
await store.write([old, old_ok])
t = store.table
async with store.engine.begin() as conn:
await conn.execute(update(t).values(outcome=None))
emb = EmbeddingSink(FakeEmbedder(), store)
counts = await backfill_embeddings(emb)
assert counts["call_error"] == 1
[row] = await rows(store)
assert row["source_id"] == old.id
assert row["text"] == "render: TimeoutError: boom from before outcome existed"
async def test_same_text_hash_and_id_as_live_sink(self, db):
records = mixed_records() + [
ev(9, "job.failed", error={"zeta": 1, "alpha": {"b": 2, "a": [1, 2]}}),
]
backfilled, live = db("backfilled"), db("live")
await backfilled.write(records)
await backfill_embeddings(EmbeddingSink(FakeEmbedder(), backfilled))
live_emb = EmbeddingSink(FakeEmbedder(), live)
await live_emb.write(records)
await live_emb.flush()
assert live_emb.errors == 0
def shape(stored):
return sorted(
(r["source_type"], r["source_id"], r["text"], r["text_hash"])
for r in stored
)
assert shape(await rows(backfilled)) == shape(await rows(live))
assert len(await rows(live)) == 7
async def test_live_records_dedupe_against_backfill(self, db):
records = mixed_records() + [
ev(9, "job.failed", error={"zeta": 1, "alpha": {"b": 2, "a": [1, 2]}}),
]
store = db()
await store.write(records)
await backfill_embeddings(EmbeddingSink(FakeEmbedder(), store))
before = await count_rows(store)
embedder = FakeEmbedder()
live = EmbeddingSink(embedder, store)
# The records as they come back from the database, as a live sink
# in another process would have seen them before storing.
await live.write(records)
await live.flush()
assert embedder.calls == []
assert live.embedded == 0 and live.skipped == 7
assert await count_rows(store) == before
async def test_second_run_embeds_nothing(self, db):
store = db()
await store.write(mixed_records())
embedder = FakeEmbedder()
emb = EmbeddingSink(embedder, store)
await backfill_embeddings(emb)
calls = len(embedder.calls)
again = await backfill_embeddings(emb)
assert again == {
"feedback": 0,
"call_error": 0,
"event": 0,
"llm": 0,
"skipped": 6,
"errors": 0,
}
assert len(embedder.calls) == calls
async def test_changed_text_replaces(self, db):
store = db()
record = error_call(1, message="first")
await store.write([record])
emb = EmbeddingSink(FakeEmbedder(), store)
await backfill_embeddings(emb)
async with store.engine.begin() as conn:
await conn.execute(
update(store.table).values(error_message="second"),
)
counts = await backfill_embeddings(emb)
assert counts["call_error"] == 1
[row] = await rows(store)
assert row["text"] == "render: TimeoutError: second"
async def test_sources(self, db):
store = db()
await store.write(mixed_records())
emb = EmbeddingSink(FakeEmbedder(), store)
counts = await backfill_embeddings(emb, sources=["feedback", "llm"])
assert counts == {"feedback": 1, "llm": 1, "skipped": 0, "errors": 0}
assert {r["source_type"] for r in await rows(store)} == {"feedback", "llm"}
async def test_defaults_to_sink_sources(self, db):
store = db()
await store.write(mixed_records())
emb = EmbeddingSink(FakeEmbedder(), store, sources=["call_error"])
counts = await backfill_embeddings(emb)
assert counts == {"call_error": 2, "skipped": 0, "errors": 0}
async def test_since_until(self, db):
store = db()
await store.write([error_call(m, message=f"error {m}") for m in range(10)])
emb = EmbeddingSink(FakeEmbedder(), store)
counts = await backfill_embeddings(emb, since=at(3), until=at(6))
assert counts["call_error"] == 3
texts = sorted(r["text"] for r in await rows(store))
assert texts == [f"render: TimeoutError: error {m}" for m in (3, 4, 5)]
# A naive datetime is UTC.
naive = (T0 + timedelta(minutes=8)).replace(tzinfo=None)
counts = await backfill_embeddings(emb, since=naive)
assert counts["call_error"] == 2
async def test_limit_takes_oldest_and_resumes(self, db):
store = db()
await store.write([error_call(m, message=f"error {m}") for m in range(10)])
emb = EmbeddingSink(FakeEmbedder(), store)
counts = await backfill_embeddings(emb, limit=4, batch_size=3)
assert counts["call_error"] == 4
texts = sorted(r["text"] for r in await rows(store))
assert texts == [f"render: TimeoutError: error {m}" for m in range(4)]
counts = await backfill_embeddings(emb, limit=4, batch_size=3)
assert counts["call_error"] == 4 and counts["skipped"] == 4
counts = await backfill_embeddings(emb, limit=0)
assert counts["call_error"] == 0
async def test_pages_through_many_rows(self, db):
store = db()
# Identical timestamps too, so paging must break ties on id.
records = [error_call(m // 5, message=f"error {m}") for m in range(53)]
records += [ev(m // 5, "job.failed", error=f"job error {m}") for m in range(41)]
await store.write(records)
embedder = FakeEmbedder()
emb = EmbeddingSink(embedder, store, batch_size=8)
counts = await backfill_embeddings(emb, batch_size=7)
assert counts["call_error"] == 53 and counts["event"] == 41
assert await count_rows(store) == 94
assert len(embedder.seen) == 94 # nothing read twice
assert max(len(c) for c in embedder.calls) <= 8
async def test_embedder_errors_are_counted(self, db):
store = db()
await store.write([error_call(m, message=f"error {m}") for m in range(5)])
emb = EmbeddingSink(FakeEmbedder(fail=True), store, batch_size=2)
counts = await backfill_embeddings(emb)
assert counts["errors"] == 5 and counts["call_error"] == 0
assert await count_rows(store) == 0
emb.embedder.fail = False
counts = await backfill_embeddings(emb)
assert counts["call_error"] == 5 and counts["errors"] == 0
async def test_wrong_vector_count_is_counted(self, db):
class Short(FakeEmbedder):
async def embed(self, texts):
return (await super().embed(texts))[:-1]
store = db()
await store.write([error_call(1), error_call(2, message="other")])
counts = await backfill_embeddings(EmbeddingSink(Short(), store))
assert counts["errors"] == 2
async def test_redaction(self, db):
store = db()
leaked = error_call(1, message=f"auth failed with {SECRET}")
custom = ev(2, "job.failed", error="ticket ACME-1234 failed")
await store.write([leaked, custom])
embedder = FakeEmbedder()
emb = EmbeddingSink(
embedder, store, redactor=Redactor(extra_patterns=[r"ACME-\d+"])
)
await backfill_embeddings(emb)
texts = sorted(r["text"] for r in await rows(store))
assert SECRET not in "".join(embedder.seen + texts)
assert "ACME-1234" not in "".join(embedder.seen + texts)
assert f"render: TimeoutError: auth failed with {REDACTED}" in texts
assert f"ticket {REDACTED} failed" in texts
async def test_truncation(self, db):
store = db()
await store.write([error_call(1, message="x" * 500)])
emb = EmbeddingSink(FakeEmbedder(), store, max_chars=50)
await backfill_embeddings(emb)
[row] = await rows(store)
assert len(row["text"]) == 50
async def test_dry_run_writes_nothing(self, db):
store = db()
await store.write(mixed_records())
embedder = FakeEmbedder()
emb = EmbeddingSink(embedder, store)
counts = await backfill_embeddings(emb, dry_run=True)
assert counts["call_error"] == 2 and counts["feedback"] == 1
assert embedder.calls == []
assert await rows(store) == []
async def test_missing_tables_raise(self, db):
store = db(create_tables=False)
emb = EmbeddingSink(FakeEmbedder(), store)
with pytest.raises(RuntimeError, match="does not exist"):
await backfill_embeddings(emb)
with pytest.raises(RuntimeError, match="does not exist"):
await backfill_embeddings(emb, dry_run=True)
async def test_bad_arguments(self, db):
emb = EmbeddingSink(FakeEmbedder(), db())
with pytest.raises(ValueError, match="unknown sources"):
await backfill_embeddings(emb, sources=["nope"])
with pytest.raises(ValueError):
await backfill_embeddings(emb, batch_size=0)
with pytest.raises(ValueError):
await backfill_embeddings(emb, limit=-1)
class TestBackfillFeedback:
async def test_matches_feedback_event(self, db):
own, events = db("own"), db("events")
items = [
("bug-1", " Render hangs ", "Stuck at 99%"),
("bug-2", "No description", None),
("bug-3", "", ""),
]
counts = await backfill_feedback(EmbeddingSink(FakeEmbedder(), own), items)
assert counts == {"feedback": 2, "skipped": 0, "errors": 0}
live = EmbeddingSink(FakeEmbedder(), events)
await live.write(
[
EventRecord(
kind="feedback.submitted",
occurred_at=datetime.now(UTC),
key=ref,
attrs={"title": title, "description": desc},
)
for ref, title, desc in items
]
)
await live.flush()
def shape(stored):
return sorted((r["source_id"], r["text"], r["text_hash"]) for r in stored)
assert shape(await rows(own)) == shape(await rows(events))
assert shape(await rows(own))[0][1] == "Render hangs\nStuck at 99%"
async def test_idempotent_and_dedupes_with_events(self, db):
store = db()
emb = EmbeddingSink(FakeEmbedder(), store)
await backfill_feedback(emb, [("bug-1", "Render hangs", "at 99%")])
await store.write(
[
ev(
1,
"feedback.submitted",
"bug-1",
title="Render hangs",
description="at 99%",
)
]
)
counts = await backfill_embeddings(emb)
assert counts["feedback"] == 0 and counts["skipped"] == 1
counts = await backfill_feedback(
emb, iter([("bug-1", "Render hangs", "at 99%")]), batch_size=1
)
assert counts == {"feedback": 0, "skipped": 1, "errors": 0}
async def test_redacted(self, db):
store = db()
emb = EmbeddingSink(FakeEmbedder(), store)
await backfill_feedback(emb, [("bug-1", "Leak", f"my key is {SECRET}")])
[row] = await rows(store)
assert SECRET not in row["text"] and REDACTED in row["text"]
class TestCli:
async def seeded(self, db):
store = db()
await store.write(mixed_records())
await store.ensure_embeddings()
return store
async def test_dry_run_main(self, db, capsys):
store = await self.seeded(db)
url = store.engine.url.render_as_string(hide_password=False)
code = await _in_thread(
cli_main,
[
"--database-url",
url,
"--dim",
str(DIM),
"--dry-run",
"--sources",
"call_error,feedback",
],
)
assert code == 0
out = json.loads(capsys.readouterr().out.strip().splitlines()[-1])
assert out == {
"dry_run": True,
"feedback": 1,
"call_error": 2,
"skipped": 0,
"errors": 0,
}
assert await rows(store) == []
async def test_dry_run_subprocess(self, db):
store = await self.seeded(db)
url = store.engine.url.render_as_string(hide_password=False)
env = {k: v for k, v in os.environ.items() if k != "FFB_EMBED_API_KEY"}
proc = await _in_thread(
subprocess.run,
[
sys.executable,
"-m",
"fastmcp_feedback.instrumentation.backfill",
"--database-url",
url,
"--dim",
str(DIM),
"--dry-run",
],
capture_output=True,
text=True,
env=env,
timeout=60,
)
assert proc.returncode == 0, proc.stderr
out = json.loads(proc.stdout.strip().splitlines()[-1])
assert out["call_error"] == 2 and out["llm"] == 1 and out["event"] == 2
assert "RuntimeWarning" not in proc.stderr
async def test_needs_embed_url_without_dry_run(self, capsys):
with pytest.raises(SystemExit) as exc:
cli_main(["--database-url", "sqlite+aiosqlite:///x.db"])
assert exc.value.code == 2
assert "--embed-base-url" in capsys.readouterr().err
async def test_no_api_key_flag(self, capsys):
with pytest.raises(SystemExit):
cli_main(["--database-url", "sqlite+aiosqlite:///x.db", "--api-key", "k"])
async def test_setup_error_exit_code(self, tmp_path, capsys):
url = f"sqlite+aiosqlite:///{tmp_path / 'empty.db'}"
code = await _in_thread(cli_main, ["--database-url", url, "--dry-run"])
assert code == 2
assert "does not exist" in capsys.readouterr().err
async def test_api_key_from_env(self, db, monkeypatch):
seen = {}
class Recorder(FakeEmbedder):
def __init__(self, base_url, api_key=None, model="m", dim=DIM, **kw):
super().__init__(dim=dim, model=model)
seen["api_key"] = api_key
seen["base_url"] = base_url
store = await self.seeded(db)
url = store.engine.url.render_as_string(hide_password=False)
monkeypatch.setattr(
"fastmcp_feedback.instrumentation.backfill.cli.OpenAIEmbedder", Recorder
)
monkeypatch.setenv("MY_KEY", "secret-value")
code = await _in_thread(
cli_main,
[
"--database-url",
url,
"--dim",
str(DIM),
"--embed-base-url",
"https://embed.example/v1",
"--api-key-env",
"MY_KEY",
],
)
assert code == 0
assert seen == {
"api_key": "secret-value",
"base_url": "https://embed.example/v1",
}
assert await count_rows(store) == 6
async def _in_thread(fn, *args, **kwargs):
# The CLI runs its own event loop, which cannot start on the test's.
import asyncio
return await asyncio.to_thread(fn, *args, **kwargs)
+90
View File
@@ -0,0 +1,90 @@
"""The instrumentation works on a base install, without the database extras.
The worker runs in a fresh interpreter with greenlet and aiosqlite made
unimportable, which is what ``pip install fastmcp-feedback`` leaves out.
"""
import json
import subprocess
import sys
from pathlib import Path
import pytest
WORKER = Path(__file__).parent.parent / "fixtures" / "base_install_worker.py"
@pytest.fixture(scope="module")
def report(tmp_path_factory) -> dict:
proc = subprocess.run(
[sys.executable, str(WORKER), str(tmp_path_factory.mktemp("base"))],
capture_output=True,
text=True,
timeout=60,
)
assert proc.returncode == 0, proc.stderr[-4000:]
return json.loads(proc.stdout.strip().splitlines()[-1])
def test_tool_calls_are_recorded(report):
assert report["result"] == "5"
assert report["tools"] == ["add", "submit_feedback"]
assert report["jsonl_lines"] == 4 # two calls, two events
def test_events_are_recorded(report):
assert report["event_kinds"] == ["feedback.submitted", "job.done"]
assert report["events_for"] == 1
def test_feedback_tools_work_alongside(report):
assert report["feedback"]["success"] is True
assert report["feedback"]["linked_calls"] == 0
def test_database_helpers_return_empty_defaults(report):
assert report["capture"] == []
assert report["similar"] == []
assert report["similar_feedback"] == []
assert report["feedback_context"] == []
assert report["link_feedback"] is False
assert report["database_sink"] is True
assert report["embedding_sink"] is True
def test_constants_import_without_the_extra(report):
assert report["prefix"] == "ffb_"
def test_storage_modules_stay_unloaded(report):
assert report["loaded"] == []
@pytest.mark.parametrize(
("name", "extra"),
[
("DatabaseSink", "[instrumentation]"),
("build_metadata", "[instrumentation]"),
("EmbeddingSink", "[instrumentation,embeddings]"),
("backfill_feedback", "[instrumentation,embeddings]"),
],
)
def test_database_names_explain_the_missing_extra(report, name, extra):
# Asked twice: SQLAlchemy reports a missing greenlet only on first import.
for message in report["errors"][name]:
assert message is not None
assert "greenlet" in message
assert f'uv add "fastmcp-feedback{extra}"' in message
def test_lazy_names_resolve_with_the_extra():
import fastmcp_feedback.instrumentation as inst
from fastmcp_feedback.instrumentation import DatabaseSink, storage
assert DatabaseSink is storage.DatabaseSink
assert set(inst._LAZY) <= set(inst.__all__)
assert set(inst.__all__) <= set(dir(inst))
for name in inst.__all__:
assert getattr(inst, name) is not None
with pytest.raises(AttributeError):
_ = inst.not_a_name
+101
View File
@@ -0,0 +1,101 @@
"""soft_error_classifier: message_keys and dotted paths into nested results."""
import pytest
from fastmcp_feedback.instrumentation import soft_error_classifier
from fastmcp_feedback.instrumentation.classify import MESSAGE_KEYS, lookup
# mcardupilot: a wait that returned because the SITL died, the cause nested.
SITL = soft_error_classifier(
status_keys=("status", "state", "reason"),
extra_statuses=("died",),
message_keys=("snapshot.error", "error", "message", "detail", "reason"),
)
class TestMcardupilot:
def test_died_reads_the_nested_cause(self):
result = {"reason": "died", "snapshot": {"error": "SITL exited: segfault"}}
assert SITL("wait_until", result) == "reason=died: SITL exited: segfault"
def test_timeout_stays_ok(self):
result = {"reason": "timeout", "snapshot": {"error": None, "alt": 12.0}}
assert SITL("wait_until", result) is None
def test_reached_stays_ok(self):
assert SITL("wait_until", {"reason": "reached", "snapshot": {}}) is None
def test_died_without_a_cause_falls_back(self):
assert SITL("wait_until", {"reason": "died"}) == "reason=died: died"
def test_dead_session_state(self):
result = {"state": "died", "error": "lost heartbeat"}
assert SITL("snapshot", result) == "state=died: lost heartbeat"
def test_default_message_keys_cannot_reach_it(self):
plain = soft_error_classifier(
status_keys=("status", "state", "reason"), extra_statuses=("died",)
)
result = {"reason": "died", "snapshot": {"error": "SITL exited: segfault"}}
assert plain("wait_until", result) == "reason=died: died"
class TestLookup:
@pytest.mark.parametrize(
"result",
[
{},
{"snapshot": None},
{"snapshot": "a string"},
{"snapshot": ["error"]},
{"snapshot": 3},
{"snapshot": {}},
{"snapshot": {"other": 1}},
],
)
def test_missing_or_non_dict_step_is_absent(self, result):
assert lookup(result, "snapshot.error") is None
def test_deep_path(self):
assert lookup({"a": {"b": {"c": "x"}}}, "a.b.c") == "x"
def test_literal_dotted_key_wins(self):
result = {"snapshot.error": "literal", "snapshot": {"error": "nested"}}
assert lookup(result, "snapshot.error") == "literal"
def test_plain_key(self):
assert lookup({"error": "x"}, "error") == "x"
assert lookup({"error": None}, "error") is None
def test_odd_paths_never_raise(self):
for key in ("", ".", "a.", ".a", "a..b"):
lookup({"a": {"": {"b": 1}}}, key)
class TestOptions:
def test_default_message_keys(self):
assert MESSAGE_KEYS == ("error", "message", "detail", "reason")
def test_message_keys_order(self):
classify = soft_error_classifier(message_keys=("detail", "error"))
result = {"status": "failed", "error": "e", "detail": "d"}
assert classify("t", result) == "status=failed: d"
def test_dotted_error_keys(self):
classify = soft_error_classifier(error_keys=("result.error",))
assert classify("t", {"result": {"error": " broke "}}) == "result.error: broke"
assert classify("t", {"result": {"error": ""}}) is None
assert classify("t", {"result": "error"}) is None
def test_dotted_error_keys_respect_explicit_success(self):
classify = soft_error_classifier(error_keys=("result.error",))
assert classify("t", {"ok": True, "result": {"error": "warn"}}) is None
def test_nested_dict_message_searched_with_same_keys(self):
classify = soft_error_classifier(message_keys=("problem",))
result = {"status": "failed", "problem": {"problem": "inner"}}
assert classify("t", result) == "status=failed: inner"
def test_is_error_uses_message_keys(self):
classify = soft_error_classifier(message_keys=("why",))
assert classify("t", {"isError": True, "why": "nope"}) == "isError: nope"
+159
View File
@@ -0,0 +1,159 @@
"""After aclose() the dispatcher is closed: late records are dropped, never written."""
import asyncio
import logging
import threading
from fastmcp import Client, FastMCP
from fastmcp_feedback.instrumentation import MemorySink, instrument
from fastmcp_feedback.instrumentation.dispatch import CLOSED
class ClosingSink(MemorySink):
def __init__(self):
super().__init__()
self.closes = 0
async def aclose(self):
self.closes += 1
def make_app():
sink = ClosingSink()
app = FastMCP("closed")
@app.tool
def echo(text: str) -> dict:
return {"echo": text}
mw = instrument(app, [sink])
return app, sink, mw
async def started(mw):
"""A middleware whose dispatcher has a loop and a running task."""
assert mw.record_event("boot", key="b")
await mw.flush()
def kinds(sink):
return [e.kind for e in sink.events]
class TestAfterClose:
async def test_record_event_dropped_and_counted(self):
app, sink, mw = make_app()
await started(mw)
await mw.aclose()
before = mw.dispatcher.dropped
assert mw.record_event("late", key="k") is False
await asyncio.sleep(0.01)
assert kinds(sink) == ["boot"]
assert mw.dispatcher.dropped == before + 1
assert mw.dispatcher.drop_reasons == {CLOSED: 1}
assert [e.kind for e in mw.recent_events] == ["boot"]
assert mw.dispatcher.closed
async def test_thread_after_close_dropped(self):
app, sink, mw = make_app()
await started(mw)
await mw.aclose()
out = []
thread = threading.Thread(target=lambda: out.append(mw.record_event("late")))
thread.start()
await asyncio.to_thread(thread.join)
await asyncio.sleep(0.01)
assert out == [False] and kinds(sink) == ["boot"]
assert mw.dispatcher.drop_reasons[CLOSED] == 1
async def test_tool_call_after_close_runs_but_is_not_written(self):
app, sink, mw = make_app()
async with Client(app) as client:
await client.call_tool("echo", {"text": "before"})
await mw.aclose()
result = await client.call_tool("echo", {"text": "after"})
assert result.data == {"echo": "after"}
await asyncio.sleep(0.01)
assert [r.args_size for r in sink.calls] == [len('{"text": "before"}')]
assert len(mw.recent) == 1
assert mw.dispatcher.drop_reasons[CLOSED] == 1
async def test_close_twice_is_a_no_op(self):
app, sink, mw = make_app()
await started(mw)
await mw.aclose()
await mw.aclose()
assert sink.closes == 1 and mw.dispatcher.drop_reasons == {}
async def test_close_before_any_record(self):
app, sink, mw = make_app()
await mw.aclose()
assert mw.record_event("late") is False
assert mw.dispatcher._task is None and mw.dispatcher._loop is None
async def test_never_rebinds_or_starts_a_task(self):
app, sink, mw = make_app()
await started(mw)
home = mw.dispatcher._loop
await mw.aclose()
mw.dispatcher.bind_loop()
def other_loop():
async def submit():
return mw.record_event("elsewhere")
return asyncio.run(submit())
assert await asyncio.to_thread(other_loop) is False
assert mw.dispatcher._task is None and mw.dispatcher._queue is None
assert mw.dispatcher._loop is home
assert mw.dispatcher.drop_reasons == {CLOSED: 1}
async def test_warns_once(self, caplog):
app, sink, mw = make_app()
await started(mw)
await mw.aclose()
with caplog.at_level(logging.WARNING, "fastmcp_feedback"):
for _ in range(5):
mw.record_event("late")
warnings = [r for r in caplog.records if "after aclose" in r.getMessage()]
assert len(warnings) == 1
assert mw.dispatcher.drop_reasons == {CLOSED: 5}
class TestHandoffAtClose:
"""A record a thread handed to the loop just before aclose(), made
deterministic by blocking the loop while the thread hands it over."""
def hand_over(self, mw, name):
out = []
thread = threading.Thread(target=lambda: out.append(mw.record_event(name)))
thread.start()
thread.join() # blocks the loop: the callback is scheduled, not run
assert out == [True]
async def test_landing_after_the_flush_is_dropped(self):
# Nothing queued: the flush returns without yielding, the dispatcher
# closes, and the handed-over record lands on a closed dispatcher.
app, sink, mw = make_app()
await started(mw)
self.hand_over(mw, "late")
await mw.aclose()
await asyncio.sleep(0.01)
assert kinds(sink) == ["boot"]
assert mw.dispatcher.drop_reasons == {CLOSED: 1}
assert "late" not in [e.kind for e in mw.recent_events]
async def test_landing_during_the_flush_is_delivered(self):
# Something still queued: the flush yields, the record lands while it
# runs, and the flush delivers it before the dispatcher closes.
app, sink, mw = make_app()
await started(mw)
assert mw.record_event("queued") # on the loop, not yet delivered
self.hand_over(mw, "handed")
await mw.aclose()
assert kinds(sink) == ["boot", "queued", "handed"]
assert mw.dispatcher.drop_reasons == {}
await asyncio.sleep(0.01)
assert kinds(sink) == ["boot", "queued", "handed"]
+524
View File
@@ -0,0 +1,524 @@
"""Statistics labels, event identity and logging, as fixed in 2026.10.01.4.
Statistics run on SQLite and, with FFB_TEST_DATABASE_URL set, on Postgres
through the feedback tools' own synchronous engine (psycopg2).
"""
import asyncio
import contextlib
import json
import os
import subprocess
import sys
import tomllib
from pathlib import Path
import pytest
from fastmcp import Client, FastMCP
from fastmcp.exceptions import ToolError
from packaging.requirements import Requirement
from sqlalchemy import String, select, text, type_coerce
from sqlalchemy.engine import make_url
import fastmcp_feedback
from fastmcp_feedback import (
FeedbackDatabase,
ManagementMixin,
RetrievalMixin,
SubmissionMixin,
SubmitFeedbackRequest,
get_database_stats,
)
from fastmcp_feedback.database import enum_label
from fastmcp_feedback.instrumentation import REDACTED, MemorySink, instrument
from fastmcp_feedback.models import Feedback, FeedbackStatus, FeedbackType
ROOT = Path(__file__).resolve().parents[2]
# Statistics
@pytest.fixture
async def feedback_db(db_backend, db_url):
"""A FeedbackDatabase on an empty database of the current backend."""
url = make_url(db_url("feedback"))
if db_backend == "postgres":
pytest.importorskip("psycopg2")
url = url.set(drivername="postgresql+psycopg2")
else:
url = url.set(drivername="sqlite")
db = FeedbackDatabase(url.render_as_string(hide_password=False))
await db.initialize()
yield db
await db.close()
async def seed(db):
"""Two bugs (one in progress) and a feature, through the tools."""
submit = SubmissionMixin(db)
ids = []
for kind in ("bug", "bug", "feature"):
result = await submit.submit_feedback(
SubmitFeedbackRequest(
type=kind, title=f"a {kind}", description="d", submitter="t"
)
)
assert result["success"], result
ids.append(result["feedback_id"])
updated = await ManagementMixin(db).update_feedback_status(
ids[0], FeedbackStatus.IN_PROGRESS
)
assert updated["success"], updated
return ids
class TestStatisticsLabels:
async def test_database_stores_enum_names(self, feedback_db):
# Why the mapping exists: the columns hold names on both backends.
await seed(feedback_db)
async with feedback_db.session() as session:
stored = session.execute(
select(type_coerce(Feedback.type, String)).distinct()
).scalars()
assert set(stored) == {"BUG", "FEATURE"}
async def test_statistics_keyed_by_value(self, feedback_db):
await seed(feedback_db)
stats = await RetrievalMixin(feedback_db).get_feedback_statistics()
assert "error" not in stats, stats
assert stats["total_count"] == 3
assert stats["by_type"] == {"bug": 2, "feature": 1}
assert stats["by_status"] == {"open": 2, "in_progress": 1}
async def test_statistics_keys_match_list_feedback(self, feedback_db):
await seed(feedback_db)
retrieval = RetrievalMixin(feedback_db)
stats = await retrieval.get_feedback_statistics()
listed = (await retrieval.list_feedback(per_page=100))["feedback"]
assert set(stats["by_type"]) == {f["type"] for f in listed}
assert set(stats["by_status"]) == {f["status"] for f in listed}
async def test_filter_by_statistics_key(self, feedback_db):
await seed(feedback_db)
retrieval = RetrievalMixin(feedback_db)
stats = await retrieval.get_feedback_statistics()
for kind, count in stats["by_type"].items():
listed = await retrieval.list_feedback(type_filter=kind)
assert listed["total_count"] == count
async def test_database_stats_keyed_by_value(self, feedback_db):
await seed(feedback_db)
stats = await get_database_stats(feedback_db)
assert stats["by_type"] == {"bug": 2, "feature": 1}
assert stats["by_status"] == {"open": 2, "in_progress": 1}
async def test_empty_database(self, feedback_db):
stats = await RetrievalMixin(feedback_db).get_feedback_statistics()
assert stats["by_type"] == {} and stats["by_status"] == {}
class TestEnumLabel:
@pytest.mark.parametrize(
("enum_cls", "stored", "label"),
[
(FeedbackType, FeedbackType.BUG, "bug"),
(FeedbackType, "BUG", "bug"),
(FeedbackType, "bug", "bug"),
(FeedbackStatus, "IN_PROGRESS", "in_progress"),
(FeedbackStatus, "in_progress", "in_progress"),
(FeedbackStatus, "archived", "archived"),
],
)
def test_label(self, enum_cls, stored, label):
assert enum_label(enum_cls, stored) == label
async def test_rows_written_as_values_are_merged(self, tmp_path):
# A row holding the value instead of the name (written by hand, or by
# another tool) is counted under the same key, not dropped.
db = FeedbackDatabase(f"sqlite:///{tmp_path}/legacy.db")
await db.initialize()
try:
await seed(db)
async with db.session() as session:
session.execute(
text(
"INSERT INTO feedback (type, title, description, submitter,"
" status, created_at, updated_at) VALUES ('bug', 't', 'd',"
" 's', 'open', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)"
)
)
session.commit()
stats = await RetrievalMixin(db).get_feedback_statistics()
assert stats["by_type"] == {"bug": 3, "feature": 1}
assert stats["by_status"] == {"open": 3, "in_progress": 1}
finally:
await db.close()
# Event identity
def make_app(resolver=None, **kwargs):
sink = MemorySink()
app = FastMCP("identity")
holder = {"spawned": []}
@app.tool
async def submit(job_id: str) -> dict:
holder["mw"].record_event("job.submitted", key=job_id)
return {"job_id": job_id}
@app.tool
def submit_sync(job_id: str) -> dict:
holder["mw"].record_event("job.submitted", key=job_id)
return {"job_id": job_id}
@app.tool
async def submit_as(job_id: str) -> dict:
holder["mw"].record_event(
"job.submitted",
key=job_id,
user_sub="given-user",
caller_kind="given-kind",
client_id="given-client",
)
return {"job_id": job_id}
@app.tool
async def chat() -> str:
holder["mw"].record_llm_call(model="m", provider="p")
return "ok"
@app.tool
async def fails() -> str:
holder["mw"].record_event("job.failed")
raise RuntimeError("no")
@app.tool
async def spawn(job_id: str) -> dict:
"""Start a task that records after this call has returned."""
started = asyncio.Event()
async def watcher():
started.set()
await asyncio.sleep(0.05)
holder["mw"].record_event("job.started", key=job_id)
holder["spawned"].append(asyncio.get_running_loop().create_task(watcher()))
await started.wait()
return {"job_id": job_id}
mw = instrument(app, [sink], identity_resolver=resolver, **kwargs)
holder["mw"] = mw
return app, sink, mw, holder["spawned"]
async def call(app, mw, *calls):
async with Client(app) as client:
for tool, args in calls:
with contextlib.suppress(ToolError):
await client.call_tool(tool, args)
await mw.flush()
class Resolver:
"""Counts its invocations; returns a different user each time."""
def __init__(self, extra=None):
self.calls = 0
self.extra = extra or {}
def __call__(self, context):
self.calls += 1
return {
"user_sub": f"user-{self.calls}",
"caller_kind": "pat",
"client_id": "addon",
"org": "acme",
**self.extra,
}
class AsyncResolver(Resolver):
async def __call__(self, context):
await asyncio.sleep(0)
return Resolver.__call__(self, context)
IDENTITY = ("user-1", "pat", "addon")
def ident(record):
return (record.user_sub, record.caller_kind, record.client_id)
class TestEventIdentity:
@pytest.mark.parametrize("resolver_cls", [Resolver, AsyncResolver])
async def test_event_in_call_gets_resolver_identity(self, resolver_cls):
who = resolver_cls()
app, sink, mw, _ = make_app(who)
await call(app, mw, ("submit", {"job_id": "j1"}))
[rec] = sink.calls
[ev] = sink.events
assert ident(ev) == IDENTITY
assert ident(rec) == IDENTITY
assert ev.call_id == rec.id
assert rec.identity == {"org": "acme"}
async def test_sync_tool_event_gets_identity(self):
app, sink, mw, _ = make_app(Resolver())
await call(app, mw, ("submit_sync", {"job_id": "j1"}))
await asyncio.sleep(0.01)
await mw.flush()
[ev] = sink.events
assert ident(ev) == IDENTITY
async def test_llm_call_event_gets_identity(self):
app, sink, mw, _ = make_app(Resolver())
await call(app, mw, ("chat", {}))
[ev] = sink.events
assert ev.kind == "llm.call" and ident(ev) == IDENTITY
async def test_failing_call_event_and_record_share_identity(self):
app, sink, mw, _ = make_app(Resolver())
await call(app, mw, ("fails", {}))
[rec] = sink.calls
[ev] = sink.events
assert rec.outcome == "error"
assert ident(ev) == ident(rec) == IDENTITY
async def test_resolver_runs_once_per_call(self):
who = Resolver()
app, sink, mw, _ = make_app(who)
await call(
app,
mw,
("submit", {"job_id": "a"}),
("chat", {}),
("fails", {}),
("submit", {"job_id": "b"}),
)
assert who.calls == 4
assert [r.user_sub for r in sink.calls] == [
"user-1",
"user-2",
"user-3",
"user-4",
]
# Each event carries its own call's identity.
assert [(e.user_sub, e.call_id) for e in sink.events] == [
(r.user_sub, r.id) for r in sink.calls
]
async def test_explicit_identity_wins(self):
app, sink, mw, _ = make_app(Resolver())
await call(app, mw, ("submit_as", {"job_id": "j1"}))
[ev] = sink.events
assert ident(ev) == ("given-user", "given-kind", "given-client")
assert ident(sink.calls[0]) == IDENTITY
async def test_task_after_call_gets_no_identity(self):
who = Resolver()
app, sink, mw, spawned = make_app(who)
await call(app, mw, ("spawn", {"job_id": "j1"}))
await asyncio.gather(*spawned)
await mw.flush()
[rec] = sink.calls
[ev] = sink.events
assert ident(rec) == IDENTITY
assert ev.kind == "job.started"
assert ev.call_id is None and ev.session_id is None
assert ident(ev) == (None, None, None)
assert who.calls == 1
async def test_outside_call_unchanged(self):
who = Resolver()
app, sink, mw, _ = make_app(who)
mw.record_event("job.finished", key="j1")
mw.record_event("job.finished", key="j2", user_sub="u9", client_id="c9")
await mw.flush()
first, second = sink.events
assert ident(first) == (None, None, None) and first.call_id is None
assert ident(second) == ("u9", None, "c9")
assert who.calls == 0
async def test_identity_redacted_for_events(self):
token = "bmcp_abcdefgh12345678"
app, sink, mw, _ = make_app(Resolver({"caller_kind": f"pat:{token}"}))
await call(app, mw, ("submit", {"job_id": "j1"}))
[ev] = sink.events
assert ev.caller_kind == f"pat:{REDACTED}"
assert sink.calls[0].caller_kind == f"pat:{REDACTED}"
async def test_failing_resolver_leaves_identity_empty(self):
calls = []
def boom(context):
calls.append(1)
raise RuntimeError("resolver bug")
app, sink, mw, _ = make_app(boom)
await call(app, mw, ("submit", {"job_id": "j1"}))
[rec] = sink.calls
[ev] = sink.events
assert rec.ok and ev.call_id == rec.id
assert ev.user_sub is None and rec.user_sub is None
# An empty first answer gets a second run after the tool; it fails too.
assert calls == [1, 1]
async def test_slow_resolver_cut_off_before_tool(self):
async def slow(context):
await asyncio.sleep(5)
return {"user_sub": "late"}
app, sink, mw, _ = make_app(slow, hook_timeout=0.05)
await call(app, mw, ("submit", {"job_id": "j1"}))
[rec] = sink.calls
[ev] = sink.events
assert rec.user_sub is None and ev.user_sub is None and rec.ok
async def test_mode_off_runs_no_resolver(self):
who = Resolver()
app, sink, mw, _ = make_app(who, mode="off")
await call(app, mw, ("submit", {"job_id": "j1"}))
assert who.calls == 0 and sink.calls == [] and sink.events == []
async def test_excluded_tool_runs_no_resolver(self):
who = Resolver()
app, sink, mw, _ = make_app(who, exclude_tools={"submit"})
await call(app, mw, ("submit", {"job_id": "j1"}))
[ev] = sink.events
assert who.calls == 0 and ev.user_sub is None and ev.call_id is None
# Logging
def run_python(code, *args, env=None):
return subprocess.run(
[sys.executable, "-c", code, *args],
capture_output=True,
text=True,
timeout=60,
env=env,
)
class TestLogging:
def test_import_configures_nothing_and_prints_nothing(self):
proc = run_python(
# FastMCP 2.x and 3.0 reset warning filters on import and authlib
# then warns; drop warnings so stderr holds only logging output.
"import warnings\nwarnings.showwarning = lambda *a, **k: None\n"
"import logging, fastmcp_feedback, fastmcp_feedback.instrumentation\n"
"root = logging.getLogger()\n"
"print(len(root.handlers), root.level)\n"
"print([type(h).__name__ for h in"
" logging.getLogger('fastmcp_feedback').handlers])\n"
)
assert proc.returncode == 0, proc.stderr
assert proc.stderr == ""
handlers, names = proc.stdout.splitlines()
assert handlers == f"0 {30}" # no handler, level still WARNING
assert names == "['NullHandler']"
def test_import_logs_nothing_above_debug(self):
# A host that set up logging first sees only the DEBUG line.
proc = run_python(
"import logging, json\n"
"seen = []\n"
"class H(logging.Handler):\n"
" def emit(self, r):\n"
" if r.name.startswith('fastmcp_feedback'):\n"
" seen.append((r.levelname, r.getMessage()))\n"
"logging.basicConfig(level=logging.DEBUG, handlers=[H()])\n"
"import fastmcp_feedback\n"
"print(json.dumps(seen))\n"
)
assert proc.returncode == 0, proc.stderr
seen = json.loads(proc.stdout.splitlines()[-1])
assert [level for level, _ in seen] == ["DEBUG"]
assert "initialized" in seen[0][1]
def test_setup_logging_is_opt_in(self):
proc = run_python("import fastmcp_feedback\nfastmcp_feedback.setup_logging()\n")
assert proc.returncode == 0
assert "fastmcp_feedback - INFO - FastMCP Feedback v" in proc.stderr
def test_package_info_matches_pyproject(self):
pyproject = ROOT / "pyproject.toml"
if not pyproject.exists():
pytest.skip("not running from a source checkout")
project = tomllib.loads(pyproject.read_text())["project"]
info = fastmcp_feedback.PACKAGE_INFO
assert info["version"] == fastmcp_feedback.__version__ == project["version"]
def reqs(items):
return {str(Requirement(r)) for r in items}
assert reqs(info["dependencies"]) == reqs(project["dependencies"])
extras = project["optional-dependencies"]
assert set(info["optional_dependencies"]) == set(extras)
for name, items in extras.items():
if any(r.startswith("fastmcp-feedback") for r in items):
continue # self-references are expanded in the metadata
assert reqs(info["optional_dependencies"][name]) == reqs(items), name
assert info["classifiers"] == sorted(project["classifiers"])
class TestBackfillCliLogging:
@pytest.fixture
def stored_db(self, tmp_path):
"""A calls database with one error, written by the CLI's own sink."""
url = f"sqlite+aiosqlite:///{tmp_path}/calls.db"
code = (
"import asyncio, sys\n"
"from datetime import UTC, datetime\n"
"from fastmcp_feedback.instrumentation import DatabaseSink,"
" ToolCallRecord\n"
"async def main():\n"
" sink = DatabaseSink(sys.argv[1], create_tables=True)\n"
" await sink.write([ToolCallRecord(tool='t',"
" started_at=datetime.now(UTC), duration_ms=1, ok=False,"
" outcome='error', mode='meta', error_type='E',"
" error_message='broke')])\n"
" await sink.ensure_embeddings()\n"
" await sink.engine.dispose()\n"
"asyncio.run(main())\n"
)
proc = run_python(code, url)
assert proc.returncode == 0, proc.stderr
return url
def backfill(self, url, *flags):
env = {k: v for k, v in os.environ.items() if k != "FFB_EMBED_API_KEY"}
return subprocess.run(
[
sys.executable,
"-m",
"fastmcp_feedback.instrumentation.backfill",
"--database-url",
url,
"--dry-run",
*flags,
],
capture_output=True,
text=True,
env=env,
timeout=60,
)
def test_verbose_logs_at_info(self, stored_db):
proc = self.backfill(stored_db, "-v")
assert proc.returncode == 0, proc.stderr
assert "INFO fastmcp_feedback.instrumentation.backfill: Backfill dry run" in (
proc.stderr
)
assert json.loads(proc.stdout.splitlines()[-1])["call_error"] == 1
def test_quiet_without_verbose(self, stored_db):
proc = self.backfill(stored_db)
assert proc.returncode == 0, proc.stderr
assert "INFO" not in proc.stderr
assert "initialized" not in proc.stderr
+5 -5
View File
@@ -140,10 +140,10 @@ class TestDatabaseAdvancedOperations:
stats = await get_database_stats(db)
assert stats["total_count"] == 2
assert stats["by_type"]["BUG"] == 1
assert stats["by_type"]["FEATURE"] == 1
assert stats["by_status"]["OPEN"] == 1
assert stats["by_status"]["RESOLVED"] == 1
assert stats["by_type"]["bug"] == 1
assert stats["by_type"]["feature"] == 1
assert stats["by_status"]["open"] == 1
assert stats["by_status"]["resolved"] == 1
assert stats["database_url"] == db.database_url
assert stats["is_initialized"] is True
@@ -629,7 +629,7 @@ class TestDatabasePerformanceAndLimits:
stats = await get_database_stats(db)
assert stats["total_count"] == 1000
assert len(stats["by_type"]) >= 1
assert stats["by_type"]["IMPROVEMENT"] == 1000
assert stats["by_type"]["improvement"] == 1000
await db.close()
+320
View File
@@ -0,0 +1,320 @@
"""The identity resolver's second run, after the tool, for identity the tool creates.
Modelled on blender-mcp: an addon calls register_client, and only after that
call can the resolver find the bus client (bus_client_uuid, bus_role) for the
caller's session. The first run, before the tool, already knows the user.
"""
import asyncio
import contextlib
import time
from fastmcp import Client, FastMCP
from fastmcp.exceptions import ToolError
from fastmcp_feedback.instrumentation import REDACTED, MemorySink, instrument
class Bus:
"""Who is registered, as the tool leaves it; read by the resolvers."""
def __init__(self):
self.client = None
class BlenderResolver:
"""blender-mcp's identify(): user and role always, bus keys once registered."""
def __init__(self, bus):
self.bus = bus
self.calls = 0
def __call__(self, context):
self.calls += 1
out = {"user_sub": "user-1", "caller_kind": "addon", "client_id": "dcr-9"}
if self.bus.client is not None:
out["bus_client_uuid"] = self.bus.client
out["bus_role"] = "interactive"
return out
class EmptyUntilRegistered:
"""Nothing before registration, every field after."""
def __init__(self, bus):
self.bus = bus
self.calls = 0
def __call__(self, context):
self.calls += 1
if self.bus.client is None:
return {}
return {
"user_sub": "user-1",
"caller_kind": "addon",
"client_id": "dcr-9",
"bus_client_uuid": self.bus.client,
}
def make_app(resolver_cls, *, late=(), sample=None, **kwargs):
bus = Bus()
resolver = resolver_cls(bus)
sink = MemorySink()
app = FastMCP("late-identity")
holder = {}
@app.tool
async def register_client(client_uuid: str) -> dict:
holder["mw"].record_event("bus.registering", key=client_uuid)
bus.client = client_uuid
return {"status": "ok"}
@app.tool
async def register_then_fail(client_uuid: str) -> dict:
bus.client = client_uuid
raise RuntimeError("registered, then broke")
@app.tool
async def quiet() -> str:
return "ok"
@app.tool
async def ping() -> str:
holder["mw"].record_event("bus.ping")
return "pong"
kwargs.setdefault("identity_resolver", resolver)
if sample is not None:
kwargs.update(
sample_rates={"ping": sample, "quiet": sample}, random=lambda: 0.99
)
mw = instrument(app, [sink], late_identity_keys=late, **kwargs)
holder["mw"] = mw
return app, sink, mw, resolver
async def call(app, mw, *calls):
async with Client(app) as client:
for tool, args in calls:
with contextlib.suppress(ToolError):
await client.call_tool(tool, args)
await mw.flush()
def columns(record):
return (record.user_sub, record.caller_kind, record.client_id)
class TestRegistration:
async def test_blender_shaped_registration_row_gets_bus_identity(self):
app, sink, mw, who = make_app(BlenderResolver, late=("bus_client_uuid",))
await call(app, mw, ("register_client", {"client_uuid": "c-1"}))
[rec] = sink.calls
assert columns(rec) == ("user-1", "addon", "dcr-9")
assert rec.identity == {"bus_client_uuid": "c-1", "bus_role": "interactive"}
assert who.calls == 2
# The event recorded during the call saw the first answer only.
[ev] = sink.events
assert ev.call_id == rec.id and columns(ev) == ("user-1", "addon", "dcr-9")
async def test_blender_shaped_without_late_keys_runs_once(self):
# The first answer is not empty, so without late_identity_keys there
# is no second run and the row lacks the bus keys. Opt-in by design.
app, sink, mw, who = make_app(BlenderResolver)
await call(app, mw, ("register_client", {"client_uuid": "c-1"}))
[rec] = sink.calls
assert rec.identity == {} and "bus_client_uuid" not in rec.identity
assert columns(rec) == ("user-1", "addon", "dcr-9")
assert who.calls == 1
async def test_registered_caller_runs_once(self):
app, sink, mw, who = make_app(BlenderResolver, late=("bus_client_uuid",))
await call(
app,
mw,
("register_client", {"client_uuid": "c-1"}),
("ping", {}),
("ping", {}),
)
assert who.calls == 2 + 1 + 1
assert [r.identity.get("bus_client_uuid") for r in sink.calls] == [
"c-1",
"c-1",
"c-1",
]
async def test_empty_first_answer_filled_after(self):
app, sink, mw, who = make_app(EmptyUntilRegistered)
await call(app, mw, ("register_client", {"client_uuid": "c-2"}))
[rec] = sink.calls
[ev] = sink.events
assert columns(rec) == ("user-1", "addon", "dcr-9")
assert rec.identity == {"bus_client_uuid": "c-2"}
assert columns(ev) == (None, None, None) # not rewritten
assert who.calls == 2
async def test_tool_that_raises_gets_second_run_too(self):
app, sink, mw, who = make_app(EmptyUntilRegistered)
await call(app, mw, ("register_then_fail", {"client_uuid": "c-3"}))
[rec] = sink.calls
assert rec.outcome == "error"
assert columns(rec) == ("user-1", "addon", "dcr-9")
assert who.calls == 2
class TestMerge:
async def test_first_answer_wins_on_conflict(self):
answers = iter(
[
{"user_sub": "early", "caller_kind": None, "org": None},
{
"user_sub": "late",
"caller_kind": "k",
"client_id": "c",
"org": "acme",
"team": "qa",
},
]
)
app, sink, mw, _ = make_app(
EmptyUntilRegistered,
identity_resolver=lambda c: next(answers),
late=("team",),
)
await call(app, mw, ("ping", {}))
[rec] = sink.calls
[ev] = sink.events
assert columns(rec) == ("early", "k", "c")
assert rec.identity == {"org": "acme", "team": "qa"}
assert columns(ev) == ("early", None, None)
async def test_second_answer_redacted(self):
token = "bmcp_abcdefgh12345678"
answers = iter([{}, {"user_sub": "u", "caller_kind": f"pat:{token}"}])
app, sink, mw, _ = make_app(
EmptyUntilRegistered, identity_resolver=lambda c: next(answers)
)
await call(app, mw, ("ping", {}))
assert sink.calls[0].caller_kind == f"pat:{REDACTED}"
class TestFailures:
async def test_second_run_raising_keeps_first_answer(self):
runs = []
def who(context):
runs.append(1)
if len(runs) == 1:
return {"user_sub": "u1"}
raise RuntimeError("resolver bug")
app, sink, mw, _ = make_app(
EmptyUntilRegistered, identity_resolver=who, late=("bus_client_uuid",)
)
await call(app, mw, ("ping", {}))
[rec] = sink.calls
assert rec.ok and rec.user_sub == "u1" and len(runs) == 2
async def test_second_run_timing_out_keeps_first_answer(self):
runs = []
async def who(context):
runs.append(1)
if len(runs) == 1:
return {"user_sub": "u1"}
await asyncio.sleep(5)
return {"user_sub": "late", "caller_kind": "k"}
app, sink, mw, _ = make_app(
EmptyUntilRegistered,
identity_resolver=who,
hook_timeout=0.05,
late=("bus_client_uuid",),
)
await call(app, mw, ("ping", {}))
[rec] = sink.calls
assert rec.ok and rec.user_sub == "u1" and rec.caller_kind is None
assert len(runs) == 2
class McspeakResolver:
"""mcspeak's _who: user_sub and caller_kind, never client_id."""
def __init__(self, bus):
self.calls = 0
def __call__(self, context):
self.calls += 1
return {"caller_kind": "claude-code", "user_sub": "mctalkbox"}
class TestWhenItRuns:
async def test_missing_column_alone_does_not_rerun(self):
app, sink, mw, who = make_app(McspeakResolver)
await call(app, mw, ("ping", {}), ("quiet", {}), ("ping", {}))
assert who.calls == 3 and len(sink.calls) == 3
assert {columns(r) for r in sink.calls} == {("mctalkbox", "claude-code", None)}
async def test_only_none_values_count_as_empty(self):
answers = iter([{"user_sub": None}, {"user_sub": "u2", "caller_kind": "k"}])
app, sink, mw, _ = make_app(
EmptyUntilRegistered, identity_resolver=lambda c: next(answers)
)
await call(app, mw, ("ping", {}))
assert columns(sink.calls[0])[:2] == ("u2", "k")
async def test_complete_answer_runs_once(self):
app, sink, mw, who = make_app(BlenderResolver)
await call(app, mw, ("ping", {}), ("ping", {}))
assert who.calls == 2 and len(sink.calls) == 2
async def test_sampled_out_call_makes_no_second_run(self):
app, sink, mw, who = make_app(EmptyUntilRegistered, sample=0.1)
await call(app, mw, ("quiet", {}))
assert sink.calls == [] and who.calls == 1
async def test_sampled_call_kept_for_its_event_gets_second_run(self):
app, sink, mw, who = make_app(EmptyUntilRegistered, sample=0.1)
await call(app, mw, ("ping", {}))
assert len(sink.calls) == 1 and who.calls == 2
async def test_no_resolver_no_runs(self):
app, sink, mw, _ = make_app(EmptyUntilRegistered, identity_resolver=None)
await call(app, mw, ("ping", {}))
[rec] = sink.calls
assert columns(rec) == (None, None, None)
async def test_second_run_overlaps_enricher(self):
# The record is built before the result reaches the client, so the
# second run must not add its time to the enricher's.
runs = []
async def who(context):
runs.append(1)
if len(runs) > 1:
await asyncio.sleep(0.4)
return {}
async def enrich(tool, args, result, context):
await asyncio.sleep(0.4)
return {}
app, sink, mw, _ = make_app(
EmptyUntilRegistered,
identity_resolver=who,
enricher=enrich,
hook_timeout=2,
)
async with Client(app) as client:
start = time.perf_counter()
await client.call_tool("ping", {})
elapsed = time.perf_counter() - start
assert len(runs) == 2
assert elapsed < 0.75, elapsed
async def test_mode_off_runs_nothing():
app, sink, mw, who = make_app(EmptyUntilRegistered, mode="off")
await call(app, mw, ("ping", {}))
assert who.calls == 0
+1 -1
View File
@@ -509,7 +509,7 @@ class TestMixinIntegration:
# 5. Get statistics
stats = await retrieval.get_feedback_statistics()
assert stats["total_count"] >= 1
assert "IN_PROGRESS" in stats["by_status"]
assert "in_progress" in stats["by_status"]
# 6. Final status update
final_result = await management.update_feedback_status(
+250
View File
@@ -0,0 +1,250 @@
"""max_payload_bytes: stored arguments, results and event attrs are bounded."""
import json
import pytest
from fastmcp import Client, FastMCP
from fastmcp_feedback.instrumentation import (
REDACTED,
DatabaseSink,
EventRecord,
MemorySink,
instrument,
)
from fastmcp_feedback.instrumentation import payload as payload_mod
from fastmcp_feedback.instrumentation.embeddings import extract_texts
from fastmcp_feedback.instrumentation.payload import (
DEFAULT_MAX_PAYLOAD_BYTES,
PREVIEW_CHARS,
TRUNCATED,
cap_attrs,
is_truncated,
json_size,
preview_chars,
)
SECRET = "sk-" + "a1b2c3d4e5" * 4 # matches the default sk- pattern
def make_app(**kwargs):
sink = MemorySink()
app = FastMCP("cap")
holder = {}
@app.tool
def echo(blob: str) -> dict:
return {"blob": blob}
@app.tool
def big(n: int) -> dict:
return {"rows": ["x" * 100] * n}
@app.tool
def report(error: str) -> str:
holder["mw"].record_event(
"job.finished", key="j1", attrs={"ok": False, "error": error}
)
return "ok"
kwargs.setdefault("mode", "full")
mw = instrument(app, [sink], **kwargs)
holder["mw"] = mw
return app, sink, mw
async def call(app, mw, tool, args):
async with Client(app) as client:
await client.call_tool(tool, args)
await mw.flush()
class TestCallPayloads:
async def test_default_is_64_kib(self):
app, sink, mw = make_app()
assert mw.max_payload_bytes == DEFAULT_MAX_PAYLOAD_BYTES == 65536
async def test_under_the_cap_stored_whole(self):
app, sink, mw = make_app()
await call(app, mw, "echo", {"blob": "y" * 1000})
[rec] = sink.calls
assert rec.args == {"blob": "y" * 1000}
assert rec.result == {"blob": "y" * 1000}
async def test_over_the_cap_becomes_a_marker(self):
app, sink, mw = make_app()
await call(app, mw, "big", {"n": 2000})
[rec] = sink.calls
assert rec.args == {"n": 2000}
assert rec.result_size > 65536
assert set(rec.result) == {TRUNCATED, "size", "preview"}
assert rec.result[TRUNCATED] is True
assert rec.result["size"] == rec.result_size
assert len(rec.result["preview"]) == PREVIEW_CHARS
assert rec.result["preview"].startswith('{"rows": ["xxxx')
json.dumps(rec.result) # JSON-safe
async def test_args_over_the_cap(self):
app, sink, mw = make_app(max_payload_bytes=500)
await call(app, mw, "echo", {"blob": "z" * 2000})
[rec] = sink.calls
assert rec.args["size"] == rec.args_size and is_truncated(rec.args)
# Under a small cap the preview shrinks so the marker fits.
assert len(rec.args["preview"]) == preview_chars(500) == 125
assert json_size(rec.args) <= 500
async def test_none_means_unlimited(self):
app, sink, mw = make_app(max_payload_bytes=None)
await call(app, mw, "big", {"n": 20000})
[rec] = sink.calls
assert len(rec.result["rows"]) == 20000
async def test_meta_mode_stores_no_payload_but_sizes(self):
app, sink, mw = make_app(mode="meta")
await call(app, mw, "big", {"n": 2000})
[rec] = sink.calls
assert rec.args is None and rec.result is None
assert rec.result_size > 65536
@pytest.mark.parametrize("before_end", [40, 15])
async def test_redaction_happens_before_the_cap(self, before_end):
# At 15 the preview ends inside the secret: cutting the raw JSON there
# would leave a fragment too short for the pattern to match.
app, sink, mw = make_app(max_payload_bytes=4000)
pad = preview_chars(4000) - len('{"blob": "') - before_end
blob = "p" * pad + " " + SECRET + " " + "q" * 5000
await call(app, mw, "echo", {"blob": blob})
[rec] = sink.calls
for stored in (rec.args, rec.result):
assert is_truncated(stored)
assert SECRET[:8] not in stored["preview"]
assert "[REDACT" in stored["preview"]
async def test_secret_key_redacted_in_preview(self):
app, sink, mw = make_app(max_payload_bytes=200)
app.tool(lambda api_key, pad: "ok", name="login")
await call(app, mw, "login", {"api_key": "hunter2-secret", "pad": "x" * 500})
[rec] = sink.calls
assert "hunter2" not in rec.args["preview"]
assert REDACTED in rec.args["preview"]
@pytest.mark.parametrize("bad", [0, -1, True, "64k", 1.5])
def test_invalid_cap_rejected(self, bad):
with pytest.raises(ValueError, match="max_payload_bytes"):
make_app(max_payload_bytes=bad)
class TestNoSecondEncoding:
async def test_payloads_encoded_once(self, monkeypatch):
# The size columns and the cap share one measurement per payload,
# whether or not the payload is over the cap.
app, sink, mw = make_app(max_payload_bytes=1000)
measured = []
original = payload_mod.json_size
def counting(value):
measured.append(1)
return original(value)
monkeypatch.setattr(
"fastmcp_feedback.instrumentation.middleware.json_size", counting
)
await call(app, mw, "echo", {"blob": "small"})
await call(app, mw, "big", {"n": 5000})
assert len(measured) == 4 # args and result, two calls
assert is_truncated(sink.calls[1].result)
def test_preview_stops_early(self, monkeypatch):
chunks = []
encoder = payload_mod._ENCODER
class Counting:
def iterencode(self, value):
for chunk in encoder.iterencode(value):
chunks.append(chunk)
yield chunk
monkeypatch.setattr(payload_mod, "_ENCODER", Counting())
huge = {"rows": ["x" * 100] * 100_000} # about 10 MB
text = payload_mod.preview(huge)
assert len(text) == PREVIEW_CHARS
assert len(chunks) < 50
class TestEventAttrs:
async def test_large_attr_replaced_small_ones_kept(self):
app, sink, mw = make_app(mode="meta")
await call(app, mw, "report", {"error": "E" * 100_000})
[ev] = sink.events
assert ev.attrs["ok"] is False
assert is_truncated(ev.attrs["error"])
assert ev.attrs["error"]["size"] == json_size("E" * 100_000)
assert json_size(ev.attrs) <= 65536
async def test_small_attrs_untouched(self):
app, sink, mw = make_app(mode="meta")
await call(app, mw, "report", {"error": "short"})
assert sink.events[0].attrs == {"ok": False, "error": "short"}
async def test_attrs_redacted_before_the_cap(self):
app, sink, mw = make_app(mode="meta", max_payload_bytes=4000)
error = "x" * (preview_chars(4000) - 30) + f" token {SECRET} " + "y" * 9000
await call(app, mw, "report", {"error": error})
stored = sink.events[0].attrs["error"]
assert SECRET[:8] not in stored["preview"] and REDACTED in stored["preview"]
async def test_outside_a_call_and_unlimited(self):
app, sink, mw = make_app(mode="meta", max_payload_bytes=None)
assert mw.record_event("job.log", key="j", attrs={"log": "L" * 200_000})
await mw.flush()
assert sink.events[0].attrs["log"] == "L" * 200_000
async def test_llm_call_attrs_capped_text_bounded(self):
app, sink, mw = make_app(mode="meta", capture_llm_text=True)
assert mw.record_llm_call(
model="m",
prompt="P" * 100_000,
attrs={"messages": ["M" * 1000] * 100},
)
await mw.flush()
[ev] = sink.events
assert len(ev.attrs["prompt"]) == mw.max_llm_text_chars
assert is_truncated(ev.attrs["messages"])
assert ev.attrs["model"] == "m" and ev.attrs["ok"] is True
def test_many_small_values_become_one_marker(self):
attrs = {f"k{i}": "v" * 50 for i in range(500)}
capped = cap_attrs(attrs, 1000)
assert is_truncated(capped) and capped["size"] == json_size(attrs)
def test_tiny_cap(self):
capped = cap_attrs({"a": "x" * 100}, 10)
assert is_truncated(capped) and len(capped["preview"]) == 2
class TestDownstream:
async def test_marker_round_trips_through_database(self, db_url):
sink = DatabaseSink(db_url("cap"), create_tables=True)
try:
app, _, mw = make_app(mode="meta")
mw.dispatcher.sinks.append(sink)
await call(app, mw, "report", {"error": "E" * 100_000})
rows = await mw.events_for("j1")
[row] = [r for r in rows if r["kind"] == "job.finished"]
assert row["attrs"]["error"][TRUNCATED] is True
assert row["attrs"]["ok"] is False
finally:
await sink.engine.dispose()
def test_embedding_text_uses_the_preview(self):
from datetime import UTC, datetime
marker = payload_mod.marker("disk full: " + "z" * 5000, 5013, 100)
record = EventRecord(
kind="job.failed",
occurred_at=datetime.now(UTC),
attrs={"error": marker},
)
[text] = extract_texts([record])
assert text.text.startswith('"disk full: zzz')
assert "_truncated" not in text.text
+6 -3
View File
@@ -57,7 +57,7 @@ def make_app(sinks=None, **kwargs):
@app.tool
def render(status: str) -> dict:
return {"status": status, "error": f"render said {status}"}
return {"status": status, "detail": f"render said {status}"}
@app.tool
def crash() -> dict:
@@ -155,7 +155,10 @@ class TestSampling:
# A sampled-out call still returns its result.
assert results[0].structured_content == {"status": "idle"}
async def test_sampled_out_ok_calls_skip_hooks(self):
async def test_sampled_out_ok_calls_skip_enricher(self):
# The identity resolver runs before the tool, when sampling cannot
# know yet whether the call will fail or record events; the enricher
# runs after, and only for calls that are kept.
seen = []
def who(context):
@@ -174,7 +177,7 @@ class TestSampling:
)
await call(app, mw, ("poll", {}))
assert sink.records == []
assert seen == []
assert seen == ["identity"]
async def test_unsampled_tools_recorded_without_rate_or_roll(self):
rolls = Rolls()
+157 -1
View File
@@ -59,7 +59,7 @@ def make_app(sinks=None, **kwargs):
@app.tool
def render(status: str) -> dict:
return {"status": status, "error": f"render said {status}"}
return {"status": status, "detail": f"render said {status}"}
@app.tool
def respond(payload: dict) -> dict:
@@ -119,6 +119,19 @@ class TestClassifier:
{"is_error": True, "content": [{"type": "text", "text": "a"}]},
"isError: a",
),
# mcspeak's failures: an error string and no status or flag key.
(
{"played": False, "error": "audio output stalled after 30s"},
"error: audio output stalled after 30s",
),
(
{"error": "Not listening: microphone stopped"},
"error: Not listening: microphone stopped",
),
({"error": " padded "}, "error: padded"),
({"status": "unhealthy", "error": "db gone"}, "error: db gone"),
# Earlier checks still win, with their own message.
({"success": False, "error": "x"}, "success=False: x"),
],
)
def test_flagged(self, result, expected):
@@ -135,6 +148,16 @@ class TestClassifier:
{"result": False},
{"state": 3},
{"isError": False},
{"played": True, "error": None},
{"error": ""},
{"error": " "},
{"error": 0.02},
{"error": 0},
{"error": False},
# A structured error is left to custom classifiers.
{"error": {"code": 1}},
{"error": ["a"]},
{"failure_reason": "x"},
[{"status": "failed"}],
"failed",
None,
@@ -159,6 +182,88 @@ class TestClassifier:
assert classify("t", {"passed": False}) == "passed=False"
assert classify("t", {"status": "failed", "ok": False}) is None
def test_unknown_status_with_error_string(self):
"""blender-mcp's shape: flagged by default, labelled by status when known."""
result = {"status": "unknown_target", "error": "no object named Cube"}
assert DEFAULT_RESULT_CLASSIFIER("t", result) == "error: no object named Cube"
classify = soft_error_classifier(extra_statuses=("unknown_target",))
assert classify("t", result) == "status=unknown_target: no object named Cube"
def test_custom_error_keys(self):
classify = soft_error_classifier(error_keys=("failure_reason",))
assert classify("t", {"failure_reason": "no gpu"}) == "failure_reason: no gpu"
assert classify("t", {"error": "ignored"}) is None
assert classify("t", {"failure_reason": ""}) is None
def test_empty_error_keys_disable(self):
classify = soft_error_classifier(error_keys=())
assert classify("t", {"played": False, "error": "stalled"}) is None
assert classify("t", {"ok": False, "error": "x"}) == "ok=False: x"
class TestExplicitSuccess:
"""A flag key that is exactly True turns an error string into a warning."""
@pytest.mark.parametrize(
"result",
[
{"ok": True, "error": "deprecated arg"},
{"success": True, "error": "x"},
{"ok": True, "status": "unknown_target", "error": "fell back"},
],
)
def test_true_flag_with_error_string_is_ok(self, result):
assert DEFAULT_RESULT_CLASSIFIER("t", result) is None
@pytest.mark.parametrize(
"result, expected",
[
({"ok": False, "error": "x"}, "ok=False: x"),
# Not exactly True, so not an explicit success.
({"ok": 1, "error": "x"}, "error: x"),
({"ok": "yes", "error": "x"}, "error: x"),
({"success": 1.0, "error": "x"}, "error: x"),
# No flag key: mcspeak's shape, as in 2026.10.01.
(
{"played": False, "error": "audio output stalled"},
"error: audio output stalled",
),
# A False flag beats a True one.
({"ok": True, "success": False, "error": "x"}, "success=False: x"),
# A failing status is more specific than the flag, so it still flags.
({"success": True, "status": "failed"}, "status=failed"),
({"ok": True, "state": "error", "error": "x"}, "state=error: x"),
({"ok": True, "isError": True, "content": ["boom"]}, "isError: boom"),
],
)
def test_still_flagged(self, result, expected):
assert DEFAULT_RESULT_CLASSIFIER("t", result) == expected
def test_custom_flag_keys(self):
classify = soft_error_classifier(flag_keys=("passed",))
assert classify("t", {"passed": True, "error": "slow"}) is None
assert classify("t", {"passed": False, "error": "x"}) == "passed=False: x"
# ok is not a flag key here, so it does not suppress the error check.
assert classify("t", {"ok": True, "error": "x"}) == "error: x"
def test_no_flag_keys(self):
classify = soft_error_classifier(flag_keys=())
assert classify("t", {"ok": True, "error": "x"}) == "error: x"
async def test_recorded_as_ok(self):
record = await record_for(
"respond", {"payload": {"ok": True, "error": "deprecated arg"}}
)
assert (record.outcome, record.ok, record.error_message) == ("ok", True, None)
async def test_truthy_flag_recorded_as_soft_error(self):
record = await record_for("respond", {"payload": {"ok": 1, "error": "x"}})
assert (record.outcome, record.ok, record.error_message) == (
"soft_error",
False,
"error: x",
)
class TestMiddleware:
async def test_status_failed_is_soft_error(self):
@@ -186,6 +291,33 @@ class TestMiddleware:
assert REDACTED in record.error_message
assert len(record.error_message) == 60
async def test_error_string_is_soft_error(self):
record = await record_for(
"respond",
{"payload": {"played": False, "error": f"stalled, key {BMCP_TOKEN}"}},
)
assert (record.outcome, record.ok, record.error_type) == (
"soft_error",
False,
"SoftError",
)
assert record.error_message == f"error: stalled, key {REDACTED}"
@pytest.mark.parametrize("value", [None, "", 0.02])
async def test_empty_or_numeric_error_stays_ok(self, value):
record = await record_for(
"respond", {"payload": {"played": True, "error": value}}
)
assert (record.outcome, record.ok, record.error_message) == ("ok", True, None)
async def test_error_keys_disabled(self):
record = await record_for(
"respond",
{"payload": {"error": "stalled"}},
result_classifier=soft_error_classifier(error_keys=()),
)
assert (record.outcome, record.ok) == ("ok", True)
async def test_extra_statuses(self):
record = await record_for(
"render",
@@ -393,3 +525,27 @@ class TestStorageAndCorrelation:
assert record.error_message == (
f"success=False: database down, password={REDACTED}"
)
async def test_list_feedback_failure_is_soft_error(self, monkeypatch):
"""list_feedback reports failure as {"feedback": [], "error": ...}."""
app = FastMCP("fb")
sink = MemorySink()
mw = instrument(app, [sink])
add_feedback_tools(app, database_url="sqlite:///:memory:", instrumentation=mw)
def broken(*args, **kwargs):
raise RuntimeError("database down")
from fastmcp_feedback.database import FeedbackDatabase
monkeypatch.setattr(FeedbackDatabase, "session", broken)
async with Client(app) as client:
result = await client.call_tool("list_feedback", {})
await mw.flush()
assert result.data["error"] == "database down"
[record] = sink.records
assert (record.tool, record.outcome, record.error_message) == (
"list_feedback",
"soft_error",
"error: database down",
)
Generated
+62 -47
View File
@@ -601,7 +601,7 @@ wheels = [
[[package]]
name = "fastmcp-feedback"
version = "2026.9.30"
version = "2026.10.5"
source = { editable = "." }
dependencies = [
{ name = "fastmcp" },
@@ -630,6 +630,7 @@ dev = [
{ name = "asyncpg" },
{ name = "mypy" },
{ name = "pgvector" },
{ name = "psycopg2-binary" },
{ name = "pytest" },
{ name = "pytest-asyncio" },
{ name = "pytest-cov" },
@@ -655,6 +656,7 @@ test = [
{ name = "aiosqlite" },
{ name = "asyncpg" },
{ name = "pgvector" },
{ name = "psycopg2-binary" },
{ name = "pytest" },
{ name = "pytest-asyncio" },
{ name = "pytest-cov" },
@@ -676,7 +678,9 @@ requires-dist = [
{ name = "pgvector", marker = "extra == 'dev'", specifier = ">=0.5.0" },
{ name = "pgvector", marker = "extra == 'embeddings'", specifier = ">=0.5.0" },
{ name = "pgvector", marker = "extra == 'test'", specifier = ">=0.5.0" },
{ name = "psycopg2-binary", marker = "extra == 'dev'", specifier = ">=2.9.0" },
{ name = "psycopg2-binary", marker = "extra == 'postgresql'", specifier = ">=2.9.0" },
{ name = "psycopg2-binary", marker = "extra == 'test'", specifier = ">=2.9.0" },
{ name = "pydantic", specifier = ">=2.11.7" },
{ name = "pymysql", marker = "extra == 'mysql'", specifier = ">=1.1.0" },
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.2.0" },
@@ -1355,54 +1359,65 @@ wheels = [
[[package]]
name = "psycopg2-binary"
version = "2.9.11"
version = "2.9.13"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/ac/6c/8767aaa597ba424643dc87348c6f1754dd9f48e80fdc1b9f7ca5c3a7c213/psycopg2-binary-2.9.11.tar.gz", hash = "sha256:b6aed9e096bf63f9e75edf2581aa9a7e7186d97ab5c177aa6c87797cd591236c", size = 379620, upload-time = "2025-10-10T11:14:48.041Z" }
sdist = { url = "https://files.pythonhosted.org/packages/ed/76/7b4383014be0fcc6c1c0e24292845a14e1672cf17fca62ca0a2bd5f4563d/psycopg2_binary-2.9.13.tar.gz", hash = "sha256:e324ecf60f952d21dd11413b8bbed0951bbd99579a06fd06f28bfc37737cd373", size = 378112, upload-time = "2026-09-10T00:06:12.199Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c7/ae/8d8266f6dd183ab4d48b95b9674034e1b482a3f8619b33a0d86438694577/psycopg2_binary-2.9.11-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:0e8480afd62362d0a6a27dd09e4ca2def6fa50ed3a4e7c09165266106b2ffa10", size = 3756452, upload-time = "2025-10-10T11:11:11.583Z" },
{ url = "https://files.pythonhosted.org/packages/4b/34/aa03d327739c1be70e09d01182619aca8ebab5970cd0cfa50dd8b9cec2ac/psycopg2_binary-2.9.11-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:763c93ef1df3da6d1a90f86ea7f3f806dc06b21c198fa87c3c25504abec9404a", size = 3863957, upload-time = "2025-10-10T11:11:16.932Z" },
{ url = "https://files.pythonhosted.org/packages/48/89/3fdb5902bdab8868bbedc1c6e6023a4e08112ceac5db97fc2012060e0c9a/psycopg2_binary-2.9.11-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2e164359396576a3cc701ba8af4751ae68a07235d7a380c631184a611220d9a4", size = 4410955, upload-time = "2025-10-10T11:11:21.21Z" },
{ url = "https://files.pythonhosted.org/packages/ce/24/e18339c407a13c72b336e0d9013fbbbde77b6fd13e853979019a1269519c/psycopg2_binary-2.9.11-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:d57c9c387660b8893093459738b6abddbb30a7eab058b77b0d0d1c7d521ddfd7", size = 4468007, upload-time = "2025-10-10T11:11:24.831Z" },
{ url = "https://files.pythonhosted.org/packages/91/7e/b8441e831a0f16c159b5381698f9f7f7ed54b77d57bc9c5f99144cc78232/psycopg2_binary-2.9.11-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2c226ef95eb2250974bf6fa7a842082b31f68385c4f3268370e3f3870e7859ee", size = 4165012, upload-time = "2025-10-10T11:11:29.51Z" },
{ url = "https://files.pythonhosted.org/packages/0d/61/4aa89eeb6d751f05178a13da95516c036e27468c5d4d2509bb1e15341c81/psycopg2_binary-2.9.11-cp311-cp311-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a311f1edc9967723d3511ea7d2708e2c3592e3405677bf53d5c7246753591fbb", size = 3981881, upload-time = "2025-10-30T02:55:07.332Z" },
{ url = "https://files.pythonhosted.org/packages/76/a1/2f5841cae4c635a9459fe7aca8ed771336e9383b6429e05c01267b0774cf/psycopg2_binary-2.9.11-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:ebb415404821b6d1c47353ebe9c8645967a5235e6d88f914147e7fd411419e6f", size = 3650985, upload-time = "2025-10-10T11:11:34.975Z" },
{ url = "https://files.pythonhosted.org/packages/84/74/4defcac9d002bca5709951b975173c8c2fa968e1a95dc713f61b3a8d3b6a/psycopg2_binary-2.9.11-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:f07c9c4a5093258a03b28fab9b4f151aa376989e7f35f855088234e656ee6a94", size = 3296039, upload-time = "2025-10-10T11:11:40.432Z" },
{ url = "https://files.pythonhosted.org/packages/6d/c2/782a3c64403d8ce35b5c50e1b684412cf94f171dc18111be8c976abd2de1/psycopg2_binary-2.9.11-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:00ce1830d971f43b667abe4a56e42c1e2d594b32da4802e44a73bacacb25535f", size = 3043477, upload-time = "2025-10-30T02:55:11.182Z" },
{ url = "https://files.pythonhosted.org/packages/c8/31/36a1d8e702aa35c38fc117c2b8be3f182613faa25d794b8aeaab948d4c03/psycopg2_binary-2.9.11-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:cffe9d7697ae7456649617e8bb8d7a45afb71cd13f7ab22af3e5c61f04840908", size = 3345842, upload-time = "2025-10-10T11:11:45.366Z" },
{ url = "https://files.pythonhosted.org/packages/6e/b4/a5375cda5b54cb95ee9b836930fea30ae5a8f14aa97da7821722323d979b/psycopg2_binary-2.9.11-cp311-cp311-win_amd64.whl", hash = "sha256:304fd7b7f97eef30e91b8f7e720b3db75fee010b520e434ea35ed1ff22501d03", size = 2713894, upload-time = "2025-10-10T11:11:48.775Z" },
{ url = "https://files.pythonhosted.org/packages/d8/91/f870a02f51be4a65987b45a7de4c2e1897dd0d01051e2b559a38fa634e3e/psycopg2_binary-2.9.11-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:be9b840ac0525a283a96b556616f5b4820e0526addb8dcf6525a0fa162730be4", size = 3756603, upload-time = "2025-10-10T11:11:52.213Z" },
{ url = "https://files.pythonhosted.org/packages/27/fa/cae40e06849b6c9a95eb5c04d419942f00d9eaac8d81626107461e268821/psycopg2_binary-2.9.11-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f090b7ddd13ca842ebfe301cd587a76a4cf0913b1e429eb92c1be5dbeb1a19bc", size = 3864509, upload-time = "2025-10-10T11:11:56.452Z" },
{ url = "https://files.pythonhosted.org/packages/2d/75/364847b879eb630b3ac8293798e380e441a957c53657995053c5ec39a316/psycopg2_binary-2.9.11-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ab8905b5dcb05bf3fb22e0cf90e10f469563486ffb6a96569e51f897c750a76a", size = 4411159, upload-time = "2025-10-10T11:12:00.49Z" },
{ url = "https://files.pythonhosted.org/packages/6f/a0/567f7ea38b6e1c62aafd58375665a547c00c608a471620c0edc364733e13/psycopg2_binary-2.9.11-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:bf940cd7e7fec19181fdbc29d76911741153d51cab52e5c21165f3262125685e", size = 4468234, upload-time = "2025-10-10T11:12:04.892Z" },
{ url = "https://files.pythonhosted.org/packages/30/da/4e42788fb811bbbfd7b7f045570c062f49e350e1d1f3df056c3fb5763353/psycopg2_binary-2.9.11-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:fa0f693d3c68ae925966f0b14b8edda71696608039f4ed61b1fe9ffa468d16db", size = 4166236, upload-time = "2025-10-10T11:12:11.674Z" },
{ url = "https://files.pythonhosted.org/packages/3c/94/c1777c355bc560992af848d98216148be5f1be001af06e06fc49cbded578/psycopg2_binary-2.9.11-cp312-cp312-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a1cf393f1cdaf6a9b57c0a719a1068ba1069f022a59b8b1fe44b006745b59757", size = 3983083, upload-time = "2025-10-30T02:55:15.73Z" },
{ url = "https://files.pythonhosted.org/packages/bd/42/c9a21edf0e3daa7825ed04a4a8588686c6c14904344344a039556d78aa58/psycopg2_binary-2.9.11-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:ef7a6beb4beaa62f88592ccc65df20328029d721db309cb3250b0aae0fa146c3", size = 3652281, upload-time = "2025-10-10T11:12:17.713Z" },
{ url = "https://files.pythonhosted.org/packages/12/22/dedfbcfa97917982301496b6b5e5e6c5531d1f35dd2b488b08d1ebc52482/psycopg2_binary-2.9.11-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:31b32c457a6025e74d233957cc9736742ac5a6cb196c6b68499f6bb51390bd6a", size = 3298010, upload-time = "2025-10-10T11:12:22.671Z" },
{ url = "https://files.pythonhosted.org/packages/66/ea/d3390e6696276078bd01b2ece417deac954dfdd552d2edc3d03204416c0c/psycopg2_binary-2.9.11-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:edcb3aeb11cb4bf13a2af3c53a15b3d612edeb6409047ea0b5d6a21a9d744b34", size = 3044641, upload-time = "2025-10-30T02:55:19.929Z" },
{ url = "https://files.pythonhosted.org/packages/12/9a/0402ded6cbd321da0c0ba7d34dc12b29b14f5764c2fc10750daa38e825fc/psycopg2_binary-2.9.11-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:62b6d93d7c0b61a1dd6197d208ab613eb7dcfdcca0a49c42ceb082257991de9d", size = 3347940, upload-time = "2025-10-10T11:12:26.529Z" },
{ url = "https://files.pythonhosted.org/packages/b1/d2/99b55e85832ccde77b211738ff3925a5d73ad183c0b37bcbbe5a8ff04978/psycopg2_binary-2.9.11-cp312-cp312-win_amd64.whl", hash = "sha256:b33fabeb1fde21180479b2d4667e994de7bbf0eec22832ba5d9b5e4cf65b6c6d", size = 2714147, upload-time = "2025-10-10T11:12:29.535Z" },
{ url = "https://files.pythonhosted.org/packages/ff/a8/a2709681b3ac11b0b1786def10006b8995125ba268c9a54bea6f5ae8bd3e/psycopg2_binary-2.9.11-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:b8fb3db325435d34235b044b199e56cdf9ff41223a4b9752e8576465170bb38c", size = 3756572, upload-time = "2025-10-10T11:12:32.873Z" },
{ url = "https://files.pythonhosted.org/packages/62/e1/c2b38d256d0dafd32713e9f31982a5b028f4a3651f446be70785f484f472/psycopg2_binary-2.9.11-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:366df99e710a2acd90efed3764bb1e28df6c675d33a7fb40df9b7281694432ee", size = 3864529, upload-time = "2025-10-10T11:12:36.791Z" },
{ url = "https://files.pythonhosted.org/packages/11/32/b2ffe8f3853c181e88f0a157c5fb4e383102238d73c52ac6d93a5c8bffe6/psycopg2_binary-2.9.11-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8c55b385daa2f92cb64b12ec4536c66954ac53654c7f15a203578da4e78105c0", size = 4411242, upload-time = "2025-10-10T11:12:42.388Z" },
{ url = "https://files.pythonhosted.org/packages/10/04/6ca7477e6160ae258dc96f67c371157776564679aefd247b66f4661501a2/psycopg2_binary-2.9.11-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:c0377174bf1dd416993d16edc15357f6eb17ac998244cca19bc67cdc0e2e5766", size = 4468258, upload-time = "2025-10-10T11:12:48.654Z" },
{ url = "https://files.pythonhosted.org/packages/3c/7e/6a1a38f86412df101435809f225d57c1a021307dd0689f7a5e7fe83588b1/psycopg2_binary-2.9.11-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5c6ff3335ce08c75afaed19e08699e8aacf95d4a260b495a4a8545244fe2ceb3", size = 4166295, upload-time = "2025-10-10T11:12:52.525Z" },
{ url = "https://files.pythonhosted.org/packages/f2/7d/c07374c501b45f3579a9eb761cbf2604ddef3d96ad48679112c2c5aa9c25/psycopg2_binary-2.9.11-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:84011ba3109e06ac412f95399b704d3d6950e386b7994475b231cf61eec2fc1f", size = 3983133, upload-time = "2025-10-30T02:55:24.329Z" },
{ url = "https://files.pythonhosted.org/packages/82/56/993b7104cb8345ad7d4516538ccf8f0d0ac640b1ebd8c754a7b024e76878/psycopg2_binary-2.9.11-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ba34475ceb08cccbdd98f6b46916917ae6eeb92b5ae111df10b544c3a4621dc4", size = 3652383, upload-time = "2025-10-10T11:12:56.387Z" },
{ url = "https://files.pythonhosted.org/packages/2d/ac/eaeb6029362fd8d454a27374d84c6866c82c33bfc24587b4face5a8e43ef/psycopg2_binary-2.9.11-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:b31e90fdd0f968c2de3b26ab014314fe814225b6c324f770952f7d38abf17e3c", size = 3298168, upload-time = "2025-10-10T11:13:00.403Z" },
{ url = "https://files.pythonhosted.org/packages/2b/39/50c3facc66bded9ada5cbc0de867499a703dc6bca6be03070b4e3b65da6c/psycopg2_binary-2.9.11-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:d526864e0f67f74937a8fce859bd56c979f5e2ec57ca7c627f5f1071ef7fee60", size = 3044712, upload-time = "2025-10-30T02:55:27.975Z" },
{ url = "https://files.pythonhosted.org/packages/9c/8e/b7de019a1f562f72ada81081a12823d3c1590bedc48d7d2559410a2763fe/psycopg2_binary-2.9.11-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:04195548662fa544626c8ea0f06561eb6203f1984ba5b4562764fbeb4c3d14b1", size = 3347549, upload-time = "2025-10-10T11:13:03.971Z" },
{ url = "https://files.pythonhosted.org/packages/80/2d/1bb683f64737bbb1f86c82b7359db1eb2be4e2c0c13b947f80efefa7d3e5/psycopg2_binary-2.9.11-cp313-cp313-win_amd64.whl", hash = "sha256:efff12b432179443f54e230fdf60de1f6cc726b6c832db8701227d089310e8aa", size = 2714215, upload-time = "2025-10-10T11:13:07.14Z" },
{ url = "https://files.pythonhosted.org/packages/64/12/93ef0098590cf51d9732b4f139533732565704f45bdc1ffa741b7c95fb54/psycopg2_binary-2.9.11-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:92e3b669236327083a2e33ccfa0d320dd01b9803b3e14dd986a4fc54aa00f4e1", size = 3756567, upload-time = "2025-10-10T11:13:11.885Z" },
{ url = "https://files.pythonhosted.org/packages/7c/a9/9d55c614a891288f15ca4b5209b09f0f01e3124056924e17b81b9fa054cc/psycopg2_binary-2.9.11-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:e0deeb03da539fa3577fcb0b3f2554a97f7e5477c246098dbb18091a4a01c16f", size = 3864755, upload-time = "2025-10-10T11:13:17.727Z" },
{ url = "https://files.pythonhosted.org/packages/13/1e/98874ce72fd29cbde93209977b196a2edae03f8490d1bd8158e7f1daf3a0/psycopg2_binary-2.9.11-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:9b52a3f9bb540a3e4ec0f6ba6d31339727b2950c9772850d6545b7eae0b9d7c5", size = 4411646, upload-time = "2025-10-10T11:13:24.432Z" },
{ url = "https://files.pythonhosted.org/packages/5a/bd/a335ce6645334fb8d758cc358810defca14a1d19ffbc8a10bd38a2328565/psycopg2_binary-2.9.11-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:db4fd476874ccfdbb630a54426964959e58da4c61c9feba73e6094d51303d7d8", size = 4468701, upload-time = "2025-10-10T11:13:29.266Z" },
{ url = "https://files.pythonhosted.org/packages/44/d6/c8b4f53f34e295e45709b7568bf9b9407a612ea30387d35eb9fa84f269b4/psycopg2_binary-2.9.11-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:47f212c1d3be608a12937cc131bd85502954398aaa1320cb4c14421a0ffccf4c", size = 4166293, upload-time = "2025-10-10T11:13:33.336Z" },
{ url = "https://files.pythonhosted.org/packages/4b/e0/f8cc36eadd1b716ab36bb290618a3292e009867e5c97ce4aba908cb99644/psycopg2_binary-2.9.11-cp314-cp314-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e35b7abae2b0adab776add56111df1735ccc71406e56203515e228a8dc07089f", size = 3983184, upload-time = "2025-10-30T02:55:32.483Z" },
{ url = "https://files.pythonhosted.org/packages/53/3e/2a8fe18a4e61cfb3417da67b6318e12691772c0696d79434184a511906dc/psycopg2_binary-2.9.11-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fcf21be3ce5f5659daefd2b3b3b6e4727b028221ddc94e6c1523425579664747", size = 3652650, upload-time = "2025-10-10T11:13:38.181Z" },
{ url = "https://files.pythonhosted.org/packages/76/36/03801461b31b29fe58d228c24388f999fe814dfc302856e0d17f97d7c54d/psycopg2_binary-2.9.11-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:9bd81e64e8de111237737b29d68039b9c813bdf520156af36d26819c9a979e5f", size = 3298663, upload-time = "2025-10-10T11:13:44.878Z" },
{ url = "https://files.pythonhosted.org/packages/97/77/21b0ea2e1a73aa5fa9222b2a6b8ba325c43c3a8d54272839c991f2345656/psycopg2_binary-2.9.11-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:32770a4d666fbdafab017086655bcddab791d7cb260a16679cc5a7338b64343b", size = 3044737, upload-time = "2025-10-30T02:55:35.69Z" },
{ url = "https://files.pythonhosted.org/packages/67/69/f36abe5f118c1dca6d3726ceae164b9356985805480731ac6712a63f24f0/psycopg2_binary-2.9.11-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:c3cb3a676873d7506825221045bd70e0427c905b9c8ee8d6acd70cfcbd6e576d", size = 3347643, upload-time = "2025-10-10T11:13:53.499Z" },
{ url = "https://files.pythonhosted.org/packages/e1/36/9c0c326fe3a4227953dfb29f5d0c8ae3b8eb8c1cd2967aa569f50cb3c61f/psycopg2_binary-2.9.11-cp314-cp314-win_amd64.whl", hash = "sha256:4012c9c954dfaccd28f94e84ab9f94e12df76b4afb22331b1f0d3154893a6316", size = 2803913, upload-time = "2025-10-10T11:13:57.058Z" },
{ url = "https://files.pythonhosted.org/packages/7b/03/639c96ff8ffb933868252308a9917ff4170d8c7f4bd16cd0ea2536814126/psycopg2_binary-2.9.13-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:d19aec88857d2a52f99eefcefdbbb45921fb2f777bee5186a355a23d9cf8a0b9", size = 3725651, upload-time = "2026-09-09T23:54:29.277Z" },
{ url = "https://files.pythonhosted.org/packages/53/5e/d50eb688e7e6dfd1499e68cf2d02f44a348b88f381cbc2bebeed15e345f4/psycopg2_binary-2.9.13-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:32cd049095135d2b69e824aea9056745a4aaaa9115a9febbc65584793665d0d0", size = 3817384, upload-time = "2026-09-09T23:54:31.118Z" },
{ url = "https://files.pythonhosted.org/packages/6c/f3/4004cfbbfc52b9b13ffd499f2103eb05246e92828c7237d2c198e028b95c/psycopg2_binary-2.9.13-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:6e696297891b56ff0115f0665de6ad774e1e301e4f60745b8d5024001ae7c2f6", size = 4585497, upload-time = "2026-09-09T23:54:33.169Z" },
{ url = "https://files.pythonhosted.org/packages/97/63/057c65532bd12cdf9d4f568e59c2a078a38e9ba8f7f251292968dc781905/psycopg2_binary-2.9.13-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:930e7e58b33a4f9c39e7532d7a40147925cf3372baed4229cbebe0cf3ba9ce6b", size = 4281172, upload-time = "2026-09-09T23:54:35.747Z" },
{ url = "https://files.pythonhosted.org/packages/43/4b/9fd928eaea9ec1e8d74fed83c9e82826f830506ba0d8c58a8fd41ca93656/psycopg2_binary-2.9.13-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3aea95340825f5ff236e7b40f0b5602c2c77a1e95943f71fae34909834043d29", size = 5900973, upload-time = "2026-09-09T23:54:37.866Z" },
{ url = "https://files.pythonhosted.org/packages/f8/2b/59e1519a22622169e2244f12227b3acde6114ea531a349292f455ab8503f/psycopg2_binary-2.9.13-cp311-cp311-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:27e539b4cafd5e03dcd32921db1b12dd72fe549dd06bae6d4d2a5b5838465f24", size = 4118611, upload-time = "2026-09-09T23:54:39.739Z" },
{ url = "https://files.pythonhosted.org/packages/87/c7/c3d84e330d1584efa0560b756914f9128b1b6fa2aba93fde54bf101d4f65/psycopg2_binary-2.9.13-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:0a6444ac48e2c04f691c2ddd542b38ba30c89463a2d446b3d74ec7d8fc90c964", size = 3659643, upload-time = "2026-09-09T23:54:41.882Z" },
{ url = "https://files.pythonhosted.org/packages/af/fc/317d248503aa29a5051ee49a7253daffe1c57e43ba174020e00bbd87c479/psycopg2_binary-2.9.13-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:8cb734989420c18ca1b71a82da880e11988f5ff3fcdaadd669161de3e98794ac", size = 3305145, upload-time = "2026-09-09T23:54:43.818Z" },
{ url = "https://files.pythonhosted.org/packages/9d/d2/8b23c57591c6d29a463748ffc634401ce14719dba7e78ab91bcbeac70934/psycopg2_binary-2.9.13-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:f47f23db2d70db39cfb714b64fd5df76595b51b2ec0a669710a78f2dceb0c3f8", size = 3051614, upload-time = "2026-09-09T23:54:45.548Z" },
{ url = "https://files.pythonhosted.org/packages/4f/f2/b10a046cc19ab1e01b92226eeb6e8653be7e6691939991b2f347995831b1/psycopg2_binary-2.9.13-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f28b5f2fa8154d0d97e97a664136f58d1639ca008d45d6e09e69fff24826abee", size = 3353133, upload-time = "2026-09-09T23:54:47.285Z" },
{ url = "https://files.pythonhosted.org/packages/40/2c/dd379facaa4bd41d7b04711ff30931ca62981c00a91502250ffb49da08a7/psycopg2_binary-2.9.13-cp311-cp311-win_amd64.whl", hash = "sha256:70d091f5c3a6177fac50c0da20181ce0e0c053f1e43c872d5f75bd6d9429c020", size = 2771441, upload-time = "2026-09-09T23:54:49.016Z" },
{ url = "https://files.pythonhosted.org/packages/fb/d1/d0125c56b865e3bc9f318d84930b2df71a729229dbb0ce12de748a82a6d7/psycopg2_binary-2.9.13-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:2bf9f97a6df69a5d89d054b8cf5257a0916096c479800715fbfe7974dbcb3a26", size = 3725735, upload-time = "2026-09-09T23:54:51.182Z" },
{ url = "https://files.pythonhosted.org/packages/54/a5/b5a73d0910555e38ee12c49c1740855f8a1e9776e87d65f0c51e1bab762a/psycopg2_binary-2.9.13-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:07b7bd9f410650c34c3532162cc329f112368d78a3fc8668cb1ea9df61bc11bf", size = 3818000, upload-time = "2026-09-09T23:54:53.229Z" },
{ url = "https://files.pythonhosted.org/packages/3d/43/3e4783f62ae3f4fc19a5acf8d1c394df54458f1336febe188f317556d2a7/psycopg2_binary-2.9.13-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:0463c00f946517f3e69192a59e6601e023ff9de45ad0a875eda3d6b1bebeb7ce", size = 4585760, upload-time = "2026-09-09T23:54:55.313Z" },
{ url = "https://files.pythonhosted.org/packages/8d/c4/a9a67ae65ad3d567eb0fc9cdf9a5a2783b779aecdcdc8945f1807b13d99e/psycopg2_binary-2.9.13-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:e3861eba31f8ea8663fd876166b032fd89179e42aa63764d6feb281f13f9eb60", size = 4282380, upload-time = "2026-09-09T23:54:57.362Z" },
{ url = "https://files.pythonhosted.org/packages/b3/db/9d459d3da12e0b841cf1596579455aaa27e9e593e3e9a5a4ded5a55a7c15/psycopg2_binary-2.9.13-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3dc3372b3731b3ef23407fe06b94f640ef87a2bda242fa386033d5589c87514a", size = 5902039, upload-time = "2026-09-09T23:55:01.955Z" },
{ url = "https://files.pythonhosted.org/packages/d6/53/21079c10a581c50b6817498eda7c3481c1b3cdb41482bd08ebfccd3664c4/psycopg2_binary-2.9.13-cp312-cp312-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:0405dd4d97720e7ab177aa02e493f524907c4cb3c445ac173e2627948d3d0528", size = 4119840, upload-time = "2026-09-09T23:55:04.336Z" },
{ url = "https://files.pythonhosted.org/packages/c4/ce/71e8d9e1b4f3e78157b49a5abdff50d915e95f2812550f6c9b4f2e4d5e94/psycopg2_binary-2.9.13-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:b6ae51708201f501a171b02419d0c30878a743c369c9054eb1289f0f8d5979e2", size = 3661077, upload-time = "2026-09-09T23:55:06.118Z" },
{ url = "https://files.pythonhosted.org/packages/d9/54/b17616472f09a0fae96f8852692948b7eaa7c971d7629696da0e5932d996/psycopg2_binary-2.9.13-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:81682c227cc1849c4a6adf7b85274229073bb4c9d6ad5697222c695dcea5a8a7", size = 3307202, upload-time = "2026-09-09T23:55:08.061Z" },
{ url = "https://files.pythonhosted.org/packages/77/c7/d9737e222a377dac67a0ce0a2c73e7231a57f5cf18bb35a65d5c8d45d5d2/psycopg2_binary-2.9.13-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:13d955f6054a705a19554364fe9888d0a6e8b0746dc7ebc08a447c7b4fd4145c", size = 3052893, upload-time = "2026-09-09T23:55:10.209Z" },
{ url = "https://files.pythonhosted.org/packages/7d/3d/c406c9f698f518c264381192c2bdf8952ee84e469ffa9f82db1411f57385/psycopg2_binary-2.9.13-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:7e2405196a8cfe6cd3e54172a54452dcf85c241eaf2e9dde7190d7469f7f5ef7", size = 3355210, upload-time = "2026-09-09T23:55:11.883Z" },
{ url = "https://files.pythonhosted.org/packages/27/64/6e3a96699770af2d0d49a2002f722c69b656fc27623ff89281cf2b109644/psycopg2_binary-2.9.13-cp312-cp312-win_amd64.whl", hash = "sha256:376ebf7d8aee4b7386b2bac31fdc27911e7e57cd0a88f1e038b8b149398ac008", size = 2767784, upload-time = "2026-09-09T23:55:13.823Z" },
{ url = "https://files.pythonhosted.org/packages/82/0a/795f2869788373cf7d08410341a444196e8ccebbac07a70a8f9a1f60e72f/psycopg2_binary-2.9.13-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:4d66bfd44a46eb88cff0287929a4193fb45166b6c1f84bb1b233cc17ece0813c", size = 3725723, upload-time = "2026-09-09T23:55:15.887Z" },
{ url = "https://files.pythonhosted.org/packages/b5/63/5a9633f4563a73beba69b20a846ddd14c1c6ac072f5e8aab0da97ffabc2a/psycopg2_binary-2.9.13-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:f818161d2302b3b3e9c75d5a1d0a5c5679e92e45cfec6432b9d5432dde5ff1f1", size = 3817976, upload-time = "2026-09-09T23:55:18.025Z" },
{ url = "https://files.pythonhosted.org/packages/6c/e2/b2e3b3a4331dc8b58e328cda30f3d0cc43a94b7aaf0c8383efd53dd10e95/psycopg2_binary-2.9.13-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:31db6cba66df5231dfd91d9f69188bec3fe6c8baae384e93a0ce792067ee2d98", size = 4585813, upload-time = "2026-09-09T23:55:20.112Z" },
{ url = "https://files.pythonhosted.org/packages/56/5c/87daea77c4132114d1a5da3a4928dd59446c3b3cc73d288cae08cf0b91a6/psycopg2_binary-2.9.13-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f04ada42bcd537adbaf8b7f3140237a204e452a88d0c1831cfce69f7d2e59f4e", size = 4282438, upload-time = "2026-09-09T23:55:22.329Z" },
{ url = "https://files.pythonhosted.org/packages/91/e5/56f9efdc9337acbd1a75798d97163183b63a1babc17602f7163009506c96/psycopg2_binary-2.9.13-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:aa37089795bd9701576edc2eb5849ce77a439eda9dfdfa47857449332cfa5292", size = 5902064, upload-time = "2026-09-09T23:55:24.37Z" },
{ url = "https://files.pythonhosted.org/packages/e4/15/f7ed0b90b47b73a9087306b42267eccfd919f92c0fb057e46bd2fa2efa4d/psycopg2_binary-2.9.13-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:41c2eb569ebd0e1b02d30d361a46932923b193fe1b5e641fb4d547c75e218955", size = 4119848, upload-time = "2026-09-09T23:55:26.433Z" },
{ url = "https://files.pythonhosted.org/packages/42/08/3091347b9fc5766e979aba6b0756ad14ce867a6bb245f3d69ac71fb768c6/psycopg2_binary-2.9.13-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:3f699a5225094a5c61402984e2fc1eca20e940223e76767c88189efb0c313f69", size = 3661129, upload-time = "2026-09-09T23:55:28.449Z" },
{ url = "https://files.pythonhosted.org/packages/34/c4/4f9a84d55484c9794b364548eb6e1fe10a57f123afd19729e5a1cc8ad7fc/psycopg2_binary-2.9.13-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:5f04ae99c9fbb94c3197ec88599ed7db921f6adcddfe83687a74c7ead4037c22", size = 3307262, upload-time = "2026-09-09T23:55:30.384Z" },
{ url = "https://files.pythonhosted.org/packages/83/42/6eba8306a61dc890805ae475a9e71790a1c5461ccacbd4f0a1f3f57b40f0/psycopg2_binary-2.9.13-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:81404c37e0344ebcf10aac127d33d35137e5dbab1daf9f3deee46188fd5879c2", size = 3052898, upload-time = "2026-09-09T23:55:32.961Z" },
{ url = "https://files.pythonhosted.org/packages/b3/5d/42a8935ab280e8dcd7c07a655c0c3d25d62e9e242be1961ac14630f1294a/psycopg2_binary-2.9.13-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:feb7b1856f6ca805cc0e08739858f6cdfed8ce903390126af30343c62899a389", size = 3355265, upload-time = "2026-09-09T23:55:35.071Z" },
{ url = "https://files.pythonhosted.org/packages/87/c2/0e0ffb4caeb651631cbc6c8ead83e2a16457750b1d2eb7f5ef111c1f4d36/psycopg2_binary-2.9.13-cp313-cp313-win_amd64.whl", hash = "sha256:691da68ae5dd7c3ac77514357d35ece7b1ba8b5f3e6c92735198aa6159c355c8", size = 2767914, upload-time = "2026-09-09T23:55:37.14Z" },
{ url = "https://files.pythonhosted.org/packages/5f/32/897c074cb99fbdda7d34b0a2546097a59162bb3d04c0d546ae4ec82345e3/psycopg2_binary-2.9.13-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:2ca263643ae37998ae04d18e431df34d0d61f12b47640dab585f14b6dbe00798", size = 3725788, upload-time = "2026-09-09T23:55:39.04Z" },
{ url = "https://files.pythonhosted.org/packages/0f/f4/e3a789de34c9ac25d20b25c2be583da16394a2ba0926da1c863653831f41/psycopg2_binary-2.9.13-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:4c0214c7da18a28d108aa7108c8a3cca8035c7911ec97ef9ec0827569c9a2720", size = 3818111, upload-time = "2026-09-09T23:55:40.979Z" },
{ url = "https://files.pythonhosted.org/packages/72/29/647724c43ac510dbc59b80e20e85d439deb94f5d5a024153c32330fa041d/psycopg2_binary-2.9.13-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:5d89e064bb12b40cad696cf4975e6da86f8c60f14cd06cb6c1bc0a7f5d01761f", size = 4586235, upload-time = "2026-09-09T23:55:43.012Z" },
{ url = "https://files.pythonhosted.org/packages/91/ad/7f52f92cc65c23778daff7eec4ee2099236694a0a4723a5f180d0708b607/psycopg2_binary-2.9.13-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:190c18b97d9ef72f2e88c451b6588af90d6bd7bf54cb94b963280dc86a2c7076", size = 4282424, upload-time = "2026-09-09T23:55:44.843Z" },
{ url = "https://files.pythonhosted.org/packages/3d/2a/1a472059b198942d99651656e2bc610575584478bfe68d297ecabbd4887f/psycopg2_binary-2.9.13-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c00ebe9a2f31151aade0db233dc1446513a95e92c39ce055ee097af0ae86be1c", size = 5902365, upload-time = "2026-09-09T23:55:46.619Z" },
{ url = "https://files.pythonhosted.org/packages/91/1a/171ea5dac7b3a0fa57b3cb59c2ad6d7b8bc60732368fecfd2ed1f1288392/psycopg2_binary-2.9.13-cp314-cp314-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:5085f7ff7b1e890f279577cedeb8c628957869a340fa34a39f7f406500b3c916", size = 4119938, upload-time = "2026-09-09T23:55:49.381Z" },
{ url = "https://files.pythonhosted.org/packages/41/ce/3c6d4ad71853a59eee6a575fe36df4bb40752a9735a27bd62af66b454ed5/psycopg2_binary-2.9.13-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:4e55357d1943673d491bbabb171c891704fc6a22441fea539e05a5c27a79ea3c", size = 3661557, upload-time = "2026-09-09T23:55:51.269Z" },
{ url = "https://files.pythonhosted.org/packages/10/a3/1819a01bf951eab2afb5ca2a3d11f50500bf536fecff088154372a8d1985/psycopg2_binary-2.9.13-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:3e60b06ec7f9dc3e5f1106d12706514b6d6b92c3dc438fcdf4e43e65cc660d1b", size = 3307770, upload-time = "2026-09-09T23:55:53.196Z" },
{ url = "https://files.pythonhosted.org/packages/4e/df/22f4aec952cd5b2dd02f438399583ed69f7d04b90e7c31659d9571bbe188/psycopg2_binary-2.9.13-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:dde942b46ce20f6c4464cdf551f3293207f803f4e4354454eb1f5599c3eb1fa1", size = 3052935, upload-time = "2026-09-09T23:55:55.117Z" },
{ url = "https://files.pythonhosted.org/packages/95/42/aab651bc22bafa961806ca3b21027bb0739a2730b0e6f7f0778baeb95e67/psycopg2_binary-2.9.13-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:215777c62ce81c3b487cefdb6a41969944eb982309f91349ff3ca0323d6f17ed", size = 3355387, upload-time = "2026-09-09T23:55:57.366Z" },
{ url = "https://files.pythonhosted.org/packages/bc/af/3b8220633eaf955e95ea7be67d76e81a0d1cd3c76362ea504b91ffa079db/psycopg2_binary-2.9.13-cp314-cp314-win_amd64.whl", hash = "sha256:f3088eb80f58ed933c62d87128741d31e786edc862e23266d3c286763d646de0", size = 2863890, upload-time = "2026-09-09T23:55:59.056Z" },
{ url = "https://files.pythonhosted.org/packages/6e/f1/377d17fc8425220d17552691cd2b97aa232da92173f5dead71278b83f8ab/psycopg2_binary-2.9.13-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:38397def2d794ffde9db80f63d6820253e61b17483112652a318355f51a56f50", size = 3725913, upload-time = "2026-09-09T23:56:00.736Z" },
{ url = "https://files.pythonhosted.org/packages/67/64/27208e67cd6e663f69bf7bf905cf69db066a015c90ac9ca948a56a8e9d78/psycopg2_binary-2.9.13-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:dff5c70ed9789ccb0d97ff4a7da51dc523a255c4ec95df188fa5d44adcae4ea8", size = 3818303, upload-time = "2026-09-09T23:56:02.551Z" },
{ url = "https://files.pythonhosted.org/packages/6b/98/67d2f34a1d18367b5f655bdd101759f8474286c74ffe701b7d6e3abd7fda/psycopg2_binary-2.9.13-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:08d3b81a6a91775c937abf97d4c58fc9142e8e35fb91c387d24f81d15c98e6cf", size = 4586508, upload-time = "2026-09-09T23:56:04.706Z" },
{ url = "https://files.pythonhosted.org/packages/bb/47/46c227deaf322dceafa0b7b321b4e5de9cc797014b7a353349b2e09b1118/psycopg2_binary-2.9.13-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:541a487a9ccd72b5e38f37f27b0ce78cb7eb3e336e7b5277d45463010c03a7a8", size = 4282383, upload-time = "2026-09-09T23:56:06.678Z" },
{ url = "https://files.pythonhosted.org/packages/f4/3c/e8705ffa381160d842eaf06a8446e8416f1a2497dd70a7e62277f3be6e7a/psycopg2_binary-2.9.13-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:562fe2a43b30e781848dce63d9080c15414c777c96df348c4342558338cc7bf3", size = 5902765, upload-time = "2026-09-09T23:56:08.634Z" },
{ url = "https://files.pythonhosted.org/packages/53/cc/359821c18317228b8032456a3740c98045b719ed003a594b9ebac9330b86/psycopg2_binary-2.9.13-cp315-cp315-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:dddfe650e7dda464d676c27fbedb5061f1ad05e1604627f54c770d7f799d36e9", size = 4122849, upload-time = "2026-09-09T23:56:10.671Z" },
{ url = "https://files.pythonhosted.org/packages/17/e5/4d935acb6d3258c7a767b3d527e54c0b537649101b55002a5dbcfe747e2a/psycopg2_binary-2.9.13-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:4ff0f575cbb14f30445858dcfdd751e043486f5290915df78a9818bc74042eff", size = 3662084, upload-time = "2026-09-09T23:56:12.316Z" },
{ url = "https://files.pythonhosted.org/packages/89/56/9e9bbc7c773c5de7bb25dd35d7f041c2a6f0fcfa9207a1ceaf01a1bc687c/psycopg2_binary-2.9.13-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:d79530b4c1af657d5620a1d21b8e39f2996aa06821d5564d05b22d6b8cd413d0", size = 3308077, upload-time = "2026-09-09T23:56:15.262Z" },
{ url = "https://files.pythonhosted.org/packages/36/fa/ed742cd4e5dbddcb44702f9c4a97f7f5b62d97e3d9d00907ecc8ac750ef4/psycopg2_binary-2.9.13-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:6ede8595767e19d30a7e8a84a7d47bfde6176d45d194fed08dbb68d1584a780b", size = 3055143, upload-time = "2026-09-09T23:56:17.168Z" },
{ url = "https://files.pythonhosted.org/packages/d5/3a/5c2cb71a844ee236be2ce91b286d797e34a21489909357c7cfba0f5c0197/psycopg2_binary-2.9.13-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:0ebcf3c4266a695df9d0ef51296155f60c86ac51cf82f0d0dd2e827255a891c5", size = 3355374, upload-time = "2026-09-09T23:56:18.793Z" },
{ url = "https://files.pythonhosted.org/packages/e8/30/3991c9fdcca90a5a1e55435292f4d74d176da2be15f3998f6858da3658cc/psycopg2_binary-2.9.13-cp315-cp315-win_amd64.whl", hash = "sha256:1752b9821f1377404d65ac43af03d59a1eccc57fb2c1eb8305f9a3fe8eb7a8ba", size = 2867246, upload-time = "2026-09-09T23:56:20.501Z" },
]
[[package]]