04 — Backend Roadmap¶
Can this generalize past Mongo — to any database, or at least a steadily
growing list? Yes, by design. The FilterAST + QueryCompiler boundary
(doc 03) exists exactly for this. The front half of the library — model introspection, parameter generation,
OpenAPI, validation — is 100% backend-neutral. Adding a database is writing one
adapter class, not touching the core.
The contract every backend implements¶
class QueryCompiler(Protocol):
name: str
supported_ops: frozenset[str]
capabilities: frozenset[Capability] # structural features beyond operators
# (shipped: NESTED_PATHS, ELEM_MATCH)
def compile_where(self, group: Group) -> Any: ...
def compile_order(self, order: list[Sort]) -> Any: ...
def compile_page(self, page: PageSpec) -> Any: ...
Rejection is two-tier (shipped): compile time, always — an AST containing
an operator or structural feature the adapter did not declare raises
CompilationError naming the operator and backend; and registration time,
opt-in — FilterDepends(Model, backend=compiler) intersects the generated
parameter surface with the declaration and raises ConfigurationError naming
every offending parameter before the app serves traffic.
Two ideas make this robust across very different databases:
-
Capability declaration. Not every store supports every operator (a key-value store has no
regex; a SQL table has no array$elemMatchunless it's JSONB). The adapter declaressupported_opsandcapabilities; asking for something the backend can't do fails loudly per the two-tier rejection above. We never silently drop a filter. -
Graceful capability tiers. A backend can advertise a subset and still be first-class.
list[str]array operators light up only where the backend can express them; everything else still works.
Phased backend expansion¶
Tier 1 — MongoDB (the launch backend)¶
- Driver-agnostic output:
compile_wherereturns a plaindict, usable withpymongoandmotoralike (sync and async). We do not depend on a driver; we emit the query, you run it. - Optional thin conveniences for
motor/pymongo(q.paginate(collection)), behind an extra:pip install fast-pager[mongo]. - Beanie / ODMantic (Pydantic-native Mongo ODMs) are a natural early add: their documents are Pydantic models, so introspection works unchanged and we can emit native ODM query expressions. Strong fit, likely Tier 1.5.
Tier 2 — SQL via SQLAlchemy¶
The highest-value second backend by far (largest FastAPI audience).
- Compiles
Condition→ SQLAlchemyColumnElementboolean expressions (column >= value,column.in_(...)). Shipped semantics: thecontains-family usesautoescape=TrueLIKE (literal%/_, mirroring the Mongore.escapeguarantee);i*variants are guaranteed case-insensitive, plain variants are database-native LIKE (SQLite's ASCII-insensitivity and MySQL collations documented);existsis rejected rather than aliased (SQL columns always exist — useisnull); array operators,elem,has_key,regex, andtext_searchare declared unsupported rather than approximated. - Input model can be a Pydantic model paired with a SQLAlchemy model, or a SQLModel class (which is both). The introspector already understands Pydantic; the adapter maps field paths → columns.
- Nested-model filtering maps to JOINs or JSON column access depending on schema; shipped support is flat tables + JSON/JSONB columns (tuple-path extraction with value-type-driven CASTs; datetimes as ISO strings, enums by value), with relationship JOINs as a follow-up.
pip install fast-pager[sqlalchemy].
Tier 3 — Elasticsearch / OpenSearch¶
- A filtering+search engine is an excellent fit:
contains/text_searchmap to real analyzed queries rather than collection-scanning regex. - Compiles to the ES query DSL (
bool/must/filter/range/terms).
Tier 4 — community backends¶
Once the adapter contract is stable and documented, additional stores
(DynamoDB, Postgres-via-asyncpg-raw-SQL, Redis search, etc.) can be third-party
packages (fast-pager-dynamodb) that just implement QueryCompiler. We
publish the adapter authoring guide and a conformance test suite so external
adapters can prove correctness.
How the user selects a backend¶
The backend is chosen at the call site, not baked into the generated
parameter surface (the shipped design; an app-global
configure(backend=...)/run(q) dispatch remains a possible future
convenience):
# Mongo:
return await db.users.find(q.to_mongo()).to_list(None)
# SQLAlchemy — same q, same params:
return session.execute(q.apply_sqlalchemy(select(UserRow))).scalars().all()
# optional early validation:
q: FilterQuery[User] = FilterDepends(User, backend=SQLAlchemyCompiler(UserRow))
Switching backends — or running two backends in one app — does not touch the endpoint signatures or the generated parameters. This is the payoff of the AST boundary: the database is a deployment detail, not an API-design decision.
A conformance suite is the real product moat¶
To make "any DB" credible, we ship a backend conformance test suite
(fast_pager.conformance, shipped): a fixed battery of FilterAST inputs
covering every registry operator, with backend-neutral semantics enforced by
the runner — declared cases must compile, undeclared cases must raise
CompilationError naming the operator, invalid inputs must be rejected —
while each backend supplies its own expected-output table. Any adapter —
first-party or community — runs the suite to claim compatibility. This keeps quality high as the
backend list grows and turns external contributions into a strength rather than a
support burden.
Honest scope note¶
We will not try to be a universal query abstraction that papers over every database difference. Some operators simply don't exist everywhere, and pretending otherwise produces leaky, surprising behavior. The capability model embraces the differences: every backend does what it genuinely can, declares the rest, and the user is told at startup. That honesty is more valuable than a false promise of total portability.
Continue to 05 — Roadmap & Release Plan.