Skip to content

Commander Spellbook Backend — Developer Docs

Welcome. This is the developer documentation for the Commander Spellbook Backend, the engine and REST API behind commanderspellbook.com — a combo database, wiki, and search engine for Magic: The Gathering.

These pages are for contributors. If you read them top to bottom you should be able to set up the project, understand how it fits together, and open your first pull request. They complement (not replace) the root README and CONTRIBUTING.md.

What this project does

Editors describe combos (small interactions between Magic cards that produce an effect). The backend then automatically generates every concrete card combination — a variant — that achieves a result, by walking a graph of cards, features, and combos. Those variants are served through a REST API that the React frontend, the chat bots, and third-party tools consume.

The heart of the project is therefore not CRUD — it is the variant generation engine.

The stack at a glance

Component Technology Role
Database PostgreSQL (SQLite for local dev) Stores cards, combos, features, generated variants
Backend Django + Django REST Framework Domain model, admin panel, REST API, variant engine
Worker django-tasks (db_worker) Runs long jobs: variant generation, Scryfall card sync
Clients Generated from OpenAPI Python & TypeScript SDKs published to PyPI / npm
Bots Discord, Reddit, Telegram Standalone services that consume the API via the Python client
Frontend React (separate repo) The website; not in this repository

Documentation map

Start here and follow the order:

  1. Getting Started — install dependencies, run the stack, run the tests and the linter.
  2. Architecture — the repository layout and how the pieces (backend project, spellbook app, website app, common, clients, bots) connect.
  3. Domain Model — Cards, Features, Combos, Templates, Variants, and Suggestions: the vocabulary everything else is built on.
  4. Variant Generation — the combo graph and the algorithm that turns editor-authored combos into concrete variants.
  5. API & Clients — the REST endpoints, authentication, the OpenAPI schema, and the generated SDKs.
  6. Git Flow & Versioning — branching, semantic versioning, and how a release ships.

Reference material:

  • uv Vademecum — the dependency and environment workflow: the projects, lockfiles, dependency groups, and how versions are derived.
  • The Minimal Set of Multisets ADT — the data structure behind the engine's minimality pruning.

Getting help

Ask on the Commander Spellbook Discord — the #website channel is the best place for backend questions. Maintainers are happy to help you get started.

Variant generation: remaining macro-optimizations

This document collects the macro-level optimization ideas for variant generation that have
not been implemented yet. They complement the ones already in place:

  • delta writes: the save phase only writes variants and relationship rows that actually changed;
  • incremental generation: entity fingerprints (VariantGenerationFingerprints) detect what changed
    since the last run, and only the affected generator combos are regenerated;
  • lightweight data loading: the variant relation tables are loaded as slim rows instead of Django
    model instances, and full Variant instances are only hydrated for the variants being written;
  • parallel generation: the graph and restore phases fan out across forked worker processes on
    platforms that support the fork start method (production containers do);
  • element-indexed variant set entries and packed-integer entry encoding (see below);
  • element-indexed BFS unblocking in the results ("up") phase (see below).

One idea was implemented and then deliberately removed: a persistent per-combo variant set cache
(ComboVariantSetCache). It only skipped the "down" phase (Graph.variants()), while the results
discovery ("up") phase — which dominates the runtime — still ran in full for every combo, so the
cache did not pay for its own complexity (an extra table, input hashing over structural
fingerprints, and serialization of every variant set).

Already implemented: variant set entry representation and up-phase indexing

Profiling showed the "up" phase (Graph.results → _card_nodes_up) running roughly 15–20x slower
than the "down" phase: it runs once per variant instead of once per combo, and each call installs a
per-variant filter that lazily re-filters every node's cached variant set. Almost all of that time
sat in MinimalSetOfMultisets.subtree() (invoked by VariantSet.filter), doing a linear scan of
every entry with a Python-level multiset issubset on each — tens of millions of comparisons per
run. The following changes address it directly.

Element index over MinimalSetOfMultisets

MinimalSetOfMultisets keeps an element → entries containing it index alongside its set of
entries. Every subset of a probed entry, and every superset, shares at least one element with it
(the empty entry is the sole exception, handled explicitly), so subtree(), add()'s dominance
check, and its superset removal all scan only the union of the relevant buckets instead of the whole
collection. This turns the previously O(n) scans output-sensitive: add() building an n-entry set
drops from O(n²) toward roughly linear, and the per-variant filter()/subtree() cost collapses.
The change is confined to minimal_set_of_multisets.py (plus its .pxd/.pyi) and is transparent
to the rest of the pipeline.

Packed-integer entries (PackedEntry)

Variant set entries are no longer FrozenMultiset (a dict plus a wrapper hashed via a frozenset
of its items). PackedEntry stores an entry as a sorted tuple of element * COUNT_LIMIT + count
integers. Subset tests and merges become linear merge-walks over sorted integers, and hashing and
equality are plain tuple operations — all of which Cython compiles to tight C. Negative elements
(templates, encoded as negated ids by VariantSet.ingredients_to_entry) decode correctly through
Python floor-division/modulo. The encoding is confined behind
VariantSet.ingredients_to_entry/entry_to_ingredients, so the blast radius is small; the visible
behavioral change is that entry_to_ingredients now yields ingredients in ascending-id order.
The save path (_restore_variant) was adjusted to compute order-dependent fields such as the
variant name from the ingredients in their final persisted display order rather than from entry
iteration order, so the stored name stays consistent with what Variant.get_recipe() reads back.

Element-indexed BFS unblocking in the up phase

_card_nodes_up's BFS previously parked combos whose feature requirements were not yet satisfiable
in two flat lists, and re-enqueued every parked combo whenever any new countable/uncountable
feature became available — latently quadratic on deep feature chains. Combos are now indexed by the
specific feature nodes that block them (_uncountable_feature_blockers /
_countable_feature_blockers report which nodes could unblock a stalled combo), and only the combos
actually waiting on a produced feature are woken. The enqueue guards were also reordered to test the
cheap issuperset multiset check before the expensive lazy variant-set filter.

Measured on a synthetic graph (90 cards, 58 combos, 6405 variants), these changes together took the
up phase from ~11.0s to ~5.7s and the down phase from ~0.72s to ~0.41s, with byte-identical recipe
output.

The remaining ideas below are ordered by expected impact.

1. Faster bulk writes on the PostgreSQL side

These matter for runs that still produce large create/update volumes (first generation, full
regenerations after wide-reaching changes). Background for each part:

  • COPY instead of multi-row INSERT for bulk creates. Django's bulk_create sends batched
    INSERT INTO ... VALUES (...), (...), ... statements. Each batch has to be parsed, planned, and
    executed as a regular statement, and every value travels through the SQL text/bind protocol.
    PostgreSQL's COPY FROM is a dedicated bulk-load path: rows are streamed in a compact format
    with almost no per-row protocol or parsing overhead. At hundreds of thousands of rows it is
    typically several times faster than batched INSERTs. psycopg 3 exposes it as
    cursor.copy('COPY table (cols) FROM STDIN') + copy.write_row(...), so the create half of
    perform_bulk_saves could stream CardInVariant/TemplateInVariant/... rows directly.
    Caveat: COPY cannot do upserts and reports conflicts as hard errors, so it only replaces the
    plain-insert paths.
  • Merge the create and update passes with an upsert. bulk_create(..., update_conflicts=True, unique_fields=..., update_fields=...) compiles to INSERT ... ON CONFLICT (...) DO UPDATE,
    letting one statement per table handle both new and changed rows instead of separate
    bulk_create + bulk_update passes. bulk_update is the slower of the two because it builds
    large CASE WHEN pk=... THEN ... expressions per batch; ON CONFLICT DO UPDATE avoids that
    entirely. (This is how the variant set cache upserts its rows already.)
  • SET LOCAL synchronous_commit = off inside the save transaction. By default PostgreSQL
    waits for the WAL to be flushed to disk before acknowledging each commit. With
    synchronous_commit = off, the commit returns as soon as the WAL record is written to memory;
    the flush happens up to ~1s later in the background. The transaction is still atomic and
    consistent — the only risk is losing acknowledged work if the server crashes within that
    window, which is acceptable here because a crashed generation is simply re-run. SET LOCAL
    scopes the setting to the current transaction, so nothing else is affected. This mainly helps
    when the save phase issues many statements/batches in sequence.
  • ANALYZE the variant tables after large writes, so the query planner sees fresh statistics
    before the site starts querying the new data (autovacuum gets there eventually, but only after a
    delay proportional to the write volume).

With delta writes and incremental generation in place, routine runs write little, so this item
pays off mainly for the worst-case runs.

API & Clients

The backend exposes a REST API built with Django REST Framework. This page covers the endpoints, authentication, the OpenAPI schema, and the generated SDKs.

Exploring the API

With the server running (see Getting Started):

  • http://localhost:8000/ — the browsable API root
  • http://localhost:8000/schema/swagger/ — Swagger UI
  • http://localhost:8000/schema/redoc/ — ReDoc
  • http://localhost:8000/schema/ — the raw OpenAPI document

Responses use camelCase keys (a middleware converts Django's snake_case), which is what the generated clients and the frontend expect.

Endpoints

Routes are wired in backend/spellbook/urls.py, backend/website/urls.py, and the project urls.py.

Core (spellbook)

Endpoint Description
GET /variants/ The generated variants — the main read endpoint. Supports the search query language.
GET /cards/ Cards.
GET /features/ Features.
GET /templates/ Templates.
GET/POST /find-my-combos Given a decklist, returns the combos it can assemble (the engine's up phase).
GET/POST /estimate-bracket Estimates the power bracket of a decklist.
… /variant-suggestions/ Community-submitted combos awaiting review.
… /variant-update-suggestions/ Suggested edits to existing variants.
… /variant-aliases/ Redirects from alternative ids to canonical variants.

Site support (website)

Endpoint Description
GET /properties/ Site-wide configurable properties.
GET /card-list-from-url Parse a decklist from a supported deckbuilder URL (Moxfield, Archidekt, Deckstats, TappedOut).
GET/POST /card-list-from-text Parse a decklist from pasted text.

Users & auth

/users/, plus the authentication endpoints below.

Authentication

Two mechanisms, both configured in the project urls.py:

  • JWT (simplejwt):
  • POST /token/ — obtain an access/refresh pair
  • POST /token/refresh/ — refresh an access token
  • POST /token/verify/ — verify a token

Send the access token as Authorization: Bearer <token>.
- Social login (social-auth) — Discord OAuth, enabled when DISCORD_CLIENTID / DISCORD_CLIENTSECRET are set.

Most read endpoints are public; writing and reviewing require authentication and the appropriate permissions. Editors work primarily through the admin panel (/admin), not the API.

The search query language

variants (and template matching) accept a Scryfall-style search query — e.g. ci:temur mana result:"infinite mana". The grammar is defined with Lark in spellbook/parsers/ and turned into ORM filters by the transformers in spellbook/transformers/. Extend the query language by editing the .lark grammar and its transformer together.

OpenAPI schema

The schema is generated from the code by drf-spectacular. It is the contract the clients and frontend depend on, so keep it accurate: add serializer annotations and @extend_schema hints when you add or change an endpoint.

Regenerate the committed schema with:

cd client
./generate-openapi.sh   # writes client/openapi.yaml

The script runs manage.py spectacular … --fail-on-warn --validate, so a schema warning is treated as an error — the CI does the same.

Generated clients

The SDKs are generated from openapi.yaml with openapi-generator (run via Docker, so Docker must be running):

cd client
./generate-openapi.sh              # 1. refresh the schema
./generate-client-python.sh        # 2a. Python client  -> client/python/
./generate-client-typescript.sh    # 2b. TypeScript client -> client/typescript/

The CI regenerates and publishes both on release; you only need to run these locally when changing the API and testing a client against it.

Architecture

Commander Spellbook is a monorepo. This page explains the moving parts and how they depend on one another, so you know where a change belongs.

The big picture

UML diagram image

The backend exposes a REST API and an admin panel. Editors author combos in the admin; the worker runs the variant generation engine to derive concrete variants; players read the results through the API — directly, through the website, or through the bots.

Repository layout

Path What lives here
backend/ The Django project. See Backend internals.
common/ Pure-Python utilities shared by the backend and the bots (text/color helpers, constants). Kept dependency-light so bots can import it.
client/ OpenAPI schema plus the scripts and generated Python/TypeScript SDKs. See API & Clients.
bot/ The discord/, reddit/, and telegram/ bots — independent services that consume the API via the generated Python client.
demo/ Fixtures and sample data dumps for local experimentation.
docs/ These pages, plus the combo-graph explainer assets.
Root docker-compose*.yml, deploy.sh, git-release, CI workflows under .github/.

Backend internals

Everything under backend/ is one self-contained Django project made of three apps.

backend — the project package

The Django project itself: settings, root URL configuration, authentication, the admin site chrome, and the OpenAPI wiring.

  • settings.py — local/dev settings (SQLite, DEBUG=True), so a clone runs with no external database.
  • production_settings.py — Postgres via SQL_* env vars, used in Docker/prod.
  • worker_settings.py — production settings plus a DB statement timeout, used by the background worker.
  • urls.py — mounts the app routers, JWT and social auth, the admin, and the drf-spectacular schema/Swagger/Redoc endpoints.

spellbook — the core app

The domain model, the variant engine, and the primary REST API. Notable subpackages:

Subpackage Responsibility
models/ The domain model: Card, Feature, Combo, Template, Variant, Suggestions, and their join tables.
variants/ The variant generation engine: the combo graph and set algebra. Ships .pxd stubs for optional Cython compilation.
views/ DRF viewsets and API views (variants, cards, features, find-my-combos, estimate-bracket, suggestions, ...).
serializers/ DRF serializers, including pre-serialized/denormalized variant output for fast reads.
tasks/ django-tasks background jobs: variant generation, Scryfall card sync, exports, notifications.
management/commands/ manage.py entry points that enqueue those tasks (update_variants, update_cards, export_variants, combo_of_the_day).
parsers/ & transformers/ Lark grammars and transformers for the Scryfall-style search query language used by the API and by Template matching.
admin/ The admin panel where editors author and review combos.

website — site-support API

Extra endpoints that back the website but are not part of the core combo model — site properties, and card-list parsing helpers (card-list-from-url, card-list-from-text).

Background work

Long-running operations do not block API requests; they are enqueued as django-tasks jobs and executed by a separate worker process (python manage.py db_worker, the worker Docker target). The heavy jobs are variant generation and the periodic Scryfall card sync. Task results are visible in the admin panel.

Data flow: from combo to variant

  1. An editor creates Cards, Features, Templates, and Combos in the admin panel.
  2. update_variants enqueues a generation job. The worker builds the combo graph and derives every valid Variant.
  3. Variants are denormalized/pre-serialized and stored for fast reads.
  4. The API serves variants; the website and bots consume them.

Read the Domain Model next for the vocabulary, then Variant Generation for step 2 in depth.

Domain Model

Everything in Commander Spellbook is built from a small vocabulary. Learn these six concepts and the rest of the codebase reads easily. The models live in backend/spellbook/models/.

The core concepts

Card

A Magic card, mirrored from Scryfall and keyed by oracle_id (synced by the update_cards task). A Card entity mirrors an oracle card — the game-rules identity shared by every printing — not a specific reprint: there is one Card per oracle card, regardless of how many times it has been printed. Beyond its name it carries the Magic characteristics used for filtering and validation — color identity, mana value, type line, oracle text, keywords — inherited from the abstract Playable base. A card can produce features directly (e.g. a card that by itself is "an extra turn").

Feature

A named effect or result — the abstraction that lets the engine chain things together. Examples: Infinite mana, Untap target permanent, Win the game. Cards and combos produce features; combos need features. A feature's status decides how it is treated and shown:

Status Meaning
Hidden / Public utility Intermediate building block used only by the engine (public ones are visible to combo submitters)
Helper A reusable effect meant to be exploited by other combos
Contextual Situational effect
Standalone A meaningful, usually game-impacting result

Features can be marked uncountable (only ever one copy — this also speeds up generation).

Template

A placeholder for "any card matching a query", e.g. "a creature with power 4 or greater". A template is defined either by a Scryfall-style search query or by an explicit list of concrete replacements (cards known to satisfy it) — not both. Either way the replacements are ordinary Cards. Templates let a combo be written generically; the engine expands them into real cards.

Combo

The editor-authored interaction — the input to the engine. A combo is a recipe that declares:

  • uses — cards it needs (CardInCombo)
  • requires — templates it needs (TemplateInCombo)
  • needs — features it consumes as prerequisites (FeatureNeededInCombo)
  • produces — features it results in (FeatureProducedInCombo)
  • removes — features it invalidates (FeatureRemovedInCombo)

plus editorial text (mana needed, prerequisites, step-by-step description, notes). A combo's status controls its role in generation — most importantly GENERATOR (a combo that variants are generated from) versus UTILITY (a building block that only exists to be chained into others).

Variant

A concrete, fully-resolved card combination produced by the engine — the primary object the API serves. Where a combo may say "a mana dork + Isochron Scepter", a variant names the exact cards. Each variant records the generator combos it is of, the combos it includes, the cards it uses, the templates it requires, and the features it produces, and adds derived data: color identity, mana cost, popularity, a power bracket estimate, and a review status (only OK and EXAMPLE variants are public). Variant output is pre-serialized/denormalized on save so reads are fast.

Suggestions

Community-submitted content that waits for editor review before it becomes canonical:

  • VariantSuggestion — a combo submitted by a user. Editors review it and, if accepted, turn it into a real Combo.
  • VariantUpdateSuggestion — proposed edits to existing variants.
  • VariantAlias — a redirect from an old or alternative id to a canonical variant, so links never break.

How they relate

UML diagram image

The feature is the pivot: a card or combo produces a feature, and another combo needs it. Chaining "produces → needs" across the graph is exactly what the variant generation engine walks.

Shared building blocks

A few abstract models and join tables recur throughout:

  • Recipe — the abstract uses / requires / produces structure and automatic name generation, shared by Combo, Variant, and VariantSuggestion.
  • Playable — Magic characteristics (color identity, mana value, type line, …) shared by Card and Variant.
  • Ingredient / IngredientInCombination — the through-model base that carries per-item data such as quantity and starting zone locations (hand, battlefield, graveyard, …).

The [[name]] reference syntax

Text fields (descriptions, prerequisites) can reference a feature by name with [[name]]. Two modifiers exist: [[name|alias]] gives it a reusable alias, and [[name$number]] selects one of several copies. This lets editorial prose refer to generated pieces without hardcoding card names.

Where to go next

The Variant Generation page explains how the engine turns these combos into variants.

Getting Started

This page gets a contributor from a fresh clone to a running backend, a passing test suite, and a clean linter.

Prerequisites

  • Python 3.14+ — each project declares its own floor via requires-python in its pyproject.toml. You do not need to install Python yourself: uv downloads and manages a matching interpreter for you.
  • uv — the Python package & environment manager used throughout the repo. Install it with pip install uv, or get it together with the shared dev tooling by running pip install -r requirements.txt from the repository root. See the uv Vademecum for the full workflow.
  • Docker + Docker Compose — required to run the full stack and to generate the API clients. Not needed for the SQLite-only inner loop below.
  • Git.
  • (Optional) VS Code with the Python extension — the repo ships a .vscode/launch.json with run/debug and pytest configurations.

Two ways to run the backend

Option A — Docker Compose (full stack)

Brings up PostgreSQL, the Django web server behind nginx, and the background worker exactly as they run in the demo environment:

docker compose up --build

The site is served on http://localhost (override the port with the PORT environment variable). The compose file uses a throwaway Postgres database with demo credentials — see docker-compose.yml.

To also run one or all of the bots, enable their compose profiles:

docker compose --profile bot up      # all bots
docker compose --profile discord up  # just Discord

Option B — manage.py with SQLite (backend only)

Fastest inner loop for backend work. Local development uses a file-based SQLite database by default (configured in backend/backend/settings.py), so you need no Postgres and no Docker — Django creates the database file on first migrate. This is enough for most day-to-day development.

cd backend
uv sync                       # create .venv and install the locked dependencies
uv run manage.py migrate
uv run manage.py createsuperuser
uv run manage.py runserver

uv run keeps the project environment locked and synced automatically before each command, so there is no separate "activate the virtualenv" step. Then open:

  • http://localhost:8000/ — the browsable REST API and OpenAPI docs
  • http://localhost:8000/admin — the Django admin panel (where editors author combos)

Note: some features (full-text search, certain indexes) are Postgres-only and are skipped on SQLite. Use Option A, or point SQL_* environment variables at a Postgres instance, when you need production-faithful behaviour. See Configuration below.

Installing dependencies

The repository is a monorepo with several independent uv projects — backend/, client/python/, and the three bot/* — each with its own pyproject.toml and a committed, platform-independent uv.lock. Install the one you are working on with uv sync from its folder:

# The Django backend
cd backend && uv sync

# A bot (installs its own dependency set)
cd bot/discord && uv sync

Dependencies are declared in each project's pyproject.toml and pinned in uv.lock. Add or change one with uv add <package> / uv remove <package> — these update pyproject.toml and uv.lock together. Never edit uv.lock by hand; the lockfiles are verified in CI with uv lock --check. The uv Vademecum covers dependency groups, versioning, and the rest of the workflow.

The repository root keeps a plain requirements.txt for shared dev/test tooling (including uv itself); install it with pip install -r requirements.txt.

Interpreter selection (VS Code): each project has its own .venv, and the backend one is the default. When working on a different project (a bot, the client), run Python: Select Interpreter and pick that project's */.venv.

Running the tests

Tests are standard Django tests. Run them from the backend folder; common must be on the Python path:

cd backend
uv run python -Wd manage.py test --no-input --parallel auto --pythonpath ../common

pytest also works (configuration lives in pytest.ini); run it inside the backend environment, whose dev group also carries what the client tests need:

uv run --project backend pytest backend client/python common

Every code change must ship with tests — the CI enforces the suite across Linux, Windows, and macOS.

Linting

The project follows PEP 8, enforced by flake8. The CI lints three folders; run the same locally before pushing:

uvx flake8 backend
uvx flake8 common
uvx flake8 bot

A failing lint fails the build.

Optional: Cython acceleration

The variant generation code ships with .pxd type stubs so it can be compiled with Cython for a large speed-up on big generation runs. It is pure Python by default; compile it only if you are profiling or running full generations locally:

cd backend
uv sync --group cython
uv run cythonize -i 'spellbook/variants/*.py'

The CI runs the test suite both with and without Cython to guarantee behaviour is identical.

Generating the API clients (optional)

The Python and TypeScript SDKs are generated from the live OpenAPI schema. You need Docker running. See API & Clients for the scripts and details.

Configuration

Settings are read from environment variables (with sensible development defaults in backend/backend/settings.py). The ones you are most likely to touch:

Variable Purpose Default
SECRET_KEY Django secret key insecure dev key
SQL_ENGINE / SQL_DATABASE / SQL_USER / SQL_PASSWORD / SQL_HOST / SQL_PORT Database connection (production settings) SQLite
VERSION Version string shown in the admin/API dev
DISCORD_WEBHOOK_URL Webhook for notifications unset
MOXFIELD_USER_AGENT User agent for Moxfield deck imports unset
DISCORD_CLIENTID / DISCORD_CLIENTSECRET Discord social login (OAuth) unset

Local development uses backend/backend/settings.py (SQLite, DEBUG = True). Docker and production use backend/backend/production_settings.py (Postgres via the SQL_* variables); the background worker uses worker_settings.py, which adds a statement timeout on top of the production settings.

Next steps

You have a running backend. Read the Architecture to learn how the pieces fit together, then the Domain Model to learn the vocabulary.

Git Flow and Versioning

This page describes the branching model used for the Commander Spellbook Backend project, based on Git. Before contributing, also read Getting Started (setup, tests, linting) and CONTRIBUTING.md.

Trunk based development

It is suggested to follow and contribute to the project using the trunk based development model. Basically, every commit is done on master. This is the simplest model, and it is suggested to use it for small projects.

If a new feature is being developed, a new branch can be created from master, but it is not required. The feature branch name should be prefixed with feature/. When the feature is ready, it can be merged into master, and the feature branch can be deleted.

Semantic versioning and annotated tags

Versioning is done using semantic versioning. Release branches are not allowed. Instead, a new version is created by tagging the commit on master, using a git annotated tag.

When a new version tag is added, it must represent a higher version than any previous tag. The version must be updated according to semantic versioning rules. The tag name must be prefixed with v, and it must be followed by the version number. For example, v1.0.0 is a valid tag name.

These tags are the single source of the Python packages' versions: no version number is written down in the repository, it is computed from the tags at build time. See uv Vademecum for the details.

Continuous delivery

Whenever a new annotated tag is pushed to the remote repository, the continuous delivery pipeline will be triggered. The pipeline is defined in .github/workflows/ci.yml: it builds and tests the backend, worker, and bot images, generates and publishes the API clients, and deploys to the production Kubernetes cluster. Pull requests run the same build, lint, and test jobs without releasing. These developer docs deploy separately via .github/workflows/docs.yml on any change under docs/.

Helper scripts

You can download a MIT Licensed semantic version helper script from here and put it in your path. If you do so, you can use the following commands to create a new version tag:

git release major
git release minor
git release patch

Each of these commands will increment the corresponding part of the version number, and create a new annotated tag. In addition to that, it will also sync the current branch with the remote repository, and push the new tag afterwards.

The Minimal Set of Multisets Abstract Data Type

Introduction

The Minimal Set of Multisets (MSM) is a data structure that represents a set of multisets. No multiset in a MSM is a subset of another multiset in the MSM. It is the antichain that backs the minimality pruning of the variant generation engine. When you add a new multiset to a MSM, one of two things happens:

  1. If the new multiset is a superset of an existing multiset, nothing happens and the MSM remains unchanged.
  2. Otherwise, if the new multiset is not a superset of any existing multiset, it is added to the MSM. Then, any multiset that is a superset of the added multiset is removed from the MSM.

Example

Suppose we have a MSM with the following multisets:

{1: 1, 2: 1, 3: 2}
{1: 1, 2: 2, 4: 1}
{1: 1, 3: 1, 5: 1}

If we add the multiset {1: 3, 2: 3, 3: 1, 4: 1}, nothing happens and the MSM remains unchanged, because the new multiset is a superset of the second multiset.

If we add the multiset {1: 1, 2: 1, 6: 1}, it is added to the MSM:

{1: 1, 2: 1, 3: 2}
{1: 1, 2: 2, 4: 1}
{1: 1, 3: 1, 5: 1}
{1: 1, 2: 1, 6: 1}

If we add the multiset {1: 1, 2: 2}, it is added to the MSM, and all its supersets are removed:

{1: 1, 2: 1, 3: 2}
{1: 1, 3: 1, 5: 1}
{1: 1, 2: 1, 6: 1}
{1: 1, 2: 2}

If we add the multiset {1: 1, 2: 1}, it is added to the MSM, and all its supersets are removed:

{1: 1, 3: 1, 5: 1}
{1: 1, 2: 1}

Implementation

You can find the implementation of the MSM in the minimal_set_of_multisets.py file.

References

These are the papers/links to refer for the implementation of an optimized MSS (Minimal Set of Sets) data structure:

These are the papers/links to refer for the implementation of an optimized MSM (Minimal Set of Multisets) data structure:

Considerations

Cython works way better with simple Python code than with over-engineered data structures, so the currently fastest implementation of the MSM is the naive one, which uses a Python set to store the multisets and checks for supersets/subsets using simple loops.

uv Vademecum

All Python dependencies and environments in this repository are managed with
uv. This page is the quick reference: what lives where, the commands
you need day to day, and the conventions the CI enforces. For a first-time setup walkthrough see
Getting Started.

Installing uv

uv is installed through pip, and it is listed in the repository-root requirements.txt together
with the shared dev tooling:

pip install -r requirements.txt   # uv + flake8, pytest-django, ...

CI and the Dockerfiles bootstrap it the same way (pip install uv -c requirements.txt), so if you
ever need to constrain uv's version, do it in that one file rather than hardcoding it elsewhere.

The layout

This is a monorepo of independent uv projects. Each has its own pyproject.toml, its own
committed uv.lock, and its own .venv — there is no shared virtualenv.

Project Path What it is
Backend backend/ The Django backend, REST API and variant engine
Python client client/python/ The OpenAPI-generated SDK (spellbook_client)
Discord bot bot/discord/ Standalone bot service
Reddit bot bot/reddit/ Standalone bot service
Telegram bot bot/telegram/ Standalone bot service

Two things are deliberately not uv projects:

  • common/ — shared source with no third-party dependencies. It is put on PYTHONPATH
    (--pythonpath ../common), not installed.
  • the repository root — its requirements.txt is a plain pip file for dev tooling.

spellbook_client is likewise consumed from source (via PYTHONPATH, and copied into the bot
images), not installed as a package. That is why the bots — and the backend's dev group — repeat
the client's runtime dependencies.

Everyday commands

Run these from inside a project folder, or from anywhere with --directory <project>.

Command What it does
uv sync Create/update .venv to exactly match uv.lock
uv run <cmd> Run a command in the project env, locking & syncing first
uv add <pkg> Add a dependency (updates pyproject.toml and uv.lock)
uv remove <pkg> Remove a dependency
uv lock Re-resolve and refresh uv.lock
uv lock --check Fail if uv.lock is stale (what CI runs)
uv lock --upgrade-package <pkg> Bump a single package
uvx <tool> Run a one-off tool without installing it (e.g. uvx flake8 .)

The most common ones in this project:

cd backend
uv sync                                   # install the backend env
uv run manage.py migrate                  # any manage.py command
uv run manage.py runserver
uv run python -Wd manage.py test --no-input --parallel auto --pythonpath ../common

Automatic environment upkeep

uv run locks and syncs the environment before every command, so the env is always current with
pyproject.toml/uv.lock — there is no "activate the virtualenv" step and no stale-dependency
class of bug. This is why the VS Code tasks and the OpenAPI script call uv run rather than
python.

To opt out (CI and Docker do, for reproducibility):

Flag Effect
--locked Error instead of updating a stale lockfile
--frozen Use the lockfile as-is, without checking it
--no-sync Run without touching the environment
--no-install-project Install only dependencies, not the project itself

Dependency groups

Dependencies default to the dev group being installed. The backend defines three groups:

Group Contents Used by
dev (default) tblib, flake8, pytest-django, django-debug-toolbar, plus the client's runtime deps local dev, tests, CI
prod gunicorn, psycopg[binary] the Docker images only
cython cython, setuptools the Cython build 
uv sync --group cython        # dev + cython
uv sync --no-dev --group prod # what the Docker builder installs

Lockfiles

uv.lock is committed and is a universal, platform-independent resolution: the same lockfile
serves Linux, macOS and Windows, so it replaces the old pip-compile output entirely.

  • Never edit uv.lock by hand — change pyproject.toml (or use uv add) and re-lock.
  • CI has a dedicated lockfile job running uv lock --check in every project; a stale lockfile
    fails the build. If it fails, run uv lock and commit the result.
  • Dependabot updates pyproject.toml + uv.lock through its uv ecosystem.

Versioning

Package versions are not written down anywhere — they are computed from the git tags described in
Git Flow & Versioning by
uv-dynamic-versioning, a hatchling plugin:

[build-system]
requires = ["hatchling", "uv-dynamic-versioning"]
build-backend = "hatchling.build"

[tool.hatch.version]
source = "uv-dynamic-versioning"

On the v5.6.0 tag a build produces 5.6.0; off-tag it produces a dev version, and with no tags at
all it falls back to 0.0.0.

uv version does not work here — it refuses dynamic versions. To see the computed version,
run uv build and read the resulting wheel's filename.

The Django backend's runtime version string (shown in the admin and the API schema) is separate:
it comes from the VERSION environment variable, which the CI passes to the Docker build.

Supply chain

Every project sets a resolution cutoff matching the Dependabot cooldown, so a freshly published
(potentially compromised) release is never picked up immediately:

[tool.uv]
exclude-newer = "1 day"

It only affects new resolutions (uv lock, uv add, upgrades). It does not make uv lock --check
time-dependent, because uv does not re-resolve when merely validating a lockfile.

uv in Docker and CI

Docker — each image installs uv with pip, then installs only dependencies from the lockfile
into a virtualenv at the fixed path /opt/venv (fixed so it stays valid when copied between build
stages), which is then put on PATH:

ENV UV_COMPILE_BYTECODE=1 \
    UV_LINK_MODE=copy \
    UV_PYTHON_DOWNLOADS=0 \
    UV_PROJECT_ENVIRONMENT=/opt/venv

RUN uv sync --locked --no-install-project --no-dev --group prod
Variable Why
UV_COMPILE_BYTECODE=1 Write .pyc files at build time, so the first request does not pay for compilation. Worth it in an image, which is built once and started many times.
UV_LINK_MODE=copy Copy files out of uv's cache instead of hardlinking. The cache and /opt/venv are on different layers, where hardlinks cannot be made; without this uv warns and falls back anyway.
UV_PYTHON_DOWNLOADS=0 Never fetch a managed interpreter — the image must use the python:3.14-alpine one it is built on.
UV_PROJECT_ENVIRONMENT See the fixed-path note above.

The applications run from source (manage.py, spellbook_<bot>.py), so the project itself is
never installed into the image, and .git is never needed at build time.

CI — installs uv with pip, then uv sync --locked + uv run --no-sync, so jobs can never
silently drift from the committed lockfile.

Troubleshooting

Symptom Cause / fix
VIRTUAL_ENV=... does not match the project environment A different virtualenv is active in your shell. Harmless — uv correctly uses the project's .venv.
uv lock --check fails in CI The lockfile is stale. Run uv lock in that project and commit.
ModuleNotFoundError for constants, text_utils, … Root common/ is missing from the path; pass --pythonpath ../common.
ModuleNotFoundError: spellbook_client You are in a project whose env lacks the client's deps, or client/python is not on PYTHONPATH.
VS Code uses the wrong interpreter Each project has its own .venv (the backend's is the default). Run Python: Select Interpreter.
flake8 reports errors inside .venv The project's .flake8 must exclude .venv.

Variant Generation

This is the heart of the project. Editors author a few hundred combos; the engine derives the tens of thousands of concrete variants that the site serves. The code lives in backend/spellbook/variants/.

There is an animated explainer of the algorithm — a good visual companion to this page:

The problem

A combo like "Isochron Scepter + Dramatic Reversal" really means "Isochron Scepter imprinting an instant that untaps your mana rocks, plus enough rocks to make net mana". Cards produce features, combos need and produce features, and features can be satisfied many different ways. Enumerating every valid, minimal card set that reaches a result by hand is impossible. So we model it as a graph and compute it.

The combo graph

The engine builds a graph (combo_graph.py) whose nodes are Cards, Features, Combos, and Templates, wired by the domain model relationships:

UML diagram image

Every node is associated with a VariantSet: the set of concrete card/template combinations that can "reach" that node. Because a card may be needed in multiple copies, combinations are multisets (multiset.py), not plain sets.

The set algebra

Two operations combine variant sets (variant_set.py):

  • OR = union. A feature can be produced by any of several providers (cards or combos), so a feature's variant set is the union of its providers' sets.
  • AND = cross-product. A combo needs all of its ingredients at once, so its variant set is the Cartesian product of the variant sets of everything it needs — each combination merged into one card set.

These two operations, applied recursively over the graph, produce every card combination that satisfies a combo.

The two phases

Down phase — generate variants (DFS from a target)

For each generator combo, _combo_nodes_down performs a depth-first traversal from the desired result down toward cards: to satisfy a combo, satisfy every feature it needs; to satisfy a feature, take the union of everything that produces it; recurse until you reach cards and templates. Combining unions and cross-products along the way yields the combo's full variant set. This is what variants_generator.py runs to populate the database.

Up phase — find combos from a hand (BFS from cards)

Given a set of cards a player owns, the up phase propagates forward: mark the features those cards produce, find combos whose needs are now met, mark the features they produce, and repeat until nothing new is reachable. This is the "Find My Combos" feature.

Minimality: keep subsets, drop supersets

The down phase can produce redundant results — a card set that works but includes cards it does not actually need. The engine prunes these with an antichain: among all produced multisets, discard any that is a superset of another, keeping only the minimal ones (minimal_set_of_multisets.py). A variant should be the smallest card set that achieves the result.

This pruning is backed by a dedicated data structure, the Minimal Set of Multisets (MSM) — a collection that automatically keeps only its minimal members as elements are added. It has its own reference page: The Minimal Set of Multisets ADT.

Guardrails against combinatorial explosion

Feature chaining is powerful enough to blow up. The engine caps it:

  • Card limit — a variant may use at most a fixed number of cards (DEFAULT_CARD_LIMIT, or HIGHER_CARD_LIMIT for combos flagged allow_many_cards).
  • Variant limit — a single combo may spawn at most a fixed number of variants (DEFAULT_VARIANT_LIMIT, lowered to LOWER_VARIANT_LIMIT when the higher card limit is in effect).
  • Uncountable features and singleton-only combos shrink the search space further.

These constants live in spellbook/models/constants.py.

Running generation

Generation is a background job, not a request-time operation. Enqueue it with:

cd backend
python manage.py update_variants   # enqueues the generation task

The worker picks up the job and writes the resulting variants. On large datasets this is expensive — compile the engine with Cython for a substantial speed-up (the .pxd files next to each module are the type stubs that make this possible).

Reading the code

A suggested order:

  1. variant_data.py — how model data is loaded into the engine's in-memory form.
  2. combo_graph.py — nodes, the graph, and the two phases.
  3. variant_set.py & multiset.py — the set algebra.
  4. minimal_set_of_multisets.py — antichain pruning.
  5. variants_generator.py — the orchestration that ties it together and writes to the database.