Content pipelines

Most puzzles here are made by code the moment you play them. Some things can’t be: no program invents good chess tactics, picks odd words that are fun to bluff about, or draws a fox. For those the site keeps content (a word bank, a bundle of chess puzzles, two sets of pictures), built once by a script in tools/, checked and committed.

Every pipeline follows the same rules:

• Nothing downloaded lives in the repository. Sources (6 GB of Google Books counts, a 1 GB CSV of chess puzzles, an icon set) go in a new, empty directory outside the repo. Only what’s made from them is committed.
• Sources are read as data, never run. CSVs are parsed by the standard library, SVGs are drawn as an <img> in headless Chrome so no script inside can run, and dictionaries are matched with regular expressions.
• The same input makes the same output. Sampling and codes come from hashes, and output comes in a fixed order. A rebuild gives the same files (except the date in the chess index), so a diff shows only real changes and a code or daily stays the same puzzle.
• Licenses travel with what’s made, in each output’s header or manifest. Share-alike material is kept in a file of its own.
• What’s built is checked by a script or the test suite before it’s committed.

This chapter stops at the files. How a game plays its content is in the game’s own chapter.

Fictionary’s word bank#

Fictionary needs real words nobody at the table knows, each with a short, true definition that blends in with players’ fakes. The bank, tables/fictionary/content/words.toml, holds a little over two thousand [[word]] tables, best first:

[[word]]
id = "brisance"
word = "brisance"
pos = "noun"
definition = "The shattering force of an explosion"
rating = 97
scores = { obscurity = 94, guessable = 0, fun = 93, definition = 100 }
zipf = 0.9
topic = "phenomenon"
source = "wn30:n11412179"
verified = "wordnet-only"

A script proposes candidates, a person rewrites or drops each one, and the script joins the two into the bank.

The sources#

Princeton WordNet 3.0 gives the headwords, parts of speech, senses, glosses, subject domains (law, chemistry) and links between related words. tools/fictionary/wordnet.py reads its database files (index.noun, data.noun, …) as plain text. Each sense is a Synset, keyed by part of speech and byte offset (n11412179), which becomes an entry’s source.

@dataclass
class Synset:
    key: str  # "n00001740": part of speech and byte offset, WordNet's own id
    pos: str  # n, v, a (s, satellites, folded in), r
    lexname: str  # its lexicographer file: noun.plant, verb.motion, …
    words: list  # its lemmas as written (case kept: "Aberdeen" is a name)
    pointers: list  # (symbol, target key, source word no., target word no.)
    gloss: str
    domains: list = field(default_factory=list)  # keys of ;c ;r ;u domain synsets

Google Books’ English 1-grams (2020 release, CC BY 3.0) say how often each word was printed. tools/fictionary/ngrams.py streams the 24 shards (about 6 GB), keeps only WordNet words, and sums their counts over books from 1950 to 2019, split by case (capitalized is mostly names) and by Google’s part-of-speech tags. A word everyday in 1850 but forgotten now counts as forgotten. The bank keeps one number from these counts, zipf: the log of uses per billion words.

def zipf(count: int) -> float:
    return math.log10(max(count, 1) * 1e9 / TOTAL_TOKENS)

A Zipf of 3 is once in a million words. Every bank word is at most 2.3 (about one in five million). Brisance, at 0.9, appears about once in 125 million.

What’s thrown out#

build.py starts from every single-word, letters-only WordNet lemma. The cheap filters come first, in _excluded:

def _excluded(word, senses, synsets, counts, inflected, lemmas):
    if not (LENGTH[0] <= len(word) <= LENGTH[1]):
        return "length"
    if not re.search(r"[aeiouy]", word):
        return "no vowel"
    if BANNED_WORD.search(word):
        return "rude"
    if word in inflected:
        return "inflected"
    for end, base in (("ies", "y"), ("es", ""), ("s", ""), ("ed", ""), ("ed", "e"), ("ing", ""), ("ing", "e")):
        if word.endswith(end) and word[: -len(end)] + base in lemmas and len(word) - len(end) >= 3:
            return "inflected"
    row = counts.get(word)
    if not row or row[0] < MIN_COUNT:
        return "too rare to be sure of"
    if row[1] > row[0]:
        return "mostly capitalized"
    if zipf(row[0] + row[1]) > MAX_ZIPF:
        return "too common"
    return None

A word must be 4 to 14 letters with a vowel, not rude (BANNED_WORD), and not an inflected form (WordNet’s *.exc lists, or -s, -es, -ed or -ing on another headword). It must have been printed at least 400 times since 1950 (rarer may be a scanning slip), written in lower case more often than capitalized (not a name), and no commoner than MAX_ZIPF.

Then the sense a dictionary would list first is chosen: the word’s most-used part of speech (WordNet’s sense counts, then Google’s tags), then its commonest sense. It’s dropped if WordNet marks it as a trade name, slur, abbreviation, plural, taxonomic family, or about sex or the toilet (BANNED_DOMAINS, BANNED_GLOSS; it’s a family site). “Of or relating to …” adjectives and “-ly” adverbs go too, being just another word with a suffix.

Families: one word each#

Defenestrate and defenestration are one joke. families builds a union-find over every lemma, joining words WordNet links by derivation, pertainym or participle, spelling variants (caliph, calif), words with the same rough stem (stem strips up to two endings), and words one ending apart (enatic, enation). After scoring, one_per_family keeps one word per family and per sense, preferring one a person has already read:

def one_per_family(scored, first=()):
    """One word per family, and per sense (stridulate and clitter are one
    meaning): a word in `first` (already read by hand) if the family has one,
    else its best."""
    first = set(first)
    seen = set()
    best = []
    for c in sorted(scored, key=lambda c: (c["word"] not in first, -c["rating"], c["word"])):
        if c["family"] in seen or c["source"] in seen:
            continue
        seen.update((c["family"], c["source"]))
        best.append(c)
    best.sort(key=lambda c: (-c["rating"], c["word"]))
    return best

How a word is rated#

score_word gives each word four scores from 0 to 100. Its rating is their weighted sum, with guessability counted against it:

• Obscurity is the Zipf value on a straight line: 100 at 0.8 or rarer, 0 at 2.4.
• Guessable is a penalty for how easily a table would land on the meaning: familiar affixes around a known word (transparent: un·rent·able), a compound that means what its parts say (a footbath, but not a sheepshank, which is a knot), a famous relative (coronate), a classical root from ROOTS (-phobia, pyro-), a gloss that echoes the word, and “capable of being …” adjectives.
• Fun is whether it sounds like an English word that could mean anything: a letter-trigram model of everyday words (Trigrams), a sweet spot of seven to nine letters, and less for four consonants in a row or foreign-looking spellings.
• Definition rewards one clear, short sense: few senses, a gloss of 3 to 15 words, no parentheses, not “any of various …”.

A worked example, brisance: a Zipf of 0.9 gives obscurity (2.4 − 0.9) ÷ 1.6 × 100 ≈ 94. Nothing is guessable (0, so the full 100 counts), it scores 93 for fun, and it has one short sense (100). With the weights above: 28.2 + 35 + 13.95 + 20 = 97.15, a rating of 97.

The weights were tuned by reading what they picked, and can be retuned freely: the bank is rebuilt from the same hand review, and ids never change.

Candidates, the hand pass, and the bank#

python3 -I tools/fictionary/build.py candidates WORDNET_DICT COUNTS_DIR candidates.json -n 2000
python3 -I tools/fictionary/build.py bank WORDNET_DICT COUNTS_DIR

candidates writes the best-scored words, from families nobody has read yet, with their glosses. select takes them best first, skips any with a guessable score over 55, and caps the share of jargon and of plants and animals:

def select(best, n):
    """The top `n`, with heavy jargon at most 4% and plants and animals at
    most 12%; nothing too guessable."""
    caps = {"technical": 0.04, "nature": 0.12}
    picked = []
    taken = Counter()
    for c in best:
        if len(picked) == n:
            break
        if c["scores"]["guessable"] > MAX_GUESSABLE:
            continue
        cap = caps.get(c["topic"])
        if cap is not None and taken[c["topic"]] >= cap * n:
            continue
        taken[c["topic"]] += 1
        picked.append(c)
    return picked

A person reads every candidate and gives it a line in tools/fictionary/house.tsv: the word, its source sense, keep or drop, and either the reason for dropping (“guessable from ampere”) or a definition rewritten in the house voice (plain words, at most 15, sentence case, no final period, nothing sharing the word’s first five letters). About three in five were dropped. house.tsv is the bank’s real source: edit it, never the bank.

bank scores everything again and writes the words house.tsv keeps, with their definitions. If a retune changes a kept word’s first sense, the word is left out and named for rereading, so it never ships with a definition of the wrong sense. The bank’s header carries WordNet’s license and the Google Books credit (write_bank).

Verifying the definitions#

A rewrite can drift from the meaning, so every kept definition was reread by hand against its WordNet gloss and, where there is one, Webster’s 1913 (the Revised Unabridged Dictionary, via GCIDE; read, never copied). The verdict is in tools/fictionary/verified.tsv and the bank’s verified field: wordnet+webster when a 1913 sense agrees, wordnet-only when none does or there’s no entry (kept as WordNet has them, the recorded source).

tests/test_fictionary_verified.py and tests/test_fictionary_words.py check that every word has a verdict, the bank matches verified.tsv and holds exactly what house.tsv keeps, every definition reads like a player’s, and jargon and nature stay a small share.

Note: select caps technical words at 4% of each candidate list, while the test (and docs/fictionary.md) allow 5% of the bank. The build never offers more than 4%; 5% is what fails the suite.

Fictionary’s pronunciations#

Beside each word the table shows a dictionary respelling (\ˈbrē-zäⁿs), and a speaker button sounds out a plain one (“BREE-zahns”). tools/fictionary/ipa.py builds both from three sources in strict order of trust, and leaves a word with nothing rather than guess:

1. Webster’s 1913, through GCIDE (GPL 3 or later). GCIDE writes marked letters as entities (<amac/ is a with a macron). webster changes only the display: entities become the key’s letters, stress marks move before their syllable, and syllables are joined by hyphens. An entry is dropped whole if the scan lost a letter (<?/), it’s a bare ending, or its syllable count doesn’t match the headword’s.
2. Wiktionary’s enPR respelling (CC BY-SA 4.0), from the Wiktextract dump at kaikki.org.
3. Worked out by rule from IPA: CMUdict’s (BSD-style; ARPAbet turned into General American IPA by from_arpa), else Wiktionary’s.

ipa.py fetch DATA words.toml downloads cmudict.dict and its license, GCIDE 0.53, and one small Wiktextract page per bank word CMUdict lacks. ipa.py build reads only those and writes three files, each under its source’s license:

The share-alike and GPL files are kept apart on purpose: nothing else is made from them, so their licenses reach no further. The server reads them in tables/fictionary/words.py (_pronunciations), Webster’s winning where it has the word. tests/test_fictionary_pronunciation.py checks the shapes, the order of trust, and that each source stays in its own file.

respell makes “BREE-zahns” from ˈbɹiːzɑ̃s. It splits the IPA into phonemes (tokens), then syllables, giving the second syllable the longest onset English allows (syllables). It spells each sound from the SPELL table and capitalizes the stressed syllable. A sound the table can’t spell means no respelling.

Fictionary’s spelling list#

“Check spelling before sending” runs in the browser against games/static/games/fictionary/spell/words.txt, made by tools/fictionary/spell.py:

• ENABLE (about 173,000 words with their forms, public domain) is what passes.
• CMUdict only marks which words are everyday, so recieve suggests receive, not a Scrabble word one edit away.
• tools/fictionary/spell-extras.txt adds modern words, interjections and proper nouns (email, podcast, countries, months).
• The bank’s headwords pass with their endings but are never suggested.

The file has four sorted sections (common words, other ENABLE words, words that take endings, and the commonest words, which break ties), each front-coded: a line is one base-36 digit for how many letters it shares with the line before, then the rest.

def front_code(words):
    lines, prev = [], ""
    for w in sorted(words):
        n = 0
        while n < min(len(w), len(prev), 35) and w[n] == prev[n]:
            n += 1
        lines.append("0123456789abcdefghijklmnopqrstuvwxyz"[n] + w[n:])
        prev = w
    return "\n".join(lines)

So 0aardvark then 8s means aardvarks. It gzips to about a third of a megabyte, and the page fetches it only once someone starts writing. npm run check:fictionary (tools/fictionary/spell.test.mjs) checks the real list decodes, catches common misspellings, and suggests the right word.

Rebuilding any of it#

All three are rebuilt in a new, empty directory outside the repo, following docs/fictionary.md (“Rebuilding the bank”, “Rebuilding the pronunciations and the spelling list”): ngrams.py, build.py candidates, new lines in house.tsv and verified.tsv, build.py bank, ipa.py fetch and build, spell.py. Then run uv run python manage.py test tests.test_fictionary_words tests.test_fictionary_verified tests.test_fictionary_pronunciation and npm run check:fictionary. Word ids are forever (ADR 0019): a word can be retired, never renamed. Dealing is in Fictionary’s chapter.

The bots’ pool of fakes#

Fictionary’s bots write mostly from tables/fictionary/content/fakes.toml: 20,000 made-up definitions in the word bank’s mix of parts of speech, each tagged with its part of speech, kind (cat), frame and content words. tools/fictionary/fakes.py builds it from tools/fictionary/frames/ with a fixed seed, in about twenty seconds. Nothing downloaded goes in.

A frame is a sentence with blanks, plus groups saying which values fit together. Any value of a blank in a group goes with any value of the others; values that only work as a pair are given together ("tool+task"):

def expand(fr, rng):
    """Every definition a frame makes, group by group (a sample of a big
    group's), each with the frame's id."""
    slots = list(dict.fromkeys((m.group(2).lower(), m.group(3) or "1") for m in kit.SLOT.finditer(fr.text)))
    out = []
    for gi, group in enumerate(fr.groups):
        lists, fills = _lists(fr, group, gi, slots)
        total = math.prod(len(v) for v in lists)
        if total > PER_GROUP:
            combos = set()
            while len(combos) < PER_GROUP:
                combos.add(tuple(rng.randrange(len(v)) for v in lists))
            picks = [tuple(lists[i][j] for i, j in enumerate(c)) for c in sorted(combos)]
        else:
            picks = list(itertools.product(*lists))
        for combo in picks:
            values = {}
            for members, vals in zip(fills, combo):
                values.update(zip(members, vals))
            by_name = collections.defaultdict(list)
            for (name, _), v in values.items():
                by_name[name].append(v)
            if any(len(vs) != len(set(vs)) for vs in by_name.values()):
                continue
            text = render(fr, values)
            low = text.lower()
            if any(all(a.lower() in low for a in pair) for pair in fr.avoid):
                continue
            out.append({"pos": fr.pos, "cat": fr.cat, "frame": fr.id, "text": text})
    rng.shuffle(out)
    return out

Every combination is made (sampled, for a huge group), alongside definitions written whole: about 311,000. The gates keep about 27,000. In order: the house voice (at most 15 words and 100 characters, a capital, no final period, “To” for a verb, no word twice); never essentially a real definition in the bank; no near-twin of one already in (content-word Jaccard of 0.5, found fast by indexing each definition’s rarest words); no content word or frame over its cap.

def gate(cands, bank, tg, scale=None):
    """The candidates that pass, in order, and why the rest didn't.
    `scale` shrinks the caps for a module checked alone."""
    word_caps, frame_caps = caps(bank, tg)
    if scale:
        word_caps = {p: (max(8, round(b * scale.get(p, 1))), {w: max(8, round(c * scale.get(p, 1))) for w, c in m.items()})
                     for p, (b, m) in word_caps.items()}
        frame_caps = {p: max(FRAME_FLOOR, round(c)) for p, c in frame_caps.items()}
    reals = Index()
    real_texts = []
    for w in bank:
        reals.add(likeness.words(w["definition"]))
        real_texts.append(w["definition"])
    pools = {p: Index() for p in kit.POS}
    keys = set()
    words_n = {p: collections.Counter() for p in kit.POS}
    frames_n = collections.Counter()
    kept, why = [], collections.Counter()
    why_by_frame = collections.defaultdict(collections.Counter)
    for c in cands:
        pos = c["pos"]
        ws = likeness.words(c["text"])
        reason = voice(c)
        if not reason and reals.near(ws, NEAR):
            reason = "like a real one"
        k = key(c["text"])
        if not reason and k in keys:
            reason = "duplicate"
        if not reason and pools[pos].near(ws, NEAR, NEAR_SHARED, NEAR_SHARED_J):
            reason = "near-twin"
        if not reason:
            base, per = word_caps[pos]
            if any(words_n[pos][w] >= per.get(w, base) for w in ws):
                reason = "word over-used"
        if not reason and c["frame"] and frames_n[c["frame"]] >= frame_caps[pos]:
            reason = "frame full"
        if reason:
            why[reason] += 1
            why_by_frame[c["frame"] or "hand"][reason] += 1
            continue
        keys.add(k)
        pools[pos].add(ws)
        words_n[pos].update(ws)
        if c["frame"]:
            frames_n[c["frame"]] += 1
        kept.append({**c, "words": sorted(ws)})
    # (rules.close catches what Jaccard alone might not: the same words reordered.)
    return kept, why, why_by_frame

Hand-written definitions go first, then frames take turns one definition at a time so each keeps its share. Then each part of speech takes its share, and any surplus framed ones furthest from real definitions’ length give way (choose). An id is a hash of the words, so it survives rebuilds.

fakes.py check <module> builds one module and prints its yield, what each gate removed, its most-used words and a sample. review N prints N random definitions, as for the hand check. tests/test_fictionary_fakes.py holds the pool to its gates and frames. How bots use it is in Fictionary’s chapter.

The chess bundle#

Chess Puzzles plays positions from real games, from the Lichess puzzle database: about six million puzzles, each a position, its solving line, and a rating from millions of plays. It’s CC0 (public domain), though the game credits Lichess and links each puzzle anyway. ADR 0028 bundles a slice as static files rather than calling Lichess or using a database: static files cache, Endless needs no server, the server reads the same files, and a remade bundle is a commit, not a migration.

The filter#

tools/chess/bundle.py reads the decompressed CSV (about 1.1 GB) row by row. There are five bands by Lichess rating, 8,000 puzzles each:

A candidate needs a sure rating (deviation at most 80), popularity at least 90 of 100, and at least 1,000 plays. Its line has an even number of moves (the opponent moves first, then solver and opponent alternate), with one to four solver moves in these shares:

Each band is split into hundred-point slices with equal shares, so it isn’t bunched where Lichess has most puzzles. A puzzle’s motif is the first idea on the MOTIFS priority list among its themes (smothered mate before mate in two, mate in two before fork). No motif fills more than 25% of a slice, so a band isn’t all mates in two. Only themes a player can read something into (SHOWN: “Fork”, “Middlegame”, “Rook endgame”) are kept for the result sheet.

Sampling by hash#

As it streams, candidates keeps a heap per (band, hundred, length) of the 8,000 passing puzzles whose ids have the smallest SHA-256:

def order(puzzle_id):
    """A stable, well-mixed order to sample by."""
    return hashlib.sha256(puzzle_id.encode()).digest()[:8]

choose takes each slice’s quota from that pool, setting aside puzzles whose motif is over its cap unless the slice can’t fill otherwise. Strong one-movers are rare, so a short hundred is topped up with two- and three-movers. A hundred with too few good puzzles at all (Beginner’s 600s) leaves its share to the band’s other hundreds.

Because the order is a hash of the id, a newer dump keeps every puzzle still there that still passes, and a band added at either end leaves the others exactly as they were. That’s how Beginner and Expert were added without changing the first three bands’ dailies.

Shards#

The bundle is written to games/static/games/chess/puzzles/: 16 shards per band (easy-00.json to easy-15.json, about 500 puzzles and 70 KB each), and an index.json with each band’s range, shard and puzzle counts, and provenance:

def shard_of(puzzle_id, shards=SHARDS):
    """Which shard holds a puzzle: FNV-1a of its id. The game (bundle.js)
    and the server (chesspuzzles/bundle.py) work it out the same way, so a
    puzzle's address needs only its band and id."""
    h = 0x811C9DC5
    for byte in puzzle_id.encode():
        h = ((h ^ byte) * 0x01000193) & 0xFFFFFFFF
    return h % shards

A puzzle lives in the shard its id hashes to (FNV-1a, mod the shard count). The filter, the server (chesspuzzles/bundle.py) and the game (games/static/games/chess/engine/index.js) compute it the same way, so a puzzle’s address is just its band and id: #/p/medium/0JMFA. FNV-1a of 0JMFA is 2,901,343,555, which is 3 mod 16, so it’s in medium-03.json:

["0JMFA", "r2q3r/pp6/2np4/4p3/1PB1P2k/P4PQb/2P2KP1/b7 b - - 3 28", "h4h5 c4f7 h5h6 g3g6", 1778, "mate mateIn2 middlegame"]

That’s the id, the position (FEN), the line in UCI, the rating and the themes. Black’s king steps to h5 (played for you). The solver’s moves are every other one from the second: Bc4–f7, check, and Qg3–g6, mate. tests/test_chess.py runs the filter’s shard_of against the server’s and keeps the bundle under 6 MB.

Checking it, and the dailies it feeds#

npm run check:chess (tools/chess/check.mjs) replays every puzzle with the game’s rules and the vendored chess.js. Each must be in its hashed shard, rated inside its band, a legal position with a legal line of two to eight moves, and solved by its own solution but not one move short, one move long, or with a wrong first move. Run it after remaking the bundle or touching the engine.

The server reads the shards once per process. A band’s daily comes only from puzzles of two or more solver moves, so it’s never over in one:

@cache
def daily_pool(name) -> tuple[str, ...]:
    """The puzzles a band's daily is picked from, in id order: two moves or
    more, so a day's puzzle is never over in one. (Duplicates, `retired`, stay
    in it, so the order every day's pick goes by doesn't move; `pick` passes
    them over.)"""
    return tuple(sorted(p.id for p in band(name).values() if len(p.line) >= 4))

Dailies covers how the day’s band is weighted and its puzzle chosen, in an order only the site’s secret key can produce, with no repeat within three years. Dailies and solves are kept by Lichess id, so a remade bundle never changes a played day. A dropped puzzle just can’t be posted any more.

Duplicates, retired#

Lichess has a few puzzles twice, from different games reaching the same place. tools/chess/duplicates.mjs treats two puzzles as the same when the solver sees the same position (pieces and side to move, after the opponent’s first move) and plays the same moves. The lowest-rated stays. The rest are retired, listed in index.json as duplicates with the id that stays:

/** { retired: kept } for every duplicate in the bundle in `index`. */
export function findDuplicates(index) {
  const groups = new Map();
  for (const [band, info] of Object.entries(index.bands)) {
    for (let k = 0; k < info.shards; k++) {
      for (const row of JSON.parse(readFileSync(new URL(`${band}-${String(k).padStart(2, '0')}.json`, DIR))).puzzles) {
        const p = fromRow(row, band);
        const seen = positionAt(p, 1).fen().split(' ').slice(0, 2).join(' ');
        const key = `${seen}|${p.line.filter((_, i) => i % 2 === 1).join(' ')}`;
        if (!groups.has(key)) groups.set(key, []);
        groups.get(key).push({ id: p.id, rating: p.rating });
      }
    }
  }
  const out = {};
  for (const group of groups.values()) {
    if (group.length < 2) continue;
    group.sort((a, b) => a.rating - b.rating || (a.id < b.id ? -1 : 1));
    for (const dup of group.slice(1)) out[dup.id] = group[0].id;
  }
  return Object.fromEntries(Object.entries(out).sort(([a], [b]) => (a < b ? -1 : 1)));
}

A retired puzzle stays in its shard, so an old daily, a shared link or a half-played puzzle still opens. It’s just never drawn again, by Endless or a daily (retired() in chesspuzzles/bundle.py).

check:chess fails if the list isn’t exactly what the script finds, so a remade bundle can’t bring a duplicate back unseen. node tools/chess/duplicates.mjs --write rewrites the list and raises the bundle’s version if it changed. Retiring them raised it to 2. A remade bundle carries the old version and duplicates over and raises the version if any shard changed (carried in tools/chess/bundle.py), so the version only goes up.

The pieces#

The pieces are Colin M.L. Burnett’s set from Wikimedia Commons, used under the BSD license (games/static/games/chess/pieces/LICENSE.txt). tools/chess/pieces.mjs turns the twelve SVGs into one module, games/static/games/chess/js/pieces.js, removing only wrappers and whitespace. chess.js is vendored with its license (games/static/games/chess/engine/vendor/LICENSE-chess.js.txt).

Low Poly’s pictures#

In Low Poly you join dots, and each triangle fills with its color until a picture appears. The pictures are low-poly versions of Microsoft’s Fluent Emoji, “Flat” style (MIT). Flat art beats photographs: one subject, a clear background, a few strong colors, so even 25 dots read as a fox. SUBJECTS in tools/low-poly/subjects.mjs lists each subject’s Fluent folder, the title shown once solved, and its kind. SOURCE beside it has the full MIT notice.

From picture to dots and triangles#

tools/low-poly/build.mjs takes a local copy of the Fluent Emoji assets. For each subject it:

1. Draws it into pixels. tools/low-poly/raster.mjs draws the SVG on a 256 × 256 canvas in headless Chrome (tools/house/chrome.mjs) and reads back the RGBA pixels.
2. Lights it. Flat art has no shading, and a low-poly picture is its facets. lit in tools/low-poly/mesh.mjs brightens toward the top left and darkens toward the bottom right, giving flat parts a slope for triangles to follow.
3. Places dots by greedy insertion (Garland and Heckbert, 1995). From the frame’s four corners, it keeps adding the dot that most lowers the error, maintaining a Delaunay triangulation (tools/low-poly/delaunay.mjs). The error is each pixel’s distance from its triangle’s flat color, weighted by alpha so outlines count more. Dots are tried where the picture changes (an edge, a shadow, an eye) and, more sparsely, inside it. Per-row prefix sums make a try cost only the triangles it replaces.
4. Keeps it fair to play. A dot is refused if it’s within the size’s gap of another (too close for a fingertip), if a line would pass too near a dot it doesn’t end at, or if it makes a sliver (a corner under 14°) in a triangle likely to be kept. A few slivers get through (Low Poly’s known issues).
5. Drops the background. Mostly transparent triangles go. One whose three sides all touch kept triangles would look like a mistake that never fills, so fillHoles puts it back in its neighbors’ color.
6. Colors each triangle with the mean of its opaque pixels.

The three sizes are three densities of dots, and the gap sets a board’s zoom unit:

// Per size: how many dots, and the least gap between two (px of the S × S
// source). A board unit is the gap / 0.7, so at the kit's zoom a dot's
// nearest neighbor is at least 0.7 of a cell away; a line keeps 0.42 of a
// unit off any dot it doesn't end at, clear of the dot's 0.3-unit disc.
export const TIERS = {
  easy: { vertices: 25, gap: 22 },
  medium: { vertices: 55, gap: 14 },
  hard: { vertices: 100, gap: 10 },
};

A subject that doesn’t read with Easy’s few dots gets more (an optional fourth SUBJECTS entry, { easy: 35 }), or no Easy at all ({ easy: 0 }). Every subject was eye-checked on a contact sheet (--sheet out.html) before it was kept.

Codes, files and the manifest#

A picture’s code is six letters from its folder name, so adding pictures never changes another’s code:

/** A picture's code: six letters from its folder name (FNV-1a), the same every time. */
export function codeOf(name, salt = 0) {
  let h = 0x811c9dc5 ^ salt;
  let out = '';
  for (let round = 0; out.length < 6; round++) {
    for (const ch of `${name}#${round}`) h = Math.imul(h ^ ch.charCodeAt(0), 0x01000193) >>> 0;
    out += ALPHABET[h % 32] + ALPHABET[(h >>> 5) % 32] + ALPHABET[(h >>> 10) % 32];
  }
  return out.slice(0, 6);
}

Fox is 76XLFK. build.mjs writes pictures/<code>.json per picture, with all its sizes (integer dots, triangles in a fixed order, #rrggbb colors), plus:

• games/static/games/low-poly/pictures/codes.js: every code, and WITHOUT, the pictures each size leaves out (codesAt(size)).
• games/static/games/low-poly/pictures/index.json: the source and notice; each picture’s code, title, kind, sizes and source URL; and daily, the order dailies take.

The daily order takes kinds in turn, each kind shuffled by a hash of its codes and spread evenly, so neighbors are seldom the same kind (a few are, all “thing”, the biggest kind):

// The daily order: the kinds taken in turn, each kind's pictures in an order
// of their own (by a hash of their codes), so neighbors are seldom alike (a
// few are, all "thing", the biggest kind). Dailies before one daily a game
// stepped through it; since, the server keeps its kinds in turn but shuffles
// each kind's pictures by the site's key (puzzles/low_poly.py).
function dailyOrder(pictures) {
  const kinds = new Map();
  for (const p of pictures) (kinds.get(p.kind) || kinds.set(p.kind, []).get(p.kind)).push(p);
  for (const list of kinds.values()) list.sort((a, b) => codeOf(a.code, 7).localeCompare(codeOf(b.code, 7)));
  const lists = [...kinds.values()].sort((a, b) => b.length - a.length);
  const total = pictures.length;
  // Each kind spread evenly through the order: its k-th picture lands at (k + ½) / n of the way.
  const slots = [];
  for (const list of lists) list.forEach((p, k) => slots.push([(k + 0.5) / list.length, lists.indexOf(list), p.code]));
  slots.sort((a, b) => a[0] - b[0] || a[1] - b[1]);
  const order = slots.map(s => s[2]);
  if (order.length !== total) throw new Error('daily order lost a picture');
  return order;
}

The server (puzzles/low_poly.py) deals dailies from pictures that come in every size. With three dailies a day, it walked this public order from a start picked by the site’s secret key. With one daily a game, it keeps the kinds in turn but shuffles each kind by the key. See Dailies and Low Poly.

Remaking a picture gives the same dots, lines and colors, so a code keeps its puzzle. A maker change that moves anything needs an ENGINE_VERSION bump. npm run check:puzzles holds fixed pictures’ fingerprints and checks every picture at every size against the engine’s problems(): no crossing lines, no dot on a line, no two dots touching (audit in picture.js, which build.mjs also runs). tests/test_low_poly.py checks every picture names its source and license.

Griddles’ pictures#

Griddles is the family’s name for a nonogram: row and column clues (“3 1 2”) describe a black-and-white picture. A good one has exactly one answer, reached by logic, and looks like something when done. Most are icon silhouettes from Phosphor Icons (fill) and Tabler Icons (filled), both MIT. A few dozen were drawn here, mostly at 10×10, where icons lose the most. Credits and licenses are in docs/griddles.md and the source file’s header.

The source: pictures.txt#

tools/griddles/pictures.txt holds the pictures as hand-editable text, # filled and . empty, under a header with size, id, title and source:

== 10 airplane Airplane | phosphor:airplane
....##....
....##....
....##....
....##....
..######..
##########
##########
....##....
...####...
...####...

tools/griddles/pictures.mjs reads and writes it, refusing a row of the wrong width (it would shift every later row into the next picture).

Within a size, a picture’s position is its place, which codes name. So new pictures go at the end of their size, and changing, moving or removing one needs an ENGINE_VERSION bump. Past dailies keep their pictures regardless (Griddles). A weak picture is retired, not removed (| retired at the end of its header): its codes still open it, but it’s never dealt again (Retired pictures).

Sizes are the difficulty#

Each size’s pictures need exactly one tier of the line solver’s technique (games/static/games/griddles/engine/solver.js): the simple slide at 10×10 (where a run’s leftmost and rightmost places overlap, fill those cells), the edges at 15×15 (a run held by a filled cell it must touch), and the whole line at 20×20 (keep what every placement of the line’s runs agrees on). No picture needs the fourth technique, trial and error. It’s in the solver only to prove that.

export const TIERS = { easy: 1, medium: 2, hard: 3 };

From icon to grid#

1. Rasterize (tools/griddles/raster.mjs). In headless Chrome, each icon is drawn at 512 px, cropped to its ink, and fitted to each grid size, plain or stretched up to 10%, at six small nudges. Each cell’s ink share is thresholded at seven levels from 0.35 to 0.65. A candidate must be 22% to 80% filled, with no empty row or column inside the picture (it would be in two pieces). Candidates are tried closest to the icon first. The first the solver finishes at exactly the size’s tier wins, else the closest it finishes at all, kept with its ink shares for touching up:

// Each size and the tier of technique its pictures must need (solver.js): 10×10
// simple sliding alone, 15×15 the edges, 20×20 the whole line.
const SIZES = [[10, 1], [15, 2], [20, 3]];

2. Look it over (tools/griddles/sheet.mjs). PNG contact sheets of numbered thumbnails. A person keeps a picture only if it reads at that size. Fewer than a fifth of about 480 subjects read at 10×10, which is why so many 10×10s are hand-drawn.
3. Touch up (tools/griddles/touch.mjs). A picture too easy for its size gets outline cells flipped, those with ink nearest half so it changes least, until the solver needs the tier and still finishes without guessing: one cell, else two, else a bounded, deterministic search of three or four. --write rewrites pictures.txt, and changes are looked over again.
4. Build and prove (tools/griddles/build.mjs). Each picture must pass three independent tests, or the build refuses it and names its line: the line solver finishes this picture needing exactly its size’s tier; a plain search (games/static/games/griddles/engine/count.js, sharing no code with the solver) finds exactly one answer; and it’s 15% to 85% filled. It then writes games/static/games/griddles/engine/pictures.js: per size, [id, title, cells as hex, measure], where the measure counts each technique’s line steps (“s31 e6 l2”), followed by "retired" for a retired picture (still proved, since it can still be played).

The solver and search agreeing is the uniqueness proof. npm run check:puzzles runs build.mjs --check, which re-proves every picture and fails if pictures.js isn’t exactly what pictures.txt makes, so a hand edit to the output can’t slip through.

A code names a place in a size’s list (its letters as a number, mod 1024, mod the list’s length). The server (puzzles/griddles.py) picks each day’s picture so none is a daily twice at its size until every one still dealt has been; see Dailies.

Question banks and crowd seeds#

Party games run on questions, prompts and cards. ADR 0019 settles how they’re kept: a TOML file in the game’s module, tables/<slug>/content/*.toml, a list of tables with stable ids, read by tables/content.py:

[[q]]
id = "window-or-aisle"
ask = "Window or aisle?"

TOML is in Python’s standard library (tomllib) and diffs cleanly. The bank ships with a deploy and is the same for every game, so reading it doesn’t break the rule that game rules do no IO (the table service). Bank.load caches one Bank per file per process, in file order, so a seeded pick repeats exactly.

def pick(self, rng: random.Random, n: int, exclude=(), apart=None) -> list[dict]:
    """`n` items at random, none of them in `exclude` (ids). `apart(a, b)`,
    if given, says two items mustn't be dealt together (near-twins): an
    item is passed over if it's apart from one already picked. Raises
    ValueError if the bank can't supply that many."""
    skip = set(exclude)
    pool = [item for item in self.items if item["id"] not in skip]
    rng.shuffle(pool)
    picked = []
    for item in pool:
        if len(picked) == n:
            break
        if apart and any(apart(item, other) for other in picked):
            continue
        picked.append(item)
    if len(picked) < n:
        raise ValueError(f"The bank has only {len(picked)} items to pick from, not {n}.")
    return picked

apart(a, b) keeps near-twins out of one deal. Ids are forever, since anything counted about an item is kept under its id: retire an item, never rename it. Bank refuses a file with a missing or repeated id. Today only Fictionary uses a bank (Bank.load(path, table="word"), with its own weighted dealing in tables/fictionary/words.py). tests/test_content.py checks the bank.

Crowd seeds#

Some questions want to show what people really answer: “only 22% pick the duck”. The crowd app keeps anonymous tallies, one count per (game, question, choice), with nothing about who answered or where. The bank gives each question a designer’s guess, its seed (a share per choice). crowd/__init__.py blends the two as if the seed were 20 answers already in:

def blend(seed: dict, live: dict, prior: float = PRIOR) -> dict:
    """The share for each choice: the seed counted as `prior` answers, then
    the live ones. Pure."""
    choices = sorted(set(seed) | set(live))
    total = sum(live.get(c, 0) for c in choices)
    weight = prior + total
    if not choices or weight <= 0:
        return {}
    return {c: (prior * seed.get(c, 0.0) + live.get(c, 0)) / weight for c in choices}

A new question seeded 70/30 shows 70/30. After 20 real answers split 10/10, it shows (20 × 0.7 + 10) ÷ 40 = 60%. After hundreds, the seed hardly matters. The seed is always the bank’s current one.

Rules never touch the database. A game asks for shares in prepare, before the deal, and leaves {"sink": "crowd", …} records in its state for the service to write in the same transaction, at most 2,000 answers per table per day. No game on the shelf uses tallies yet; they were designed for Called It, a this-or-that party game not yet on the shelf (ADR 0019). The machinery and its tests (tests/test_crowd.py) are ready.

Fonts#

Every face is an open font under the SIL Open Font License, self-hosted: a Latin-subset .woff2 in the look’s fonts/ directory, loaded by an @font-face, with its license beside it (OFL-<family>.txt, or OFL.txt for a one-family directory). No font service is used, so pages make no third-party request and work the same on a slow line.

A new look adds faces the same way (AGENTS.md, “Adding a multiplayer game”). Fonts stay out of covers lists (docs/guide/README.md), since a font can’t change what a chapter says.

Adding content#

What each check proves, and when to run the rest, is in Testing and checks.