How the indexer works

Every number the API returns — token summaries, trades, holders, candles, burns, graduations — comes from a Cloudflare D1 database (HARTII_LABS_DB) that a single Pages Function, functions/api/indexer-run.js, keeps in sync with the chain. This page documents its exact behavior so you can reason about data freshness and the money-math guarantees the API relies on.

#What gets scanned

Each run issues quai_getLogs over three log sources:

1. Factory TokenLaunched — from every configured TokenFactory generation (HARTII_LABS_FACTORY_ADDRESS, plus optional V2/V3 addresses). TokenLaunched is byte-identical across factory generations, so one multi-address filter picks up launches from all of them in a single pass.
   solidityCopy

   event TokenLaunched(address indexed token, address indexed curve, address indexed creator, string name, string symbol, bytes32 metadataHash);

2. Curve Buy / Sell / Graduated / CreatorFeesWithdrawn — from every known BondingCurve clone address (every curve_address already in D1, plus any curve launched during this same run), all four topics filtered together in one call.
3. Token Transfer — from every known LaunchToken clone address. This is the only source holder balances are derived from (see below); it also carries the launch mint (from = 0x0), plain wallet-to-wallet moves, burns (to = 0x0), and the graduation move — everything that changes a balance, not just trades.

A graduated token's internal-pool trades continue to emit the same Buy/Sell event shapes as the bonding-curve phase, so nothing here has to special-case a token after graduation.

#Block windows and reorg safety

MAX_BLOCK_SPAN = 2000        // bounded per-run scan window, keeps each run fast
REORG_BUFFER_BLOCKS = 20     // never scan past (head - 20)

Each run computes toBlock = min(head - 20, fromBlock + 1999) and only commits its cursor for blocks actually scanned. The 20-block buffer is the entire reorg strategy — there is no rollback/undo logic for a block that turns out to have been orphaned; the indexer simply never treats a block that close to head as final in the first place. If you need finality guarantees stronger than "20 confirmations behind head," treat that as the floor, not a promise beyond it.

Block-timestamp lookups (quai_getBlockByNumber, one per distinct block touched by a matched log) run with a concurrency cap of 8 in-flight requests, deduplicated by block number first.

#Cursor and resume

The cursor lives in D1, table indexer_state, one row per network:

last_scanned_block INTEGER

On startup: fromBlock = cursor !== null ? cursor + 1 : HARTII_LABS_FACTORY_START_BLOCK (or 0). The cursor is only advanced (setCursor) after a scan succeeds — a run that throws partway through leaves the cursor exactly where it was, so the next run re-scans the same range rather than skipping it.

#Advisory lock

Two triggers can call /api/indexer-run at any moment — a Cron Worker firing every minute, and a traffic-driven backup on the busiest read route — and holder-balance recomputation is a read-modify-write, not an atomic operation, so an overlap could double-apply or drop a delta. Both triggers therefore run against a fenced compare-and-swap lock on indexer_state:

lock_until   INTEGER   -- ms epoch; NULL/past = free
lock_owner   TEXT       -- random id, minted fresh on every acquire attempt
LOCK_TTL_MS = 90_000     -- 90s

Acquire: UPDATE indexer_state SET lock_until=?, lock_owner=? WHERE network=? AND (lock_until IS NULL OR lock_until < now). A losing caller skips cleanly ({"skipped":"another indexer run in progress"} — see api.md). Release (in a finally block, regardless of success/failure) is fenced to the same owner id: ... WHERE lock_owner = mine — a run that gets pre-empted by a stale-lock takeover can never clobber the new owner's lock, it just fails to release (no-op) and lets the TTL be the backstop. setCursor is fenced the same way, so a runner that loses its lock mid-flight throws rather than committing a cursor it no longer has authority over.

#Holder balances

For every Transfer(from, to, value) on a known token:

if (from !== ZERO_ADDRESS) delta(from, -value)   // mint: nothing leaves 0x0
if (to   !== ZERO_ADDRESS) delta(to,   +value)    // burn: nothing arrives at 0x0

Zero-value transfers are skipped. Deltas are applied as next = max(0, current + delta) per (network, token_address, holder), upserted into the holders table.

Replay safety. Because both the cron worker and the traffic backup can retry a range after losing the lock, the same block range can legitimately be re-scanned. A per-token watermark table guards against double-applying it:

CREATE TABLE holder_apply_state (
  network TEXT, token_address TEXT,
  applied_through_block INTEGER NOT NULL DEFAULT -1,
  updated_at TEXT,
  PRIMARY KEY (network, token_address)
)

• If applied_through_block >= toBlock for this run, the whole range is a pure replay → skipped entirely ({written:0, skipped:true, reason:'already-applied'}).
• A partial overlap (a retry after losing the lock keeps the same fromBlock but reaches a larger toBlock because head moved on) is not caught by that check alone — applied >= toBlock is false even though part of the range was already folded in. The indexer filters out only the already-applied portion, block by block (event.block > applied), rather than reapplying the whole range.
• A gap between the watermark and fromBlock (blocks never folded in at all) is surfaced as a warning in the run's response rather than silently skipped.
• The watermark write and the balance upserts happen in the same db.batch() call, so they commit atomically together or not at all — a re-scan of an already-applied range is a true no-op, never a silent double-count.

#Burns

Any Transfer to the zero address on a known token is a burn, recorded in token_burns keyed by (network, tx_hash, log_index) — the same idempotency key shape as trades, so an overlapping re-scan can't inflate a burn total. This counts every burn: creator-triggered, holder-triggered, and a V2 curve's auto-buyback — /api/burns and a token's burnedWei field are platform-wide totals, not just "official" burns.

#Money-math rules

Every wei-denominated value — quai_amount, token_amount, price_wei, balance, volume_quai, and every derived *Wei/*_wei column — is stored in D1 as TEXT, never a numeric column. CAST(... AS INTEGER) overflows SQLite's signed 64-bit integer at roughly 9.2 QUAI (2^63 wei), which is trivially exceeded by a token's total supply or cumulative volume. The rule enforced throughout this codebase:

• Never CAST a wei column to a number in SQL for anything that has to be exact (a sum, a balance, a price). All exact math happens in application code with BigInt, guarded by a /^\d+$/ regex on the raw string before conversion (a malformed/empty string is treated as 0 rather than throwing).
• Sorting a wei column without an exact numeric cast is done with ORDER BY LENGTH(col) DESC, col DESC — for non-negative integer strings with no leading zeros, a longer string is always numerically larger, and equal-length strings sort correctly under plain lexicographic comparison. This is exact and BigInt-safe with zero conversion cost, and it's how /api/token/:address/holders orders its rows.
• CAST(...AS REAL) does appear in a few places (top-traders leaderboard ranking, dev-activity display, wallet-stats display) — always explicitly for ranking or display purposes where float precision loss above ~15 significant digits is an accepted, documented trade-off, never for a value the API represents as exact.
• Price is computed as (quaiAmount * 1e18) / tokenAmount, integer division on BigInts, so it stays an exact integer (wei per whole token) rather than a floating point ratio.

If you're consuming this API from a language with a native big-integer type, parse every *Wei field into one rather than a double/float, for the same reason.

#String sanitization

Token name and symbol are attacker-controlled — a creator can put arbitrary UTF-8 in these fields on-chain, and they flow into D1, every API response, and server-rendered OG/badge images. At ingestion (functions/_lib/eventDecode.js):

function sanitizeOnchainText(str, maxLen) {
  return str
    .replace(/[\u0000-\u001F\u007F-\u009F

]/g, ' ') // C0/C1 controls, line/para separators
    .replace(/\s+/g, ' ')
    .trim()
    .slice(0, maxLen);
}

applied as name: sanitize(name, 96), symbol: sanitize(symbol, 32) — name is capped at 96 characters, symbol at 32, both with all control characters and Unicode line/paragraph separators collapsed to a single space. This happens once, at ingestion — nothing downstream (including this documentation's examples) has to trust raw chain data.

#Chain params

ecosystem-stats and the creator dashboard need the factory's creationFee() and each curve's feeBps() — both owner-adjustable via contract setters, so they can't be hardcoded. Reading them live on every request was the single most expensive thing this API did (measured: ~69 requests/hour, 68 of them over 500ms, average 2.6s, worst case 9.9s). Instead, the indexer refreshes a small D1 cache after each run:

CREATE TABLE chain_params (network TEXT, key TEXT, value TEXT, updated_at TEXT, PRIMARY KEY (network, key))

keyed creation_fee_wei and fee_bps:<curve address>, self-throttled to refresh at most once per 30 minutes, bounded to 40 curves per refresh pass. Request-path routes read this cache first and only fall back to a live eth_call for a curve that isn't cached yet.

#Freshness guarantees a consumer can rely on

• Trigger cadence: a Cloudflare Cron Worker (hartii-labs-indexer-cron) POSTs /api/indexer-run every minute (* * * * *), independent of site traffic. A traffic-driven backup (functions/_lib/autoIndex.js) also fires on hits to the busiest read route, throttled to at most once per 60 seconds (an in-isolate timestamp check plus a D1 compare-and-swap on a small throttle table, so the cross-colo throttle costs no KV reads). Both triggers are just cheap pre-filters — the actual concurrency safety is the advisory lock described above, so an occasional double-fire is a harmless early exit, never a correctness issue.
• Practical staleness: with the cron worker running, expect data to lag chain head by roughly one run interval (a minute) plus the 20-block reorg buffer. Poll GET /api/status and compare indexer.lastIndexedBlock/lastIndexedAt against your own chain-head read if you need to detect an indexer stall rather than assume freshness — see integrate-api.
• Per-token isolation: a failure recomputing one token's holders/candles/summary does not abort the run or block the cursor from advancing — it's caught, recorded in the run's warnings array, and every other token still gets processed. A single bad token can't brick indexing platform-wide.
• Graduation is double-covered: besides the Graduated event itself, every run also reads each touched curve's on-chain graduated() flag as a defensive check against a missed log, and a bounded backfill pass (healGraduationOutbox, 10 rows per run) repairs any token whose graduation was recorded in tokens.status but never made it into the graduation_outbox feed that /api/graduations reads.
• Board summaries decay honestly: besides tokens touched this run, each pass also refreshes up to 5 of the stalest already-summarized tokens, so a token that pumped and then went quiet doesn't freeze at a stale change24h/volume24hWei forever.

#Related

• API reference — the read surface this data powers, including the exact wei-string convention every field above feeds into.
• Integration recipes — polling etiquette that assumes the cadence described on this page.