Skip to content
QuranBench
Editorial · describes v0.8.0

How this is built

Do not accept our interpretation. This page tells you exactly where every field comes from, what we compute, what we merely assert, and where the edges are — so you can reproduce any figure on the site and judge it for yourself.

Try it yourself — a worked example

Editorial

You do not need Arabic to check a claim on this site. Here is one walked end to end, in English, using only links.

  1. Suppose you read that “mercy” is central to the Quran and want to see for yourself. Start from the English side: the reverse lookup for “mercy” shows every Arabic word that translators render with that English word — and that they are not one word but several, from different roots.
  2. Follow the most common one to its root page, ر ح م. Everything there is computed from the corpus: how often the root occurs, in which forms, in which surahs — with every occurrence linked to its verse.
  3. Reproduce the count yourself: run the search root:ر ح م. The result states the corpus version and parameters it was computed with; the number on the root page and the number here must agree, and you can watch them do so.
  4. See how translators disagree: compare the translations of 1:1, where two words from this one root are rendered differently by every edition on file.

Every number on this site can be interrogated this way. If a page shows a figure you cannot reproduce by clicking through to the search that generated it, that is a bug — report it.

The text edition

Quranic text

The Arabic is the Tanzil Quran Text (Uthmani) (Uthmani 1.1), published by the Tanzil Project under CC-BY-3.0. It is the immutable source of truth. We never modify Quranic text: normalised, tashkeel-stripped and segmented forms are stored as separate labelled fields, and the source text is always attributed to Tanzil.

Its sha256 is recorded in the manifest, so you can confirm the exact bytes we ingested:

4b91f9e6e8ac645d039e4ed85b3be492e795232a31cd22d668ac58238722e26f

The morphology source

Morphology

Root, lemma, part-of-speech and sub-word segmentation come from the Quranic Arabic Corpus — Morphology (mustafa0x fork of Leeds QAC v0.4), licensed GPL-2.0-or-later. This is an external annotation. It never alters a token’s id, position or text, and it carries its own copyleft licence — which is why the tokenised tokens.jsonl is distributed under the GPL while the plain text stays CC-BY.

Each token records how the Leeds word aligned onto it, so a questionable annotation is traceable rather than anonymous.

How normalisation works

Computed

Search runs over a normalised form derived from the Uthmani text by a fixed, disclosed set of rules. Normalisation only ever produces a new labelled field — it never rewrites the source.

  1. strip-tashkeel

    Remove Arabic diacritical marks.

  2. remove-tatweel

    Remove tatweel (kashida) elongation.

  3. unify-alef

    Unify alef variants to bare alef.

  4. normalise-ta-marbuta

    Map teh marbuta to heh.

  5. normalise-alif-maqsura

    Map alef maksura to yeh.

  6. to-simple

    Plain-orthography form: remove Quranic recitation/pause signs and unify alef wasla to bare alef, keeping harakat. Applied to derive token text_simple, which cannot be sourced from the Simple edition because the editions do not tokenise word-for-word.

The basmala

Computed

The basmala is handled as separated: in most surahs it is a separated opening, stored as its own token group that is not a counted verse row. Surah 1 counts it as a verse; surah 9 has none. Because it is a parameter, a computation discloses whether it counts the basmala (basmala=1 or 0) rather than baking one choice in.

Numbering is a parameter

Computed

Verse numbering is a recorded parameter, not a fact baked into identifiers. The active scheme is kufan (6236 counted verses). Each scheme is a data file; the active scheme’s counted segments are the rows of verses.jsonl. An identifier is scheme-qualified, so it means the same thing regardless of which scheme you view under.

Computed versus asserted

Computed

Computed: every count, frequency, form list, root distribution and search result. These are derived deterministically from the loaded corpus and carry the exact parameters that produced them. Re-running the same query on the same version gives the same number.

Asserted from an external source: the morphology (Leeds QAC) and the human translation editions. We attribute these, never present them as our own computation, and never present them as Quranic text.

Asserted by us (editorial): investigations and notes. These are labelled editorial and are the only layer that is our opinion.

How to reproduce any figure

Every number on the site links to the query and version that produced it. To reproduce one independently:

  1. Download the corpus version from /data and verify its sha256.
  2. Re-run the query against the API — for example /api/v1/search?q=root:z-k-w. The response echoes the parsed query, the corpus version and the computation parameters.
  3. Compare the total to what the page shows. They must match.

Similar verses — the measure

Computed

On a single verse page, the 8 most similar verses are ranked by the Jaccard index of the two verses’ sets of morphological roots: J(A, B) = |A ∩ B| / |A ∪ B|. Jaccard is symmetric and bounded in [0, 1], and — unlike a raw shared-root count — is not inflated by verse length, so a short verse and a long one are compared fairly. Only verses that share at least one root are candidates, gathered through the root→verses index, so the cost is bounded by the target verse’s roots rather than the whole corpus.

Before the measure runs, a stoplist of ubiquitous roots is removed from every verse’s set. A root that occurs in a large fraction of verses carries no discriminating signal: left in, it would make almost every verse look similar to every other. The stoplist is defined precisely and reproducibly as every root occurring in 500 or more distinct verses. In v0.8.0 that is these 8 roots:

  • أ ل هe-l-h
  • ق و لq-w-l
  • ك و نk-w-n
  • ر ب بr-b-b
  • أ م نe-m-n
  • ع ل مaa-l-m
  • ق و مq-w-m
  • أ ي يe-y-y

The score and the exact shared roots are shown on every result, so you can see what drove the ranking and disagree with it. Verses stoplisted out of the target are listed too. The computation is per-request over the loaded corpus and stays within budget; no similarity is precomputed.

Root co-occurrence — connected roots

Computed

On a root page, the 12 most connected roots are shown. Window: the verse — two roots co-occur when they appear in the same counted verse. Measure: the number of distinct verses in which both roots occur. Raw shared-verse count is used rather than an association score because it is stable, needs no smoothing for rare roots, and reproduces exactly from tokens.jsonl.

The same stoplist above is applied to the results (not to the subject root), so the list surfaces genuinely connected concepts rather than the same handful of ubiquitous roots under every entry. Separated basmalas are excluded, as they are not counted verses.

Known limitations

This page is worth more than any feature, so it is blunt:

  • Verse slot labels remain Kufan-derived. The counted verse rows follow the Kufan tradition. Other counting traditions exist and are modelled as alternative schemes, but the default slots are Kufan and should not be read as tradition-neutral.
  • Translation alignment is verse-level only. The 4 translation editions align to the Arabic by verse identifier, never word-for-word. Word-level alignment data does not exist for these editions and is never fabricated — we do not claim which English word renders which Arabic word.
  • Morphology is a third-party annotation. The Leeds corpus is a scholarly reconstruction with its own editorial choices; disagreements with it are legitimate and traceable to the aligned source.
  • Token text_simple is computed, not sourced. It is derived from the Uthmani text because the Simple edition does not tokenise word-for-word — see the to-simple normalisation rule above.

The generated translation

One edition in the reader — QuranBench (generated) — is not a human translation. It is produced from a table of word-by-word decisions: each Arabic word (by root, lemma and part of speech) maps to one English rendering, decided from how the word is used across the whole text, and the build substitutes those decisions verse by verse. Arabic word order is preserved. No language model writes any of it, and no tafsir, hadith or existing translation was consulted; the machine indexes and substitutes, a person decides.

Every decision carries a grade. Settled means the text itself fixes the sense; supported means the evidence is thinner; judgement means the word occurs once in the Quran and its rendering rests on general Arabic rather than the corpus. Judgement words are underlined in the reader, and the full list is reviewable at /review. The edition is hash-locked to the exact decision table that produced it, so identical inputs always yield an identical text.

The complete method — twenty-two rules, each named for the error it prevents — is in the repository as docs/translation-method.md.