probes
Data license: CC BY-SA 4.0 (records) · Apache 2.0 (tools) · Data source: Loxyn-Korela/fault-atlas
1 row where form_id = "form-183"
This data as json, CSV (advanced)
| id ▼ | form_id | form_name | file | kind | corpus_id | written | run | result_count | result_of | result_date | matches_excerpt | note | code |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 69 | Half of the in-force acts have no structured text in the official dump form-183 | Half of the in-force acts have no structured text in the official dump | probes/2026-09/form-183-dump-coverage.py | python | corpus-2026-09-eurlex | 2026-09-09 | python3 probes/2026-09/form-183-dump-coverage.py <dir holding the three dump .zip> | 27792 | 58686 | 2026-09-09 | 1 | Re-run on the 2026-09-06 archives (stream read, nothing extracted): 58,686 works; 30,047 with EN Formex; 29,892 with FR; 27,792 without any (47 %); 22,293 .tif in the EN archive. Same figures. | #!/usr/bin/env python3 """Fault Atlas probe — form-183: half of the in-force acts have no structured text in the official dump. Input: the three archives of the Publications Office data dump for legal acts in force (sector 3), as served on 2026-09-06 behind EU Login (https://datadump.publications.europa.eu/, "LEG" collection): LEG_MTD_20260906_01_00.zip metadata, one tree_non_inferred.rdf per work LEG_EN_FMX_20260906_01_00.zip Formex XML sections, English LEG_FR_FMX_20260906_01_00.zip Formex XML sections, French Nothing is extracted: the archives are read as streams, their content is not rewritten. Adapted from the collector's inventory script (Alexandrie, Loxyn, 2026-09-09). Run: python3 probes/2026-09/form-183-dump-coverage.py <dir holding the three .zip> Result on the 2026-09-06 dump: 58,686 works in the metadata, 30,047 with at least one EN Formex file, 29,892 with FR, 27,792 (47 %) with no Formex in either language. On the EN side the archive carries 22,293 .tif files: old acts scanned, never converted. The dump is complete for metadata and half complete for text. """ import sys, os, re, zipfile, collections def main(dump_dir): zips = sorted(x for x in os.listdir(dump_dir) if x.endswith(".zip")) mtd = [z for z in zips if "_MTD_" in z] fmx = {z.split("_")[1].lower(): z for z in zips if "_FMX_" in z} if not mtd or not fmx: sys.exit("need one *_MTD_*.zip and at least one *_FMX_*.zip in " + dump_dir) works_with_text = {} tif = collections.Counter() for lang, name in fmx.items(): z = zipfile.ZipFile(os.path.join(dump_dir, name)) per_work = collections.Counter() for n in z.namelist(): if "/" not in n: continue per_work[n.split("/")[0]] += 1 if n.lower().endswith(".tif"): tif[lang] += 1 works_with_text[lang] = set(per_work) z.close() z = zipfile.ZipFile(os.path.join(dump_dir, mtd[0])) works = {n.split("/")[0] for n in z.namelist() if n.endsw… |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE probes(id integer primary key, form_id text references forms(id), form_name text, file text, kind text, corpus_id text, written text, run text, result_count int, result_of int, result_date text, matches_excerpt int, note text, code text);