Evidence: Measured Results & Benchmarks
Measured project results, with sources, dates and scope. Unmeasured capabilities are listed separately.
How to read these numbers
Measured
Empirical scores tested against transcribed ground truth, benchmark corpuses, or live system routing tables.
Collected
Total automated tests gathered by a test runner (e.g. pytest --collect-only). This reflects test suite breadth, not a percentage pass rate.
Not measured yet
Explicit boundaries and unbenchmarked aspects of shipped systems. Documented plainly to prevent false assumptions.
What wasmeasured
Project measurements, where they came from and when. The table also records where another tool did better.
| Project | Measurement | Compared with | Value | Source |
|---|---|---|---|---|
| B.L.A.S.T. | Character error rate, 14 hand-checked pages from one 1959 book | Tesseract 4.1.1 scored 0.177 and won 9 of 14 | 0.108 | data/benchmarks/2026-10-20-ocr/result_C.json (OCR repo 43590fd)Mean over 14 hand-transcribed pages from one 1959 book, CPU only. Tesseract 4.1.1 scored 0.177 on the same pages and won 9 of 14.20 Oct 2026 |
| The first version, same pages | EasyOCR, before the gutter split | 0.499 | OCR repo docs/adr/0002 and 0003 (phase 0, EasyOCR, same 14 pages)10 Aug 2026 | |
| Tests in the suite | Collected, not a pass rate | 914 | pytest --collect-only in the OCR repo at 43590fdCollected tests, not a CI pass rate.20 Oct 2026 | |
| Character error rate, Darwin scan exhibit | RapidOCR alone 0.725; Tesseract 0.002 | 0.011 | data/benchmarks/2026-10-20-scan-demo (OCR repo 43590fd)Clean typeset spread from Darwin (1859); illustrative exhibit.20 Oct 2026 | |
| Marketplace Monitor | Tests in the suite | Collected, not a pass rate | 304 | pytest --collect-only in freelancermap-monitorCollected tests, not a CI pass rate.20 Oct 2026 |
| CI matrix, Python versions × operating systems | 3.10 to 3.13 on Ubuntu and Windows | 4x2 | freelancermap-monitor .github/workflows/ci.yml17 Aug 2026 | |
| UET GPT | Model providers in the routing table | Not an uptime figure | 3 | uet-gpt convex/providers/registry.ts (gpt-oss-20b, Gemini 2.5 Flash and embeddings, Cohere rerank)Providers in the production routing table, not uptime.15 Sept 2026 |
| Menu Data Pipeline | Restaurant sites pulled concurrently | One validated sheet out | 17 | github.com/Ibrahim-Salman19/foodmenu_scraper README21 Aug 2026 |
| Quantum Singularity | GPU particles, fastest device tier | About 80,000 on mid-range machines | 110,000 | quantum-singularity index.html:1606 (28k / 48k / 82k / 110k by device tier)The count is chosen from the device at load; mid-range machines run about 80,000.24 Aug 2026 |
OCR benchmark summary
The B.L.A.S.T. OCR pipeline was tested against out-of-the-box Tesseract 4.1.1 and raw RapidOCR across 14 hand-checked pages from one 1959 book on CPU hardware.
Tesseract won 9 of 14 pages and ran at 3.7 seconds per page, compared with 16.5 for B.L.A.S.T. B.L.A.S.T.'s lower mean CER (0.108 vs 0.177) is driven by two near-empty pages. Without p096 and p097, mean CER is 0.074 for Tesseract and 0.077 for B.L.A.S.T. Tesseract also has the lower mean WER: 0.279 vs 0.449. These results cover one book.
What is not measured yet
These are capabilities supported by the codebase or infrastructure, but not yet verified against an objective, published benchmark.
Table accuracy
Table extraction and normalization are supported in the B.L.A.S.T. pipeline (exporting to Markdown, DOCX, and JSON), but table recognition accuracy (such as a TEDS score) has not yet been benchmarked against a ground-truth table dataset.
RAG answer accuracy
UET GPT operates in production with multi-provider routing and explicit refusal logic, but no formal answer accuracy, retrieval recall percentage, or faithfulness score has been published yet.
GPU OCR timing
All published OCR execution timings (3.7s for Tesseract, 16.5s for B.L.A.S.T.) were measured on a 4-core laptop CPU (i5-10210U). High-throughput GPU inference latency has not been benchmarked.
Uptime and availability
While UET GPT uses multi-provider fallback and Marketplace Monitor runs fail-closed retry loops, no continuous 99.9% uptime SLA or availability percentage is claimed.
Correction policy
Each measurement listed here includes its source and scope. When reporting a rerun, include the new source data, measurement date and comparison.
Need to verify a specific metric or review a project brief? See engagement details on the hire page or send a project inquiry.