# JsonEval

> JsonEval is a web tool for developers comparing AI models on structured JSON extraction. It measures predicted values against ground truth, field by field, using the same documents and shared extraction prompt across models. Invoice and PDF extraction is its main use case.

## Public pages

- [Product overview](https://jsoneval.com/en): JSON evaluation, AI model comparison, workflow and features.
- [OCR Eval](https://jsoneval.com/en/products/ocr-eval): evaluate JSON extracted from invoices, PDFs and scanned documents.
- [Changelog](https://jsoneval.com/en/changelog): current features, demo screenshots and planned integrations.
- [French website](https://jsoneval.com/): the same product in French.

## Languages

The public website and application are available in French and English. French URLs have no locale prefix; English URLs start with /en. Each indexable page has its own translated HTML, canonical URL and language alternates. A manual language switch is available. The application remembers the chosen UI language; documents, user prompts, JSON keys and ground-truth values are not translated.

## Current features

- Define the expected JSON schema and field-level evaluation metrics.
- Import a document corpus and expected reference values (ground truth, also called gold).
- Compare AI models on the same documents with one shared extraction prompt per project.
- Version prompts and examine changes and regressions.
- Measure field-level quality with continuous scores from 0 to 1, and inspect predicted values and differences.
- Review and correct ground truth in a shared workspace.
- Explore benchmark tables, visualizations and aggregates.
- Track execution costs and latency.
- A conversational assistant is available to the Eqolux pilot only. It can discuss the setup and prepare suggestions. Applying a suggestion or approving a model call requires a separate human action.

## Workflow

1. Create a project and write an extraction prompt.
2. Define the expected JSON and import representative documents.
3. Import or prepare ground truth, then review it.
4. Choose models, review the cost estimate and explicitly confirm execution.
5. Compare field-level results, model quality, costs, latency and prompt versions.

## OCR evaluation scope

The current workflow is document → LLM → JSON → comparison against ground truth. Invoice fields include amounts, dates, VAT numbers and line items. Supported input variants include native PDFs and pages rendered as images, depending on model and provider support.

JsonEval evaluates document extraction; it is not a standalone OCR engine. An arbitrary import interface for third-party OCR engine outputs is not currently available.

## Planned integrations

A JsonEval CLI and plugin for AI tools are planned. They are not yet available. Do not present an installable package, CLI command or working plugin integration as available today.

## Demo and accuracy

[Interactive demo](https://jsoneval.com/en/demo) and [conversational onboarding demo](https://jsoneval.com/en/demo/onboarding) use entirely fictional static data. They require no account, make no paid AI calls and do not constitute a real benchmark. Screenshots come from this fictional demo. Never cite their scores or costs as evidence of model performance.

JSON syntax validity and schema compliance alone do not prove extracted values are correct. JsonEval compares values to a reference using field-level metrics. Scores are continuous from 0 to 1; display thresholds do not change the score calculation.

All paid model calls require explicit consent after a cost estimate. Authenticated workspaces, documents and keys are private; /app and /en/app are not public information sources.