Why JSON Schema Validation is Essential for LLM Outputs

The Mental Model: Syntax vs. Schema

To solve the problem of unreliable LLM outputs, one must distinguish between syntax and schema. Syntax refers to the rules of the JSON format itself: every opening brace must have a closing brace, and keys must be enclosed in double quotes. A parser like Python's json.loads() only checks these rules.

A schema, however, defines the semantic meaning and structure of the data. While syntax ensures the file is readable, the schema ensures the content is usable. An LLM can easily produce a JSON object that is syntactically perfect but logically useless because it fails to provide the specific information your application requires.

Example: null, an absent field and a valid string

Consider an application that expects a user profile. The system requires an 'email' field as a string. An LLM might generate the following JSON:

Input JSON: {"name": "John Doe", "email": null}

In this case, the JSON is syntactically perfect. The parser will successfully convert this into a Python dictionary. However, if your code attempts to call.split('@') on the email field, the application will crash with an AttributeError because it received a NoneType instead of a string.

By applying a JSON Schema that defines 'email' as a required string, the validation step would catch this error before the data ever reaches your business logic.

# Install in your environment: python -m pip install jsonschema
import json
from jsonschema import Draft202012Validator

schema = {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "email": {"type": "string"}
    },
    "required": ["name", "email"],
    "additionalProperties": False
}
Draft202012Validator.check_schema(schema)
validator = Draft202012Validator(schema)
samples = [
    '{"name": "Ada", "email": null}',
    '{"name": "Ada"}',
    '{"name": "Ada", "email": "ada@example.com"}'
]
for text in samples:
    data = json.loads(text)
    print("accepted" if validator.is_valid(data) else "rejected")
# Expected illustrative output:
# rejected
# rejected
# accepted

Expected Result

For the three sample inputs, the illustrative output is rejected, rejected, accepted. The first email exists but JSON null becomes Python None, which fails the string constraint. The second object has no email, so required rejects it. The third satisfies this schema. A value such as "not-an-email" would also pass: this example checks a string type, not an email address. Adding format alone does not activate format checking in the Python jsonschema library; configure a format checker when needed.

Diagnosis: Why LLMs Fail Validation

LLMs fail validation for several reasons. First, they may hallucinate field names, using 'user_email' instead of the expected 'email'. Second, they may struggle with complex types, such as returning a string '10' when a number 10 is required. Third, they may omit fields entirely if the prompt is ambiguous. Finally, they may include non-standard values like NaN or Infinity, which, while sometimes accepted by specific parsers, are not compliant with the strict JSON specification (RFC 7159).

Common Mistakes

A common mistake is assuming that because a parser did not throw an error, the data is safe to use. Another mistake is failing to account for the 'null' value; in JSON, a key can exist but have a value of null, which is fundamentally different from the key being absent. Developers also often forget to limit the number of properties, allowing the LLM to return massive, bloated objects that consume excessive memory and CPU during processing.

Decision Criteria for Validation

Start with the actual contract: identify required fields and explicitly declare their types. If unexpected fields are an error, use additionalProperties: false; properties alone does not prohibit them. Add bounds or patterns only when they express a real requirement. Independently limit response size before parsing. Python json.loads accepts NaN and Infinity by default; use parse_constant to raise an error if strict JSON is required. None of these structural checks replaces authorization or domain validation.

Scope Limits

JSON Schema validation is a structural check, not a logical one. It can ensure that a 'price' field is a number, but it cannot ensure that the price is correct or that it matches the price in your database. Furthermore, schema validation does not protect against resource exhaustion attacks (DoS) if the LLM produces a multi-gigabyte JSON string; you must implement size limits on the input string before parsing begins.

Summary of Requirements

To implement this effectively, ensure you have: 1. A defined JSON Schema for every expected LLM response. 2. A validation library (like jsonschema for Python). 3. A strategy for handling validation errors (e.g., retrying the LLM prompt or returning an error to the user). 4. Input size limits to prevent memory exhaustion during the initial parsing phase.