Loading structured JSON
In this section, we assume the JSON data is inNDJSON (Newline delimited JSON) format, known as JSONEachRow in ClickHouse, and well structured i.e. the column names and types are fixed. NDJSON is the preferred format for loading JSON due to its brevity and efficient use of space, but others are supported for both input and output.
Consider the following JSON sample, representing a row from the Python PyPI dataset:
Prefer static schemas where possibleIn cases where your columns have fixed names and types, and new columns aren’t expected, always prefer a statically defined schema in production.The JSON type is preferred for highly dynamic data, where the names and types of columns are subject to change. This type is also useful in prototyping and data exploration.
Ordering keysWe have selected an ordering key here via the
ORDER BY clause. For further details on ordering keys and how to choose them, see here.*.json.gz files in the bucket. ClickHouse automatically infers the format is JSONEachRow (ndjson) from the file extension and contents. A format can be manually specified through parameter functions in case ClickHouse is unable to detect it.
Compressed filesThe above files are also compressed. This is automatically detected and handled by ClickHouse.
INSERT INTO SELECT:
FORMAT clause e.g.
JSONEachRow format. Other common JSON formats are supported, with examples of loading these provided here.
Loading semi-structured JSON
Our previous example loaded JSON which was static with well known key names and types. This is often not the case - keys can be added or their types can change. This is common in use cases such as Observability data. ClickHouse handles this through a dedicatedJSON type.
Consider the following example from an extended version of the above Python PyPI dataset dataset. Here we have added an arbitrary tags column with random key value pairs.
tags column of type JSON:
When to use the JSON type
Use the JSON type when your data:- Has unpredictable keys that can change over time.
- Contains values with varying types (e.g., a path might sometimes contain a string, sometimes a number).
- Requires schema flexibility where strict typing isn’t viable.
- A flat structure with known keys: use standard column types e.g. String.
- Predictable nesting: use Tuple, Array, or Nested types for these structures.
- Predictable structure with varying types: consider Dynamic or Variant types instead.