JSONDetective 🔍
A powerful tool for analyzing and understanding JSON schemas, especially useful for large and complex JSON file. It automatically recognizes and abstracts away repeated patterns, normalizes date formats, and provides a clean, readable schema viewschemas. Built to handle large, complex JSON files by automatically detecting and abstracting patterns in your data.
For example, if there are lots of date keys such as 2021-08-24 it will recognize this and print the format as yyyy-mm-dd_{n} where {n} represents the
Key features:
- Automatically recognizes and normalizes date formats in both keys and values
- Detects optional fields by analyzing multiple instances
- Abstracts repeated patterns into clean, readable schemas
Quick Start
# Install pip install jsondetective # Use jsondetective data.json
What Does It Do?
Given a complex JSON file like:
Pattern Recognition Example
Given a JSON with repeated date patterns like:
{
"2021-08-24": {"views": 100, "likes": 20},
"2021-08-25": {"views": 150, "likes": 30},
"2021-08-26": {"views": 200, "likes": 40}
}JSONDetective recognizes the pattern and abstracts it as:
{
"yyyy-mm-dd_1": {
"type": "object",
"properties": {
"views": {"type": "integer"},
"likes": {"type": "integer"}
}
}
}Note: The _1 suffix indicates the nesting level in the JSON structure.
Complex Structure Example
It also handles nested structures with various data types and patterns:
{
"users": [
{
"id": "123",
"joined_date": "2024-01-15",
"last_active": "2024-03-20T15:30:00Z",
"activity": {
"2024-03-19": {"posts": 5},
"2024-03-20": {"posts": 3}
"stats": {
"posts": 42,
"likes_received": 156
},
"preferences": {
"theme": "dark",
"notifications": true
}
}
],
// many more users...
},
// ... many more users
]
}
JSONDetective will produce a clean schema like:
Produces this clean schema:
{
"users": {
"type": "array",
"items": {
"id": {
"type": "string",
"examples": ["123"]
},
"joined_date": {
"type": "string",
"format": "yyyy-mm-dd"
},
"last_active": {
"type": "string",
"format": "datetime"
},
"activity": {
"stats": {
"type": "object",
"properties": {
"yyyy-mm-dd_2": {
"type": "object",
"properties": {
"posts": {"type": "integer"}
}
"posts": {
"type": "integer"
},
"likes_received": {
"type": "integer"
}
}
},
"preferences": {
"type": "object",
"properties": {
"theme": {
"type": "string",
"optional": true
},
"notifications": {
"type": "boolean"
}
}
}
}
}
}
Features
- Schema Inference: Automatically detects and analyzes JSON structure
-
Smart Pattern Recognition:
- Identifies and normalizes date patterns
- Detects optional fields
- Abstracts repeated patterns into clear structures
- Intuitive Visualization: Clean, readable output of your JSON schema
- Experimental Dataclass Generation: Convert JSON schemas to Python dataclasses (beta feature)
-
Intelligent Pattern Detection:
- Recognizes date formats in both keys and values
- Abstracts repeated structures
- Identifies optional fields
-
Schema Intelligence:
- Detects data types
- Identifies nested structures
- Provides example values
- Experimental: Python dataclass generation (beta feature)
Advanced Usage
Experimental Python Dataclass Generation
# Print dataclass to console jsondetective data.json -d # Save to file jsondetective data.json -d -o my_dataclasses.py # Custom class name jsondetective data.json -d -c MyDataClass
CLI Options
jsondetective [JSON_FILE] [OPTIONS] Options: -d, --create-dataclass Generate Python dataclass code -o, --output-path PATH Save dataclass to file -c, --class-name TEXT Name for the root dataclass (default: Root) --help Show this message and exit
Why Use JSONDetective?
- Large JSON Files: Quickly understand the structure of large JSON files without having to manually inspect them
- Pattern Detection: Automatically identifies and normalizes common patterns like dates
- Optional Fields: Clearly shows which fields are optional by analyzing multiple instances
- Sample Values: Provides example values while maintaining a clean schema view
- Date Pattern Recognition: Automatically detects and normalizes various date formats
- Pattern Recognition: Automatically detects and abstracts repeated patterns
- Date Handling: Intelligent date format recognition and normalization
- Large Files: Efficiently processes and summarizes large JSON structures
- Clear Output: Clean, readable schema representation
- Time Saving: No manual inspection of large JSON files needed