Understanding JSON Formats: JSON, JSONL, NDJSON, and More

JSON (JavaScript Object Notation) is ubiquitous in modern applications, but did you know there are several variations of JSON formats designed for different use cases? Let's explore the most common JSON formats and when to use each one.

1. Standard JSON

The classic JSON format we all know and love - a single, complete JSON document.

{
  "users": [
    {
      "id": 1,
      "name": "Alice Johnson",
      "email": "alice@example.com",
      "active": true
    },
    {
      "id": 2,
      "name": "Bob Smith", 
      "email": "bob@example.com",
      "active": false
    }
  ],
  "metadata": {
    "total": 2,
    "page": 1
  }
}

Best for:

• API responses
• Configuration files
• Small to medium datasets
• When you need the entire document structure

2. JSON Lines (JSONL)

Also known as Newline Delimited JSON (NDJSON), each line contains a separate JSON object.

{"id": 1, "name": "Alice Johnson", "email": "alice@example.com", "active": true}
{"id": 2, "name": "Bob Smith", "email": "bob@example.com", "active": false}
{"id": 3, "name": "Carol Davis", "email": "carol@example.com", "active": true}

Interactive DiagramView Fullscreen
Click diagram or fullscreen button for better viewing • Press ESC to exit fullscreen

Best for:

• Large datasets
• Streaming data
• Log files
• Machine learning training data
• When you need to process records individually

Advantages:

• Memory efficient - process one line at a time
• Streaming friendly
• Easy to append new records
• Each line is independently valid JSON

3. JSON Streaming

For real-time data streams where JSON objects are sent continuously.

{"event": "user_login", "user_id": 123, "timestamp": "2025-09-01T10:00:00Z"}
{"event": "page_view", "user_id": 123, "page": "/dashboard", "timestamp": "2025-09-01T10:00:15Z"}
{"event": "user_logout", "user_id": 123, "timestamp": "2025-09-01T10:05:30Z"}

Interactive DiagramView Fullscreen
Click diagram or fullscreen button for better viewing • Press ESC to exit fullscreen

Best for:

• Real-time applications
• WebSocket connections
• Server-Sent Events (SSE)
• Live data feeds

4. Concatenated JSON

Multiple JSON objects concatenated without separators (less common but used in some APIs).

{"id": 1, "name": "Alice"}{"id": 2, "name": "Bob"}{"id": 3, "name": "Carol"}

Best for:

• Specific API designs
• Compact data transmission
• Legacy system compatibility

5. JSON Patch Format

Used for describing changes to JSON documents (RFC 6902).

[
  { "op": "replace", "path": "/name", "value": "Alice Johnson-Smith" },
  { "op": "add", "path": "/phone", "value": "+1-555-0123" },
  { "op": "remove", "path": "/temp_field" }
]

Interactive DiagramView Fullscreen
Click diagram or fullscreen button for better viewing • Press ESC to exit fullscreen

Best for:

• API updates (PATCH requests)
• Version control for JSON documents
• Minimal data transmission for changes

Practical Examples and Use Cases

Example 1: Processing Large User Data

Standard JSON (Memory Intensive):

// Loads entire file into memory
const data = JSON.parse(fs.readFileSync('users.json', 'utf8'));
data.users.forEach(user => processUser(user));

JSONL (Memory Efficient):

// Process line by line
const readline = require('readline');
const fileStream = fs.createReadStream('users.jsonl');
const rl = readline.createInterface({ input: fileStream });

rl.on('line', (line) => {
  const user = JSON.parse(line);
  processUser(user);
});

Example 2: API Design Patterns

REST API with Standard JSON:

// GET /api/users?page=1&limit=10
{
  "data": [...],
  "pagination": {
    "page": 1,
    "total_pages": 50,
    "total_records": 500
  }
}

Streaming API with JSONL:

// GET /api/users/stream
// Response: text/plain with application/x-ndjson
{"id": 1, "name": "Alice", "created_at": "2025-01-01T00:00:00Z"}
{"id": 2, "name": "Bob", "created_at": "2025-01-01T00:01:00Z"}

Performance Comparison

Interactive DiagramView Fullscreen
Click diagram or fullscreen button for better viewing • Press ESC to exit fullscreen

Best Practices

When to Use Each Format

1. Standard JSON

   • Small to medium datasets (< 100MB)
   • Configuration files
   • API responses with nested structures
   • When you need the complete document structure

2. JSONL/NDJSON

   • Large datasets (> 100MB)
   • Log processing
   • Machine learning datasets
   • When memory efficiency is crucial
   • Data that naturally represents a sequence of records

3. JSON Streaming

   • Real-time applications
   • Live data feeds
   • Chat applications
   • Monitoring dashboards

Implementation Tips

Reading JSONL in Python:

import json

with open('data.jsonl', 'r') as file:
    for line in file:
        record = json.loads(line.strip())
        process_record(record)

Writing JSONL in Node.js:

const fs = require('fs');

const writeStream = fs.createWriteStream('output.jsonl');
users.forEach(user => {
    writeStream.write(JSON.stringify(user) + '\n');
});
writeStream.end();

Streaming JSON in React:

useEffect(() => {
  const eventSource = new EventSource('/api/stream');
  eventSource.onmessage = (event) => {
    const data = JSON.parse(event.data);
    setLiveData(prev => [...prev, data]);
  };
  
  return () => eventSource.close();
}, []);

Cloud Platform Support

Different cloud platforms have varying support for JSON formats:

Interactive DiagramView Fullscreen
Click diagram or fullscreen button for better viewing • Press ESC to exit fullscreen

Security Considerations

• Always validate JSON input before processing
• Set size limits for JSON payloads to prevent DoS attacks
• Use streaming for large datasets to avoid memory exhaustion
• Sanitize data when converting between formats

Conclusion

Understanding different JSON formats helps you choose the right tool for the job:

• Use standard JSON for most API responses and configurations
• Choose JSONL for large datasets and efficient processing
• Implement JSON streaming for real-time applications
• Consider JSON Patch for efficient updates

The key is matching the format to your specific use case, considering factors like data size, memory constraints, and processing requirements.

Want to learn more about data formats and API design? Check out our other tutorials on REST API best practices and cloud data processing patterns.