Dive into JSON:API specification

How do you standardize API communication in a complex platform? How do you handle relationships, pagination, and error responses consistently? This article explores our journey with the JSON:API specification, examining both its strengths and challenges.

The Challenge of API Standardization

In the Pulse platform, we needed a consistent approach to API design that would provide a predictable experience for clients while avoiding common pitfalls of custom API implementations. We chose to implement the JSON:API specification as our standard for client-platform communication while using an event bus for most inter-service communication (with some synchronous JSON:API RPC for specific needs).

Our Solution: JSON:API Specification

When designing the Pulse platform, we wanted to adopt a well-defined standard for our APIs to avoid common pitfalls and shortcomings of custom API designs. JSON:API offered a comprehensive specification that addressed many of the challenges we anticipated:

1. Standardized Format: A consistent structure for requests and responses across all endpoints

2. Relationship Handling: Built-in support for complex data relationships

3. Pagination and Filtering: Standardized approaches for handling large datasets

4. Error Handling: Consistent error reporting format

Implementation in Pulse

Our implementation of JSON:API is encapsulated in the verse-jsonapi gem, which provides:

1. Renderer: Transforms Ruby objects into JSON:API compliant responses

2. Deserializer: Parses JSON:API requests into Ruby objects

3. Exposition DSL: Simplifies the creation of basic JSON:API endpoints

The exposition layer in each microservice uses this DSL to define endpoints:

json_api Model::TalentRecord do
  index do
    allowed_filters :search
  end
  show
  create
  update
  delete
end

This generates standardized CRUD endpoints that follow the JSON:API specification, handling relationships, pagination, filtering, and error responses automatically.

Advantages of JSON:API in Pulse

1. Standardization of the API

JSON:API provides a consistent structure across all our microservices. This standardization offers several benefits:

• Predictable Responses: Clients know exactly what structure to expect

• Consistent Patterns: Similar operations work the same way across different resources

• Documentation: The specification serves as built-in documentation

2. Handling of Relationships

One of the strongest features of JSON:API is its approach to relationships:

• Compound Documents: Related resources can be included in a single request

• Resource Linkage: Relationships are clearly defined with type and ID

• Inclusion Control: Clients can specify which relationships to include

• Circular Reference Handling: The specification addresses circular relationship issues

This is particularly valuable in domains with highly interconnected data models where relationships between entities are complex and multi-layered.

3. Structured Pagination and Filtering

JSON:API defines standard approaches for:

• Pagination: Using page[number] and page[size] parameters

• Filtering: Using filter[attribute] syntax

• Sorting: Using the sort parameter with comma-separated fields

• Sparse Fieldsets: Allowing clients to request only needed fields

This standardization simplifies client implementation and server-side query optimization.

4. Error & Validation Handling

JSON:API provides a structured error format that includes:

• Status Codes: HTTP status codes for each error

• Error Types: Categorization of errors

• Detailed Messages: Human-readable error descriptions

• Source Pointers: References to the specific part of the request that caused the error

This comprehensive error handling improves debugging and client-side error management.

Disadvantages of JSON:API in Pulse

1. Protocol Complexity

While JSON:API appears straightforward at first glance, the specification is actually quite complex:

• Learning Curve: Developers need time to understand all the nuances

• Edge Cases: The specification covers many edge cases that add complexity

• Verbose Syntax: The format can be verbose compared to simpler alternatives

• Specification Size: The full specification is extensive and takes time to master

2. Implementation Complexity

Serializing and deserializing model objects to and from JSON:API format introduces complexity:

• Relationship Mapping: Correctly mapping complex relationships is challenging

• Circular References: Handling circular references requires careful implementation

• Custom Extensions: Extending the specification for custom needs can be difficult

Our verse-jsonapi gem helps abstract much of this complexity, but it still requires careful implementation.

3. Overhead

JSON:API responses tend to be more verbose than custom-tailored JSON:

• Response Size: Responses include more metadata and structural elements

• Processing Overhead: Parsing and generating the format requires more processing

• Network Traffic: Larger payloads mean more network traffic

• Client-Side Processing: Clients need more complex parsing logic. Strangely, we haven't found any simple Typescript package that can handle JSON:API responses out of the box.
  There exist some packages and libraries, but they are very opinionated and don't work well with our use case.

4. Flat API Structure

Compared to more action-oriented CRUD approaches, JSON:API's resource-centric approach can be less intuitive:

• Resource-Centric: Everything is modeled as a resource, which doesn't always map cleanly to actions

• Action Limitations: Custom actions don't fit neatly into the specification

• Batch Operations: The specification doesn't offer guidance for batch operations

• Complex Queries: Some complex queries might be difficult to express in the JSON:API filtering syntax, although we tackled this problem by offering custom filtering capabilities in our data layer.

Example: Invoice and Invoice Lines

Consider a system with invoices and invoice lines. In a traditional RESTful API, you might have:

GET /invoices/123            # Get invoice details
GET /invoices/123/lines      # Get lines for a specific invoice
POST /invoices/123/lines     # Add a line to an invoice

This hierarchical structure makes the relationship between invoices and lines explicit and intuitive.

In JSON:API's flat structure, you would instead have:

GET /invoices/123                          # Get invoice details
GET /invoice_lines?filter[invoice_id]=123  # Get lines for a specific invoice
# or
GET /invoices/123?included[]=lines         # Include relationship into the output
POST /invoice_lines                        # Add a line (must include invoice relationship)

While this works, it's less intuitive and requires more complex filtering to express the same hierarchical relationship. The flat structure makes it less obvious how resources relate to each other, especially for developers new to the API.

Conclusion: Mixed Feelings

Overall, we have mixed feelings about our adoption of JSON:API in the Pulse platform. While it has provided valuable standardization and solved many common API design problems, it has also introduced complexity and overhead that sometimes feels unnecessary.

The specification does its job well, providing a consistent approach across our API endpoints and handling complex relationships effectively. However, the learning curve and implementation complexity have sometimes slowed development and created challenges.

If we were starting again, we might have considered a more simple RESTful approach with less complexity.

Despite these considerations, JSON:API has served as a solid foundation for our API architecture, providing consistency and solving many common API design challenges. The standardization benefits have generally outweighed the complexity costs, particularly as our team has become more familiar with the specification over time.

Are you using JSON:API in your projects? We'd love to hear about your experiences and how you've addressed some of the challenges we've encountered. Share your thoughts or reach out to our team with questions about our implementation.

---

This article is part of our series on API design and microservice architecture. Stay tuned for more insights into how we've built a scalable, maintainable system.