Zod,

I am a big fan of TypeScript, but one thing that I still find lacking compared to a real strongly type language: the runtime validation of the data.

While a static type checker esnures everything works, eventually we have to interact with the scary outside world.

This can include user input, API responses, streamed data, reading data from disk or even loading environment variables.

Lying interfaces

One thing that is more harmful than no interface, is a lying interface. This will set you —and your team— up for a world of unnecessary debug sessions.

Let's say you have a neat generic function that fetches data from an API:

async function fetchData<T>(url: string): Promise<T> {
    const response = await fetch(url);
    return response as T;
}

You have properly declared an interface for the data you expect:

type ApiData = {
  foo: string;
  bar: number;
}

And when you call the function you expect to get the data you want:

const response = await fetchData<ApiData>("example.api.org"); // ApiData

At first glance the interface looks fine, the function has explicit type inputs and outputs.

But theres a catch — or rather 2 lies:

1. The function can return anything, not just ApiData.
2. The function can throw an error, but you do not know that from the interface. You are left in the dark about what could go wrong.

You can easily spot those lies in source code whenever you see an as T which should be read as as LIE.

Removing the lies

We can enhance the interface by adding a runtime validation library like Zod¹. This requires some initial extra typing, but it pays off in terms of new type-safe superpowers inside your IDE.

import { SafeParseReturnType } from 'zod';
 
type Response<T extends ZodSchema> =
  | SafeParseReturnType<T, z.output<T>>
  | { success: false; error: unknown };
 
async function fetchData<T extends ZodSchema>(url: string, schema: T): Promise<Response<T>> {
  try {
    const response = await fetch(url);
    return schema.safeParse(response);
  } catch (error) {
    return { success: false, error };
  }
}

Instead of defining a type directly, we define a schema with Zod and infer the type from it.

import { z } from "zod";
 
const apiData = z.object({
  foo: z.string(),
  bar: z.number(),
});
 
// Direct replacement for the previous type
type ApiData = z.infer<typeof apiData>;

Now, when we call the function, we are sure that the data we get is according to the schema. And if not, we get proper typed errors.

export async function getData() {
  const response = await fetchData("example.api.org", apiData);
 
  if (!response.success) {
    console.log(response.data); // undefined
    if (response.error instanceof ZodError) {
      // The data we got from the API is not according to the schema
      console.log(response.error); // ZodError
      return null;
    }
 
    // The API request failed
    console.log(response.error); // unknown
    return null;
  }
 
  console.log(response.error); // undefined
  return response.data; // ApiData
}

Zod in the real world

For my client, Growy, we are dealing with unstructured data from various sources.

Growy is revolutionizing vertical farming by heavily relying on (hundreds of) IoT devices sending and receiving data to/from the cloud.

Our software is handling data from two primary sources: DynamoDB and MQTT messages.

In the software, there was efford put into data-classes that allow for creating data objects for static type-safety.

export class GrowthCycle {
  id: string;
  type: string;
  amount: number;
 
  constructor(id: string, type: string, amount: number) {
    this.id = id;
    this.type = type;
    this.amount = amount;
  }
 
  static fromDynamoRecord(data: Record<string, string | number | undefined | null>) {
    return new GrowthCycle(
      data.id as string,
      data.type as string,
      data.amount as number
    );
  }
 
  // ... other methods
}

For DynamoDB queries, there were functions like:

function getRecord(id: string) {
  const response = await dynamoClient.send(<QUERY>);
  const Item = response.Item; // Record<string, any>
 
  return GrowthCycle.fromDynamoRecord(Item); // GrowthCycle
}

And for MQTT messages, there were handlers that parsed data² as follows:

mqqClient.onMessage(message => {
  const data = JSON.parse(message.payload.toString()); // any
 
  const example = new GrowthCycle(data.id, data.type, data.amount); // GrowthCycle
 
  // ... do something with the data
})

But, you might have noticed some as LIEs in the code.

When parsing a record from DynamoDB, we are assuming that the data is always there and of the correct type. Similarly, when parsing an MQTT message, we are assuming that the payload is always a stringified JSON object containing the correct data.

While all interfaces and static types are correct, runtime data could still be (and sometimes was) wrong.

The problem

As we tested our platform with simulated (IoT) devices, everything seemed to work smoothly. But when the simulated devices were being replaced by their actual real-world counterparts, that is when things got interesting.

When we started to get actual data from the real world, bugs started to surface. The data we were expecting and worked with in our simulated environment was not always the correct data. The interfaces from “the hardware team” and “the software team” were not always in sync.

Some bugs were found immediately, as the services started breaking down when the wrong data was received:

TypeError: Cannot read property 'foo' of undefined or NaN values after calculations were not uncommon.

But some bugs were more subtle. Wrongly received data was stored in the DynamoDB table and only processed in later stages, making it harder to identify the root cause.

Adding runtime validation

To prevent services from crashing and data from being stored incorrectly, we introduced runtime validation to our data objects.

When I joined the project, which was already quite large, it had adopted a heavy Object-Oriented Programming (OOP) style. The data objects were used exetensively throughout the codebase. As a result, we decided to not go full-in on the zod way.

Instead, we opted for a hybrid approach, adding zod parsing capabilities to the existing classes. This involved some TypeScript magic, but allowed us to leverage the benefits of runtime validation without disrupting the overall codebase structure.

The following Parsable class serves as the foundation for our runtime validation framework:

import { SafeParseReturnType, ZodSchema, ZodTypeDef } from 'zod'
 
export interface GeneratedParsable<Output = unknown, TypeDef extends ZodTypeDef = ZodTypeDef, Input = Output> {
  schema: ZodSchema<Output, TypeDef, Input>
  // passing `unknown` instead of `any` will beak TS
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  parse<TFinal extends new (data: any) => InstanceType<TFinal>>(this: TFinal, value: unknown): InstanceType<TFinal>
  // passing `unknown` instead of `any` will beak TS
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  safeParse(data: unknown): SafeParseReturnType<Input, Output>
  new (input: Input): Output
}
 
export function Parsable<Output = unknown, TypeDef extends ZodTypeDef = ZodTypeDef, Input = Output>(schema: ZodSchema<Output, TypeDef, Input>) {
  const Generated = class {
    static schema = schema
    static parse<T extends typeof Generated>(this: T, data: unknown) {
      return new this(data as Input)
    }
    static safeParse(data: unknown) {
      const result = schema.safeParse(data)
 
      if (!result.success) {
        console.warn('Failed to parse data', { data, error: result.error })
      }
 
      return result
    }
 
    constructor(input: Input) {
      const result = Generated.safeParse(input)
 
      if (!result.success) {
        throw result.error
      }
 
      Object.assign(this, result.data)
    }
  }
 
  return Generated as unknown as GeneratedParsable<Output, TypeDef, Input>
}

Okay, busted. In our code we also use some as LIEs, but let me break it down.

Our constructor requires an Input type and does not allow the unknown data we pass to the parse method.

A double LIE! To make your IDE happy we need to return any form of our GeneratedParsable interface.

As the shape of the schema can be (almost) anything, typescript will infer everything from the genericsOutput, TypeDef and Input and use them in theGeneratedParsable interface.

Luckely, all of the above happens behind the scenes and we can extend the Parsable in the data objects to give it runtime validation capabilities.

import { z } from 'zod'
 
const schema = z.object({
  id: z.string(),
  type: z.string(),
  amount: z.number()
})
 
export class GrowthCycle extends Parsable(schema) {
  // ... other methods
}

With this setup, we can now use the parse and safeParsemethods to validate any unknown data.

const validInput = { id: '123', type: "foo", amount: 42 }
const invalidInput = { id: '123', type: false, amount: 42 } // type should be a string
 
GrowthCycle.parse(validInput) // GrowthCycle
GrowthCycle.parse(invalidInput) // Throws a `ZodError`
// or
GrowthCycle.safeParse(validInput) // Returns a `GrowthCycle`
GrowthCycle.safeParse(input) // Gracefully returns a `ZodError`

On top of this, even new instances of any Parsable class will be validated using the schema and giving type-hints in the IDE. Adding some extra static type checking magic to the mix.

const invalidInput = { id: '123', type: false, amount: 42 }
 
const example = new GrowthCycle(invalidInput) 
// Argument of type '{ id: string; type: boolean; amount: number; }' is not assignable to parameter of type '{ type: string; id: string; amount: number; }'.
//  Types of property 'type' are incompatible.
//    Type 'boolean' is not assignable to type 'string'.

From this moment onwards we can be sure the data we are working with is in the expected format, no more need for as LIEs.

When receiving faulty data, we will be better equipped to pinpoint and handle it gracefully.

The aftermath

After implementing runtime validation, services started reporting a lot more faulty data inputs. Initially, the team was not very pleased with the constant need for fixing unnecessary errors. Instead of adding new features the focus was on data integrity for nearly a full month.

However, as time passed, received data became increasingly in line with the expected format. This led to better discussions between teams about data formats and interfaces, as faulty data would no longer be accepted.

As a result, the platform is now more stable, and the reliability of the data has improved significantly.

Edit

Before leaving the project, I was surprised with a gift from the team. They had named a robot after me, forever engraving zod (and myself) in the Growy system.

Room for improvement

While the hybrid approach worked well for this project, it is not perfect. For instance, it does not work with more complex types like like unions and intersections.

When starting a new project, I would personally be going full-in on the zod way. This would remove the need for the Parsable entirely.

import { z } from "zod";
 
export const GrowthCycle = z.object({
  id: z.string(),
  type: z.string(),
  amount: z.number()
});
 
export type GrowthCycle = z.infer<typeof GrowthCycle>;

You could then use the GrowthCycle type when only type information is needed.

import type { GrowthCycle } from "./GrowthCycle";
 
function handler(growthCycle: GrowthCycle) { /* ... */ }

Or pass the GrowthCycle zod object when data validation is required.

import { GrowthCycle } from "./GrowthCycle";
 
export class Handler {
  handle(data: unkown) {
    const { success, data, error} = GrowthCycle.safeParse(data);
 
    // ... handle the result
  }
}