Return JSON from OpenAI to build AI enhanced APIs - WunderGraph

Jens Neuse
CEO & Co-Founder at WunderGraph
July 17, 2023·5min read
Last updated on September 9, 2025

Archive Notice
This article is archived and no longer maintained. It describes an earlier version of WunderGraph and its Agent SDK, which is no longer part of the current product architecture. The examples and implementation details may not work as described. For current documentation and guidance, see WunderGraph Cosmo.

The Problem: How to return structured data (JSON) from OpenAI?

Let's say we want to build an API that returns the weather of a given country. We don't want to manually write the integration code but rather use OpenAI to generate the response. However, LLMs like GPT-3 are simply returning plain text, not structured data (JSON). So how can we force OpenAI to return an answer that conforms to a JSON Schema so that we can expose it as an API, documented using OpenAPI?

The Solution: With zod / JSON Schema and OpenAI Functions, you can return structured data (JSON) from OpenAI

OpenAI has a new feature called "Functions". Functions are a way to define Operations that can be called from within an LLM. Functions can be described using JSON Schema.

What happens though is that the LLM will not call the function directly, but rather generate inputs for the function and return them to you. So you can create a prompt and add the available functions as context. You then call the OpenAI API and the response "might" contain instructions to call a function with certain inputs.

This is a bit hard to understand, so let's look at an example.

const completions = await openai.createChatCompletion({
  model: this.model,
  messages: [
    {
      role: 'user',
      // this is the actual prompt from the user
      content: 'What is the weather like in Berlin?',
    },
    {
      role: 'agent',
      // this is the assumed response from the agent (LLM) in text form
      content: 'The weather in Berlin is sunny and 25 degrees',
    },
    {
      role: 'user',
      // this is our "injected" prompt to trigger the agent to "send" the result to the out function
      content: 'Set the result to the out function',
    },
  ],
  functions: [
    {
      name: 'out',
      // here we define our function to get the result from the agent in JSON form
      description: 'This is the function that returns the result of the agent',
      // we use zod and a zod-to-json-schema converter to define the JSON Schema very easily
      parameters: zodToJsonSchema(
        z.object({
          temperature: z.number(),
          city: z.string(),
          description: z.string(),
        })
      ),
    },
  ],
})
const structuredResponse = JSON.parse(
  completions.data.choices[0].message!.function_call!.arguments!
)
const validatedOut = this.outputZodSchema.parse(structuredResponse)
console.dir(validatedOut) // { temperature: 25, city: 'Berlin', description: 'sunny' }

So, what happens here? We ask OpenAI to create a chat completion for a previously answered prompt. We want the LLM to use a function to "send" the result to. The parameters of the function are defined using JSON Schema (zod).

As a result of this prompt, we get a response from OpenAI that it wants to call the function with a JSON encoded string as input (completions.data.choices[0].message!.function_call!.arguments!).

This string can be parsed using JSON.parse and then validated using zod. After that, we can be sure that the response is valid on following the schema we've defined.

What's left is that we put all of the pieces together, add code generation on top of it and we have a fully automated way to build APIs on top of OpenAI.

Final solution to expose an AI-enhanced API via OpenAPI

The WunderGraph Agent SDK does all of this for you out of the box. Define an Operation using TypeScript, add an agent to execute your prompt, and you're done. The framework will infer the JSON Schema from the TypeScript types and generates the OpenAPI documentation for you.

We can now use this Operation using any OpenAPI client, like Postman, or even just curl.

The response will be a JSON object that conforms to the schema we've defined.

You OpenAPI documentation will be generated in the following directory:

Conclusion

OpenAI is a powerful tool that can be used to build APIs on top of it. With the new Functions feature, we can even return structured data (JSON) from OpenAI. This allows us to build APIs on top of OpenAI that can be consumed by other machines. We've also demonstrated how to use the WunderGraph Agent SDK to write up the agents and generate OpenAPI documentation automatically.

You can check out the source code on GitHub and leave a star if you like it. Follow me on Twitter, or join the discussion on our Discord server.

Jens Neuse is the CEO and one of the co-founders of WunderGraph, where he builds scalable API infrastructure with a focus on federation and AI-native workflows. Formerly an engineer at Tyk Technologies, he created graphql-go-tools, now widely used in the open source community. Jens designed the original WunderGraph SDK and led its evolution into Cosmo, an open-source federation platform adopted by global enterprises. He writes about systems design, organizational structure, and how Conway's Law shapes API architecture.