Skip to content

Chapter 13: Building a Weather & Stock Assistant

Theoretical Foundations

The paradigm of building user interfaces with Large Language Models (LLMs) represents a fundamental shift from static, pre-determined UI generation to dynamic, on-the-fly rendering. In previous chapters, we explored the mechanics of React Server Components (RSC) and their ability to decouple the component tree from the client-side bundle. We established that RSCs allow us to render complex, data-heavy components exclusively on the server, sending only the necessary UI payload to the client.

In this chapter, we extend that concept into the realm of generative UI. We are no longer simply fetching data and rendering a static chart; we are instructing an LLM to act as a UI designer and data fetcher, orchestrating a stream of UI chunks that assemble themselves in the browser. To understand this, we must dissect the three pillars of this architecture: Streaming UI, Tool Invocation via JSON Schema, and the Stateful Checkpointer.

The Streaming UI Paradigm: From Packets to Pixels

Imagine a traditional web application as a postal service. When a user requests a page, the server packages the entire response (HTML, CSS, JS) into a single, large box and ships it. The user sees nothing until the entire box arrives and is unpacked. Even with partial hydration, the structure is largely static.

Generative UI with streaming transforms this into a live video feed. Instead of sending a finished box, the server sends a continuous stream of frames. Each frame represents a discrete unit of UI—a paragraph, a chart component, a button. The client renders these frames as they arrive, creating a perceived instantaneous experience even while the backend is still processing complex data.

In the context of the Vercel AI SDK, this is handled by the useChat hook. The hook manages the WebSocket or HTTP stream connection. Under the hood, this utilizes the Web Streams API. The server (running Next.js App Router) generates a ReadableStream of text. However, in our specific application, we are not just streaming text; we are streaming React Server Components.

When an LLM decides to render a weather chart, it doesn't send a description of the chart. It sends the serialized RSC payload. The client receives this stream, parses the JSON-like structure, and progressively renders the React tree. This is akin to a printer that prints line-by-line rather than waiting for the whole page to be composed.

Tool Invocation and JSON Schema Output

To make this streaming useful, the LLM needs to interact with the real world—fetching weather data or stock prices. We cannot rely on the LLM's internal knowledge, which is static and prone to hallucination. We need to give the LLM "hands" in the form of tools.

When we ask an LLM to perform a task, we often need its response to conform to a strict structure so our code can parse it reliably. This is where JSON Schema Output comes in.

A JSON Schema is a blueprint for a JSON object. It defines exactly what keys are allowed, what data types they hold, and the required fields. When we instruct an LLM to adhere to a JSON Schema, we are essentially putting it in a straitjacket—it can think freely, but its output must fit the mold we defined.

Analogy: The Restaurant Order Form Think of an LLM as a waiter taking an order. If you ask, "What do you recommend?", the waiter might give a free-form paragraph describing dishes. This is unstructured text. If you hand the waiter a specific order form with checkboxes for "Appetizer," "Main Course," and "Dessert," you force the waiter to structure their response into predictable fields. In our app, the JSON Schema is that order form. We define a schema that requires a tool field (e.g., "getWeather") and an input field (e.g., "city: New York"). The LLM, acting as the waiter, fills out this form. Our code then reads the form and executes the action.

This is critical for reliability. Without JSON Schema, we would have to parse natural language to extract parameters, which is error-prone. With the schema, we can use a validation library like Zod to parse the LLM's output instantly. If the LLM deviates from the schema, we catch the error immediately.

The Checkpointer: State as a First-Class Citizen

In a simple chat, the state is just an array of messages. But in a complex agent that performs multi-step reasoning (e.g., "First, get the weather. Second, analyze the stock trend. Third, generate a chart"), the state becomes a graph of execution.

This is where the Checkpointer becomes vital. In the context of LangGraph (or similar agentic frameworks), the Checkpointer is the persistence layer. It saves the complete graph state after every node execution.

Analogy: The Video Game Save System Imagine playing a complex RPG. You defeat a boss (Node 1), then solve a puzzle (Node 2), and finally enter a new zone (Node 3). If the game crashes after Node 2, you don't want to restart from the beginning. You reload your "save file." The Checkpointer is that save file. It captures the exact state of the world (variables, memory, previous decisions) at specific intervals.

In our Weather & Stock Assistant, the Checkpointer allows us to: 1. Resume: If a user closes the browser during a long data fetch, we can reopen the chat and the agent resumes exactly where it left off. 2. Debug: We can inspect the state at any point in the graph to see why the agent made a specific decision. 3. Branch: We can theoretically fork a state to try a different path without affecting the original execution.

The Checkpointer is usually backed by a fast key-value store like Redis or a SQL database. It maps a thread_id (the conversation) and a checkpoint_id (the specific step) to the serialized state of the graph.

Visualizing the Data Flow

To visualize how these concepts interact in the Weather & Stock Assistant, consider the flow of data from user input to rendered UI.

A diagram shows how a unique thread_id and checkpoint_id map to the serialized graph state, tracing the data flow from user input to the final rendered UI.
Hold "Ctrl" to enable pan & zoom

A diagram shows how a unique `thread_id` and `checkpoint_id` map to the serialized graph state, tracing the data flow from user input to the final rendered UI.

The Architecture of the Agent

In this chapter, we are building an agent that routes requests. The agent is not a single monolithic function; it is a graph of nodes.

  1. The Router Node: This is the first step. The LLM looks at the user's message and decides which tool to invoke. It outputs a JSON object (constrained by schema) indicating the intent.
  2. The Execution Node: Based on the router's output, the system executes the specific tool (e.g., fetchWeather). This is an asynchronous operation.
  3. The Synthesis Node: Once the data is fetched, the LLM is prompted again, this time given the raw data. Its task is to generate the UI. We instruct the LLM to return an RSC. This is where the generative magic happens—the LLM writes code (in the form of a serialized React component) that visualizes the data.

Why RSCs for the Output? We could simply return JSON data to the client and let the client render it. However, by returning an RSC, we offload the rendering logic to the server. The LLM can generate complex UI structures (like a grid of cards or a specific chart library component) without the client needing to know the underlying logic. The client simply receives the "pixels" (the component tree) and displays them.

Under the Hood: The Streaming Protocol

When the useChat hook receives the stream, it processes it chunk by chunk. The stream contains special control characters and delimiters to distinguish between: 1. Text tokens: For displaying conversational text. 2. Tool invocation markers: Signals that a tool is about to be called. 3. RSC payloads: Serialized React components (usually in a custom binary format or base64-encoded JSON).

The client-side useChat hook maintains a local state array of messages. As it consumes the stream, it appends these chunks to the message object. When a complete RSC payload is received, the client uses a hydration function (provided by the AI SDK) to turn that payload into actual React elements that can be rendered in the DOM.

Theoretical Foundations

In this chapter, we are not merely fetching data. We are orchestrating a conversation between the user, the LLM, and external APIs, mediated by a strict structural contract (JSON Schema) and persisted via a state management system (Checkpointer). The result is a UI that is not built, but generated—streaming live from the server to the client, component by component. This architecture leverages the strengths of the modern stack: the safety of TypeScript, the component model of React, and the intelligence of LLMs, all glued together by the streaming capabilities of Next.js and Vercel.

Basic Code Example

This example demonstrates a minimal SaaS-style web application that uses the Vercel AI SDK to generate a structured JSON response from an LLM. We will build a simple "User Profile Generator" where the AI takes a free-text description and outputs a structured user profile adhering to a strict JSON schema. This illustrates the core concept of JSON Schema Output combined with the useChat hook.

The application consists of two parts: 1. Client Component (Frontend): A React component using the useChat hook to send user input and stream the AI's response. 2. API Route (Backend): A Next.js Server Action (or API route) that invokes the LLM, enforces the JSON schema, and streams the structured data back to the client.

The Core Concept: Enforcing Structure

When building reliable AI features, raw text is often insufficient. We need the AI to return data we can programmatically parse and use (e.g., to populate a database or render a chart). By using JSON Schema Output, we instruct the LLM to strictly adhere to a predefined structure. The Vercel AI SDK handles the complexity of parsing this stream and validating the structure, often using libraries like Zod under the hood.

JSON Schema Output provides a predefined structure that the LLM must follow, and the Vercel AI SDK uses libraries like Zod to parse the stream and validate that the structure is strictly adhered to.
Hold "Ctrl" to enable pan & zoom

JSON Schema Output provides a predefined structure that the LLM must follow, and the Vercel AI SDK uses libraries like Zod to parse the stream and validate that the structure is strictly adhered to.

Code Implementation

We will use a Next.js App Router structure. The code is split into the Server Action (API) and the Client Component.

1. The Server Action (app/actions.ts)

This file contains the server-side logic. It defines the schema and invokes the model.

// app/actions.ts
'use server';

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

/**
 * SCHEMA DEFINITION
 * We use Zod to define the JSON schema. This serves two purposes:
 * 1. It acts as the runtime validation schema.
 * 2. The Vercel AI SDK can extract the JSON Schema definition from this Zod object
 *    to pass to the LLM as a constraint.
 */
const userProfileSchema = z.object({
  name: z.string().describe("The user's first name"),
  age: z.number().min(0).max(120).describe("The user's age in years"),
  interests: z.array(z.string()).describe("List of hobbies or interests"),
});

/**
 * SERVER ACTION
 * This function runs on the server. It accepts a prompt from the client
 * and returns the structured JSON result.
 * 
 * @param prompt - The user's free-text description.
 * @returns The parsed JSON object adhering to the schema.
 */
export async function generateUserProfile(prompt: string) {
  try {
    // The 'generateText' function is used here. 
    // In a streaming UI context (like useChat), we usually use 'streamText',
    // but for this specific "Basic Code Example" focusing on JSON Schema validation,
    // 'generateText' is often simpler to demonstrate the structured output first.
    // However, to match the "Streaming UI" context of the chapter, we will simulate
    // a stream or use 'streamText' if strictly required. 
    // Let's stick to 'generateText' for the pure JSON logic, 
    // as streaming JSON parsing is complex to show in a single snippet.

    const { text, finishReason } = await generateText({
      model: openai('gpt-4o-mini'),
      prompt: `Generate a user profile based on this description: ${prompt}`,

      // CRITICAL: This is where we enforce the JSON Schema.
      // The AI SDK converts the Zod schema into a JSON Schema definition
      // and passes it to the LLM's 'response_format' parameter.
      experimental_providerMetadata: {
        openai: {
          response_format: {
            type: 'json_schema',
            json_schema: {
              name: 'user_profile_schema',
              strict: true,
              schema: userProfileSchema,
            },
          },
        },
      },
    });

    // Parse the result using the Zod schema to ensure validity
    const parsedData = userProfileSchema.parse(JSON.parse(text));

    return parsedData;

  } catch (error) {
    console.error("Error generating profile:", error);
    throw new Error("Failed to generate user profile.");
  }
}

2. The Client Component (app/page.tsx)

This is the frontend interface using useChat to interact with the server action.

// app/page.tsx
'use client';

import { useChat } from 'ai/react';

export default function UserProfileGenerator() {
  // useChat hook handles the message state, input, and API communication.
  // We point it to our custom API endpoint (which we will define in step 3).
  const { messages, input, handleInputChange, handleSubmit, isLoading, error } = useChat({
    api: '/api/generate-profile', // The route handling the AI logic
  });

  return (
    <div style={{ padding: '2rem', fontFamily: 'sans-serif' }}>
      <h1>AI User Profile Generator</h1>
      <p>Enter a description (e.g., "25 year old developer from NYC who loves hiking")</p>

      {/* Message Display Area */}
      <div style={{ marginBottom: '1rem', border: '1px solid #ccc', padding: '1rem', minHeight: '100px' }}>
        {messages.map((msg, index) => (
          <div key={index} style={{ marginBottom: '0.5rem' }}>
            <strong>{msg.role === 'user' ? 'You: ' : 'AI: '}</strong>
            {/* 
              If the AI returns structured JSON, we might want to pretty-print it.
              Note: In a real app, you would parse this JSON and render specific UI elements.
            */}
            <pre style={{ whiteSpace: 'pre-wrap' }}>
              {msg.content}
            </pre>
          </div>
        ))}
        {isLoading && <div style={{ color: 'gray' }}>Generating...</div>}
        {error && <div style={{ color: 'red' }}>Error: {error.message}</div>}
      </div>

      {/* Input Form */}
      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={input}
          onChange={handleInputChange}
          placeholder="Describe the user..."
          style={{ width: '300px', padding: '0.5rem', marginRight: '0.5rem' }}
          disabled={isLoading}
        />
        <button type="submit" disabled={isLoading}>
          {isLoading ? 'Generating...' : 'Generate Profile'}
        </button>
      </form>
    </div>
  );
}

3. The API Route (app/api/generate-profile/route.ts)

While useChat can call Server Actions directly in the latest Next.js versions, using a standard API Route is the most robust way to handle streaming responses. This route acts as the bridge between the client useChat hook and the server-side AI generation.

// app/api/generate-profile/route.ts
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';

// Must export for Next.js App Router
export const runtime = 'edge';

export async function POST(req: Request) {
  const { messages } = await req.json();

  // Get the latest user message
  const lastMessage = messages[messages.length - 1].content;

  // Define the schema locally or import it
  const userProfileSchema = z.object({
    name: z.string(),
    age: z.number(),
    interests: z.array(z.string()),
  });

  // Use streamText for streaming UI updates
  const result = await streamText({
    model: openai('gpt-4o-mini'),
    prompt: `Generate a user profile based on this description: ${lastMessage}`,

    // JSON Schema configuration for streaming
    experimental_providerMetadata: {
      openai: {
        response_format: {
          type: 'json_schema',
          json_schema: {
            name: 'user_profile_schema',
            strict: true,
            schema: userProfileSchema,
          },
        },
      },
    },
  });

  // Convert the stream to a Response object
  return result.toAIStreamResponse();
}

Detailed Line-by-Line Explanation

1. The Schema Definition (app/actions.ts & app/api/...)

const userProfileSchema = z.object({
  name: z.string().describe("The user's first name"),
  age: z.number().min(0).max(120).describe("The user's age in years"),
  interests: z.array(z.string()).describe("List of hobbies or interests"),
});
  • z.object({...}): We use the Zod library to define a strict object shape. This is the foundation of our JSON Schema.
  • .describe(...): This is crucial. LLMs work better when they understand the intent of a field. Providing descriptions helps the model map the user's vague input ("25 year old dev") to the specific field age.
  • z.number().min(0).max(120): We add runtime constraints. If the LLM hallucinates an age of -5 or 200, Zod will throw an error when we parse the result.

2. The Server Logic (Invoking the LLM)

const { text, finishReason } = await generateText({
  model: openai('gpt-4o-mini'),
  prompt: `...`,
  experimental_providerMetadata: { ... },
});
  • generateText vs streamText: In the server action example, we used generateText for simplicity. It waits for the full response before returning. In the API route, we used streamText to enable the "streaming UI" experience where tokens appear as they are generated.
  • experimental_providerMetadata: This is the Vercel AI SDK's way of passing provider-specific configuration.
    • response_format: We are telling OpenAI (via the SDK) that we expect JSON.
    • json_schema: We pass the schema definition directly to the model. The model is instructed not to chat, but to output a JSON object matching this structure.

3. The Client Hook (useChat)

const { messages, input, handleInputChange, handleSubmit } = useChat({
  api: '/api/generate-profile',
});
  • useChat: This hook abstracts the WebSocket or HTTP fetch logic.
  • api prop: By default, useChat looks for /api/chat. We override this to point to our custom endpoint /api/generate-profile.
  • Streaming Behavior: When the API route streams back JSON tokens (e.g., { "name": "J... ohn", ... }), useChat appends these chunks to the messages array. The UI re-renders continuously, showing the JSON building up in real-time.

Common Pitfalls

When implementing JSON Schema Output with the Vercel AI SDK, developers often encounter these specific issues:

1. Hallucinated JSON Structure * The Issue: Even with a schema defined, the LLM might return a valid JSON object that doesn't match the keys defined in your schema (e.g., returning user_age instead of age). * The Fix: Use the strict: true flag in the OpenAI JSON schema definition (as shown in the code). This forces the model to adhere strictly to the schema. Additionally, always use Zod to parse the result on the server before sending it to the client. * Code Safety: 

// Always wrap in a try/catch
try {
  const parsed = userProfileSchema.parse(JSON.parse(text));
  return parsed;
} catch (e) {
  // Handle parsing errors (LLM hallucination)
  return { error: "The AI produced invalid data." };
}

2. Vercel Edge Timeouts * The Issue: Vercel's Edge functions (used in the API route example) have a default timeout of 10 seconds. Generating complex JSON structures, especially if the LLM is slow, can hit this limit. * The Fix: * For simple data, ensure your prompts are concise. * If the generation takes longer, switch the runtime from 'edge' to 'nodejs' in your route configuration (export const runtime = 'nodejs';). Node.js functions on Vercel have a longer timeout (up to 10s on Hobby, 15s on Pro). * Alternatively, use Server Actions with revalidatePath for background processing if immediate streaming isn't strictly required.

3. Async/Await Loops in Server Components * The Issue: When using React Server Components (RSC), developers might try to await a stream inside a component render function. This blocks the rendering of the entire page until the AI finishes generating, defeating the purpose of streaming. * The Fix: RSCs can stream, but you must use the specific RSC syntax (returning Promises or using await only at the top level of the component). For interactive streaming, useChat (Client Component) is the standard approach. Do not mix await inside loops of interactive client components.

4. Zod Parsing Mismatch * The Issue: The LLM returns a string "25" for a number field. Zod expects a number type 25. JSON.parse converts numbers correctly, but if the LLM wraps numbers in quotes (a common hallucination), z.number() will fail. * The Fix: Be explicit in your schema descriptions. Add instructions in the prompt: "Ensure all numeric fields are raw numbers, not strings." Also, rely on Zod's coercion if necessary, though it's better to correct the prompt.

The chapter continues with advanced code, exercises and solutions with analysis, you can find them on the ebook on Leanpub.com or Amazon

Loading knowledge check...

Code License: All code examples are released under the MIT License. Github repo.

Content Copyright: Copyright © 2026 Edgar Milvus | Privacy & Cookie Policy. All rights reserved.

All textual explanations, original diagrams, and illustrations are the intellectual property of the author. To support the maintenance of this site via AdSense, please read this content exclusively online. Copying, redistribution, or reproduction is strictly prohibited.