(* Just kidding it will be amazing and engaging.)

Not how to load data, but instead we’ll take a step back, and look at where to load data. Not in any particular framework, either, this is going to be more broadly about data loading in different web application architectures, and paricularly how that impacts performance.

We’ll start with client-rendered sites and talk about some of the negative performance characteristics they may have. Then we’ll move on to server-rendered apps, and then to the lesser-known out-of-order streaming model. To wrap up, we’ll talk about a surprisingly old, rarely talked about way to effectively load slow data in a server-rendered application. Let’s get started!

Client Rendering

Application metaframeworks like Next and SvelteKit have become incredibly popular. In addition to offering developer conveniences like file system-based routing and scaffolding of API endoints, they also, more importantly, allow you to server render your application.

Why is server rendering so important? Let’s take a look at how the world looks with the opposite: client-rendered web applications, commonly referred to as “single page applications” or SPAs. Let’s start with a simplified diagram of what a typical request for a page looks like in an SPA.

The browser makes a request to your site. Let’s call it yoursite.io. With an SPA, it usually sends down a single, mostly empty HTML page, which has whatever script and style tags needed to run the site. This shell of a page might display your company logo, your static header, your copyright message in the footer, etc. But mostly it exists to load and run JavaScript, which will build the “real” site.

This is why these sites are called “single page” applications. There’s a single web page for the whole app, which runs code on the client to detect URL changes, and request and render whatever new UI is needed.

Back to our diagram. The inital web page was sent back from the web server as HTML. Now what? The browser will parse that HTML and find script tags. These script tags contain our application code, our JavaScript framework, etc. The browser will send requests back to the web server to load these scripts. Once the browser gets them back, it’ll parse, and execute them, and in so doing, begin executing your application code.

At this point whatever client-side router you’re using (i.e. react-router, Tanstack Router, etc) will render your current page.

But there’s no data yet!

So you’re probably displaying loading spinners or skeleton screens or the like. To get the data, your client-side code will now make yet another request to your server to fetch whatever data are needed, so you can display your real, finished page to your user. This could be via a plain old fetch, react-query, or whatever. Those details won’t concern us here.

SSR To The Rescue

There is a pretty clear solution here. The server already has has the URL of the request, so instead of only returning that shell page, it could (should) request the data as well, get the page all ready to go, and send down the complete page.

Somehow.

This is how the web always worked with tools like PHP or asp.net. But when your app is written with a client-side JavaScript framework like React or Svelte, it’s surprisingly tricky. These frameworks all have API’s for stringifying a component tree into HTML on the server, so that markup can be sent down to the browser. But if a component in the middle of that component tree needs data, how do you load it on the server, and then somehow inject it where it’s needed? And then have the client acknowledge that data, and not re-request it. And of course, once you solve these problems and render your component tree, with data, on the server, you still need to re-render this component tree on the client, so your client-side code, like event handlers and such, start working.

This act of re-rendering the app client side is called hydration. Once it’s happened, we say that our app is interactive. Getting these things right is one of the main benefits modern application meta-frameworks like Next and SvelteKit provide.

Let’s take a look at what our request looks like in this server-rendered setup:

That’s great. The user sees the full page much, much sooner. Sure, it’s not interactive yet, but if you’re not shipping down obscene amounts of JavaScript, there’s a really good chance hydration will finish before the user can manage to click on any buttons.

We won’t get into all this, but Google themselves tell you this is much better for SEO as well.

So, what’s the catch? Well, what if our data are slow to load. Maybe our database is busy. Maybe it’s a huge request. Maybe there is a network hiccup. Or maybe you just depend on slow services you can’t control. It’s not rare.

This might be worse than the SPA we started with. Even though we needed multiple round trips to the server to get data, at least we were displaying a shell of a page quickly. Here, the initial request to the server will just hang and wait as long as needed for that data to load on the server, before sending down the full page. To the user, their browser (and your page) could appear unresponsive, and they might just give up and go back.

Out of Order Streaming

What if we could have the best of all worlds. What if we could server render, like we saw. But if some data are slow to load, we ship the rest of the page, with the data that we have, and let the server push down the remaining data, when ready. This is called streaming, or more precisely, out-of-order streaming (streaming, without the out-of-order part, is a separate, much more limited thing which we won’t cover here).

Let’s take a hypothetical example where the data abd, and data xyz are slow to load.

With out-of-order streaming we can load the to-do data load on the server, and send the page with just that data down to the user, immediately. The other two pieces of data have not loaded, yet, so our UI will display some manner of loading indicator. When the next piece of data is ready, the server pushes it down:

What’s the catch?

So does this solve all of our problems? Yes, but… only if the framework you’re using supports it. To stream with Next.js app directory you’ll use Suspense components with RSC. With SvelteKit you just return a promise from your loader. Remix supports this too, with an API that’s in the process of changing, so check their docs. SolidStart will also support this, but as of writing that entire project is still in beta, so check its docs when it comes out.

Some frameworks do not support this, like Astro and Next if you’re using the legacy pages directory.

What if we’re using those projects, and we have some dependencies on data which are slow to load? Are we stuck rendering this data in client code, after hydration?

Prefetching to the rescue

The web platform has a feature called prefetching. This lets us add a <link> tag to the <head> section of our HTML page, with a rel="prefetch" attribute, and an href attribute of the URL we want to prefetch. We can put service endpoint calls here, so long as they use the GET verb. If we need to pre-fetch data from an endpoint that uses POST, you’ll need to proxy it through an endpoint that uses GET. It’s worth noting that you can also prefetch with an HTTP header if that’s more convenient; see this post for more information.

When we do this, our page will start pre-fetching our resources as soon as the browser parses the link tag. Since it’s in the <head>, that means it’ll start pre-fetching at the same time our scripts and stylesheets are requested. So we no longer need to wait until our script tags load, parse, and hydrate our app. Now the data we need will start pre-fetching immediately. When hydration does complete, and our application code requests those same endpoints, the browser will be smart enough to serve that data from the prefetch cache.

Let’s see prefetching in action

To see pre-fetching in action, we’ll use Astro. Astro is a wonderful web framework that doesn’t get nearly enough attention. One of the very few things it can’t do is out-of-order streaming (for now). But let’s see how we can improve life with pre-fetching.

The repo for the code I’ll be showing is here. It’s not deployed anywhere, for fear of this blog posting getting popular, and me getting a big bill from Vercel. But the project has no external dependencies, so you can clone, install, and run locally. You could also deploy this to Vercel yourself if you really want to see it in action.

I whipped up a very basic, very ugly web page that hits some endpoints to pull down a hypothetical list of books, and some metadata about the library, which renders the books once ready. It looks like this:

The endpoints return static data, which is why there’s no external dependencies. I added a manual delay of 700ms to these endpoints (sometimes you have slow services and there’s nothing you can do about it), and I also installed and imported some large JavaScript libraries (d3, framer-motion, and recharts) to make sure hydration would take a moment or two, like with most production applications. And since these endpoints are slow, they’re a poor candidate for server fetching.

So let’s request them client-side, see the performance of the page, and then add pre-fetching to see how that improves things.

The client-side fetching looks like this:

useEffect(() => {
  fetch("/api/books")
    .then((resp) => resp.json())
    .then((books) => {
      setBooks(books);
    });

  fetch("/api/books-count")
    .then((resp) => resp.json())
    .then((booksCountResp) => {
      setCount(booksCountResp.count);
    });
}, []);

Nothing fancy. Nothing particularly resilient here. Not even any error handling. But perfect for our purposes.

Network diagram without pre-fetching

Running this project, deployed to Vercel, my network diagram looks like this:

Notice all of the script and style resources, which need to be requested and processed before our client-side fetches start (on the last two lines).

Adding pre-fetching

I’ve added a second page to this project, called with-prefetch, which is the same as the index page. Except now, let’s see how we can add some <link> tags to request these resources sooner.

First, in the root layout, let’s add this in the head section

<slot name="head"></slot>

this gives us the ability to (but does not require us to) add content to our HTML document’s <head>. This is exactly what we need. Now we can make a PrefetchBooks React component:

import type { FC } from "react";

export const PrefetchBooks: FC<{}> = (props) => {
  return (
    <>
      <link rel="prefetch" href="/api/books" as="fetch" />
      <link rel="prefetch" href="/api/books-count" as="fetch" />
    </>
  );
};

Then render it in our prefetching page, like so

<PrefetchBooks slot="head" />

Note the slot attribute on the React component, which tells Astro (not React) where to put this content.

With that, if we run that page, we’ll see our link tags in the head

Now let’s look at our updated network diagram:

Notice our endpoint calls now start immediately, on lines 3 and 4. Then later, in the last two lines, we see the real fetches being executed, at which point they just latch onto the prefetch calls already in flight.

Let’s put some hard numbers on this. When I ran a webpagetest mobile Lighthouse analysis on the version of this page without the pre-fetch, I got the following.

Note the LCP (Largest Contentful Paint) value. That’s essentially telling us when the page looks finished to a user. Remember, the Lighthouse test simulates your site in the slowest mobile device imagineable, which is why it’s 4.6 seconds.

When I re-run the same test on the pre-fetched version, things improved about a second

Definitely much better, but still not good; but it never will be until you can get your backend fast. But with some intelligent, targetted pre-fetching, you can at least improve things.

Parting thoughts

Hopefully all of your back-end data requirements will be forever fast in your developer journeys. But when they’re not, prefetching resources is a useful tool to keep in your toolbelt.

Quick summary: Streamed HTML is as you imagine it. Rather than serving the entire HTML document at once, servers serve pieces of it. The browser gets these pieces and can start working on them, even rendering them, so the page can appear to load more quickly. It’s similar to how a progressive JPG loads or how video tends to “stream” as it plays on the web. While browsers can handle it, the rather large caveat is that not all other languages and frameworks are built to handle it.

The first I heard of it was in Taylor Hunt’s The weirdly obscure art of Streamed HTML. At a time, he was doing work for Kroger grocery stores. Every website should have a goal of being fast, but in this case there was a really direct connection that could be drawn. Kroger literally sells mobile phones, and the bestselling phone it sold was the Hot Pepper’s Poblano VLE5 ($15 on sale) and it was a reasonable assumption that “slow 3G” was what users of that phone would experience.

With Streamed HTML in place, you can see the difference:

Not all sites have my API bottlenecking issue, but many have its cousins: database queries and reading files. Showing pieces of a page as data sources finish is useful for almost any dynamic site. For example…

• Showing the header before potentially-slow main content
• Showing main content before sidebars, related posts, comments, and other non-critical information
• Streaming paginated or batched queries as they progress instead of big expensive database queries

Taylor goes on to explain other benefits, like that these chunks of HTML that arrive can be interactive immediately, an import aspect of web performance (as typified by FID or “First Input Delay” metrics, or TTI “Time To Interactive”).

Taylor found that React couldn’t help with this, wanted to use Svelte instead, but found it couldn’t support Streaming HTML. Instead, he landed on Marko. (I highly suggest reading Taylor’s article for the investigations of back and front-end technologies that largely put Streaming HTML in the back seat.)

I’d bet you can imagine why streamed HTML and JavaScript frameworks have a hard time getting along. If the framework code and your usage of that framework loads, but the bits of DOM its looking for aren’t there, well, that ain’t gonna work. Like ReactDOM.render() needs what it needs — if it can’t find the DOM element to bind to it’s just going to fail, not sit around waiting for it to potentially appear later.

This all came up in mind again as Chris Haynes blogged Streaming HTML out of order without JavaScript. The video captured quite a few people’s attention, me included:

This particular trick was done not just with streamed HTML but also incorporated Web Components and and the unique ability that <slot /> has. Meaning code that arrives in predictable HTML order might get, ahem, slotted into wherever it appears in a Shadow DOM, which may be an entirely different order.

Chris nicely puts a point on what I was trying to say earlier: you need a combination of server and client technology that supports all this to pull it off. Starting with the server. Chris is more optimistic than Taylor was:

You’re in luck here, there is pretty much universal support for this across all languages. I’ve opted for Hono as it’s a lightweight server, built on web standards, that runs on node as well as a wide variety of edge platforms.

Using Node seems to be smart here (see the code), as Node’s support seems to be the most first-class-citizen-y of all the back-end languages. Deno, a spiritual successor to Node, supports it as well, and I tried running their example and it worked great.

Then you might want something client-side to help, to make things a bit more ergonomic to work with:

In the JavaScript world, there aren’t a lot of standalone templating languages that support streaming, but a recent project called SWTL does.

It’s notable that this “Service Worker Template Language” is used here, as Service Workers are aligned with streaming. Service Workers can help do things like intercept requests to the network when offline to return cached data. That can be great for speed and access, which is what streaming HTML is also trying to help with.

This all kinda has the vibes of an old technology coming roaring back because it was a smart thing to do all along, like server-side rendering broadly. For instance, React 18’s renderToPipeableStream seems to support their server-side efforts more seriously. Solid also supports streaming. It’s not without some downsides, though. Aside from the trickiness of pairing technologies that support it Eric Goldstein notes:

Streaming has a few downsides. Once a response begins and a response code is chosen (e.g. 200), there is no way to change it, so an error occurring during a data fetch will have to let the user know another way.

I wanted to try this myself, but not bother with any of the exotic JavaScript stuff, templating stuff, or out-of-order stuff. I just wanted to see HTML stream at all, from my own servers. Most of my easy-access experimental sites run on PHP, so I tried that. Taylor had noted PHP “requires calling inscrutable output-buffering functions in a finicky order.” Best I could tell, that means to turn off output-buffering so that PHP isn’t holding on to output and instead returning it.

I couldn’t get it to work myself (demo attempt), which is essentially exactly what I was worried about. This stuff, despite being “old” isn’t particularly well documented. And since it seems a little against-the-grain right now, it’s hard to know why it doesn’t work.

<?php
// Set the content type to HTML
header('Content-Type: text/html; charset=UTF-8');

// Turn off output buffering
ob_end_flush();
ob_implicit_flush(true);

// Start the page output
echo '<!DOCTYPE html>';
echo '<html>';
echo '<head><title>Streaming Example</title></head>';
echo '<body>';
echo '<h1>Content is streaming...</h1>';

// Simulate a process that takes time, e.g., database queries or complex calculations
for ($i = 0; $i < 10; $i++) {
  // Simulate some server-side processing time
  sleep(1); // Sleep for 1 second
	
  // Send a (decently large) piece of content to the browser
  echo "<div>New content block $i loaded.</div><details>...</details>";
}

echo '</body>';
echo '</html>';
?>

So what did I do wrong? Did I screw up the PHP? That’s very plausible since I’ve literally never even tried this before and the PHP docs didn’t inspire confidence. Does the PHP host this is on (Flywheel — quite WordPress focused which as far as I know doesn’t do streaming HTML) not allow this somehow at some lower level? Is there some kind of caching in front of the PHP (NGINX reverse cache?) that is preventing this? Is the fact that Cloudflare is in front of it all messing it up? I tried bypassing all cache at this URL which seems to have worked, but that doesn’t mean something else Cloudflare-related is happening.

Anyway! If y’all have played with this or are using it and have thoughts, I’d love to hear about it.