A File Format For Static Websites
Everybody’s doing it wrong. Use this file format.
Ever hear of a “static website?” They’re the Next Big Thing in website hosting — and a return to yesteryear. Unlike today’s normal hosting companies, static-site hosting companies don’t run any of your code. Economies of scale make your website crazy-fast and crazy-cheap.
Static websites can‘t write to a database on the server. No sign-in forms; no online shopping; no photo upload. With a bit of creative programming, a static site can handle just about anything else.
And that’s wrong.
What Is A Static Page?
A static page is a web server’s response to a web browser’s request.
For instance, here’s the dialogue between a browser and a server for the world’s first web page:
GET /hypertext/WWW/TheProject.html HTTP/1.1
Host: info.cern.chHTTP/1.1 200 OK
Date: Tue, 04 Apr 2017 22:25:34 GMT
Last-Modified: Thu, 03 Dec 1992 08:37:20 GMT
<TITLE>The World Wide Web project</TITLE>
... (obsolete HTML continues...)
That’s a static page: an HTTP response. It has three components:
- path: where the file is stored on the server.
- headers: instructions the web browser needs to decode the content.
- body: the data.
Here’s what today’s static website generators would produce:
<TITLE>The World Wide Web project</TITLE>
... (more really-old HTML stuff)
That’s the static page body. The site generator will store it at a certain path — but often the wrong path. And there’s no sign of headers.
Today’s site generators do this:
- Produce static pages
- Write the static pages to files
- Upload the files to a website
They should skip step 2, because it’s problematic.
The Price Of Files
Using files means you lose your Content-Type. Something needs to calculate
Content-Type: text/html before serving the HTML to your web browser. Content-Type isn’t always easy to guess; for instance,
Content-Type: text/csv; charset=utf-8 is tricky. So who guesses the Content-Type? Is it your static site generator (while it uploads)? Is it your hosting service (as it receives files)? Is it custom code you write? How do you debug it?
Files can’t store ETag and Cache-Control. These headers let you configure some responses to be fast and cheap (like images) and other responses to be easy to overwrite at a moment’s notice (like breaking news articles).
Files aren’t aware of Content-Encoding. Some hosting companies (S3 springs to mind) won’t compress static pages for you, making them load more slowly and at a higher cost than you’d like. You can solve that problem by compressing the body and setting Content-Encoding … but your filesystem doesn’t understand this.
And with files, you can’t name a file after a directory. For instance, https://github.com/huffpostdata is a web page, and https://github.com/huffpostdata/in-memory-website is a web page. But if your static site generator generates a file called
output/huffpostdata, it can’t generate another file called
output/huffpostdata/in-memory-website because that would only work if
output/huffpostdata were a directory … but it’s a file.
Maybe your site generator works around this by storing
output/huffpostdata/index, where “index” is some special keyword. But then you can’t create a file called “index”. In general, files don’t map to web-server endpoints.
Plus, different filesystems allow different filenames. Linux filesystems can store a file named
foo:bar, but 2017’s Windows and Mac filesystems can’t.
Finally, reading and writing files is slow.
In sum: files don’t describe a static website. So static website generators shouldn’t write files.
The Solution: In-Memory Website
A website is a bunch of HTTP responses. Each response has a
headers and a
The idea is:
- All static website generators can output static websites. At HuffPostData I coded a couple of site generators: hpd-asset-pipeline and hpd-page-generator.
- You can store a static website in a file and load it later.
- You can upload a static website to a hosting service like S3.
- You can modify a static website — for instance, add ETags or gzip-compress.
- You can develop a static website using a development server that re-runs your framework every time a file changes. (My NodeJS development server even makes the browser refresh, with LiveReload.) The development server can mimic S3 almost exactly.
- You can stream a static website, if it’s too large to fit in memory. And if you must, you can always stream it into filesystem files.
- You can mix and match programming languages: pipe a StaticWebsite from your Ruby site generator to your Node S3 uploader.
I look forward to the day static website generators live in a happy community. Our first step: ditch the filesystem.