Build Your Own Stuff: Part 2

In part 1, I discussed the reasoning behind why I've decided to build my own stuff. In this part we'll discuss some of the challenges I've faced in building a static website generator and some of the changes that I'll be making to its design to better fit my needs.

Modern web development is challenging. I built a static site generator to get started, but there are many little things I need learn about. To stay true to learning in public, I’m going to write about the struggles I had getting my websites developed enough to push them to staging. I’ll also discuss the static site generator that I built and why I’m rebuilding it using a design philosophy.

I’ve worked on three of the four websites over the course of several months. I wrote the copy for the Skriptble Publishing and Skriptble Press websites on a plane to a conference. The rest of the copy has been slowly refined, most through small tweaks to the voice of the content. The design though? That was a bit more challenging. It required me to (re)learn CSS and modern web layout. In my initial research I stumbled upon EveryLayout. It has provided some of the most straightforward advice that I've come across. The simple tools, provided as both plain CSS and via custom elements, are really refreshing. I choose the ones that I wanted to use and with relatively little effort the vision I had in my mind was in the browser. I wanted something super basic. There are some writers on the internet who go with the most barebones design possible, like Dan Luu's website, but I wanted something with a bit more CSS.

Once I had written the small amount of CSS I needed and tested it in a couple browsers, I was done. But it turns out that there were a few bugs, the most noticeable one was on the publishing website, where the hero text on mobile awkwardly slid off the side of the screen. It initially confused me, because I'd written the CSS to resize dynamically. After some investigation, I realized that the font size was too large for that screen size. The fix was therefore simple: I needed to decrease the font size on smaller screens. The first solution that came to mind was to use media queries. It would be straightforward: detect the screen size, then adjust the font size accordingly. But I had already put in effort to avoid using media queries, mostly for the reasons articulated throughout EveryLayout, for example in this section on viewport units. I'm not saying I won't ever use media queries, but I'd rather not start hardcoding magic widths if I can. So how did I resolve this problem? With the very handy CSS min function. As the name implies, it selects the minimum between a set of values. I used this to choose either the font size that works well for desktop screen sizes, or a smaller dynamic font size based on viewport width that looks nice on mobile devices. Problem resolved and bug fixed!

An issue that I ran with the static site generator was how to handle links between the websites. Usually you can just make an absolute link and call it a day, but I use different domains for development, staging, and production. I didn’t want to add an HTML postprocessing step search around replacing the href attribute of anchor tags. The generator uses Markdown for copy, and I had some previous experience extending Goldmark with custom functionality, so I wrote an extension for Goldmark that handles processing all of links with a custom scheme and rewrites them to the appropriate host for a given environment. This actually worked surprisingly well. In a few hours I had a working prototype. I had to do some cleanup since I had some links within the YAML metadata portion of the Markdown files that wasn't initially handed by the extension. I do like how extensible Goldmark is, and if that was all I had to deal with, I think I would actually keep developing the static site generator using this as its base. However, there are several reasons why I do not want to do this.

The first major reason is that it’s annoying to write HTML within Markdown and have it actually be processed properly. Templating for the generator is done via Templ, which is a fantastic project and I’ve really enjoyed using it, but every time I needed some custom HTML I would either need to adjust a template or create a new one, then write some Go code to have the generator process those templates, and then write some Markdown for the copy. The entire endeavor is just a level of annoying that I do not want to deal with. Part of the reason why I wanted to build this system was to have the flexibility to have each article I write be as custom as I want it to be. This includes customizing the markup for each webpage to my liking. Having to write several Markdown files, a Templ template, and some Go code to glue the entire thing together was unappealing. What I wanted to do was write the webpages in HTML. This is the driving reason for redesigning the generator. I want to do my writing directly within HTML.

For most, this would be a nonstarter. Why would you want to write directly in HTML instead of writing in a much nicer format like Markdown? How are you going to handle the boilerplate that’s on each page? Well, I got the idea from how the HTML specification itself is developed. It too is written in HTML, well it’s written in a special form of HTML that runs through a preprocessor that creates the final HTML that is delivered to users. I don't mind having to run through a preprocsesor. I do mind running through three preprocessors, in three different languages. By using HTML directly and then preprocessing it into regular HTML through a custom preprocessor, I should be able to get the best of both worlds.¹ Something like my link rewriting system will be easy to handle by parsing the HTML and then doing a replacement of the links before rendering the DOM tree back to text. In fact, I’ll be able to do all sorts of things, like inserting boilerplate and automatically adding headers and footers. I’m planning to include the Markdown YAML metadata inside the HTML directly. I’m going to put it in the head element, likely using meta elements with some custom elements where I find the meta element lacking. That’s another reason why I decided I needed to do something differently: the YAML within Markdown used for metadata.

When I first started building this system, having YAML metadata didn’t seem that bad. I could put pretty much whatever I wanted into it and it worked well. But one day I hit a snag where I wasn’t getting proper output and I couldn’t figure out why. This is actually one of my dislikes with Goldmark, in that there isn’t any native error handling in some of the extension functionality. You wind up dumping errors into the Markdown output itself, which just gets very messy.² Anyway, the problem was that I had an array of strings I was using to configure content that appeared in different sections of a landing page that I was making. It took me a bit to first figure out that there was a YAML error and then even longer to figure out what that error was. It turned out that the YAML parser disliked that I had used tabs instead of spaces in the YAML part of the Markdown file. But it hadn’t actually given me that as an output, and instead it just complained that a line started with an invalid character. I really disliked this experience. I could have configured my editor to use spaces instead of tabs when it’s within a YAML block within a Markdown file, but that just felt really annoying. It didn't help that I had already been struggling with figuring out how to represent different data structures within YAML. Not because of YAML itself, which I could definitely represent them in, but because of the way that I would have to extract them from the metadata that is provided from Goldmark meta extension. You are given a map[string]interface{} and you need to extract things out of that bit by bit. This is extremely annoying for any deeply nested thing, or even a moderately nested thing. Instead of using an object I encoded information in strings with delimiters because it was easier to write something to parse out the values from a string than it was to write a bunch of type assertions on the plethora of empty interfaces that were shoved within the metadata map. There are probably better solutions here, like I could have just written my own YAML extension for markdown to handle this in a more custom way. But ultimately, I just didn’t like the direction any of this development was going and the overall generator design was starting to feel messy.

Those extra processing stages I mentioned earlier? Those were particularly annoying. I would often find myself forgetting to regenerate the templates or rebuild the generator application. I just want a far simpler system for all of this. I want my tools to have as minimal friction as possible. Learning how to write prose directly in HTML is going to be a challenge, I’ll have to change the default way that my editor handles formatting, because I absolutely refuse to write prose indented just because of HTML nesting. But this provides learning opportunity. Through writing prose in HTML and through using HTML as the base format for all of my websites, I’ll actually learn HTML. I’ll have to. That feels like a net positive to me. It seems like the sort of thing that someone in my position should be doing.

One of the frustrations that I’ve had with writing on the internet has been that people continue to write as if we’re publishing papers and books. The medium is often treated as if it’s going to be printed out, instead of the dynamic medium that it is. I would guess that part of this is because over time, those who write and publish on the web have become more and more disconnected from HTML, but I would also say it’s because our tooling and build systems have made it very difficult to actually directly touch the HTML on a regular basis. We should be able to get to that low level often, and our tooling should support provide that capability. The advancements that we’ve had with HTML and CSS, especially the ones related to typography, open up a huge number of opportunities for us to create compelling reading experiences for our audience. But they can only be realized if we aren’t trapped behind WYSIWYGs and alternative markup formats. I think that Markdown is a fantastic format, but if the goal is to publish specifically for the web, then HTML needs to be embraced. The argument is slightly different if you also want to target your writing at print or books, but print stylesheets have been around for a very long time, so it’s not as if browsers are incapable of providing styling and formatting for the print medium as well. While this is mainly my own decision, I hope that building this tooling eventually inspires others to do the same.