Markdown and makefiles
using Make to make websites
I wrote a makefile!
There has always been a make
I’ve always known that Make is a thing.
Rather, I know that Make has always been a thing.
It was written in 1976 which, at the time of this writing, makes it a 43 year old tool, an older tool than me. At the time of this writing.
Make is a relic from the olden times, from the days of yore. A standby of C/C++ hackers. Something that greybeard wizards mutter and chatter and argue about as they craft and maintain their own special makefiles by hand.
But my interaction with
make has always been limited to
merely reciting that age old incantation:
./configure && make && make install without
ever delving too much into what it is and what it does. I knew that it’s
something you use when building code from source, but that was the
extent of my knowledge.
I’ve always appreciated the elegance of the basic formula:
target: prerequisite recipe
Or, to use a slightly different vocabulary:
task: dependency code
Whichever you prefer, really:
target: source command
Very plain, very organized. Simple. I’ve heard people say that a well written Makefile is like documentation for a project, and I can understand a little bit why they would say that. It’s like reading a cookbook: a list of dishes, ingredients, and how to build those dishes from those ingredients:
dish: ingredient recipe
Make is a build tool
Make is a build tool, and a task runner.
You’ve encountered such things in your travels:
gradleif you’re a Java kind of person.
If you’re a java script kind of person, then you’ve written npm scripts in your
package.jsonto run your tests or start your server; and you’ve used grunt or gulp, or webpack.
rake, if you’re a ruby/rails kind of person. (Which was my first attempt at using a Make-like kind of build tool.)
It builds stuff and automates tasks.
make has over other solutions like npm
scripts is that:
It only builds a target if it needs to. (If the sources have been updated.)
No extra dependencies: it’s pre-installed on all *nix platforms, including macos.
It can wrap existing build tools! You can have
makeinstall your node_modules if they’re missing (as long as there is a package.json), run a bash script, start your server, etc.
I wrote a makefile
So I’ve always wanted an excuse to learn
And when starting this blog, I needed a way to create HTML files from markdown source files. I certainly didn’t want to install webpack, or grunt or gulp, or anything like that. Or even npm. Minimalism is the name of our game. I don’t want any other dependencies.
And so, excuse in hand, I got to cracking away at it.
It took me a minute to learn
all enough of Make’s magic
variables, macros, and built-in functions. What really took me
a minute to wrap my head around was how I had to adhere to, and work
around, the basic
task > dependency > recipe formula.
There’s no flexibility here. Declarative or bust.
For example: I was trying to write a recipe that targets a bunch of HTML files. The problem is that I didn’t have those HTML files. Nor did I necessarily know what they are. I didn’t have a list of my targets, and that was a problem.
What I did have is a list of prerequisites: a bunch of Markdown files.
That is, given Make’s strict syntax requirements, you can’t just start with a bunch of ingredients and then simply build something from them. No, you must have the target first. Something to depend on those prerequisites. That’s your entry point.
The trick was to do this:
Get a list of all the prerequisites. Easy enough. E.g.
find src/posts -name "*.md". And then,
Use make’s super weird pattern substitution function to map that list of markdown files to a list of the HTML files I wish I had (my list of targets).
And finally, assign that list to a variable, which I can then use as a list of targets.
Being declarative is hard work some times.
That whole part looks like this:
markdowns := $(shell find src/posts -type f) htmls := $(patsubst src/posts/%.md, posts/%.html, $(markdowns))
Here’s another way to do it using the built-in
macro and a different, built-in substitution short hand:
markdowns=$(wildcard src/posts/*.md) htmls=$(markdowns:src/posts/%.md=posts/%.html)
Those targets (
htmls) are strings that have the format
So now, targets in hand, we get to do this:
Create some new task that has dependencies equal to the list of htmls. We’ll call it “posts.”
Use pattern matching to create a task for any file matching
posts/*.html, which just happens to be the format of all htmls! This target has a dependency of its corresponding markdown file, and its recipe is the pandoc conversion.
That part looks like this:
posts: $(htmls) posts/%.html: src/posts/%.md pandoc --options --more-options -o $@ $<
Now I can call
make posts from the command line, and
every markdown file in
/src/posts will generate a new HTML
/posts! Assuming the following, of course:
There’s not already an existing HTML file for that markdown file, AND
the HTML file is newer than the markdown file.
That is, it will only create the HTML file if it needs to: if it is missing, or if it is out of date.
The makefile in its entirety currently looks like this:
markdowns := $(shell find src/posts -type f) htmls := $(patsubst src/posts/%.md, posts/%.html, $(markdowns)) all: index posts index: index.html index.html: src/index.md pandoc -s -c styles/reset.css -c styles/main.css -c styles/index.css -o $@ $< posts: $(htmls) posts/%.html: src/posts/%.md pandoc -s --toc -c ../styles/reset.css -c ../styles/main.css -o $@ $<
(The above snippet may be out of date even as soon as the publication of this post, but the current Makefile I’m using for this site can always be found on github.)
Other parts I didn’t go over are these:
- magic variables / “macros”
$@: the target
@<: the first dependency
@^: (not shown here) ALL dependencies
=: lazy assignment
:=: immediate assignment
- pattern matching and substitution
%: wildcard. as opposed to
*, like the rest of the civilized world uses
patsubst: a function that takes a from pattern, a to pattern, and things to do that pattern substitution on.
- other macros like
Anyway, that’s my first experience making any kind of a makefile. It’s not fantastic. It’s not easy, but it is simple. And it’s ubiquitous and transferable.
And now in the meantime I can
make index to create the
index page, or
make posts to create posts, or just
make to make anything and everything that needs making.
So there. I wrote a makefile.
I still have some TODOs and some nitpicks. For example:
I don’t have a great way at the moment, because of my github workflow, to incorporate deployment into my makefile the way I could if I was just
scping these files to a remote server.
I’d also like to figure out a way to not have to have
postsbe separate targets. That’d mean either learning how to manage subdirectories better in make (complete with relative paths for things that need them, like the stylesheets), or basically removing all directory structure and just dumping everything into one root folder. That’d make building files super simple, but it would also result in a messy, untidy pile of files, which is not something I find delightful.
- Often refers to GNU Make. An ancient, language agnostic build tool which defines tasks, dependencies, and recipes. Used by C/C++ hackers of old. Not a great tool, but an extremely useful one, and installed everywhere, much like vim. An example of the worse-is-better?
- Converts documents from one format/markup to another. Supports a large variety of formats: pandoc.org