Markdown and makefiles
using Make to make websites
2020-01-01
I wrote a makefile!
There has always been a make
Iāve always known that Make is a thing.
Rather, I know that Make has always been a thing.
It was written in 1976 which, at the time of this writing, makes it a 43 year old tool, an older tool than me. At the time of this writing.
Make is a relic from the olden times, from the days of yore. A standby of C/C++ hackers. Something that greybeard wizards mutter and chatter and argue about as they craft and maintain their own special makefiles by hand.
But my interaction with make
has always been limited to
merely reciting that age old incantation:
./configure && make && make install
without
ever delving too much into what it is and what it does. I knew that itās
something you use when building code from source, but that was the
extent of my knowledge.
Iāve always appreciated the elegance of the basic formula:
target: prerequisite
recipe
Or, to use a slightly different vocabulary:
task: dependency
code
Whichever you prefer, really:
target: source
command
Very plain, very organized. Simple. Iāve heard people say that a well written Makefile is like documentation for a project, and I can understand a little bit why they would say that. Itās like reading a cookbook: a list of dishes, ingredients, and how to build those dishes from those ingredients:
dish: ingredient
recipe
Make is a build tool
Make is a build tool, and a task runner.
Youāve encountered such things in your travels:
ant
,maven
, and/orgradle
if youāre a Java kind of person.If youāre a java script kind of person, then youāve written npm scripts in your
package.json
to run your tests or start your server; and youāve used grunt or gulp, or webpack.rake
, if youāre a ruby/rails kind of person. (Which was my first attempt at using a Make-like kind of build tool.)
It builds stuff and automates tasks.
Advantages that make
has over other solutions like npm
scripts is that:
It only builds a target if it needs to. (If the sources have been updated.)
No extra dependencies: itās pre-installed on all *nix platforms, including macos.
It can wrap existing build tools! You can have
make
install your node_modules if theyāre missing (as long as there is a package.json), run a bash script, start your server, etc.
I wrote a makefile
So Iāve always wanted an excuse to learn make
.
And when starting this blog, I needed a way to create HTML files from markdown source files. I certainly didnāt want to install webpack, or grunt or gulp, or anything like that. Or even npm. Minimalism is the name of our game. I donāt want any other dependencies.
And so, excuse in hand, I got to cracking away at it.
It took me a minute to learn all enough of Makeās magic
variables, macros, and built-in functions. What really took me
a minute to wrap my head around was how I had to adhere to, and work
around, the basic task > dependency > recipe
formula.
Thereās no flexibility here. Declarative or bust.
For example: I was trying to write a recipe that targets a bunch of HTML files. The problem is that I didnāt have those HTML files. Nor did I necessarily know what they are. I didnāt have a list of my targets, and that was a problem.
What I did have is a list of prerequisites: a bunch of Markdown files.
That is, given Makeās strict syntax requirements, you canāt just start with a bunch of ingredients and then simply build something from them. No, you must have the target first. Something to depend on those prerequisites. Thatās your entry point.
The trick was to do this:
Get a list of all the prerequisites. Easy enough. E.g.
find src/posts -name "*.md"
. And then,Use makeās super weird pattern substitution function to map that list of markdown files to a list of the HTML files I wish I had (my list of targets).
And finally, assign that list to a variable, which I can then use as a list of targets.
Being declarative is hard work some times.
That whole part looks like this:
markdowns := $(shell find src/posts -type f)
htmls := $(patsubst src/posts/%.md, posts/%.html, $(markdowns))
Hereās another way to do it using the built-in wildcard
macro and a different, built-in substitution short hand:
markdowns=$(wildcard src/posts/*.md)
htmls=$(markdowns:src/posts/%.md=posts/%.html)
Those targets (htmls
) are strings that have the format
of posts/somefilename.html
.
So now, targets in hand, we get to do this:
Create some new task that has dependencies equal to the list of htmls. Weāll call it āposts.ā
Use pattern matching to create a task for any file matching
posts/*.html
, which just happens to be the format of all htmls! This target has a dependency of its corresponding markdown file, and its recipe is the pandoc conversion.
That part looks like this:
posts: $(htmls)
posts/%.html: src/posts/%.md
pandoc --options --more-options -o $@ $<
Now I can call make posts
from the command line, and
every markdown file in /src/posts
will generate a new HTML
file in /posts
! Assuming the following, of course:
Thereās not already an existing HTML file for that markdown file, AND
the HTML file is newer than the markdown file.
That is, it will only create the HTML file if it needs to: if it is missing, or if it is out of date.
The makefile in its entirety currently looks like this:
markdowns := $(shell find src/posts -type f)
htmls := $(patsubst src/posts/%.md, posts/%.html, $(markdowns))
all: index posts
index: index.html
index.html: src/index.md
pandoc -s -c styles/reset.css -c styles/main.css -c styles/index.css -o $@ $<
posts: $(htmls)
posts/%.html: src/posts/%.md
pandoc -s --toc -c ../styles/reset.css -c ../styles/main.css -o $@ $<
(The above snippet may be out of date even as soon as the publication of this post, but the current Makefile Iām using for this site can always be found on github.)
Other parts I didnāt go over are these:
- magic variables / āmacrosā
$@
: the target@<
: the first dependency@^
: (not shown here) ALL dependencies
- assignment
=
: lazy assignment:=
: immediate assignment
- pattern matching and substitution
%
: wildcard. as opposed to*
, like the rest of the civilized world usespatsubst
: a function that takes a from pattern, a to pattern, and things to do that pattern substitution on.- other macros like
shell
andwildcard
Anyway, thatās my first experience making any kind of a makefile. Itās not fantastic. Itās not easy, but it is simple. And itās ubiquitous and transferable.
And now in the meantime I can make index
to create the
index page, or make posts
to create posts, or just
make
to make anything and everything that needs making.
Conclusion
So there. I wrote a makefile.
I still have some TODOs and some nitpicks. For example:
I donāt have a great way at the moment, because of my github workflow, to incorporate deployment into my makefile the way I could if I was just
scp
ing these files to a remote server.Iād also like to figure out a way to not have to have
index
andposts
be separate targets. Thatād mean either learning how to manage subdirectories better in make (complete with relative paths for things that need them, like the stylesheets), or basically removing all directory structure and just dumping everything into one root folder. Thatād make building files super simple, but it would also result in a messy, untidy pile of files, which is not something I find delightful.
Resources
- Links
- Glossary
Links
- https://bost.ocks.org/mike/make/
- https://www.olioapps.com/blog/the-lost-art-of-the-makefile/
- https://blog.mindlessness.life/2019/11/17/the-language-agnostic-all-purpose-incredible-makefile.html
- https://jblevins.org/log/markdown-makefiles
Glossary
- Make
- Often refers to GNU Make. An ancient, language agnostic build tool which defines tasks, dependencies, and recipes. Used by C/C++ hackers of old. Not a great tool, but an extremely useful one, and installed everywhere, much like vim. An example of the worse-is-better?
- Pandoc
- Converts documents from one format/markup to another. Supports a large variety of formats: pandoc.org