Read time: 11 mins
Custom Static Site Generator in Clojure with Org mode
Digital Homestead
Jack Rusher's post on digital homesteading inspired me to craft and own my publishing tools. I'm not sure that the spirit of the post was in line with "write another static site generator that works slightly different than others". But that is one of the things I took away from it.
There are numerous ready-made static site generators available, and the smart (and time-valuing) choice would be to use one of them. However, I have some specific niche needs that aren't met by any existing generator.
Specific Requirements
My requirements for this project were as follows:
- Compatibility with Org-Mode
I primarily use Doom Emacs as my text editor, writing in org mode is my preferred medium for content creation.
- Ease of Styling
I want to leverage Clojure hiccup and Tailwind for creating HTML without learning a new template language or an arbitrary theme plugin system.
- Efficient Load Times
A static site with minimal dependencies was essential for efficient loading times.
- Flexibility
I wanted the freedom to add features such as comments or interactive experiences on my site without having to defy platform conventions.
- Minimal Overhead
I aimed to be as close to basic HTML, CSS, and JavaScript as possible, while still utilizing Clojure, hiccup, tailwind, and org mode. I also wanted an easy process for deploying and writing, as a low barrier to content production encourages more frequent writing.
- Cost Effectiveness
A cheap or free hosting solution that wouldn't require scaling considerations was a must.
Other Blogging Solutions
Several platforms I tested were interesting but ultimately fell short of meeting my needs.
- Cryogen
A static site generator written in Clojure, served its purpose well, but I encountered some challenges. Writing in markdown was less comfortable for me than in Org-Mode, and deployment was somewhat tedious.
- Ghost
Ghost's themes were impressive and offered certain conveniences such as metadata for Twitter previews and Google result displays. However, the experience with the editor left something to be desired, and I was hesitant to learn another theme syntax to customize styles.
- Roam Research
While the idea of using Roam Research was interesting, the fact that it would create a Single Page Application (SPA) and not a static site raised concerns about load times. Additionally, setting up a custom domain appeared to be either difficult or potentially costly.
- org-static-blog
As the name implies, org-static-blog does exactly what it promises – it's a simple static blog generator for Emacs' Org-Mode. However, styling it proved to be an arduous task, as all the template HTML is embedded within a string in an Emacs configuration file.
Project Overview
I decided to start from scratch. From an empty git repo I added a directory with some org mode blog posts and a jvm clojure file. From there I let the REPL guide me. What I ended up with looked like this:
.
├── css
│ └── input.css
├── deps.edn
├── package.json
├── package-lock.json
├── pages
│ ├── now.org
│ └── resume.org
├── postcss.config.js
├── posts
│ ├── 2023-04-22-kitchen-sink.org
│ └── 2023-05-20-org-blog.org
├── README.org
├── src
│ └── org_blog
│ ├── common
│ │ ├── components.clj
│ │ ├── files.clj
│ │ └── org.clj
│ ├── core.clj
│ ├── dev_server.clj
│ ├── pages
│ │ ├── archive.clj
│ │ ├── home.clj
│ │ ├── now.clj
│ │ ├── resume.clj
│ │ └── rss.clj
│ ├── pandoc-template-body.html
│ ├── pandoc-template-toc.html
│ └── posts.clj
├── static
│ ├── archive
│ │ └── index.html
│ ├── css
│ ├── img
│ ├── index.html
│ ├── now
│ │ └── index.html
│ ├── posts
│ │ ├── 2023-04-22-kitchen-sink
│ │ │ └── index.html
│ │ └── 2023-05-20-org-blog
│ │ └── index.html
│ ├── resume
│ │ └── index.html
│ └── rss.xml
└── tailwind.config.js
css
Contains one input.css file with some tailwind `@apply`
statements for things that I can't style directly in the template src
files.
pages
Contains one off org files for pages that are not blog posts. As of now, that is just my resume.
posts
All of the org files representing my blog posts.
src
This is where all of the clojure code exists to build the site. It's
starts with core.clj. I've organically organized it loosely
into different namespaces. Right now the only directories needed are
common/ and pages/.
static
This is the directory with the static assets. All of the html, css, and images are here. I commit images right to the repository. Videos I will host in an s3 bucket and link out. When the Github repo updates the digital ocean app pulls in the changes and serves everything in this directory.
Workflow
I'm really happy with my workflow. It starts with opening Doom Emacs.
Easy writing
If I want to just write I open or create an org file in the
posts/ directory and start writing. If I want to save my
progress I can commit and push my changes to the Github repo.
Preview with the REPL
If I want to see what a post looks like I can start a repl and eval
the org-blog.core namespace. That starts up a development
webserver locally and generates all the static files. It includes a
filewatcher so any changes to source code or the org mode blog posts
trigger a re-generation of the static files.
I can then go to localhost:8080 and see the post I'm
working on. The screenshot below is not what the blog looks like
anymore. Originally I was trying a kind of Star Trek LCARS theme.
Custom pages
Not everything I want to make is an Org-Mode based blog post. For one off pages I make a clojure namespace that uses hiccup to generate an html page. All of the reusable components of the site are clojure functions that generate hiccup. Building up a page is functional and feels similar to writing Reagent components in a React based SPA.
Here is the home page generation function.
(defn gen []
(-> "Generating home (index) page" c/blue println)
(-> [:html {:lang "en"} ; Add language attribute
(comps/head)
(comps/body
[:header
(comps/nav)]
[:main
[:div.lcars-bottom-border.lcars-border-purple.pl-8.md:pl-40
[:div.p-4.w-full.rounded-tl-lg.bg-black
[:h1 "Things I've got going on"]
[:p "More stuff maybe"]
[:h2 "Recent writing"]
[:ul.grid.md:grid-cols-2.lg:grid-cols-4
(->> posts-org-dir
io/file
file-seq
(filter #(re-matches #".*\.org" (.getName %)))
(sort)
(reverse)
(take 5)
(map #(str (.getCanonicalPath %)))
(map (fn [org-file]
(let [post-name (posts/get-org-file-name org-file)]
[:a {:href (str "/posts/" post-name)} post-name]))))]]]])]
html
(->> (spit-with-path "./static/index.html"))))It's essentially one thread -> macro. It pushes some
hiccup with embedded function calls that generate other hiccup
components. The hiccup is turned into html then
spit into a file in the static/ directory. It
feels right to have the templating language (hiccup) be so
close to the programming language.
In this example I'm grabbing a directory on the file system and finding the latest 5 blog posts to generate a link on the home page. It's just right inline with the templating of the html. There are no `{{%!?? whatever ??!%}}` escape hatches. This is just plain clojure code.
✨ The templating and the language are one ✨.
Deployment
When I'm ready to deploy I just need to commit the changes in
static/.
Code
There are some areas of the codebase that are interesting.
Dev webserver
To be able to develop locally the project needed a dev webserver.
That exists in org-blog.dev-server namespace. It uses
org.httpkit.server. The heart of it is just a simple
handler function that does a little something different per content
type.
(defn handler [req]
(let [resource-path (str "static" (:uri req))
file (io/file resource-path)
ext (->> (split resource-path #"\.")
last)]
(if (.exists file)
(if (.isDirectory file)
{:status 200
:headers {"Content-Type" "text/html"}
:body (slurp (io/file (str resource-path "/index.html")))}
{:status 200
:headers {"Content-Type" (content-type-for resource-path)}
:body (if (#{"jpg" "png" "gif"} ext)
(io/input-stream file)
(slurp file))})
{:status 404
:headers {"Content-Type" "text/plain"}
:body "Not Found"})))File watcher
What makes developing the site super easy is that it automatically
re-generates static files, and reloads the repl, on any file save. In
org-blog.core namespace are a few lines of code that make
this possible. This is all built on top of the hawk library.
;; These lines are in `org-blog.dev-server`
(defn watch-source-files [dirs handler]
(hawk/watch! [{:paths dirs
:handler handler}]))
(defonce source-watchers (atom nil))
;; These lines are in `org-blog.core`
(when (nil? @dev-server/source-watchers)
(reset! dev-server/source-watchers
(dev-server/watch-source-files
["src" "posts" "pages"]
(fn [ctx e]
(when (= (:kind e) :modify)
(println "File modified:" (:file e))
;; Calling `ns-repl/refresh` in another thread (hawk must run this handler in a another thread)
;; generates an error
;; By wrapping in future, by some magic, the function calls within are scheduled on the main thread I guess
(future
(try
(println "Refreshing repl ...")
(ns-repl/refresh)
(println "Ahhhh, so refreshed!")
(regenerate-site)
(catch Exception e
(when-not (and (instance? IllegalStateException e)
;; Not sure why this error happens but the repl refreshes when it's thrown so I guess it doesn't matter
(.contains (.getMessage e) "Can't change/establish root binding of: *ns* with set"))
(println "Error refreshing repl:" e))))))))))Converting Org-Mode Files with Pandoc
In org-blog.common.org is the actual conversion of
Org-Mode content to html. It relies on pandoc and basically shells out to that
system dependency. I use some basic templates to isolate the body and
the table of contents (toc) and return a vector of the html toc and the
html body.
(defn org->html
"Requires at least pandoc 3.1.2 installed locally"
[org-file]
(let [absolute-org-file (full-path org-file)
toc-template-path (full-path "./src/org_blog/pandoc-template-toc.html")
body-template-path (full-path "./src/org_blog/pandoc-template-body.html")
toc-cmd (str "pandoc -f org -t html "
"--template=" toc-template-path " "
"--table-of-contents " absolute-org-file)
body-cmd (str "pandoc -f org -t html "
"--template=" body-template-path " "
absolute-org-file)
toc-result (shell/sh "sh" "-c" toc-cmd)
body-result (shell/sh "sh" "-c" body-cmd)]
(if (and (zero? (:exit toc-result))
(zero? (:exit body-result)))
[(:out toc-result)
(:out body-result)]
(do (println (str "Error(s):" [(:error toc-result) (:error body-result)]))
nil))))Design Choices
I leaned heavily on chatgpt to get somewhere with the UI design. I knew I wanted to draw inspiration from LCARS Star Trek interface design. I didn't use anything from the LCARS Online Template but I did look at it for awhile to figure out what I wanted.
The color scheme was entirely generated from chatgpt. I asked for a Vaporwave color palette and plugged it into palettte.app (three T's) to make some different shades. Chatgpt was useful for converting the export of Palettte to the config of TailwindCSs.
theme: {
colors: {
transparent: 'transparent',
current: 'currentColor',
black: '#000000',
white: '#ffffff',
yellow: {
100: "#FDCF70",
200: "#F9BF46",
DEFAULT: "#EDAD28",
400: "#B8820F",
900: "#865B00",
},
pink: {
100: '#FF88D1',
DEFAULT: '#FF71CE',
900: '#E064B7',
},
cyan: {
100: '#33E1FD',
DEFAULT: '#01CDFE',
900: '#01B2D6',
},
green: {
100: '#30FFB3',
DEFAULT: '#05FFA1',
900: '#05D68E',
},
purple: {
100: '#CA7FFF',
DEFAULT: '#B967FF',
900: '#A355E2',
},
red: {
100: '#FF8B8B',
DEFAULT: '#FF6B6B',
900: '#E25B5B',
},
},With that custom theme in place styling with Tailwind is really easy. Here is the code that creates the top section of the LCARS border and side panel.
[:div.lcars-top-border.lcars-border-green.pl-8.md:pl-40
[:div.p-4.rounded-bl-lg.bg-black
[:div.text-4xl.font-bold.mb-2.bg-clip-text.text-transparent.bg-gradient-to-b.from-green-100.to-cyan-100
"JGood Blog"]
;; ...
]]There are some style defined in the css file. Includin a pseudo element to create a "break up" effect on the long thin horizontal part of the border.
/* LCARS-inspired styling */
.lcars-top-border {
@apply bg-gradient-to-b pb-1;
position: relative;
border-bottom-left-radius: 2rem;
}
@media screen and (min-width: 768px) {
.lcars-top-border {
border-bottom-left-radius: 5rem;
}
}
.lcars-top-border::before {
@apply h-1;
content: "";
position: absolute;
bottom: 0;
left: 0;
width: 100%;
background-image: linear-gradient(to right,
transparent 0%, transparent 25%,
#B967FF 25%, #B967FF 26%,
transparent 26%, transparent 28%,
#000000 28%, #000000 29%, /* Black Section */
#05FFA1 29%, #05FFA1 30%,
rgba(0, 0, 0, 0.5) 30%, rgba(0, 0, 0, 0.5) 31%, /* Fade to Black Section */
#01CDFE 31%, #01CDFE 32%,
transparent 32%, transparent 37%,
#B967FF 37%, #B967FF 38%,
transparent 38%, transparent 40%,
#000000 40%, #000000 41%, /* Black Section */
#05FFA1 41%, #05FFA1 42%,
rgba(0, 0, 0, 0.5) 42%, rgba(0, 0, 0, 0.5) 43%, /* Fade to Black Section */
#01CDFE 43%, #01CDFE 44%,
transparent 44%, transparent 59%,
#B967FF 59%, #B967FF 60%,
transparent 60%, transparent 62%,
#000000 62%, #000000 63%, /* Black Section */
#05FFA1 63%, #05FFA1 64%,
rgba(0, 0, 0, 0.5) 64%, rgba(0, 0, 0, 0.5) 65%, /* Fade to Black Section */
#01CDFE 65%, #01CDFE 66%,
transparent 66%, transparent 81%,
#B967FF 81%, #B967FF 82%,
transparent 82%, transparent 84%,
#000000 84%, #000000 85%, /* Black Section */
#05FFA1 85%, #05FFA1 86%,
rgba(0, 0, 0, 0.5) 86%, rgba(0, 0, 0, 0.5) 87%, /* Fade to Black Section */
#01CDFE 87%, #01CDFE 88%,
transparent 88%, #000000 100%);
background-size: 100% 100%;
}Design Update (2023-10-21)
I've done away with the LCARS design. It was fun to play around with but I want something less distracting. I stripped all the design down to basics and copied colors and layout from a few different inspiration sources.
Handling Images
I'm using two tools for image generation. flameshot is for taking screenshots and exiftool for stripping gps data from photos.
Images are stored in the static/img/ directory and
committed right to the repo. I don't have any videos yet but I
anticipate putting those in Digital Ocean bucket and linking to them
from there.
I don't do any resizing of images yet. If I notice some issues with performance I'll think about doing that.
/End
Thanks for reading this. Hope seeing an example of someone rolling their own static site generator was helpful in some way!