Multi-page web setup

Here are several situations that might arise when setting up a multi-page website.

Options for production only

If you have a more complicated setup (e.g. you want to self host libraries) and have to change many options in production, then do the following:

  1. Create config.yaml with all your normal/development options

    base_dir: .
enable_jinja: true
...

# Local style sheets
~~css: |
  <link href="{{rel_root}}/shared/local.css" rel="stylesheet">

~~end_head: |
  {{css|safe}}
  <script async src="/usr/lib/python3.14/site-packages/livereload/vendors/livereload.js?host=127.0.0.1&port=35729"></script>

  2. Propagate these settings through the directory hierarchy if needed by creating stub config.yaml files containing:

    include_before: ../config.yaml

    This way your editor can simply make files with md-to-html %, without having to hunt for the site wide configuration.

  3. Create production.yaml with options to override for production:

    # Include normal/development options
dir_config_file: NONE
include_before: config.yaml

# Override necessary options for production
make_search_index: true
~~end_head: {{css|safe}}

You can enable a site wide search by setting search: true in your config.yaml. The search is powerd by Lunr.js and runs via JavaScript directly in the client (no server / setup required).

  • You need to generate an index to use the search. Do this by setting make_search_index: true.
  • You can set make_search_index: true only in production to speed up compilation during testing.
  • The search won’t work if you view the HTML output directly your browser; but will work when uploaded to your webserver.
  • To test the search locally set up a local web server.

Self hosting libraries

To change the CDN or to self host, use the lib option. The default is

lib:
  bootstrap='https://cdn.jsdelivr.net/npm/bootstrap@5',
  bootstrap_icons='https://cdn.jsdelivr.net/npm/bootstrap-icons@1',
  lunr='https://cdn.jsdelivr.net/npm/lunr@2.3',
  mathjax='https://cdn.jsdelivr.net/npm/mathjax@4',
  lightgallery='https://cdn.jsdelivr.net/npm/lightgallery@2',
  mark_js="https://cdn.jsdelivr.net/npm/mark.js@8",
  reveal_js: 'https://cdn.jsdelivr.net/npm/reveal.js@latest'
  reveal_js_plugins: 'https://cdn.jsdelivr.net/gh/rajgoel/reveal.js-plugins@master'

To self host the libraries, install them using npm and this package.json:

{
  "dependencies": {
    "@fortawesome/fontawesome-free": "^7.1.0",
    "@mathjax/mathjax-newcm-font": "^4.1.0",
    "bootstrap": "^5.3.3",
    "bootstrap-icons": "^1.13.1",
    "cc-icons": "file:cc-icons",
    "jquery": "^4.0.0",
    "lightgallery": "^2.8.2",
    "lunr": "^2.3.9",
    "mark.js": "^8.11.1",
    "mathjax": "^4.1.0",
    "reveal.js": "^5.1.0",
    "reveal.js-plugins": "^4.6.0"
  }
}

Then set lib to point to the library directory. You will also need to specify the [MathJax] font path (otherwise it will fetch fonts from the CDN). These can be done via:

node_modules: "https://path/to/node_modules"
~~lib:
  bootstrap: "{{node_modules}}/bootstrap"
  bootstrap_icons: "{{node_modules}}/bootstrap-icons"
  lunr: "{{node_modules}}/lunr"
  mathjax: "{{node_modules}}/mathjax"
  font_awesome: "{{node_modules}}/@fortawesome/fontawesome-free"
  jquery: "{{node_modules}}/jquery"
  lightgallery: "{{node_modules}}/lightgallery"
  reveal_js_plugins: "{{node_modules}}/reveal.js-plugins"
  reveal_js: "{{node_modules}}/reveal.js"
  mark_js: "{{node_modules}}/mark.js"

~~mathjax_config: |
  MathJax.output = {
    fontPath: '{{node_modules}}/@mathjax/%%FONT%%-font',
  };

Setting up a local web server

Python web server is good enough for most use cases.

python3 -m http.server --cgi --bind 127.0.0.1 /path/to/site

Alternately, to use lighttpd create localhost.yaml:

dir_config_file: NONE
include_before: config.yaml
make_search_index: true

# If self hosting modules
node_modules: /node_modules
~~lib:
  bootstrap: "{{node_modules}}/bootstrap"
  bootstrap_icons: "{{node_modules}}/bootstrap-icons"
  lunr: "{{node_modules}}/lunr"
  mathjax: "{{node_modules}}/mathjax"
  font_awesome: "{{node_modules}}/font-awesome"
  jquery: "{{node_modules}}/jquery"
  lightgallery: "{{node_modules}}/lightgallery"
  reveal_js_plugins: "{{node_modules}}/reveal.js-plugins"
  reveal_js: "{{node_modules}}/reveal.js"
  mark_js: "{{node_modules}}/mark.js"

~~end_head: |
  {{css|safe}}
  <script async src="/livereload.js?host=127.0.0.1&port=35729"></script>

Then create lighttpd.conf

server.modules = ( "mod_cgi", "mod_setenv" )
server.port = 8000
server.bind = "127.0.0.1"
server.document-root = "/path/to/site"
dir-listing.activate = "enable"
index-file.names = ( "index.html" )
cgi.assign = ( ".py"  => "/usr/bin/python" )
cgi.execute-x-only = 1

Then do the following:

  1. Symlink or install modules into ./node_modules
  2. Symlink livereload.js to ./livereload.js (don’t change the name)
  3. Render using md-to-html -Psc localhost.yaml .
  4. Run lighttpd -c lighttpd.conf -D
  5. Open localhost:8000 in your browser.

Uploading to the webserver via git

Some servers (e.g. codeberg / github) allow you to commit to a repository and then serve all files. To use this create site-config.yaml as follows:

base_dir: .     # Directory where sources are relative to this file
dst_dir: pages  # Directory where targets go relative to this file

# List of source files / directories. If files are supplied on command line
# then this is ignored.
sources:
  - .

# directories to exclude recursing into (glob patterns OK)
exclude_dirs:
  - /pages

# Files to avoid processing (glob patterns OK)
#exclude_files: [README.md]

# Don't inline CSS / JS
standalone: false

# Default template for all documents
template: contents-right
contents:
  page1.html: title
  page2.html: title
# Alternately use `template: navright` without contents.

~end_head: >-
  {% if env!="production" %}
    <script async src="/path/to/livereload.js?host=127.0.0.1&port=35729"></script>
  {% endif %}

This sets the output to go into the pages directory. Setup this directory to be a git work tree with the branch you want to push to:

git worktree add --orphan -b pages pages

Now create your site for uploading

md-to-html -Pc site-config.yaml -o "env: production" -fv

(For local testing, omit the -o env=production option.) Upload the site using

git -C pages commit -a && git -C pages push

(On your first push you may have to specify the branch name via git -C pages push origin pages.)

Uploading to the webserver via RSync

First setup a filter list of files you want to include/exclude in .rsync-filter:

# rSync filter rules. First matching rule applies.

# Don't upload these
- no-upload
- preprocess.py
- .git/
- __pycache__/

# Upload these
+ *.html
+ *.jpg
+ *.png
+ *.css
+ *.js
+ *.pdf
+ *.csv
+ *.py
+ .htaccess

# Recurse into folders
+ */

# Exclude all files not handled above
- *

Now use a simple upload script:

#! /usr/bin/zsh

# Parameters
target=host:target_directory
md_to_html_args=(-c site-config.yaml -o "env: production" -dPfv .)
rsync_args=(-cav --delete --delete-excluded -FF)
# Rsync filter rules are in .rsync-filter

#Terminal color codes for error messages
RE=$'\e'"[0m" ER=$'\e'"[0;31m"

# Exit script if any command fails 
set -e

cd ${0:A:h}

# Re-render before uploading
md-to-html $md_to_html_args

# Warn if there are unreadable files
unreadable=($(find . -type f -not -perm /og+r))
if (( $#unreadable > 0 )); then
    echo "${ER}Warning unreadable files found: $unreadable${RE}"
fi

# Sync files
rsync $rsync_args ./ $target

Note, with the -d option if you rename a source .md file the old .html output will be deleted. If there are .html files you want protected use the protect_files / protect_dirs options