Convert Markdown to HTML using Jinja2 templates

This converts a markdown file to a full HTML file by using a template. Can be used to make small standalone HTML documents, or larger multi-page sites. The default templates are:

  1. simple: A simple layout using Bootstrap. (See a demo here.)

  2. navright: Like simple, but includes a navigation column on the right for medium screens. (See a demo here.)

  3. revealjs: A RevealJS presentation. (See a demo here.)

  4. A LightGallery slideshow. (See a demo here.) (This script can also resize your original images and create thumbnails / web versions for your slideshow. See the demo for instructions.)

Installation

Usage

One page use case

Just run md-to-html file.md to produce file.html. This is good for creating a simple webpage, RevealJS presentation, LightGallery slideshow. Optionally choose template / set options in the YAML front matter.

Multi-page use case

To build more complicated sites, this script can do two the following:

  1. Recurse directories and create HTML files (either in the same location, or in a specified destination directory preserving the hierarchy).

  2. Resize images to create thumbnails / web-versions for your slideshows.

  3. Move JavaScript / CSS to a separate shared directory, instead of inlining them.

  4. Recurse through the destination directory, and purge all HTML files that don’t correspond to markdown source files. (You can set exclude / preserve lists.)

  5. Run external commands you can use to copy static files (e.g. CSS / JS).

  6. Run a preprocess hook, to process up YAML meta-data before rendering.

Few useful tips are described here

Example

Look in site-config.yaml for an example that generated this site

Command line options

usage: md-to-html [-h] [-c CONFIG] [-d] [-D DIR_CONFIG_FILE] [-f] [-F] [-q]
                  [-R] [-r] [-s] [-S] [-o OPTIONS] [-p]
                  [--previewer PREVIEWER] [-P] [-t THREADS] [-u] [-v]
                  [files ...]

Convert markdown files to HTML using Jinja2 templates

positional arguments:
  files                 Markdown files

options:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        Config file (YAML)
  -d, --delete-extra    Delete extra html files (default False)
  -D DIR_CONFIG_FILE, --dir-config-file DIR_CONFIG_FILE
                        Per directory config file
  -f, --force           Render whether or not source is newer (None)
  -F, --force-resize    Resize images whether or not source is newer (False)
  -q, --quiet           Suppress info messages (False)
  -R, --recurse         Don't recurse subdirectories for *.md files (True)
  -r, --resize-images   Resize images for slideshows (False)
  -s, --show-extra      Show extra html files (False)
  -S, --search-parents  Search parent directories for config file (False)
  -o OPTIONS, --option OPTIONS
                        Extra YAML to inject (e.g. -o "var: value")
  -p, --preview         Also launch preview
  --previewer PREVIEWER
                        Previewer (xdg-open {file})
  -P, --preprocess      Allow preprocessing metadata (False)
  -t THREADS, --threads THREADS
                        Number of threads. Use 0 to let the system decide
                        automatically (0).
  -u, --update          Only render if source is newer (False)
  -v, --verbose         Show debug messages (False)

Note: --delete-extra, --show-extra require dst_dir to be set in a config file that is read with -c (and not just set in the directory config file).

Configuration (from YAML config files)

Configuration / options are read in the following order:

  1. config.yaml from the installation directory
  2. config.yaml from the XDG configuration directories in order (typically /etc/xdg/md-to-html, and then ~/.config/md-to-html).
  3. All config files specified with the -c option (in order).
  4. Options specified with -o "var: value" on the command line
  5. The dir_config_file from the directory of the input file (default config.yaml)
  6. The YAML front matter in the input file, surrounded by ---. E.g.
    ---
template: simple
encoding: utf-8
enable_jinja: true
---
Markdown content starts here.

If list / dict options are prefixed with a +, then their value is added to the previous value. For lists, only new values are added.

The default config.yaml in the installation directory is:

dir_config_file: config.yaml

enable_mathjax: true
enable_codehilite: true
enable_jinja: false
read_frontmatter: true
jinja_header: >
  {%- import 'lightgallery.j2' as LightGallery -%}
yaml_jinja_depth: 10
template: simple
template_dirs: []

standalone: true
toc_depth: 2-4
encoding: utf-8
base_url: '' # Include trailing slash
shared_dir: shared/md-to-html
absolute_links: false
update: false
exclude_dirs:
  - .git
  - __pycache__
  - templates
  - /shared

exclude_files: []
protect_dirs:
  - /shared

protect_files: []

gallery_defaults:
  images:
    width: 1920
    height: 1080
    quality: 95
  thumbnails:
    width: 150
    height: 150
    quality: 95
    method: crop

Markdown Syntax and Extensions

We use python-markdown to render the HTML, with the following extensions enabled:

MathJax Macros and options

A few MathJax options and LaTex macros are defined in the simple.j2 template. To add your own macros or change the mathjax configuration, you have a few options:

  1. For new macros in one document use this at the start of the file:

    $
  \newcommand{\mymacro}{expansion}
  ...
$

  2. For per document configuration changes use the mathjax_config YAML metadata.

    ---
mathjax_config: |
  MathJax.tex.tags = 'ams'
  MathJax.tex.macros = {
    ...MathJax.tex.macros,
    sign: '\\operatorname{sign}',
    abs: [ '#1\\lvert#2#1\\rvert', 2, '' ]
  }
---

    The contents of mathjax_config are treated as JavaScript and included right after the MathJax configuration object is defined. You can, for instance, also define (or undefine) macros using MathJax.tex.macros. (If you put this in config.yaml, then it will be used for all files in that directory.)

  3. For more global settings, you can extend the template and override the mathjax block (look at tests/templates/custom.j2 for an example).

Use [[file.html|label]] to generate a link to file.html with label label. To use the file name as the label, just use [[file.html]]

Using Jinja2 expressions

In YAML frontmatter / config files.

If you start a field with ~, then it’s value is processed with Jinja2 and the ~ is removed. For instance, to only include a script when env != 'production' use:

~end_head: >-
  {% if env!="production" %}
    <script>...</script>
  {% endif %}

You can either set env in a different config file, or from the command line via -o "env: production".

To reference other meta-data variables use a ~~ prefix.

~a1: "{{b}}"    # maybe empty, since b may not have been computed yet.
~~a2: "{{b}}"   # Works.
~b: 1

Variables starting with ~~ are re-rendered through Jinja until the value stabilizes, or the yaml_jinja_depth is reached. If the value of your variable is not stable (e.g. uses random numbers, or the current time) then don’t use the ~~ prefix.

In the Markdown File

If you set enable_jinja=true, then you can use Jinja2 expressions in markdown files. For example:

---
enable_jinja: true
title: Document tile
author: Author
user_flag: true
---

# {{title}}

*{{author}}, 2023-10-06*

You can use Jinja2 expressions: 1 + 1 = {{1 + 1}}.

{% if user_flag %}
You can set conditionals in metadata
{% else %}
and render content based on the values.
{% endif %}

Note: enable_jinja=false by default so invalid Jinja2 expressions don’t surprise users.

A few extra functions are available in Jinja2 expressions:

Templates

A file is converted to HTML using a Jinja2 template. Templates should be in the templates subdirectory of one of the configuration folders, and should have a .j2 extension. A bare bones example of a template is:

<!DOCTYPE html>
<html>
  <head><title>{{title}}</title></head>
  <body>{{content}}</body>
</html>

All YAML metadata and global options are accessible as template variables. The following additional variables are defined while processing:

Extensions

Template specific options

There are various template specific options that can be set from YAML front matter (or config files). For instance, to add a local style sheet, you can use end_head, which is included at the end of the <head> element.

end_head: <link href="{{rel_root}}/shared/local.css" rel="stylesheet">

Several more are in the templates (e.g. before_toc, after_toc, lib, etc.).

Self hosting libraries

To change the CDN or to self host, use the lib option. The default is

lib:
  bootstrap: 'https://cdn.jsdelivr.net/npm/bootstrap@5.3.3'
  mathjax: 'https://cdn.jsdelivr.net/npm/mathjax@4'
  lightgallery: 'https://cdn.jsdelivr.net/npm/lightgallery'
  reveal_js: 'https://cdn.jsdelivr.net/npm/reveal.js@latest'
  reveal_js_plugins: 'https://cdn.jsdelivr.net/gh/rajgoel/reveal.js-plugins@master'

To self host the libraries, install them using npm and this package.json:

{
  "dependencies": {
    "@fortawesome/fontawesome-free": "^7.1.0",
    "@mathjax/mathjax-newcm-font": "^4.1.0",
    "bootstrap": "^5.3.3",
    "bootstrap-icons": "^1.11.3",
    "jquery": "^4.0.0",
    "lightgallery": "^2.8.2",
    "mathjax": "^4.1.0",
    "reveal.js": "^5.1.0",
    "reveal.js-plugins": "github:rajgoel/reveal.js-plugins"
  }
}

Then set lib to point to the library directory. You will also need to specify the MathJax font path (otherwise it will fetch fonts from the CDN). These can be done via:

node_modules: "https://path/to/node_modules"
~~lib:
  bootstrap: "{{node_modules}}/bootstrap"
  mathjax: "{{node_modules}}/mathjax"
  font_awesome: "{{node_modules}}/@fortawesome/fontawesome-free"
  jquery: "{{node_modules}}/jquery"
  lightgallery: "{{node_modules}}/lightgallery"
  reveal_js_plugins: "{{node_modules}}/reveal.js-plugins"
  reveal_js: "{{node_modules}}/reveal.js"
~~mathjax_config: |
  MathJax.output = {
    fontPath: '{{node_modules}}/@mathjax/%%FONT%%-font',
  };

Live Previews

You can have the web browser automatically reload the HTML preview every time you make a change. Two ways of doing this are described in examples/livereload.html.