Convert Markdown to HTML using Jinja2 templates

This converts a markdown file to a full HTML file by using a template. Can be used to make small standalone HTML documents, or larger multi-page sites. The default templates are:

simple: A simple layout using Bootstrap. (See a demo here.)
navright: Like simple, but includes a navigation column on the right for medium screens. (See a demo here.)
revealjs: A RevealJS presentation. (See a demo here.)
A LightGallery slideshow. (See a demo here.) (This script can also resize your original images and create thumbnails / web versions for your slideshow. See the demo for instructions.)

Installation

Using pip: Run pip install "md-to-html2[resize,codehilite]". (Note the extra 2; the name md-to-html was already used on PyPi 🙄.)
On Arch Linux: Install md-to-html2 from the AUR either manually, or using a helper.
Installing manually
- Clone the repository https://codeberg.org/gi1242/md-to-html.git
- Create a symlink to src/md-to-html from somewhere in your PATH.
- Install the required dependencies: python, python-frontmatter, python-jinja, python-markdown, python-pyxdg.
- Optionally install the optional dependencies: python-pillow (for resizing images), python-pygments (for syntax highlighting output).

Usage

One page use case

Just run md-to-html file.md to produce file.html. This is good for creating a simple webpage, RevealJS presentation, LightGallery slideshow. Optionally choose template / set options in the YAML front matter.

Multi-page use case

To build more complicated sites, this script can do two the following:

Recurse directories and create HTML files (either in the same location, or in a specified destination directory preserving the hierarchy).
Resize images to create thumbnails / web-versions for your slideshows.
Move JavaScript / CSS to a separate shared directory, instead of inlining them.
Recurse through the destination directory, and purge all HTML files that don’t correspond to markdown source files. (You can set exclude / preserve lists.)
Run external commands you can use to copy static files (e.g. CSS / JS).
Run a preprocess hook, to process up YAML meta-data before rendering.

Few useful tips are described here

Example

Look in site-config.yaml for an example that generated this site

Command line options

usage: md-to-html [-h] [-c CONFIG] [-d] [-D DIR_CONFIG_FILE] [-f] [-F] [-q]
                  [-R] [-r] [-s] [-S] [-o OPTIONS] [-p]
                  [--previewer PREVIEWER] [-P] [-t THREADS] [-u] [-v]
                  [files ...]

Convert markdown files to HTML using Jinja2 templates

positional arguments:
  files                 Markdown files

options:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        Config file (YAML)
  -d, --delete-extra    Delete extra html files (default False)
  -D DIR_CONFIG_FILE, --dir-config-file DIR_CONFIG_FILE
                        Per directory config file
  -f, --force           Render whether or not source is newer (None)
  -F, --force-resize    Resize images whether or not source is newer (False)
  -q, --quiet           Suppress info messages (False)
  -R, --recurse         Don't recurse subdirectories for *.md files (True)
  -r, --resize-images   Resize images for slideshows (False)
  -s, --show-extra      Show extra html files (False)
  -S, --search-parents  Search parent directories for config file (False)
  -o OPTIONS, --option OPTIONS
                        Extra YAML to inject (e.g. -o "var: value")
  -p, --preview         Also launch preview
  --previewer PREVIEWER
                        Previewer (xdg-open {file})
  -P, --preprocess      Allow preprocessing metadata (False)
  -t THREADS, --threads THREADS
                        Number of threads. Use 0 to let the system decide
                        automatically (0).
  -u, --update          Only render if source is newer (False)
  -v, --verbose         Show debug messages (False)

Note: --delete-extra, --show-extra require dst_dir to be set in a config file that is read with -c (and not just set in the directory config file).

Configuration (from YAML config files)

Configuration / options are read in the following order:

config.yaml from the installation directory
config.yaml from the XDG configuration directories in order (typically /etc/xdg/md-to-html, and then ~/.config/md-to-html).
All config files specified with the -c option (in order).
Options specified with -o "var: value" on the command line
The dir_config_file from the directory of the input file (default config.yaml)

The YAML front matter in the input file, surrounded by ---. E.g.

---
template: simple
encoding: utf-8
enable_jinja: true
---
Markdown content starts here.

If list / dict options are prefixed with a +, then their value is added to the previous value. For lists, only new values are added.

The default config.yaml in the installation directory is:

dir_config_file: config.yaml

enable_mathjax: true
enable_codehilite: true
enable_jinja: false
read_frontmatter: true
jinja_header: >
  {%- import 'lightgallery.j2' as LightGallery -%}
yaml_jinja_depth: 10
template: simple
template_dirs: []

standalone: true
toc_depth: 2-4
encoding: utf-8
base_url: '' # Include trailing slash
shared_dir: shared/md-to-html
absolute_links: false
update: false
exclude_dirs:
  - .git
  - __pycache__
  - templates
  - /shared

exclude_files: []
protect_dirs:
  - /shared

protect_files: []

gallery_defaults:
  images:
    width: 1920
    height: 1080
    quality: 95
  thumbnails:
    width: 150
    height: 150
    quality: 95
    method: crop

base_dir: Base of the source directory hierarchy. Source file outside this directory will raise an exception. It is treated as a relative path from the file it was defined in, so that base_dir: . works as expected.
base_url: Base URL of the target site. Shared files should accessible at base_url/shared_dir. If empty, then relative paths will be used for shared files.
dst_dir : All output goes in this directory, preserving the directory hierarchy up to base_dir. That is base_dir/path/file.md becomes dst_dir/path/file.html. It is treated as a relative path from the file it was defined in.
dst_file: Destination file name. If unspecified, use the source file name with .html extension.
exclude_files / exlcude_dirs: List of glob patterns of source files / directories to ignore when recursing through directories. (To append to existing values use +exclude_files / +exclude_dirs.)
include: YAML config file (or list of config files) to read settings from, after reading all settings from the current file.
include_before: YAML config file (or list of config files) to read settings from, before applying settings from the current file.
read_frontmatter: If false, then skip reading front matter in all files.
rel_src_file: Source file name relative to base_dir
rel_dst_file: Destination file name relative to dst_dir
rel_dir: Directory containing src_file relative to base_dir. (Same as directory containing dst_file relative to dst_dir.)
shared_dir: For non-standalone documents, put CSS / JS here. (Relative to dst_dir if specified, or the directory of the current file otherwise.)
src_file: Source file name
standalone: If true, all CSS / JS is included inline.
template_dirs: Extra Jinja2 template directories to add (base_dir is automatically added when specified).
preprocess: Path of python file to use to pre-process meta-data. Call the init function in this file passing the metadata and the filename as arguments. The function should modify this (in place) as needed. Useful to cleanup the YAML data in Python when Jinja templating features aren’t sufficient.
- Only processed if -P is used.
protect_files / protect_dirs: List of glob patterns of destination files / directories to protect from deletion. When -d is used, all HTML files in the destination directory that do not correspond to source markdown files are deleted. protect_files / protect_dirs can be used to protect some of these files from deletion. (To append to existing values use +protect_files / +protect_dirs.)
sources: List of source files / directories to convert to HTML / recurse into. (Ignored if files are present on command line.)
update: Only compile source if it is newer than the target HTML (overridden by -u / -f on command line)

Markdown Syntax and Extensions

We use python-markdown to render the HTML, with the following extensions enabled:

Extra: A compilation of various Python-Markdown extensions that (mostly) imitates PHP Markdown Extra. The supported extensions include: Abbreviations, Attribute Lists, Definition Lists, Fenced Code Blocks, Footnotes, Tables, Markdown in HTML.
Sane Lists: The Sane Lists extension alters the behavior of the Markdown List syntax to be less surprising.
SmartyPants: The SmartyPants extension converts ASCII dashes, quotes and ellipses to their HTML entity equivalents.
Table of Contents: The Table of Contents extension generates a Table of Contents from a Markdown document and adds it into the resulting HTML document. (Use it in templates via {{toc}}.)
Strikethrough: Produce ~~strike through~~ text using ~~text~~
CodeHilite: The CodeHilite extension adds code/syntax highlighting to standard Python-Markdown code blocks using Pygments. (Disable it with enable_codehilite: false)
Wiki Links: To use [[link.html|label]] syntax (and provide warnings if the link is to a local find and isn’t found.
Math: Render math using MathJax. For example: $\displaystyle f(z) = \frac{1}{2\pi i} \oint_{\abs{z - \zeta} = r} \frac{f(\zeta)}{z - \zeta} \, d\zeta \,. $
Jinja2 expressions if enable_jinja is true (default is false).

MathJax Macros and options

A few MathJax options and LaTex macros are defined in the simple.j2 template. To add your own macros or change the mathjax configuration, you have a few options:

For new macros in one document use this at the start of the file:
```
$
  \newcommand{\mymacro}{expansion}
  ...
$
```
For per document configuration changes use the mathjax_config YAML metadata.
```
---
mathjax_config: |
  MathJax.tex.tags = 'ams'
  MathJax.tex.macros = {
    ...MathJax.tex.macros,
    sign: '\\operatorname{sign}',
    abs: [ '#1\\lvert#2#1\\rvert', 2, '' ]
  }
---
```
The contents of mathjax_config are treated as JavaScript and included right after the MathJax configuration object is defined. You can, for instance, also define (or undefine) macros using MathJax.tex.macros. (If you put this in config.yaml, then it will be used for all files in that directory.)
For more global settings, you can extend the template and override the mathjax block (look at tests/templates/custom.j2 for an example).

WikiLinks

Use [[file.html|label]] to generate a link to file.html with label label. To use the file name as the label, just use [[file.html]]

Link to files which don’t exist produce a warning.
[[/absolute/path]] links to base_url/absolute/path.
If absolute_links is set, then all wiki-links are converted to absolute links using base_url/absolute/path.

Using Jinja2 expressions

In YAML frontmatter / config files.

If you start a field with ~, then it’s value is processed with Jinja2 and the ~ is removed. For instance, to only include a script when env != 'production' use:

~end_head: >-
  {% if env!="production" %}
    <script>...</script>
  {% endif %}

You can either set env in a different config file, or from the command line via -o "env: production".

To reference other meta-data variables use a ~~ prefix.

~a1: "{{b}}"    # maybe empty, since b may not have been computed yet.
~~a2: "{{b}}"   # Works.
~b: 1

Variables starting with ~~ are re-rendered through Jinja until the value stabilizes, or the yaml_jinja_depth is reached. If the value of your variable is not stable (e.g. uses random numbers, or the current time) then don’t use the ~~ prefix.

In the Markdown File

If you set enable_jinja=true, then you can use Jinja2 expressions in markdown files. For example:

---
enable_jinja: true
title: Document tile
author: Author
user_flag: true
---

# {{title}}

*{{author}}, 2023-10-06*

You can use Jinja2 expressions: 1 + 1 = {{1 + 1}}.

{% if user_flag %}
You can set conditionals in metadata
{% else %}
and render content based on the values.
{% endif %}

Note: enable_jinja=false by default so invalid Jinja2 expressions don’t surprise users.

Templates

A file is converted to HTML using a Jinja2 template. Templates should be in the templates subdirectory of one of the configuration folders, and should have a .j2 extension. A bare bones example of a template is:

<!DOCTYPE html>
<html>
  <head><title>{{title}}</title></head>
  <body>{{content}}</body>
</html>

All YAML metadata and global options are accessible as template variables. The following additional variables are defined while processing:

uses_math: True if math was detected when rendering the file
uses_codehilite: True if syntax highlighted code was detected when rendering the file
toc: HTML table of contents obtained from the file
title: Contents of the first <h1> element, if not previously set.
content: Document content after conversion from Markdown to HTML

Extensions

Convert the contents of the var from markdown to HTML by {{var|markdown}}.
Convert an arbitrary block of markdown to HTML using {% markdown %}... {% endmarkdown %}.

Template specific options

There are various template specific options that can be set from YAML front matter (or config files).For instance, to add a style sheet, you can use

end_head: <link href="..." rel="stylesheet" integrity="..." crossorigin="anonymous">

Live Previews

You can have the web browser automatically reload the HTML preview every time you make a change. Two ways of doing this are described in examples/livereload.html