Converting PDFs with Apple Silicon GPU Acceleration

Typically, PDF to Markdown converters either do not do a very good job converting mathematical formulas to LaTeX, or require an NVIDIA GPU to run a Transformer model. After quite a bit of work, I have discovered a way of converting PDF files, with all mathematical formulas converted to LaTeX, and using Apple silicon GPUs for acceleration.

First, create a Python virtual environment and install docling and docling[vlm]. One way to do it is to quickly create a new file pyproject.toml:

[project]
name = "pdf-convert"
version = "0.1.0"
description = "Setting up the virtual environment for converting PDFs with Apple Silicon GPUs."
requires-python = ">=3.13"
dependencies = [
    "docling",
    "docling[vlm]",
]

and then run uv sync and source .venv/bin/activate. After setting up the environment, the launch command I used was:

docling --enrich-formula --pipeline vlm --vlm-model granite_docling --image-export-mode placeholder file.pdf

This runs the Granite Docling model, with 258M parameters, on the Apple Silicon GPUs with MLX. The conversion process may take a while, but the results look excellent. I have added my setup above to a git repository so that I can use it more easily.

In case there is a need to convert an entire directory containing PDF documents, use a script — it is a bit more user-friendly than using a command-line option that docling provides:

#!/usr/bin/env bash
set -euo pipefail

output_dir="${1:-markdown}"
mkdir -p "$output_dir"

shopt -s nullglob
pdfs=( *.pdf )

if (( ${#pdfs[@]} == 0 )); then
  echo "No PDF files found in $(pwd)" >&2
  exit 1
fi

for pdf in "${pdfs[@]}"; do
  echo "Converting: $pdf"
  docling \
    --enrich-formula \
    --pipeline vlm \
    --vlm-model granite_docling \
    --image-export-mode placeholder \
    --to md \
    --output "$output_dir" \
    "$pdf"
done

echo "Done. Markdown files are in: $output_dir"