tenkana, a spider that jumps from the checkpoints of your TeX document ---------------------------------------------------------------------- /\ ______ /\ / \ / . . \ / \ _/\___/ \__|o(O)(O)o|__/ \___/\_ / \ __ | '' | __ / \ / \/ \ \______/ / \/ \ tenkana is free software that makes repeated LaTeX builds faster. Its command is `tenkana'. It needs no root access and changes neither the TeX engine nor your document. While TeX runs, tenkana keeps frozen copies ("snapshots") of the running engine: one just before \begin{document} and one before each \include. After an edit, tenkana continues from the last snapshot before the change, so the preamble, its packages and fonts, and all unchanged chapters before the edit are not processed again. You give tenkana the TeX command you would type without it, as with env or time; tenkana's own options come before it: tenkana lualatex thesis draft build (default): fast, references not updated tenkana --final lualatex thesis final build: references, citations and table of contents up to date tenkana -v pdflatex thesis also show tenkana's details (passes, snapshots, memory) tenkana pdflatex -synctex=1 thesis TeX's options go to TeX unchanged tenkana -q thesis short for `tenkana lualatex thesis' `man tenkana' has the full manual. Contents -------- 1. What you see 2. Installation 3. How it works 4. Snapshots on disk 5. Measured results 6. Building without snapshots 7. Options and environment 8. Limitations 9. Files in this package 10. Copyright and license 1. What you see --------------- TeX's own terminal output, followed by tenkana's messages: Output written on talk.pdf (14 pages, 74150 bytes). Transcript written on talk.log. tenkana: jumped from ~5.6s for a plain run to 1.8s (68% less time). Each message is one line and starts with `tenkana:', as with other Unix commands; error messages go to stderr. --silent (or TENKANA_SILENT) leaves out all but error messages. The comparison uses the time of the last full run of the document. When a build saves no time, tenkana says so and gives the reason. Every message tenkana can print is listed, with examples, in the file MESSAGES; all texts live in src/tenkana/messages.py. tenkana runs your command as you typed it, in the current folder, and adds only -recorder right after the command name: it needs the list of files TeX reads, which TeX writes to .fls. (XeTeX also gets -no-pdf, see section 3.) TeX's questions, after an error or from \typein, appear on your terminal and take your answers, and Ctrl-C reaches TeX, which answers with its "Interruption" prompt; this holds for resumed builds too. On a resumed build, tenkana first prints what TeX printed up to the snapshot (saved with it), then the live output, so the terminal transcript is the same as a plain run's with the same options and -recorder. So is the .log file, except for two memory-statistics lines at the very end (tenkana defines one extra macro). Errors, warnings and SyncTeX refer to your real file names and line numbers. 2. Installation --------------- tenkana itself needs only Python 3.9 or newer. It installs and runs without TeX, but to turn a document into a PDF it calls a TeX engine (lualatex, pdflatex, xelatex, ...), so for that you need a TeX distribution such as TeX Live, with LaTeX 2020-10 or newer. For snapshots, tenkana also needs: * GNU/Linux (tenkana uses LD_PRELOAD and /proc; it is not tested elsewhere); * a C compiler (cc, gcc or clang) for the snapshot library, which tenkana compiles from src/tenkana/tenkanasnap.c on first use and caches in ~/.cache/tenkana/ (in /tmp/tenkana-/ if that path contains a space or a colon, which LD_PRELOAD cannot handle); * an x86-64 machine. Without a C compiler or on another processor, tenkana builds with plain runs (section 6). No root access is needed: ./install.sh installs ~/.local/bin/tenkana and the man page ./install.sh --system installs into /usr/local (uses sudo) ./install.sh --uninstall removes tenkana again The `tenkana' command is a single executable file (a Python zip application). If ~/.local/bin is not on your PATH, the installer says what to add. 3. How it works --------------- The core idea of process checkpointing is to capture a running process and later continue from that exact point. Tools such as CRIU do this from outside the process, which needs root or special privileges. tenkana lets the engine copy itself instead, with fork(), which any program may do. * A small library, src/tenkana/tenkanasnap.c, is loaded into the unmodified engine with LD_PRELOAD. The engine is started with your command, as in section 1 (`lualatex -recorder talk'). * Pause points. When the engine opens your main file, the library hands it the same text with two hooks added at the start of line 1, so line numbers do not change. TeX still prints and records your file's real name. The hooks make TeX open a special file just before \begin{document} and before every \include that TeX actually runs (not those skipped by \includeonly or a false conditional). When the library sees that file opened, it notes the position of every open file and forks: - the copy freezes and becomes a snapshot. It waits for commands and shows up in ps as `tenkana-snap-'. It marks itself as the first process the kernel's out-of-memory killer should stop, so if memory runs out, the kernel ends snapshots before your other programs (other memory managers, such as systemd-oomd or earlyoom, may choose differently); - the original reports to tenkana, waits while tenkana saves copies of the output files (.pdf, .aux, .log, ...), and carries on typesetting. * Resuming. On the next run, tenkana puts the output files back as they were at the snapshot and tells that frozen copy to fork again. The new copy restores every file position and continues typesetting from the pause point; for the main file it also discards text that had been read ahead, so an edited body is read fresh. It reads its terminal input from the `tenkana' command that resumed it and gets back the engine's own signal handlers, so TeX can ask questions and Ctrl-C works as without tenkana. The frozen copy stays frozen (holding no terminal) and can be resumed again and again. * Memory. A snapshot holds what the engine holds at that point, mostly fonts and packages. fork() is copy-on-write, so further snapshots of the same run only cost the memory pages changed in between: all 13 snapshots of the 220-page test book took 330 to 400MB in our tests. `tenkana -v' shows the figure for your document. * Lifetime. Snapshots are ordinary processes, so they survive between runs of tenkana and the closing of the terminal. They end: - at a reboot, like every process; - at a logout only if the system ends a user's processes then (with systemd, when KillUserProcesses is on in logind.conf; distributions differ); - 8 hours after the document's last build (TENKANA_TTL; any build counts, even one that resumes no snapshot); - with `tenkana --clean', or within half a minute after tenkana's cache folder for the document is deleted; - if the kernel runs out of memory (see above). tenkana does not watch for any of this. At the next build it finds that the snapshots are gone, and restores the preamble snapshot from disk (section 4) or makes a full run. * Which engine. tenkana finds it from the program your command finally runs: pdflatex and pdflatex-dev run pdfTeX, lualatex runs LuaHBTeX, xelatex runs XeTeX. Any command that runs pdfTeX, LuaTeX (also LuaHBTeX, LuaJITTeX) or XeTeX works. * Which file. TeX's first line is what follows the options: usually the file name, as in `lualatex thesis'. It may also be TeX code, as in `pdflatex '\nonstopmode\input paper.ltx''; tenkana then takes the file of the first \input as the main file. Without an \input, tenkana runs the command as it is, without snapshots, and says so. The file is looked for as TeX does: in the current folder, then on TeX's search path (as kpsewhich reports it), so `tenkana pdflatex sample2e' works. If TeX would not find the file either, tenkana runs the command as it is and TeX reports the problem. * Where tenkana keeps its data. Everything goes to ~/.cache/tenkana/ ($XDG_CACHE_HOME/tenkana/ if that is set), one folder per document, readable only by you. Nothing is written next to the document except TeX's own output (including .fls), so a folder synchronised between computers (Nextcloud, Syncthing, ...) carries no snapshots; each computer keeps its own. One build of a document runs at a time: a second one (from an editor that builds on every save, say) waits for the first. * Draft and final builds. LaTeX reads the previous run's .aux (and .toc, ...) at \begin{document} and writes new ones during the run. A normal build always does one pass, so a new or moved label shows as ?? or with an old number until you run `tenkana --final'. That reruns from the preamble snapshot (taken before the .aux is read) until the .aux, .toc and similar files stop changing, like latexmk (at most 4 passes). In drafts, references keep the values read by the last build that started at \begin{document} (a full run, a final build, or an edit that resumed from the preamble snapshot); a draft that resumes from a chapter snapshot uses what that snapshot read. The text and page layout are always current. * Change tracking. Every file the engine reads is recorded (-recorder) and assigned to the part of the run that first read it: the preamble, the main file's body, or the stretch after one of the pauses. The parts follow what TeX actually did, so \includeonly, conditionals and \include inside an \input file are handled correctly. A change to a file read in the preamble needs a full run; any change to the main file's body resumes from the preamble snapshot; any other change resumes from the snapshot just before the part that first read the file. The library also records every file TeX looked for but did not find in the document's folder, so a new chapter, image, .bbl (after biber) or .ind (after makeindex) is noticed too. Files that other programs write for TeX are also watched by name: .bbl, .ind, .gls, .acr, .nls, .glstex, and glossary types declared in the .aux. More can be added with TENKANA_WATCH. * Edits during a build. A file saved while a build is running may or may not have been read by TeX already. tenkana records each input as it was when the build started, so the next build takes such an edit as a change and none is lost. * Errors. Snapshots taken before a LaTeX error stay valid, so after you fix the error tenkana resumes just before the chapter that failed. * Interrupted builds. If tenkana is ended while it builds (kill, a closed terminal), it ends the TeX engine too. The output files are then incomplete, and tenkana knows: the next build rebuilds them even if no input changed. * XeTeX runs with -no-pdf, so snapshots never involve its separate PDF converter (a second process, which a snapshot cannot hold); tenkana runs it once at the end, as XeTeX would (xdvipdfmx -q -E). If you give -no-pdf yourself, tenkana leaves the .xdv file as it is. * SyncTeX. TeX writes .synctex(busy) during the run and renames it to .synctex.gz at the end. A snapshot holds the busy file open, so the snapshot library keeps that file in tenkana's cache (it catches the file however the engine opens it, also in engines with zlib built in, as TeX Live's own binaries are), and the renaming becomes a copy to .synctex.gz. So snapshots work with SyncTeX on. For pdfTeX and LuaTeX the .synctex.gz file is the same as a plain run's; XeTeX's names the .xdv file as its output (it runs with -no-pdf), and viewers find the same positions in it. 4. Snapshots on disk -------------------- After a build that made a new preamble snapshot, a copy of that snapshot writes itself to tenkana's cache (section 3) in the background, unless it would be larger than TENKANA_SAVE_LIMIT, so it survives a reboot. Only what is needed to rebuild it is stored: * memory backed by files (the engine, its libraries, mapped fonts) is mapped again from the files; only pages that differ are stored; * empty pages are skipped; * the rest is compressed with zstd (zlib if zstd is not installed). Saving happens in a copy of the frozen engine, which reads the memory of a pristine child of its own (a parent may read its child's memory without root), so no build waits for it. Restoring needs no root either. tenkana starts the engine exactly as it started the original: same command, recorded environment, and address-space randomisation switched off for the engine (allowed for any user), so the binary, libraries, stack and vDSO land at the same addresses. Before the engine's main() runs, the snapshot library replaces the process's memory with the image, reopens its files at their positions, restores signal handlers, working directory and thread pointer, and continues from the frozen point. An image is used only on the same kernel, with the same engine binary, libraries and mapped files; otherwise tenkana makes a full run and says why. TENKANA_SAVE_LIMIT (in MB, default 20) sets the largest image kept; 0 turns saving off. Images contain the engine's memory, including its environment variables, which is why tenkana's cache is readable only by you. Measured as an ordinary user: document engine memory on disk restore article, pdfLaTeX 28MB 6.7MB 0.1s article, XeLaTeX 113MB 8.4MB 0.2s article, LuaLaTeX 96MB 13.5MB 0.2s 220-page book, LuaLaTeX 90MB 13.3MB 0.2s talk with a Japanese font 719MB 55.5MB 1.1s (over the default limit) 5. Measured results ------------------- Test files are in testbook/ (run `python3 make_chapters.py' there first). Machine: TeX Live 2023, LuaLaTeX, as an ordinary user. book.tex: 220 pages, 12 \include'd chapters with TikZ, pgfplots, siunitx, hyperref and cleveref. A plain lualatex run takes 8.5s. Medians of three draft builds after an edit: edit in tenkana compared with a plain run chapter 12 0.77s 11 times as fast chapter 11 1.45s 5.8 times as fast chapter 9 2.63s 3.2 times as fast chapter 6 4.54s 1.9 times as fast chapter 1 7.54s 1.1 times as fast preamble 9.09s 7% slower (a full run) nothing changed 0.08s build skipped article.tex (the same preamble, one chapter, 20 pages): 2.07s plain, 0.76s with tenkana after an edit (63% less time). Correctness: after `tenkana --final', the book's PDF was compared with plain lualatex run on the same sources; all 220 pages were identical, also after an edit that changes references across chapters. The same held for pdfLaTeX and XeLaTeX, and for builds restored from disk. 6. Building without snapshots ----------------------------- When snapshots are not possible (no C compiler, not an x86-64 machine), or with --no-snapshots, tenkana builds with plain runs: * pdfTeX, and XeTeX when the preamble loads no system fonts: the preamble is cached once in a private format (mylatexformat) and reused until it changes, unless TeX's first line is TeX code; * LuaTeX, and XeTeX with system fonts: these fonts cannot be stored in a format file, so the preamble is processed normally; * all engines: reruns until references settle with --final, and no run at all when no input changed. 7. Options and environment -------------------------- tenkana [tenkana options] TeX-command [TeX options] file tenkana [tenkana options] -q file tenkana's options come first; the first word that is not one of them starts the TeX command (`--' ends tenkana's options explicitly). -q, --quick the file only, with the default engine: `tenkana -q thesis' is `tenkana lualatex thesis' -F, --final rerun until references, citations and table of contents are up to date -d, --draft one pass; references not updated (the default) -v, --verbose also show passes, changed files, snapshots and their memory --silent show only TeX's output and tenkana's error messages --no-snapshots build with plain runs instead of snapshots -f, --force stop snapshots, discard cached state, build from scratch --clean stop a document's snapshots, delete its cache, and exit. Name the document as for a build (`tenkana --clean lualatex thesis') or just by its name (`tenkana --clean thesis'; thesis.tex and thesis.ltx mean the same). The file itself need not exist any more. -V, --version show version, copyright and license -h, --help show a short help text TeX's options go to TeX unchanged (-shell-escape, -synctex=1, -interaction=batchmode, -file-line-error, -jobname=NAME, ...). tenkana does not support these yet, and says so: -ini, -fmt (and &format), -output-directory, -aux-directory, -output-format and -output-driver. A change of the TeX command makes the next build a full run. TENKANA_ENGINE the command for -q instead of lualatex; it may have options ("pdflatex -shell-escape") TENKANA_SILENT if set, as --silent TENKANA_TTL seconds snapshots live after the document's last build (default 28800, i.e. 8 hours) TENKANA_SAVE_LIMIT largest preamble snapshot kept on disk, in MB (default 20; 0 = never) TENKANA_WATCH more generated-file extensions to watch TENKANA_SHIM use this snapshot library instead of building one TENKANA_DEBUG if set, explain why a snapshot could not be used 8. Limitations -------------- * Snapshots use memory while they live, roughly what the engine itself needs for your fonts and packages. Use `tenkana --clean' when done, or a lower TENKANA_TTL. * Snapshots need \begin{document} in the main file itself. Other documents -- plain TeX, or a main file that \inputs another with \begin{document} -- are built with plain runs (section 6); -v says so. * Chapter snapshots happen only at \include; documents that use only \input get the preamble snapshot. With \includeonly, only the chapters it includes get snapshots. * The TeX options -ini, -fmt, -output-directory, -aux-directory, -output-format and -output-driver are not supported yet. * Snapshots keep the output files (.pdf, .aux, ...) open. If another program replaces one of them (some synchronisation tools do), the snapshots that hold it cannot be used, and tenkana starts from an earlier one or makes a full run. * Only the preamble snapshot is saved on disk; after a reboot the first build resumes from the preamble and makes the chapter snapshots again. * Disk images need the same kernel and TeX installation as when they were saved (checked; otherwise a full run). Where processes may not read their children's memory (Yama ptrace scope 2 or 3), images cannot be saved; snapshots in memory still work. * The engine runs with address-space randomisation switched off, which restoring images requires. This affects only the engine tenkana starts. * tenkana's hooks are added to the start of line 1 of the copy of the main file that the engine reads, so a %&format line or a byte-order mark at the very start of the file is not recognised. * The pause uses read stream 15 (\openin15) for an instant; it would clash only with a document that keeps stream 15 open across \begin{document} or an \include. * \today, time stamps and \write18 side effects come from the run that made the snapshot. * The engine must be dynamically linked (true for TeX Live and the usual distribution packages), because the library is loaded with LD_PRELOAD. 9. Files in this package ------------------------ README this file COPYING the GNU General Public License, version 3 AUTHORS who wrote tenkana NEWS changes between versions MESSAGES every message, the help page and the man page tenkana.1 the man page install.sh builds the single-file `tenkana' and installs it pyproject.toml alternative installation with pip or pipx src/tenkana/cli.py command line src/tenkana/messages.py every text tenkana prints src/tenkana/snapshot.py builds with snapshots src/tenkana/tenkanasnap.c the snapshot library (LD_PRELOAD) src/tenkana/shim.py compiles and locates the snapshot library src/tenkana/fallback.py builds without snapshots src/tenkana/common.py shared helpers tools/catalogue.py regenerates MESSAGES from the source testbook/ test documents (book.tex, article.tex, talk.tex, make_chapters.py) 10. Copyright and license ------------------------- Copyright © 2026 निरंजन (Niranjan) tenkana is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. tenkana is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with tenkana. If not, see . tenkana was written by Claude, a large language model (LLM) made by Anthropic. Courts are divided on whether material generated by an LLM can be protected by copyright. Should it be found that the copyright claimant cannot hold copyright in any part of tenkana, that part is in the public domain. Wherever copyright does hold, tenkana is distributed under the terms of the GNU General Public License, version 3 or (at your option) any later version, as stated above.