LaTex: Continous Background Compilation

What I’m about to propose here is a bit unconventional, but it really works for me. I’ve been doing a lot of front and back end web stuff lately and as a result every machine I own or work with has Node installed by default. Not that I do much in node itself, but because it gives me access to some amazing front-end tools and utilities such as Bower, Yeoman and Grunt. That last one especially has become my go-to build tool as of late. Not necessarily because I love it’s syntax (it’s not great) but because of how it works.

It is a build tool unlike any other: instead of providing users with a monolithic set of build tasks and commands it is completely modularized, allowing you to mix and match both official and community made plugins. And it has a vibrant community that builds plugins for just about everything. Including LaTex.

Combining Latex and Grunt

Combining Latex and Grunt

One of my favorite features of Grunt is the ability to set up watch tasks that monitor your project files for changes and will continuously build and test your code while you work. You can combine it with a live-reload functionality which automatically refreshes your browser. When I work on web projects these days I can automatically see my changes taking effect on the second monitor. I can’t emphasize enough how much does this improve your productivity, and how cool it feels.

I decided I want that kind of setup when I work in LaTex, and it turns out that it can be accomplished using the same tools I use for web development. This is perhaps not a pure LaTex environment, and I am probably committing some horrible transgression here by altering the “proper” Tex workflow. This is definitely not how St. Knuth and St. Lamport intended their tools to be used. But it works really well.

The basic idea is to use Grunt to set up a watch task that will re-compile your document every time you save it. Web-developers probably already know where I’m going with this, but since this is a LaTex post, I’m going to assume you are not familiar with these very web-specific tools. So if you never used Grunt I’m going to explain it step by step.

First, you need to install Node, and then use the built in package manager known as npm to install the Grunt command line tool:

npm install -g grunt-cli

The -g parameter means “global”. By default npm installs packages into node_modules folder in the current directory, but the command line client must be installed globally. While npm is working and spewing all kinds of information into the console, you might go ahead and put that node_modules directory into your .gitignore file.

Now, that you are done with that, go into your project directory and install Grunt proper. Yes, you install it twice. The tool used for issuing build commands is installed globally, whereas the actual meat of the build engine along with all the plugins is installed locally in your project folder. If you think about it, it is a brilliant move as it ensures that every project gets it’s own self-contained environment. Not only that, but you can simultaneously work on two different projects: one which requires a bleeding edge version of the build tool, and one which uses deprecated logic tied to a very old version and they will never conflict with each other.

npm install grunt

It will be installed to the aforementioned local directory:

Installing Grunt

Installing Grunt

Next, you will need to install at least two plugins. One of them is the official watch plugin which will monitor files and trigger build tasks when they are changed. The other one is an excellent LaTex wrapper written by Tim von Oldenburg:

npm install grunt-contrib-watch grunt-latex

You can specify both plugins on the same line. In fact, you can install both grunt and the associated plugins in one command.

Installing Plugins

Installing Plugins

The nice thing about Tim von Oldenburg’s package is that it is configured to work with all the popular LaTex distributions so it should work on any platform. I tested it both on Windows and Linux and it was working flawlessly.

Once you have all the components of your build system in place, you can set up your “make” file, here known as a Gruntfile. In most circumstances these build files are written in Javascript and saved as Gruntfile.js in the root of your project directory. But I highly recommend using CoffeeScript instead, because it results in much smaller and readable files. Grunt does not care which of the two languages you use.

Don’t worry if you have never used CoffeeScript – the basic syntax needed to make a Gruntfile is no more complex than that of Make. Here is a very basic Gruntfile.coffee you can use for your project:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Grunt boilerplate
module.exports = ->
 
    # Set up individual tasks
    @initConfig
        latex:
            src: ['main.tex']
        watch:
            files: 'main.tex'
            tasks: ['latex']
 
    # Specify tasks you want to use
    @loadNpmTasks 'grunt-latex'
    @loadNpmTasks 'grunt-contrib-watch'
 
    # Tell grunt what to do if no arguments are specified
    @registerTask 'default', ['latex', 'watch']

Let me explain. Lines 2 and 5 are just standard boilerplate that has to be there in every Gruntfile. In a CoffeeScript they are just two trow-away lines, whereas in the Javascript version they would be two nested closures which are about seven times as scary to someone who is not used to the “sideways x-mas tree” shape that language usually ends up with. CoffeeScript flattens it out a bit and makes it quite more readable.

Lines 13 and 14 declare the plugins that you will be using. We then set up specific option for each plugin in the @initConfig block (lines 6-10). By default the name for each plugin based task is the unique part of the plugin name. So it is latex for grunt-latex and watch for grunt-contrib-watch (all the “official” plugins have the contrib prefix). You use that name to refer to them in the initialization block.

For example the latex plugin requires only one option, which is src and it is used to specify your main .tex file. The watch plugin needs to options. First specifies which files to watch, and second is which tasks to run (which in our case is the latex task).

Finally, the last line defines the “default” taks – which is the one to be run if no tasks are specified on the command line. Normally, you can run a specific task by supplying it to grunt as an argument. For example:

grunt latex

Will run the latex task. If no arguments are specified the task default is run, if it exist. So it is always a good idea to register it in your Gruntfile. I set it to compile the document once, and then watch for changes from that point on. Running grun then throws it into an infinite loop, where it waits for file changes and re-compiles my document on the fly. Observe:

Using Grunt to Live Compile Latex

Using Grunt to Live Compile Latex (click to enlarge)

To stop Grunt from it’s active watch-loop you simply hit Ctrl+C.

Most PDF readers will auto-refresh your document when it changes, so as long as you keep it open on the side, or on the second monitor you will see the changes as they stream in. Adobe Reader for some reason locks any file it has open, and prevents LaTex from overwriting it, so you cannot use it with this setup. Pretty much anything else will work fine. In the screenshot above I’m using Evince which is the default PDF viewer on Ubuntu which has a perfectly serviceable Windows version.

You should keep in mind that this method is not perfect. For example, while the grunt-latex plugin is very robust and set up to work in just about every environment, it is not very versatile. It does have some configuration options, but it does not run external commands. This means that if you are using BibTex or a similar package which requires an additional binary to be run before or after latex, the plugin won’t help you.

The good news is that Grunt is incredibly flexible, so it is easy to work around such limitations. For example, in the document I was working on I needed to build a word index. This requires you to run the makeindex command each time you compile. I figured how to make this happen using the grunt-shell plugin which lets you configure a task that will run an arbitrary shell command for you. My Gruntfile ended up looking like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
module.exports = ->
 
    @initConfig
        latex:
            src: ['main.tex']
        watch:
            files: 'main.tex'
            tasks: ['latex', 'shell', 'latex']
        shell:
            makeIndex:
                command: 'makeindex main.idx'
 
    @loadNpmTasks 'grunt-latex'
    @loadNpmTasks 'grunt-contrib-watch'
    @loadNpmTasks 'grunt-shell'
 
    @registerTask 'build', ['latex', 'shell', 'latex']
    @registerTask 'default', ['build', 'watch']

If I needed to add BibTex support I could just another entry under shell and then whenever that task was invoked both the indexing and BibTex tasks would run. Or, I could run them individually by specifying their name after the colon (shell:makeIndex instead of just shell).

What do you think of this setup? How do you compile your LaTex documents? Do you use a makefile of any sort, or do you rely on an IDE to run all the build tasks for you? And if you use an IDE, then which one? Let me know in the comments.

This entry was posted in programming and tagged . Bookmark the permalink.



7 Responses to LaTex: Continous Background Compilation

  1. gruntless FRANCE Safari Mac OS says:

    To live-compile a LaTeX document I just do: latexmk -pvc and preview in Mac OS X’s Preview.

    Reply  |  Quote
  2. Call me old fashioned, but I’d use the “watch” command for this. :-) I always set up a Makefile anyway.


    output.pdf : output.tex
    $(PDFLATEX) $<

    So running “watch -n1 make” is trivial. The Makefile ensures the PDF only updates as needed, and, as you mentioned, Evince (my favorite PDF viewer), will automatically refresh when the PDF updates. Usually I don’t want the automatic updates, though, and instead I run make from within Emacs with a keystroke, which is immediately reflected in the adjacent Evince window.

    If watch is too much of a hack for you, you can always use an inotify command line program instead.

    Reply  |  Quote
  3. Luke Maciak UNITED STATES Google Chrome Linux Terminalist says:

    @ gruntless:

    Does latexmk use pdflatex or just latex?

    @ Chris Wellons:

    Wow, that’s much simpler and much more unixy. :P Thanks.

    Btw, can you explain $< ensure that the file is not recompiled every second? I don’t think I’ve seen this makefile/shell idiom before.

    Reply  |  Quote
  4. @ Luke Maciak:

    The $< is an automatic variable that is equal to the first prerequisite (in this case it’s “output.tex”). I used it to stay DRY. Think of is like the less-than bracket in a Unix shell, providing stdin from a file. The command to build the target becomes “pdflatex output.tex” with the variables filled in.

    Note that there’s a tab not being displayed in my example Makefile code block, so that second line should be tabbed out. Here’s a more elaborate pastebin version with proper formating: http://pastebin.com/hDaacnJ1.

    The PDF isn’t recompiled every second simply because make will see that the .pdf file is newer than the .tex file. It will only perform the build when a prerequisite is updated, otherwise it just checks the timestamps and exits. This is one of make’s core features.

    Reply  |  Quote
  5. Luke Maciak UNITED STATES Google Chrome Linux Terminalist says:

    @ Chris Wellons:

    Ah, got it. Makes perfect sense now. :)

    Reply  |  Quote
  6. Philipp GERMANY Google Chrome Linux says:

    @ Luke Maciak:
    You should really learn about make and do a series about it.
    If people new more about make they would not waste so much time setting up complex build tools for small projects when a make file and some simple scripts would do the job.

    If it is powerful enough for the Linux kernel it is probably powerful enough four you.

    Reply  |  Quote
  7. Agn0sis CANADA Mozilla Firefox Linux says:

    @ Luke Maciak:

    Luke Maciak wrote:

    @ gruntless:
    Does latexmk use pdflatex or just latex?

    latexmk can use basically whatever tool you want. Many of the programs to use are going to be inferred by latexmk itself, some of them can be specified with options when executing it, and some of them can be specified in a config file. For example, to compile and watch changes each time I save the file, I create a config file named .latexmkrc with the contents:


    $pdf_previewer = 'start okular %O %S';

    where okular is the pdf viewer I’ll use, but you can use whatever you want.

    The I just run:


    latexmk -xelatex -pvc world_domination

    In my work directory, the main latex file is world_domination.tex

    -pvc is the option to tell the program to keep compiling and updating the pdf, and -xelatex is … well, an option to specify that I work with that tool instead of the pure latex.

    latexmk is usually intelligent enough to know what programs to call and how many times (for cross-referencing)

    Reply  |  Quote

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>