LaTex: Continous Background Compilation

What I’m about to propose here is a bit unconventional, but it really works for me. I’ve been doing a lot of front and back end web stuff lately and as a result every machine I own or work with has Node installed by default. Not that I do much in node itself, but because it gives me access to some amazing front-end tools and utilities such as Bower, Yeoman and Grunt. That last one especially has become my go-to build tool as of late. Not necessarily because I love it’s syntax (it’s not great) but because of how it works.

It is a build tool unlike any other: instead of providing users with a monolithic set of build tasks and commands it is completely modularized, allowing you to mix and match both official and community made plugins. And it has a vibrant community that builds plugins for just about everything. Including LaTex.

Combining Latex and Grunt

One of my favorite features of Grunt is the ability to set up watch tasks that monitor your project files for changes and will continuously build and test your code while you work. You can combine it with a live-reload functionality which automatically refreshes your browser. When I work on web projects these days I can automatically see my changes taking effect on the second monitor. I can’t emphasize enough how much does this improve your productivity, and how cool it feels.

I decided I want that kind of setup when I work in LaTex, and it turns out that it can be accomplished using the same tools I use for web development. This is perhaps not a pure LaTex environment, and I am probably committing some horrible transgression here by altering the “proper” Tex workflow. This is definitely not how St. Knuth and St. Lamport intended their tools to be used. But it works really well.

The basic idea is to use Grunt to set up a watch task that will re-compile your document every time you save it. Web-developers probably already know where I’m going with this, but since this is a LaTex post, I’m going to assume you are not familiar with these very web-specific tools. So if you never used Grunt I’m going to explain it step by step.

First, you need to install Node, and then use the built in package manager known as npm to install the Grunt command line tool:

npm install -g grunt-cli

The -g parameter means “global”. By default npm installs packages into node_modules folder in the current directory, but the command line client must be installed globally. While npm is working and spewing all kinds of information into the console, you might go ahead and put that node_modules directory into your .gitignore file.

Now, that you are done with that, go into your project directory and install Grunt proper. Yes, you install it twice. The tool used for issuing build commands is installed globally, whereas the actual meat of the build engine along with all the plugins is installed locally in your project folder. If you think about it, it is a brilliant move as it ensures that every project gets it’s own self-contained environment. Not only that, but you can simultaneously work on two different projects: one which requires a bleeding edge version of the build tool, and one which uses deprecated logic tied to a very old version and they will never conflict with each other.

npm install grunt

It will be installed to the aforementioned local directory:

Installing Grunt

Next, you will need to install at least two plugins. One of them is the official watch plugin which will monitor files and trigger build tasks when they are changed. The other one is an excellent LaTex wrapper written by Tim von Oldenburg:

npm install grunt-contrib-watch grunt-latex

You can specify both plugins on the same line. In fact, you can install both grunt and the associated plugins in one command.

Installing Plugins

The nice thing about Tim von Oldenburg’s package is that it is configured to work with all the popular LaTex distributions so it should work on any platform. I tested it both on Windows and Linux and it was working flawlessly.

Once you have all the components of your build system in place, you can set up your “make” file, here known as a Gruntfile. In most circumstances these build files are written in Javascript and saved as Gruntfile.js in the root of your project directory. But I highly recommend using CoffeeScript instead, because it results in much smaller and readable files. Grunt does not care which of the two languages you use.

Don’t worry if you have never used CoffeeScript – the basic syntax needed to make a Gruntfile is no more complex than that of Make. Here is a very basic Gruntfile.coffee you can use for your project:

# Grunt boilerplate
module.exports = ->
    
    # Set up individual tasks
    @initConfig
        latex:
            src: ['main.tex']
        watch:
            files: 'main.tex'
            tasks: ['latex']

    # Specify tasks you want to use
    @loadNpmTasks 'grunt-latex'
    @loadNpmTasks 'grunt-contrib-watch'

    # Tell grunt what to do if no arguments are specified
    @registerTask 'default', ['latex', 'watch']

Let me explain. Lines 2 and 5 are just standard boilerplate that has to be there in every Gruntfile. In a CoffeeScript they are just two trow-away lines, whereas in the Javascript version they would be two nested closures which are about seven times as scary to someone who is not used to the “sideways x-mas tree” shape that language usually ends up with. CoffeeScript flattens it out a bit and makes it quite more readable.

Lines 13 and 14 declare the plugins that you will be using. We then set up specific option for each plugin in the @initConfig block (lines 6-10). By default the name for each plugin based task is the unique part of the plugin name. So it is latex for grunt-latex and watch for grunt-contrib-watch (all the “official” plugins have the contrib prefix). You use that name to refer to them in the initialization block.

For example the latex plugin requires only one option, which is src and it is used to specify your main .tex file. The watch plugin needs to options. First specifies which files to watch, and second is which tasks to run (which in our case is the latex task).

Finally, the last line defines the “default” taks – which is the one to be run if no tasks are specified on the command line. Normally, you can run a specific task by supplying it to grunt as an argument. For example:

grunt latex

Will run the latex task. If no arguments are specified the task default is run, if it exist. So it is always a good idea to register it in your Gruntfile. I set it to compile the document once, and then watch for changes from that point on. Running grun then throws it into an infinite loop, where it waits for file changes and re-compiles my document on the fly. Observe:

Using Grunt to Live Compile Latex (click to enlarge)

To stop Grunt from it’s active watch-loop you simply hit Ctrl+C.

Most PDF readers will auto-refresh your document when it changes, so as long as you keep it open on the side, or on the second monitor you will see the changes as they stream in. Adobe Reader for some reason locks any file it has open, and prevents LaTex from overwriting it, so you cannot use it with this setup. Pretty much anything else will work fine. In the screenshot above I’m using Evince which is the default PDF viewer on Ubuntu which has a perfectly serviceable Windows version.

You should keep in mind that this method is not perfect. For example, while the grunt-latex plugin is very robust and set up to work in just about every environment, it is not very versatile. It does have some configuration options, but it does not run external commands. This means that if you are using BibTex or a similar package which requires an additional binary to be run before or after latex, the plugin won’t help you.

The good news is that Grunt is incredibly flexible, so it is easy to work around such limitations. For example, in the document I was working on I needed to build a word index. This requires you to run the makeindex command each time you compile. I figured how to make this happen using the grunt-shell plugin which lets you configure a task that will run an arbitrary shell command for you. My Gruntfile ended up looking like this:

module.exports = ->
    
    @initConfig
        latex:
            src: ['main.tex']
        watch:
            files: 'main.tex'
            tasks: ['latex', 'shell', 'latex']
        shell:
            makeIndex:
                command: 'makeindex main.idx'

    @loadNpmTasks 'grunt-latex'
    @loadNpmTasks 'grunt-contrib-watch'
    @loadNpmTasks 'grunt-shell'

    @registerTask 'build', ['latex', 'shell', 'latex']
    @registerTask 'default', ['build', 'watch']

If I needed to add BibTex support I could just another entry under shell and then whenever that task was invoked both the indexing and BibTex tasks would run. Or, I could run them individually by specifying their name after the colon (shell:makeIndex instead of just shell).

What do you think of this setup? How do you compile your LaTex documents? Do you use a makefile of any sort, or do you rely on an IDE to run all the build tasks for you? And if you use an IDE, then which one? Let me know in the comments.