Writing Vim Plugins in Python

There are few things Vim and Emacs users have in common. One of these things is the fact they can talk together about programatically extending their work environment. You can’t really expect to talk to Eclipse of Visual studio about such things, because only a small fraction of them have ever even attempted to write a plugin for their IDE. Many probably installed plugins, but scant few even entertain the notion of writing one? Why? Because it is usually a pain in the ass.

Vim and Emacs users are different – take one at random from each camp, put them in a room and sooner or later you will find them talking about that plugin/extension they wrote for their editor. Why is it that majority of people who use these two tools always eventually end up writing custom plugins for their editor, whereas most other programers never do? Perhaps Vim and Emacs simply attract certain kind of minds to them – inquisitive tinkerers and hackers who simply like to tweak and customize things. Or perhaps it is that both these editors are just so easy to tweak and modify that there is simply no barrier to entry. If you have written your .vimrc from scratch (as every Vim user should – don’t let anyone tell you otherwise) then you already know how to write plugins. In fact, you have probably already written a few without realizing it – all you would need to do now would be to dis-entangle them from your config file and generalize them a wee bit so they can be installed as a self contained module. Same goes for Emacs.

Of course, whenever Emacs user and Vim user start talking about plugin developments, the former is bound by law to smugly condescend about the quality of the respective domain specific languages employed by each editor. Emacs uses Elisp which is a Lisp flavor and therefore superior.

You see, the ancient legend says that before the universe was compiled, the creator deity pondered for a spell, and then on the first day said Let there be parentheses! and he saw they were good. Then on the second day he said Let code be data and data be code and there was much rejoicing. Then on the third day he hacked up a meta-programming macro, and fucked off for the rest of the week as the universe generated itself. Then, millions of years later man invented haskell and was like “What now god?” and god was like “Shiiiiieeet, son!” because in his omnipotence he saw it was also pretty fucking awesome, though mostly in an incomparable to lisp in an apples to barnacles kind of way.

The point is, Emacs users have a pretty cool language they can play around with. Us Vim users, on the other hand, have VimL which is… Neither pretty nor elegant. It has no OOP support, weird scoping, too many overloaded syntax level operators that behave differently based on context, no JIT, and it allows (or even encourages) you to write terrible, unreadable spaghetti code that is worse in terms of readability than Perl. You could probably argue about Elisp’s aesthetic flavor with all the parentheses and what not, but at the end of the day it is rather elegant and consistent environment. VimL is quite the opposite: it is quirky, inconsistent mess of a language that grew along with the editor developing strange warts and tentacles along the way. It is much like a giraffe developed by a committee: while it is supposed to be a giraffe, it is actually an aardvark wearing a pineapple fruit hat and a monocle. It is neither glamorous, nor particularly fun to work with – but it will get the job done if you clench your teeth and try to resist the overwhelming urge to slap it in the face for being so goofy looking.

Fortunately for us, we have options. Modern versions of Vim tend to come pre-baked with support for alternate languages such as Python or Ruby for example. So if we wish, we can get away from VimL and write code that looks and reads nicely. Of course such a thing comes at a trade-off. The general rule of plugin development is that the further you go from the native language the editor “speaks” the worse off you are. In case of Emacs, this doesn’t matter because it was largely written in Elisp (except the few small parts that were coded in C). Vim is mostly implemented in C with VimL already being second class DSL so putting another layer of abstraction between us and the editor can become problematic. So the rule of thumb is that for simple stuff you should stick to VimL and fall back on external language only when you have to do some heavy and/or convoluted processing. VimL is closely tied to Vim but not as expressive as it could be when it comes to representing complex program logic.

Python, on the other hand is. While, perhaps not as elegant as Lisp, Python is nice, strong, opinionated and beautiful language. Opinionated, because among other things, it doesn’t give a shit about your preconceived notions regarding the use of white space. Beautiful, because it is the kind of language that will force you to write code that doesn’t look like shit or die trying. It is the language you need, but perhaps not one you deserve.

Also, because it is older and more mature than Ruby and therefore you have a better chance of it being compiled in your copy of Vim. Mature only in software terms of course – after all making Monty Python references in your code is the accepted development pattern and recommended best practice after all.

How do you know if your Vim can handle Python code? Well, it is easy:

  • If you are outside of Vim you just run vim –version and look for +python which indicates you’ve got it or -python which indicates that whomever compiled your vim package is a major douchenozzle.

  • If you are inside of Vim you can just issue the :py 1 command. If nothing happens, you are good. If you get an error message complaining that py is not an editor command, then… Douchenozzle.

Once you have confirmed that your vim will consume python code, you go about writing a plugin as you would with a VimL one. First, you create a file: say helloworld.vim or something like that. Then, at the top of that file you place some contingency management code:

if !has('python')
	finish
endif

This checks if the user’s editor has the Python support baked in, and silently quits if it does not. This is a nice thing to do, since it prevents someone’s editor from blowing up. Chances are the person who will be installing your plugin doe not read your README file and thus does not realize he needs python support, and therefore his Vim will blow up with a barrage of error messages next time he launches it.

Of course we could try being more explicit, and display an error message along the lines of “Y U has no +python!??” but I personally prefer the silent failure. Don’t get me wrong, explicit is almost always good, but I hate when plugins spam me with error messages on start-up. I keep my .vim/ directory under source control and I clone it onto every machine I work on. Sometimes I don’t have the luxury of using the latest and greatest Vim with all the fancy options. I don’t like to build specialized configs for those special cases – I’d rather use my usual setup, and just have the fancier stuff fail silently and simply be unavailable, rather than vomit stream of warning messages at me every time I do something. But that’s just me.

Next we want to expose some interface to the user in VimL so they can, for example bind it to a command or a key. That part has to be done in VimL. Think of it as bridging the divide between the native language and the external one. Usually, the best way to expose some functionality to the user in an unobtrusive way is to create a publicly accessible function:

function! HeloWorld()
 
endfunc

In general, you can access the Python environment in Vim by issuing the :python command (short version :py). You could technically write an entire plugin by simply prefacing every line with that command, but that would be extremely ugly and kinda crazy. Instead we can do something like this:

function! HeloWorld()
 
python <<< EOF
 
print "Hello World!"
 
EOF
endfunc

Now you can do :source % to load it up, and call HelloWorld() to test it. If all went well, you should see the text Hello World in the status line.

Granted, this is still kinda ugly. Mixing VimL and Python in a single file is definitely non-kosher, and it completely defeats the purpose of this entire exercise: to write plugins in a more elegant language, and make them more readable. Fortunately, there is another command we can use to load up an external python file, and execute it in Vim context: :pyfile (or :pyf). So our ugly function can be reduced to something like:

function! HeloWorld()
    pyfile helloworld.py
endfunc

“That is all well and good,” I can hear you say “but what does this do for us?” To actually do anything interesting as a plugin, your Python code needs access to internal Vim stuff – like the buffer list, or the contents of said buffers. How do we get that?

Well, we get it via the magical vim module. It is magical because it isn’t really a module that you can go and download or install in the usual way. It doesn’t exist on the outside, and can only be imported when the script is called from within Vim. This is both good and bad. It is good, because it makes it completely plug-and-play. If you have Vim with +python there is nothing else you need to do: it just works. On the other hand, it is bad because it requires Vim to work. This means you can’t run your code outside the context of your editor. For example, you may not be able use your favorite Python unit-testing suite to run test on your plugin, unless you mock up the Vim module functionality first. Which in retrospect is what you probably ought to do anyway so I guess it works out.

The vim module exposes bunch of useful objects such as vim.buffers or vim.current.buffer that you can use to manipulate your buffers, files and settings. The current buffer for example is actually a zero indexed array, each element representing a line as a string. Any changes to the elements in said array are immediately reflected on the screen. I will let that sit with you, so you can just imagine the possibilities for fun and abuse here.

If you want to know all the available objects and their details, I suggest you RTFM. Just as an exercise however, lets write a quick and dirty python plugin together.

I work with Markdown files a lot, so I always run them through something like Pandoc to check how will they look in HTML format. Let’s say I wanted to create a plugin that takes current buffer and dumps it into HTML format without the need for Pandoc or any other converter to be installed on the system. Because I’m lazy and I don’t feel like writing actual Markdown converter myself I will just use this Markdown package.

So, first let’s create a VimL plugin file that will load our Python code and call it, say markdown2html.vim:

if !has('python')
	finish
endif
 
function! Markdown2HTML()
	pyfile markdown2html.py
endfunc
 
command! MKD2HTML call Markdown2HTML()

The last line actually binds my function call to a Vim command providing an easy way for the user to run it. Next let’s write our python code:

try:
    import markdown
    import vim
    import string
 
    html = string.join(vim.current.buffer, "\n")
    html = markdown.markdown(html, output_format='html5')
 
    filename = vim.current.buffer.name+".html"
 
    f = open(filename, 'w')
    f.write(html)
    f.close()
 
    print "HTML output written to "+filename
 
except ImportError, e:
    print "Markdown package not installed, please run: pip install markdown"

Bam! Instant plugin. How awesome is that?

Of course the downside of using Python to write your plugins is that the users will need to have their editor compiled with +python in order to use them. Also, every time you use a third party Python package (like I just did above) your users will need to actually install that package on the system. So if you actually try to implement the above, don’t forget to pip install markdown at some point beforehand.

Still, the ability to bring the power of Python and all of it’s packages and modules into your Vim plugins is not something one should scoff at. If you would like to see a real life example of an excellent and popular plugin written in Python, go check out Gundo. It’s actually a plugin I use all the time as I work.

This entry was posted in programming and tagged . Bookmark the permalink.



3 Responses to Writing Vim Plugins in Python

  1. JuEeHa FINLAND Links Linux says:

    I believe it should be “Writing”, not “Vriting”.

    Reply  |  Quote
  2. Luke Maciak UNITED STATES Google Chrome Linux Terminalist says:

    @ JuEeHa:

    Sigh… And I thought I was clever with the alliteration.

    VVriting :)

    Reply  |  Quote
  3. Great read! I absolutely lost it at “or -python which indicates that whomever compiled your vim package is a major douchenozzle”

    Reply  |  Quote

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>