Long Path Madness

Most people never run into the MAX_PATH issue when using windows. I run into it every other week. Windows has thins funky little quirk about long file paths. The fully qualified path of any file on your NTFS file system is supposed to be under 256 characters. It’s a weird limitation, and hell – if your path is 250 characters long, you are probably doing something very wrong, right?

No, not really. For example, consider this sample path:

C:\Documents and Settings\Johnathan Smith\My Documents\Work Documents\

Let’s say our imaginary friend John Smith keeps all his work related documents in a folder called Work Documents in his My Documents directory. Seems reasonable, isn’t it? Note that path to his work folder is already 70 characters. I’m not doing anything outrageous here – that’s just how it is.

Let’s say John is very organized and he keeps various documents in aptly named sub folders. For example he has a folder named “Reports” and inside couple of sub folders like “For Review” and “Reviewed”. Let’s say he dumps new reports in the “For Review” folder as he gets them. Since he gets several of them each day he keeps them organized by month and company name. So a typical path he might be working with on any given day could be something like this:

C:\Documents and Settings\Johnathan Smith\My Documents\Work Documents\Reports\For Review\Jan 2008\Initrode International Corporation\

If you are counting this is already whopping 134 characters. This is still a very simple hierarchy – no exceedingly huge folder names, no redundant sub folders and nothing out of oridinary. And yet, it is already huge. Now let’s assume that John gets a report in a zip file aptly named something like:

Initrode International Corp Field Report Jan-13-07 by JB and RF.zip

Yes, it’s a little bit long, but this is not an outrageous file naming convention. Some managers do insist on using a scheme like [project name] [document type] [date] [name of submitter] for all the data you send into the office. John diligently extracts this zip file and gets another hierarchy unrolled in his work folder. For example the file he is interested in might be located in following set of folders:

\Workpapers\Accounts Receivable\Dec 2007\Initrode Accounts Receivable Aging Schedule – Projected 12-07.doc

Nothing out of usual here – granted, we could shorten the file name, and instead of spelling out Accounts Receivable we could just say AR, but honest to goodness I do see file paths like that at work all the time. So let’s put the two paths together and see where it takes us:

C:\Documents and Settings\Johnathan Smith\My Documents\Work Documents\Reports\For Review\Jan 2008\Initrode International Corporation\Initrode International Corp Field Report Jan-13-07 by JB and RF\Workpapers\Accounts Receivable\Dec 2007\Initrode Accounts Receivable Aging Schedule – Projected 12-07.doc

Vim tells me that this path is exactly 302 characters, and Vim usually doesn’t lie. This puts us way beyond the MAX_PATH but windows won’t really care. It will let us create this path by unzipping, and it won’t complain. The fun starts if our friend John decides to open this file in Word and edit it. He won’t be able to save it, no matter how he tries and he will naturally call the IT department at exactly 5:59 pm on a Friday and cry that if he can’t save this file he will loose at least 7 hours of work if not more. I know this, because it happens all the time – and always on Firday as I’m walking out the door.

There is a quick workaround of course – save the file in another folder for example. And according to MSDN you can actually prepend \\?\ to the file name to force windows to use a Unicode based API call which supports path lengths up to 32,000 characters. And of course you can use the eight-dot-three path which is naturally shorter. None of this done by default though.

Part of the problem here is that prerequisite My Documents path which usually weighs in around 50-60 characters. Compare it to the default userspace path in linux which is usually /home/username and can be abreviated as ~. I hear Vista puts user files in a little bit saner directory structure.

Still, while 256 seems like a lot, it is clearly not. A simple folder hierarchy that is only 7 levels deep (at least from John’s perspective) can easily, and without warning hit that magic boundary. I’m not advocating creating complex hierarchies – they do cause problems, but that is a topic for an entirely separate rant. The point is that in the real world using verbose names for files and folders is not something uncommon. In fact it is often considered a desirable practice that improves readability and organization. Microsoft is obviously aware of this issue, since they build in that \\?\ workaround. I wonder why isn’t there some sort of a fall-back mechanism here.

Is it really this hard to catch the MAX_PATH error at the OS level, and then re-try it using the Unicode extended call? This way most of the file system is served with the legacy 256 character capable mechanic, which I suspect might be faster, or more direct. Those special cases would then be handled with the secondary call – completely in the background and transparently to the user. But I guess this is to much to ask. :P

[tags]windows, max_path, maximum path length, windows path length, windows xp[/tags]