Xcopy: Insufficient Memory

Here is an interesting issue I ran into today. I have a script which creates a local copy of one of the important folders on the server using the xcopy command. To make the long story short, after several “accidents” with people messing up or deleting files on the server, we figured that letting people to make local mirrors on their own desktops might be slightly better than restoring stuff back from the tape 4 times a day.

It was something along the lines of:

rmdir /S /Q "%HOMEPATH%\My Documents\Important"
xcopy /E /R /Y /I G:\Important "%HOMEPATH%\My Documents\Important"

The script worked fine for almost a year now. Today however it decided to fail. After copying 1 or 2 folders it would simply stop. After closer inspection of the output I noticed that the last line was:

Insufficient Memory

I immediately checked the memory. The machine had 512 MB of RAM, and over 20 GB of free disk space. The folder I was attempting to copy had only 128 MB. This could not be correct.

Quick google confirmed this – the “insufficient memory” message is very misleading. It appears that this message shows up when the fully qualified (ie. with path) name of the copied file is longer than 254 characters which seems to be Windows maximum path length. Yes – even in WinXP SP2 and Win 2k3. Thank you Microsoft!

So I’m guessing the message doesn’t really refer to the actual free memory, but to the amount of space left in the file name buffer… Or at least I hope that’s what it means. Either that or MS just decided to make “insufficient memory” one of those lovely have a “catch all exceptions and errors here” clauses.

There are essentially 3 workarounds to this:

1. fix your paths – you can try using virtual drives, junctions and all that jazz to make the source and destination paths shorter. Still, this does not mean that a trigger happy user won’t build a crazy long directory tree in your important folder in the future
2. Use xxcopy – a 3rd party xcopy clone. It was suggested by David at blogdom.org. This actually worked for me but I’m not entirely happy with this solution. The good part is that xxcopy implements almost all the regular xcopy switches. The bad part is that to use it for commercial purposes you are supposed to pay up $20 or more.
3. Use robocopy – which seems to be the optimal solution. Robocopy is a part of Windows Server 2003 Resource Kit. The kit will only install on Windows XP and 2003 but robocopy will work on any NT compatible machine. Just install the kit on some machine and copy robocopy.exe from it to all the workstations that need it

The added benefit of using robocopy is that it actually simplified my script:

robocopy \\server\share\Important "%HOMEPATH%\My Documents\Important"

Robocopy can snag the copies directly from the network share, so I do not have to worry about having correct drive letters assigned to the correct shares. This was never really a problem for me since I use login scripts to mount shares on our network, but it is a good thing nevertheless.

In addition robocopy does true mirroring – it will purge all the files in the destination folder which do not appear in the source, saving me the headache of deleting the whole folder prior to copying.

In conclusion – xxcopy works, but robocopy works better. Get it and use it.

[tags]xcopy, xcopy insufficient memory, insufficient memory, xxcopy, robocopy, windows[/tags]