Risk averse Workflows, or Why CEO’s Keep Losing Files

Let’s talk about workflows. If you are a white collar worker, chances are that you spend most of your day creating or editing digital files. Whether you are a programmer, sysadmin, accountant, salesman or a CEO, you will be spending considerable part of your day messing with data, grouped as some sort of logical entity: a document, spreadsheet, source code, etc.. Different people have different strategies and approaches for this sort of work.

For example, I have noticed that my personal workflow is very risk averse. I typically start with a git pull and ends with a git push. As I make changes to a file, I tend to save very often, which is not unusual for programming. When you write code you usually focus on small discrete, incremental changes that need to be tested in isolation. You make an edit, save the document, check if anything broke, make another change, and so on. When you finish working on a specific task or accomplish a specific goal, you commit the code encapsulating the changes into neat snapshot that could be rolled back later. Then you move on to the next thing. Multiple times per day you collect bunch of these snapshots and push them out to a remote repository.

The entire process is anchored not to the local the file system but also to a revision tracking system which provides me with backups and snapshots of my code. It is actually quite difficult for me to lose more than a few minutes of work due to a mistake or a software glitch. I always have at least 3 recent copies of the project: the working copy in storage, the local revision history, and the remote repository. More if I’m feeling adventurous, and I create feature branches which provide yet another working copy that is separate from the main one. It is a very safe way to work.

Busy CEO Workflow

Busy CEO Workflow

This is very different from what I call the “busy CEO workflow” which starts and ends within Outlook. I was recently able to observe several people using this exact Microsoft Office driven workflow and I was baffled how risky and failure prone it was. I would never actually choose to work this way, if nothing else than to save myself the stress and preserve my own sanity.

Let me try to outline this workflow for you:

  1. You start by receiving a Word/Excel document attached to an email
  2. You double click on that attachment to open it
  3. You laboriously make dozens of changes over the span of next 3 hours
  4. When finished you hit “Save and Send” button on the toolbar
  5. Outlook attaches the modified file to a new email

Note how in this particular workflow, all the work is being done almost entirely in memory. When you open a Microsoft Office document attachment from Outlook it renders it opens it directly. It probably puts a working copy somewhere in a local temp folder, but not in a way you could later track down. All the changes you add to the document may or may not be saved to that ephemeral temp file, which will go away the minute you close Outlook.

Microsoft Office does offer you a little bit of protection from glitches and software crashes in terms of the auto-recovery feature (unless of course it was switched off) which will periodically attempt to create a snapshot of your work. If the application does not close cleanly, it prompts you to recover from one of the recent snapshots. Unfortunately these backup copies are immediately deleted when the user deliberately closes application. So if you accidentally close the wrong window, you are likely to lose all the work.

The “save and send” functionality relies on a magical hand-off happening between two office applications that involves passing around references to an ephemeral, temporary file, hidden away from the user. This interaction is semi-reliable but I have seen it break in such a way that it closes the edited document and silently drops the modified file without actually ever giving the user a chance to send it.

This breakage is not an isolated fluke, by the way. The Microsoft Office interop features are known to be rather fragile. Because of their complexity Office applications often end up in weird states which may affect these sort of hand-off situations. In fact, it happened twice in a week when I was working with end users gathering specs for a project. Both times it required closing and re-opening of all Office applications to restore the functionality.

This workflow is fraught with data loss risk and has way to many points of failure:

  • There is no user-accessible “work copy” of the file with recent changes
  • Only life-line is the magical auto-recovery feature
  • The “save” feature is not guaranteed to work all the time

You have got to admit that this is quite bad. If you are a tech savvy person, you know that this is not how one is supposed to work. You are supposed to anchor your work in the storage, not in main memory. You are supposed to save often and keep multiple copies of your work to keep track of changes. And yet, this email-to-email, in place-editing workflow is baked right into the very fabric of Microsoft office. It is easy, convenient and as such it is really appealing to the busy executives who must juggle a lot of balls in the air at all times.

No amount of user education can counteract the “common sense” logic of “if you’re not supposed to use it, then why did Microsoft include it as a feature” counter-argument. Software developers of course know that this fallacious line of reasoning: we put half-baked features into our software all the time, and we don’t always have the time or resources to work through all possible use-cases and usage scenarios. Once the feature is in production, it is hard to remove it.

So the universe is full of half-baked convenience features that don’t really work right. I imagine the “save and send” feature was intended for people who just want to fix 3 typos before approving a staff memo or a courtesy letter of some sort. But but I’ve just seen someone use it to re-write an 80 page report almost entirely, over the course of almost an entire day. That file sat there, in memory when the person took their lunch break, responded to other emails, and worked with dozen other attachments. And that’s quite scary. It is putting a lot of faith in a piece of software…

Which is something I have noticed people do. As a software engineer, the best advice I can probably give you is to never assume any software you use is reliable. It isn’t. Unless it has been developed by NASA for the explicit purpose of flying a rocket into space, then the code is probably a bug ridden mess. And even NASA fucks up every once in a while.

If you consistently lose work due to accidental clicks or software glitches, and someone told you that you can avoid it by modifying your work-flow to route around the flaws in the software, would you do it? Or would you keep your workflow and just be mad at flaky software and the IT staff’s inability to make a third party application do things it was not properly designed to do?

Is there a way to eliminate the busy CEO workflow from your organization? Can you force it out of the system via infrastructure change? Granted, trying to force out Microsoft Office from your organization would be tilting at windmills so that’s probably not a good approach. You will never convince the business folk to give up Word and Excel, but you can sometimes wean people off Outlook. Especially new generations of office workers who grew up on fast, reliable webmail interfaces with endless storage capacities tend to scoff at the very idea of a dedicated email client. And that’s actually a good thing.

For all their flaws, web-mail interfaces do one thing right: they force users to anchor their work in the file system by asking them to save attachments to disk before opening them. This may seem like a major annoyance at first, but that one extra click solves so many issues.

Thoughts? Comments? Ideas?

This entry was posted in sysadmin notes. Bookmark the permalink.

5 Responses to Risk averse Workflows, or Why CEO’s Keep Losing Files

  1. Robert UNITED STATES Safari Mac OS says:

    For almost all users the file system is a horrible place full of dragons and terror. The only way they can deal with it is by double-clicking on files they see in their email or on their desktop, or through the File, Open dialogue in Word and Excel. Ask where their files are saved and you’re likely to get a response like, “In Word,” in a ‘duh, where else?’ tone.

    Reply  |  Quote
  2. Luke Maciak UNITED STATES Google Chrome Linux Terminalist says:

    @ Robert:

    I did have some luck training users to “Save to the Desktop” because files just show up there, and can be found easily. Somehow the Desktop is a more real place than “My Documents” because it can be observed directly I guess. Next step of course is teaching them to keep stuff organized.

    And yes, ten thousand files piled on the desktop without any kind of organization quickly becomes a brand new issue, but at least it is a step in the right direction.

    Reply  |  Quote
  3. some bigger companies/organisations did try to circumvent that whole “send me that file in outlook”-workflow by introducing network-shares/web-tools where documents _have_ to be stored.
    (even going as far as introducing office-plugins that make this the default save/load-location)

    And then there is that whole cloud-bullshit-stuff, where people even sometimes have a finer control about who changed what in which version. So we kinda see the situation getting better.. but in the worst possible way.

    Reply  |  Quote
  4. Luke Maciak UNITED STATES Google Chrome Linux Terminalist says:

    @ Dr. Azrael Tod:

    Network shares do have their own peculiar drawbacks:

    – Office files getting locked when someone is editing them
    – Some asshole accidentally deleting shared folder
    – Version hell when people try to collaborate “New”, “New New”, “Updated”, “Updated New”…

    Also executives love to circumvent this by asking their assistant to email them the file, then mail it back to be put in the right place because they can’t be bothered to navigate a file system.

    But yeah, I think cloud based apps do solve this problem by abstracting the file-system and enabling concurrent collaboration. You do trade off local storage, control and responsiveness of a native application for a flaky web interface that you have no direct control over. :(

    Reply  |  Quote
  5. You can’t help those who won’t help themselves. Most people aren’t really interested in improving their workflow — they don’t see the value of it — so they’re basically unhelpable when it comes to this stuff. They’ll keep doing what they do until either they retire or changes in the software ecosystem forces it (like when people used to store their data directly on the C drive before Windows security stepped it up). I’ve decided it’s not worth spending any energy on these people. You’ll recognize them when you give them a tiny bit of advice (“There’s a shortcut key for that, so you don’t need to dig through three menus all the time.”) and they completely ignore it. This wouldn’t bother me much, but unfortunately it’s usually the people that have these awful workflows that get to decide the standard corporate system configuration (like buying WinZip licenses in 2015).

    Where I’ve had the most success is when someone who is interested in improving gets to watch me in my workflow. They’ll see me do wizardry with Emacs (or some other thing I use) and start asking questions because they hadn’t realized how much easier some particular task can be (I’ve seen people spend hours doing things that should take about one minute). That’s exactly why I like following Emacs blogs and such. I pick up all sorts of new tricks from other people that makes productive work easier and faster.

    Reply  |  Quote

Leave a Reply

Your email address will not be published. Required fields are marked *