Revisiting Private Journaling

A while ago I wrote about my search for a good private journaling solution. Since then I have tried dozens of different apps and services without finding anything I would like. At one point I even wrote my own little Jounraling client thinking I could hit the sweet spot. Unfortunately I was wrong, and my own design turned out to be just as bad as the competition.

Looking at it in retrospect, now I know that I have been trying to solve the wrong problem. I was designing a generic journal tool, rather than building something specific to my own needs. Part of the problem was that I wasn’t exactly sure how exactly I was going to be using this tool. I had an idea of a long form journal I wanted to write but that’s not what I ended up doing. I guess I should have started with a journal and built a tool around it rather than the other way around.

It turns out that creating long exhaustive daily posts was not something I was enjoying or what I had time for. I seem get most of the long form rants out of my system via this blog (and the few other public sites I run) – that’s sort of where I channel most of my clever musings. After externalizing all of my interesting musings via these public outlets, the the remainder was just boring, trivial or redundant.

I don’t actually enjoy chronicling every minute of my day. While there would be some utility to keeping a log of things I have done each day for future reference, actually putting these things down on paper are needless drudgery to me.

I actually enjoy browsing archives of my own blog because it takes me back to those old moments. Since I tend to write about things I love or care about all of the posts have a good deal emotional weight attached to them. I found out that creating more detailed and personal accounts of my daily life had the opposite effect – I noticed I was mostly chronicling my mistakes and being overly critical of myself. It seemed like a good idea at the time (self improvement and all) but reading it back would just bum me out and force me dwell on past failures.

That combined with the fact that on your average day I usually don’t have the time or inclination to play a recollection game to write a five thousand word essay of useless daily trivia made me stop trying altogether. This sort of journal does not work at all for me.

What does work? Well, I realized that for a Jounral to work it must meet the following requirements:

  1. First and foremost, the entries have to be short. I’d rather write a bunch of twitter style thought-bursts throughout the day, than a long thoughtful essay at the end of the day.
  2. Entries should be positive in nature – random musings, reflections, celebrating accomplishments, writing down jokes, etc.. They should concentrate on what I’m into at the moment, and what I want to remember about a given day years from now.
  3. Because it is sort of a continuously updated stream, it should be accessible over the internet. So it ought to be either web based, or stored in the Cloud somehow.
  4. Since I don’t always have a computer with me to jot down my thoughts, the journal should also have a dedicated iPhone App or be accessible via mobile friendly web interface.
  5. It should be stored in plain text files and not tied to a single client, unless that client is open source and available on all platforms. This is especially important with respect to the iOS based journaling tools. While there are a lot of great little diary type apps in the AppStore, most of them either store your data on their own servers, or use some proprietary file format. They are also frequently either exclusive to iOS or OSX/iOS combo leaving Windows and Linux out completely.

I gave up on my encryption requirement which I mentioned in the old post for the very simple reason: I forget passwords. If I keep my accounts reasonably secure, then I shouldn’t worry about setting per-file password on bunch of text files. The aim of a journal is to have something to aid in recollection and memory retrieval long time down the road. Chances are that in a decade, I will no longer remember the passwords I was using circa 2013. So I run a risk of locking myself out of my own journals which seems counterproductive.

Looking at my list of requirements, it almost seems that the best tool for the job would simply be Dropbox and a text editor of some sort. I could use Vim when on an actual computer, and something like Plain Text App on the phone. Unfortunately this sort of solution does not have any built-in automation. I would have to date my entries myself, which may not seem like a huge issue at first. After all, this is what people have done when they used dead-tree medium to do this.

The thing is I don’t really want a traditional “journal”. I want something like Twitter, but without the character limit (so like Pownce I guess) that saves to plain text files in a Dropbox folder. I want my entries tagged with date and time automatically, so I don’t have to think about it. I just don’t want to be bothered typing in a date/time string into a text editor 6-7 times a day.

Surprisingly enough, a guy called by Matthew Lang built a tool that is almost perfect for my needs. His creation is called Journalong and it is basically just a simple text box that sends the data to Dropbox:

Journalong

Journalong

It does everything I need, but it is not without flaws:

  1. I don’t know who Matthew Lang is and how much should I trust him not to read my entries.
  2. Matthew wants you to pay $10 per year, for the privilege of using his app. This may seem like a bargain, until you actually realize how little code is needed to build the application he built.
  3. Matthew never set the DNS record for his wwww subdomain, so half the time when my browser auto-completes the URL for me I get sent into cyberspace void.
  4. Journalong has no apple touch icon, so when I put a link to it on my iPhone it looks like absolute shit.
  5. Matthew seems to be a fan of military time. I want my entries to be dated using the 12 hour format but there isn’t a way to do this.

I sent Matthew an email asking if he ever plans to do anything about the last 3 of these issues about a week ago, but it was completely ignored. This puts #1 and #2 on the list in perspective. In this day and age, majority of the shit I use on the web is free. If I was to shell out money for this simple app, I at least ought to get some support… Otherwise, why not write something like this myself.

No, seriously. I’ve been using Journalong’s free trial and I actually kinda like the functionality it provides. But it is super simple, and the more I used it the more I realized I could actually do better. So I sat down and implemented my own Dropbox Journal in approximately 150 lines of Ruby (not counting HTML markup). I called it Markdown Journal.

Markdown Journal

Markdown Journal Screenshot

How does it differ from Matthew Lang’s application?

For one, Markdown Journal is open source. You can find the code on Github and easily verify I’m not doing anything shady with your data. Secondly, it is completely free and I intend it to stay that way.

In all honesty, I would actually feel bad charging anyone a recurring fee to use something that took me about two evenings to put together, and is shorter than about 150 lines of code, plus some HTML. The webapp runs on the excellent Sinatra miniframework and uses the official Dropbox SDK gem for authentication and file uploading. It was really simple and fun to put together and I’m hosting it on a super cheep shared host, along side some other sites because it really doesn’t require that much resources, nor do I see it ever becoming terribly popular.

I also have Matthew beat on features. My date/time headings in the file are completely configurable via a simple YAML file you can drop into the same directory you will be keeping your journal files in. Granted, creating text files is scary to some people, but chances are if your choose to write a journal in Markdown format and host it on Dropbox you can probably figure out how to create a YAML file based on an example on the site.

Finally, I have properly sized apple touch icons so when you save the link on your home screen it actually shows a proper icon rather than a blank square.

Let me know what you think. Would this tool be useful to you? What do you use for note-taking and journaling? Let me know in the comments.

This entry was posted in programming and tagged . Bookmark the permalink.



17 Responses to Revisiting Private Journaling

  1. Peter SWITZERLAND Safari Mac OS says:

    When I try to authorize in Dropbox I get this error message: “Only a limited set of users can receive access tokens while this app is in development mode”

    Reply  |  Quote
  2. Daan Bakker NETHERLANDS Google Chrome Ubuntu Linux says:

    I actually followed exactly the same thought process as you, but for me there is another really important requirement: the backup has to be incremental. I don’t want my notes to be corrupted someday and have dropbox happily sync them up to the cloud. Another thing is that I don’t always have internet so I need a way to ‘merge’ notes. I then realized I had the perfect tool already installed: git! Together with a free bitbucket account, you are very sure never to lose your stuff! Another good free cloud storage provider is spideroak. They have client side encryption and incremental backup as well.

    Reply  |  Quote
  3. fundamental UNITED STATES Mozilla Firefox Linux says:

    While it does not exactly meet the requirements you setup in your previous post, I think that pencil and paper is a nice method to keep a journal with various notes. I find it preferable to any electronic solution as I don’t have a smartphone (nor will I unless energy storage tech. makes a huge leap) and it is something that is easy to keep on my person at all times.

    I don’t really track things based upon their dates as I can never quite place when events happened in the past in absolute terms, so I generally just flip through the pages until I find the information I’m looking for or if I know that I’ll look for it in perhaps a few months I put it in a dedicated section. While not automated, some journals do have tables of contents if you do want to track when you are writing about various things. Another key advantage for me is being able to draw out various ideas and thoughts as my use includes design sketches and some general doodling which tend to bring back ideas much more quickly than just my scrawl.

    I didn’t see any mention of this low low tech choice in the previous post. Any reason why you have eliminated paper/pencil as an option?

    Reply  |  Quote
  4. FX FRANCE Google Chrome Mac OS says:

    Interesting, I really have to learn some Ruby! I totally support Markdown, and I was using Pelican a while ago for something that’s somewhat close to what you did. I stored my journal under a Git mirror on my development server though, but that’s just me being used to it.

    Reply  |  Quote
  5. Luke Maciak UNITED STATES Google Chrome Linux Terminalist says:

    @ Peter:

    Thank you sir! I applied for production status today, so that request is still pending. This is actually my first time using Dropbox SDK so I’m not entirely sure how long that might take.

    @ Daan Bakker:

    Hmmm… That’s a good point. Dropbox does have some limited versioning support, but it would be easy for file corruption to occur unnoticed and persist over a long time period rendering that feature useless.

    @ fundamental:

    Actually there is a whole bunch of reasons:

    - I have terrible hand writing.
    - I actually can type faster than I can write long hand.
    - Electronic files can be sorted by date, and searched with grep. Physical notebooks cant.
    - There is no easy way to back up a paper journal.
    - There is no easy way to copy and paste from/to paper journal.
    - All my electronic devices are password protected. Paper journal is world readable.
    - I never seem to have pen/paper on me, but always have my smart phone.
    - If I ever wanted to compile my journal into an “easy to read” single document memoir I could simply concat all my markdown files and pipe them through pandoc.
    - Paper notebooks are rather volatile – one spill and half your work is gone. Digital files can potentially stay forever as long as you have redundant copies and keep moving them around.

    So yeah…

    @ FX:

    I haven’t really done much Ruby in the past. Then I was messing around with the static site generators like Jekyll and Nanoc, and got to writing custom Rake scripts to make updating and deploying these sites easier. Next thing I knew I was knee deep in Ruby and loving it. It is a really nifty language. :)

    Reply  |  Quote
  6. Luke Maciak UNITED STATES Mozilla Firefox Windows Terminalist says:

    @ Peter:

    Hey, just a follow-up on this – the app just has been approved for production so you should be able to use it now. Can you try again and let me know if it worked?

    Reply  |  Quote
  7. Peter SWITZERLAND Safari Mac OS says:

    @ Luke Maciak:

    Yes it works.
    Thank you.

    Reply  |  Quote
  8. Mitlik UNITED STATES Mozilla Firefox Windows says:

    Luke I think there is an error in your program. I just got a post I didn’t generate. It involved EA games and mustaches. ^_^

    Reply  |  Quote
  9. Luke Maciak UNITED STATES Google Chrome Linux Terminalist says:

    @ Mitlik:

    Ya, session collision. This is very un-good. :(

    Reply  |  Quote
  10. Mitlik UNITED STATES Mozilla Firefox Windows says:

    @ Luke Maciak:
    In a positive light, I tried it because it sounded neat. I don’t have to worry too much about exotic platforms not having a smartphone, but having a web front end to something that handles the file creation and time stamps to a private location is appealing.

    Reply  |  Quote
  11. Luke Maciak UNITED STATES Mozilla Firefox Windows Terminalist says:

    @ Mitlik:

    Ok, I believe I fixed it. It was a rookie mistake really. Classic example of “works on my machine so… ready for production” type attitude. Now I think sessions are properly handled so this shouldn’t happen anymore.

    Reply  |  Quote
  12. Mitlik Mozilla Firefox Linux says:

    @ Luke Maciak:
    I’ll try to leave it open in the background at work tomorrow! Also, I am CST, so if you wanna try to recreate a collision it shouldn’t be around 11am EST.

    Reply  |  Quote
  13. Dirk Krause GERMANY Google Chrome Mac OS says:

    Hi,

    I just stumbled accross https://github.com/maebert/jrnl. It works from the command line and is compatible with Day One.

    – Dirk

    Reply  |  Quote
  14. Sam UNITED KINGDOM Mozilla Firefox Windows says:

    I stumbled upon your blog today – weirdly enough your ‘Firewall Saga’ was posted on an IRC chan that I was idling on.
    Safe to say, it amused me for far too long at work!

    Anyways, your Markdownjournal is fantastic. Thought i’d give it a go for a while to see if I really do use it. Website layout if very nice too – love the bootstrap layout

    Thanks mate

    Reply  |  Quote
  15. Mitlik UNITED STATES Mozilla Firefox Windows says:

    Sorry I pestered the heck out of you after you posted this. But now that it’s seems like it’s working you can get back to teaching me how to PHP like a Pro again! ^_^

    Reply  |  Quote
  16. Luke Maciak UNITED STATES Google Chrome Linux Terminalist says:

    @ Dirk Krause:

    Thanks. Looks pretty good, but the problem with that tool is that it seems to be designed only for very short entries and it provides no spell-check and no editing capabilities. I’d rather compose my entries in a text editor or in a browser input box where I at least have inline spell-checker available.

    @ Sam:

    Thank you! Btw, if anyone is interested the Firewall Saga starts here.

    @ Mitlik:

    No, I’m actually really grateful. This was extremely helpful and we got a lot of bugs out. Thanks! :)

    Reply  |  Quote
  17. Manuel UNITED STATES Google Chrome Mac OS says:

    @ Luke Maciak:

    Author of jrnl here. You can use jrnl with any text editor you like, for example Sublime Text or even just TextEdit – there’s a config setting that tells it to launch an external editor ;-)

    Reply  |  Quote

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>