One of the major issues I had with it was using it behind corporate firewalls. When you are on a network that only allows outbound communication on port 80 and 443, BTSync is completely useless. Unlike Dropbox or Google Drive which both have absolutely no issues synchronizing folders in such environment, the BTSync client simply does not work at all.

And yes, before you say anything, there are reasons to block outbound traffic on port 22. Firstly, if on average the number of users who need to ssh out of that location approaches zero, then leaving that port open simply increases the attack space for no reason. Secondly, even if users do need to ssh out, chances are they will be communicating with known servers. Why have a wide open port that can be used an abused, when you can control connections on IP and MAC address basis, and require audit trail, and change-of-permission request documentation when devs ask for more access.

The only outbound ports that are typically wide open are HTTP and HTTPS. Your local BOFHs can’t readily lock them down as tight as they would want to, unless they set up a proxy server. Fortunately, proxies break a lot of the modern, dynamic, internet based things so chances are you might not have one. And if you do not, then you can funnel your BTSync traffic through an SSH tunnel on a HTTP/HTTPS port.

To get this working working you will need a few things:

• A functional shell with ssh on your work machine
• An internet accessible remote machine running sshd server
• Recent BTSync client (obviously)

If outbound communications on port 22 are open at your location, any server to which you have shell access will do. If you only can get out on ports 80 and 443, you will need to configure said server to run SSH daemon on one of these ports. This unfortunately requires root access.

You set this up by editing /etc/ssh/sshd_config. Search for the word “Port” and simply add another entry below, like this:

# What ports, IPs and protocols we listen for
Port 22
Port 443

Then restart ssh server:

sudo service ssh restart

Make sure you can ssh into it from behind the firewall. If your port 22 is closed, you can specify the alternate port on the command line like this:

ssh -p 443 you@your.host.net

If that works, you will now be able to create an SSH tunnel that will act as a SOCKS proxy. On the machine where you want to run the BTSync client, do the following:

ssh -D 9988 you@your.host.net -N -p 443

This will create a SOCKS proxy tunnel running on the local machine on port 9988. You don’t have to use that port number. Feel free to use any other port, as long as it is not taken by anything else. I recommend making a script with this command and saving it somewhere in your path, because you will have to run it whenever you want to enable syncing.

Finally, once you have the tunnel running open the BTSync client, go to Preferences and open up the Advanced tab. Check the “Use proxy server” box, type in the localhost ip and the port number you picked (in my case 9988). Use the default SOCKS4 proxy type:

BtSync Proxy Setup

Save the settings, then pause and restart syncing to make them take effect. Once you do this, you should see your folders syncing up as they should. Of course the sync will stop when the tunnel is closed, but it is better than nothing.

For example, I have noticed that my personal workflow is very risk averse. I typically start with a git pull and ends with a git push. As I make changes to a file, I tend to save very often, which is not unusual for programming. When you write code you usually focus on small discrete, incremental changes that need to be tested in isolation. You make an edit, save the document, check if anything broke, make another change, and so on. When you finish working on a specific task or accomplish a specific goal, you commit the code encapsulating the changes into neat snapshot that could be rolled back later. Then you move on to the next thing. Multiple times per day you collect bunch of these snapshots and push them out to a remote repository.

The entire process is anchored not to the local the file system but also to a revision tracking system which provides me with backups and snapshots of my code. It is actually quite difficult for me to lose more than a few minutes of work due to a mistake or a software glitch. I always have at least 3 recent copies of the project: the working copy in storage, the local revision history, and the remote repository. More if I’m feeling adventurous, and I create feature branches which provide yet another working copy that is separate from the main one. It is a very safe way to work.

Busy CEO Workflow

This is very different from what I call the “busy CEO workflow” which starts and ends within Outlook. I was recently able to observe several people using this exact Microsoft Office driven workflow and I was baffled how risky and failure prone it was. I would never actually choose to work this way, if nothing else than to save myself the stress and preserve my own sanity.

Let me try to outline this workflow for you:

1. You start by receiving a Word/Excel document attached to an email
2. You double click on that attachment to open it
3. You laboriously make dozens of changes over the span of next 3 hours
4. When finished you hit “Save and Send” button on the toolbar
5. Outlook attaches the modified file to a new email

Note how in this particular workflow, all the work is being done almost entirely in memory. When you open a Microsoft Office document attachment from Outlook it renders it opens it directly. It probably puts a working copy somewhere in a local temp folder, but not in a way you could later track down. All the changes you add to the document may or may not be saved to that ephemeral temp file, which will go away the minute you close Outlook.

Microsoft Office does offer you a little bit of protection from glitches and software crashes in terms of the auto-recovery feature (unless of course it was switched off) which will periodically attempt to create a snapshot of your work. If the application does not close cleanly, it prompts you to recover from one of the recent snapshots. Unfortunately these backup copies are immediately deleted when the user deliberately closes application. So if you accidentally close the wrong window, you are likely to lose all the work.

The “save and send” functionality relies on a magical hand-off happening between two office applications that involves passing around references to an ephemeral, temporary file, hidden away from the user. This interaction is semi-reliable but I have seen it break in such a way that it closes the edited document and silently drops the modified file without actually ever giving the user a chance to send it.

This breakage is not an isolated fluke, by the way. The Microsoft Office interop features are known to be rather fragile. Because of their complexity Office applications often end up in weird states which may affect these sort of hand-off situations. In fact, it happened twice in a week when I was working with end users gathering specs for a project. Both times it required closing and re-opening of all Office applications to restore the functionality.

This workflow is fraught with data loss risk and has way to many points of failure:

• There is no user-accessible “work copy” of the file with recent changes
• Only life-line is the magical auto-recovery feature
• The “save” feature is not guaranteed to work all the time

You have got to admit that this is quite bad. If you are a tech savvy person, you know that this is not how one is supposed to work. You are supposed to anchor your work in the storage, not in main memory. You are supposed to save often and keep multiple copies of your work to keep track of changes. And yet, this email-to-email, in place-editing workflow is baked right into the very fabric of Microsoft office. It is easy, convenient and as such it is really appealing to the busy executives who must juggle a lot of balls in the air at all times.

No amount of user education can counteract the “common sense” logic of “if you’re not supposed to use it, then why did Microsoft include it as a feature” counter-argument. Software developers of course know that this fallacious line of reasoning: we put half-baked features into our software all the time, and we don’t always have the time or resources to work through all possible use-cases and usage scenarios. Once the feature is in production, it is hard to remove it.

So the universe is full of half-baked convenience features that don’t really work right. I imagine the “save and send” feature was intended for people who just want to fix 3 typos before approving a staff memo or a courtesy letter of some sort. But but I’ve just seen someone use it to re-write an 80 page report almost entirely, over the course of almost an entire day. That file sat there, in memory when the person took their lunch break, responded to other emails, and worked with dozen other attachments. And that’s quite scary. It is putting a lot of faith in a piece of software…

Which is something I have noticed people do. As a software engineer, the best advice I can probably give you is to never assume any software you use is reliable. It isn’t. Unless it has been developed by NASA for the explicit purpose of flying a rocket into space, then the code is probably a bug ridden mess. And even NASA fucks up every once in a while.

If you consistently lose work due to accidental clicks or software glitches, and someone told you that you can avoid it by modifying your work-flow to route around the flaws in the software, would you do it? Or would you keep your workflow and just be mad at flaky software and the IT staff’s inability to make a third party application do things it was not properly designed to do?

Is there a way to eliminate the busy CEO workflow from your organization? Can you force it out of the system via infrastructure change? Granted, trying to force out Microsoft Office from your organization would be tilting at windmills so that’s probably not a good approach. You will never convince the business folk to give up Word and Excel, but you can sometimes wean people off Outlook. Especially new generations of office workers who grew up on fast, reliable webmail interfaces with endless storage capacities tend to scoff at the very idea of a dedicated email client. And that’s actually a good thing.

For all their flaws, web-mail interfaces do one thing right: they force users to anchor their work in the file system by asking them to save attachments to disk before opening them. This may seem like a major annoyance at first, but that one extra click solves so many issues.

Thoughts? Comments? Ideas?

Manually testing web apps is a nightmare because it forces you to pull up a web browser, refresh pages, make sure you clear your cache between tests, etc.. No one wants to fiddle with the browser all day, so automating basic testing tasks will not only save time, but also greatly improve your workflow.

Unfortunately testing web apps can be a bit tricky sometimes. They are designed to be accessed over the network, and to render in a web browser, and so they require your test framework to do network-like and browser-like things to emulate those conditions. While unit tests for classes can be easily mocked, pretending to be a web browser is definitely non trivial. When I work with PHP I usually use the excellent Codeception tool-set to do acceptance testing. When I work in Node or just build front end stuff, I typically use Grunt with Phantom.js.

When working with the Sinatra framework, most unit/acceptance style testing can be easily done by using the minitest and rack-test gems. Let me show you how.

Let’s set up a simple Sinatra app. Our folder structure ought to be something like this:

myapp/
├── app.rb
├── Gemfile
├── public/
│   └── style.css
├── Rakefile
├── tests/
│   └── app_test.rb
└── views/
    ├── layout.erb
    └── main.erb

When setting up your dependencies in your Gemfile it is a good idea to isolate the test related gems from the actual run time dependencies. You can do this by using the group keyword:

source 'https://rubygems.org'
gem 'sinatra'

group :test do
  gem 'minitest'
  gem 'rack-test'
end

When deploying to production you can exclude any group using the –without argument:

bundle install --without test

If you are deploying to Heroku, they exclude test and development groups by default, so you don’t even have to worry yourself about it.

Here is a simple Sinatra app:

require 'bundler'
require 'bundler/setup'
require 'sinatra'

get '/' do
  erb :main
end

You know how this works right? The above will render the contents of main.erb and envelop them in layout.erb which is the auto-magical default template. For the time being lets assume that the contents of the former are simply the words “Hello World” and that the later provides a basic html structure.

To test this application we need to create a test file somewhere (I put them in the test/ directory) and inside create a class derived from Minitest::Test and include the Rack::Test::Methods mixin.

Mixins are a wonderful Ruby feature that let you declare a module and then use the include keyword to inject it’s methods into a class. These methods become “mixed in” and act as if they were regular instance methods. It’s a little bit like multiple inheritance, but not really.

In the example below, this gives us access to standard Rack/Sinatra mock request methods such as get, post and etc…

ENV['RACK_ENV'] = 'test'

require 'minitest/autorun'
require 'rack/test'
require_relative '../app'

class MainAppTest < Minitest::Test
  include Rack::Test::Methods 

  def app
    Sinatra::Application
  end

  def test_displays_main_page
    get '/'
    assert last_response.ok?
    assert last_response.body.include?('Hello World')
  end
end

Once you invoke the mock request method (see line 15 above) the last_request and last_response objects become available for making assertions. The last_response object is an instance of Rack::MockResponse which inherits from Rack::Response and contains all the members and methods you could expect. For example, to check whether or not my app actually displayed "Hello World" I simply had to test if that string was somewhere inside last_response.body (see line 17).

To run this test you simply do:

ruby tests/app_test.rb

The minitest gem takes care of all the boring details. We just run the test and see the results.

Let me give you another example. Here is a bunch of tests I written when working on the Minion Academy web service. My goal here was to make sure my routing rules worked correctly, that the requested pages returned valid JSON objects, with the right number of nodes, and that no JSON would be generated if the URL was formatted wrong:

 
  def test_json_with_1
    get '/json/1'
    assert last_response.ok?
    response = JSON.parse(last_response.body)
    assert_equal 1, response.count
  end

  def test_json_with_1_trailing_slash
    get '/json/1/'
    assert last_response.ok?
    response = JSON.parse(last_response.body)
    assert_equal 1, response.count
  end

  def test_json_with_0
    get '/json/0'
    assert last_response.ok?
    response = JSON.parse(last_response.body)
    assert_equal 0, response.count
  end

  def test_json_with_100
    get '/json/100'
    assert last_response.ok?
    response = JSON.parse(last_response.body)
    assert_equal 50, response.count
  end
  
  def test_json_with_alphanumeric
    get '/json/abcd'
    assert_equal 404, last_response.status
  end

Note that those are not all of the tests I have written for this particular bit, but merely a representative sample.

The neat thing is that these tests will seamlessly integrate with other unit tests you write against regular, non-Sinatra, non-Rack related classes. You can simply dump all the test files in the tests/ directory and then add the following to your Rakefile:

require 'rake/testtask'

Rake::TestTask.new do |t|
  t.pattern = 'tests/*_test.rb'
end

This will add a test task you can run at any time that will iterate through all the files matching t.pattern and run them in a sequence.

If you're like me and you don't feel content unless successful tests are rendered in green, and errors in bright red, I recommend using the purdytest gem which colorizes the minitest output. There are many test report filters out there that make the output prettier, but Purdytest probably the simplest and least obtrusive. You simply require it at the top of your test files and then forget all about it.

There is of course nothing wrong with either of these things. Long domains may not easy to type but they can be memorable. Most people don’t even use the address box in their web browser (which used to drive me nuts until I learned to just accept it). These days it is all about on fuzzy search in Google so as long as someone can manage to type in something close to termanotely incaharet somewhere, they will find my blog.

Similarly, long permalinks are actually useful due to their descriptive nature. I like to be able to glance at a link and not only know what it is about, but also how long ago it might have been posted. I would not want to get rid of an useful feature like that.

That said, my links are just super long. Take this one for example:

http://www.terminally-incoherent.com/blog/2015/01/19/utility-spells-in -video-games/

It is an absolute beast, and I would love to have the option to shorten it when I post it on social media for example. Yes, Twitter does minify links by default but the truncated URL still looks quite ugly in your tweets.

I could use a third party minifier such as bit.ly or goo.gl but that is never a good idea. Not only does it obfuscate the links, but it also sets them up for possible breakage in the future when said 3rd party services shut down which is not uncommon. I have seen quite a few of them appear and go under in just last few years, and being backed by a big company does not seem to help. Google might not be going anywhere anytime soon, but they shut down their services all the time. Personally, I got burned twice with them. First time with Google Notebook (an Evernote precursor), and the second time with Google Reader. URL’s are supposed to be forever so I wouldn’t feel comfortable using any service that is not under my control. Ideally I would want to use my own domain for minification of links.

I noticed that WordPress already does support shortened links in a way. If you have permalinks enabled (and you should), WordPress calls your standard, long and “pretty” links the cannonical URL’s. However it also provides a so called shortlink for each post. You can see it shown in the post header on most themes:

The shortlink format is actually the default style of WordPress URL’s you get if you can’t be bothered (or you are unable) to set up permalinks. It acts as a fallback, and allows you to access blog posts by their internal database ID even if the permalinks are not working correctly.

So if I wanted to, I could link to my posts using the shortlink format like this:

http://www.terminally-incoherent.com/blog/?p=18242

If you have a short domain name, this might be enough for you. Mine is still too long, and I absolutely hate parameters in the URL. IMHO, it looks unprofessional. However, knowing that this format exists allows you to shorten and prettify it using Apache redirects. For example, I could put the following line in my .htaccess file:

RedirectMatch ^/(\b\d+\b)/?$ /blog/?p=$1

This matches any URL that contains only numerical digits followed by an optional trailing slash, and seamlessly redirects them to the shortlink style URL format. This rule allows me to use something like:

terminally-incoherent.com/18242

Because of the way WordPress handles shortlinks, users who follow the link above won’t actually see it in the address box when the page loads. Instead WordPress will automatically unfold it to the “pretty” permalink format, which is exactly what I want.

This is a huge improvement. Still to long though.

To compress my addresses even further I purchased a brand domain: tein.co Unfortunately ti.co and in.co were already taken so this is the next best thing I could think of. It’s not pronounceable, but it is short, and visually similar to my long domain.

All that was left was to set the new domain to “web forward” to my actual blog URL and I was all set. I use iwantmyname as my registrar and it took literally 60 seconds to set it up:

Web Forwarding

From there it worked pretty seamlessly. If you are feeling lazy and you don’t want to type in my full web address you can now simply use tein.co. Similarly, every post now has a genuine short link (displayed under post title) in the format:

tein.co/18242

Because the entire thing is implemented via 301 redirects, if you post one of these short links on Twitter you will still get that sweet Twitter Card action.

So there you have it. Zero effort link minification using built in WordPress addressing scheme and a single Apache redirect statement.

I figured I post about the symptoms and experience here in case anyone else decided to buy an Alienware Aurora-R4 with a dual NVIDIA GeForce GTX 780 setup only to have it die few months later.

The problem started when I was playing a game (it was FarCry 4 for reference) when it completely froze up. It was a hard lock-up with the non-responsive keyboard, and speakers stuck repeating a single bleep over and over again. The video winked out few seconds later and my monitor dutifully displayed a “NO DVI SIGNAL” message, but the speakers kept on going. I ended up having to power cycle it just to get rid of the noise.

This was kinda odd, since FarCry 4 has been rather remarkably polished and bug free (as it should be since it is basically FarCry 3 with a palette swap) so such hard crash was unexpected. But the machine rebooted just fine so I thought nothing of it. Since it was already late, I thought nothing of it, logged off and went to sleep assuming this was the universe’s way of telling me to get off the computer.

Next day I was doing something in Photoshop, and the machine did this again: all of a sudden my screen went blank, and then about 30 seconds later I saw BIOS POST screen and the computer started rebooting itself. Again, I was a bit concerned but after it powered up, it was fine again, and I was unable to reproduce the crash by just toying around in Photoshop so I wrote it off as a one time glitch.

It wasn’t until I went back to FarCry 4 that I saw a persistent issue. Every time I started the game it will load up, show me main menu, let me load a saved game, display a progress bar, and then as soon as the actual game would start the screen would go blank. I would then get the “NO DVI SIGNAL” message from my monitor, followed by a reboot shortly after. This happened every single time.

My Event Viewer

As soon as I had a reproducible issue, I started digging. First place I went was the Windows EventViewer which, unsurprisingly, was full of critical Kernel-Power errors. I checked the timing, and each of them coincided with the hard crash and reboot. They all looked more or less like this:

Log Name:      System
Source:        Microsoft-Windows-Kernel-Power
Level:         Critical
Description:
The system has rebooted without cleanly shutting down first. 
This error could be caused if the system stopped responding, crashed, or lost power unexpectedly.

BugcheckCode: 278
BugcheckParameter1: 0xfffffa80140da4e0
BugcheckParameter2: 0xfffff8800fc16828
BugcheckParameter3: 0xffffffffc000009a
BugcheckParameter4: 0x4
SleepInProgress: false
PowerButtonTimestamp: 0

This was not very helpful, but after some research I found out that Bugcheck 278 is actually equivalent to BSOD 0x116 also known as VIDEO_TDR_ERROR. The most approachable description of this issue I found was:

This indicates that an attempt to reset the display driver and recover from a timeout failed.

In other words it was a video issue that would normally result in a blue screen of death, but since it crashed the entire video processing stack said BSOD could never actually be displayed. Possible causes of this error were as follows:

• Bad video driver acting up (not unusual from nVidia)
• Bad RAM chip causing discrepancies when syncing with VRAM
• Bad video card

So I went down this list, trying to nail down the exact issue. First, I upgraded to the latest nVidia driver. I actually don’t remember which version I had when I started the process, but I knew it was slightly behind. So I downloaded the latest and greatest, and updated it. This did not solve the problem. I decided to go the other way, and tried four previous versions of the driver, as well as two previous beta versions. None of them got rid of the crashes. It’s probably worth noting I was doing “clean” installs – meaning I would uninstall, the current driver, reboot and then install another one to avoid weird conflicts.

Next I tried doing the Dell pre-boot diagnostics. It is an on-board functionality on all Dell machines and is usually available from the selective boot menu (accessed by mashing F12 during POST). It doesn’t really do anything useful, but in case of detectable hardware failures it typically spits out an error code which can be given to Dell tech support circumventing a lot of bullshit like checking if the computer is plugged in, wiggling the wires and etc. Not only that – the Dell warranty support drones usually like to tell you to run the hour long extended test anyway and refuse to stick around on the phone as you do, necessitating a call-back.

Unfortunately, the pre-boot diagnostics module gave my computer a clean bill of health. Granted, it did not really have any extended tests it could run on the video cards – it would simply check if they were present and responding. It did however confirm that there was no issues with the memory. Just to double check that, I booted into a MemTest CD and ran it for about 12 hours (started in the evening, finished next day when I came back from work) and it did not show any errors.

The Alienware machine also came with something called Alien Autopsy which is yet another diagnostic tool. This one is a bit friendlier, since it does not require you to reboot your machine, and it also has seemingly more thorough tests for the video cards. So I decided to run that as well.

Alienware Alien Autopsy

The video testing involves a thorough VRAM test and few video benchmarks during which it renders some spaceships on the screen, spins them around, and tests real time shaders, transparency, graphics pipeline and etc… As soon as I started running those, my machine started crashing and rebooting itself. It was reproducible and consistently failing about half-way through the benchmarks. I couldn’t pin down the crash to a single benchmark or test case, but I ran it about 20 times and I never managed to get through all of them without the machine shutting down on me. At this point I was fairly confident it was an issue with one of the video cards.

Armed with that evidence I phoned Dell Alienware support line and gave them all of the details outlined above. The guy on the other line listened to my spiel, looked through his notes and admitted I covered pretty much all the bases. He made me check my BIOS version to see if it needs to be updated but it turned out I had the latest and greatest one. So he agreed I need video cards replaced. I was expecting him to tell me to disable SLI and start pulling cards out to narrow down which one is the faulty one, but he just set up a dispatch to replace both of my cards.

Luckily I purchased the next business day on-site service warranty, so it only took them a week and a half to get it fixed:

Next Business Day support just became "next business week, maybe". Thank's a lot @Dell. I reported this on Sunday. pic.twitter.com/Oa3vrxEHwd

— Luke Maciak (@LukeMaciak) December 9, 2014

I’m happy to report that replacing the cards completely fixed my issue. I was a little concerned this was going to turn out to be a motherboard problem – because knowing my luck it would. But I haven’t seen the dreaded Bugcheck 278 crash since the new cards were installed. I’m currently trying to finish FarCry 4 so that I can go through some of my Steam Holiday Sale backlog, and probably Dragon Age Inquisition.

I also have a few book, and comics reviews in the pipeline, and I’ve been toying around with an idea of doing a Ravenflight style series but for a SF themed setting. So I’m not dead, do not unsubscribe from the blog yet.

I’ve been itching to try Arch for a while not but I never really had an opportunity so I figured I might as well use it now. It worked amazingly well, so I figured it would be wroth while to document the procedure for future reference. Especially considering it is slightly different from the Debian procedure. I used this guide but with some significant alterations (see below).

Logging into the PogoPlug

First, you need to figure out the IP of your plug. Best way to do this is to log into your router and match it by name or Mac address. Once you know the IP address you can ssh into it using root as the username and ceadmin as password.

Preparing the Thumb Drive

The default OS on the PogoPlug is locked down pretty tight. Pretty much the only place with write access on the device is /tmp so you won’t be able to install to the internal drive (or rather it is fairly impractical to do). Instead you want to set up Arch on a thumb drive.

First, you will need to figure out which devices is the drive recognized as. I’m fairly sure the top-most USB port on the back of the device always registers as /dev/sda but you can easily check it by plugging it in and then running:

dmesg | tail

The last few lines should reveal which device the system thinks was plugged in. I’ll assume it was /dev/sda. First thing you want to do is to repartition it using fdisk:

/sbin/fdisk /dev/sda

Create two new partitions:

• Press o to blow away all existing partitions on the drive.
• Press n to create a partition, p to set it as “primary” and 1 to designate it as first
• Hit Enter to accept the default starting point
• Specify size using the format +[size]M where [size] is an actual value in MB. For example, I used +1536M designating majority of the space on my 2GB drive for my primary partition, and leaving 512MB for swap. If you have 4GB drive use +3582 and so on.
• To set up second partition hit n, p, 2
• Hit Enter to accept the default starting point
• Hit Enter once again to use all the remaining space
• Hit t then 2 to change the filesystem on partition 2 and use 82 (Linux Swap)
• Hit a, 1 to make first partition bootable
• Hit w to write changes to the disk

When you’re done the p command should return something like:

/dev/sda1   *           1         911     3501853  83 Linux
/dev/sda2             912        1018      411308  82 Linux swap

Since arch uses ext3 file system we will want to format the primary partition /dev/sda1 as such. Unfortunately the default OS on the PogoPlug does not ship with support for ext3 so we will need to download the mke2fs tool from the arch website and then use it to format the partition:

cd /tmp
wget http://archlinuxarm.org/os/pogoplug/mke2fs
chmod +x mke2fs
./mke2fs -j /dev/sda1
mkdir alarm
mount /dev/sda1 alarm

Installing Arch

Now we are ready to download the Kirkwood Arch binaries. The latest builds are close to 200MB in size, which was too big to fit in on the PogoPlug system partition. I recommend downloading it it to the newly formatted drive instead:

cd alarm
wget http://archlinuxarm.org/os/ArchLinuxARM-kirkwood-latest.tar.gz

The official PogoPlug write-up on the Arch website tells you to use bsdtar to extract this archive. This may or may not work for you. I had major issues unpacking that way due to a locale mismatch and the UTF-8 encoding being used for file paths within the compressed bundle. Extracting the file the old fashioned way however worked just fine which is what I recommend you do:

tar -xzvf ArchLinuxARM-kirkwood-latest.tar.gz
sync
cd ..
umount alarm

Finally, download the U-Boot installer which will flash the ROM memory of the PogoPlug and force it to boot off the USB drive. Note that this step can brick the device (though I’ve done it a dozen times by now and never had any issues):

wget http://archlinuxarm.org/os/armv5te/boot/pogo_e02/pogo_e02.sh
chmod +x pogo_e02.sh
./pogo_e02.sh

Once this is done, reboot manually:

/sbin/reboot

If everything worked correctly the device should now boot into Arch. When the device reboots, log in with username root and password root.

Configuring Arch

First thing you will probably want to do is to update the system. You use the default Arch package manager pacman for that:

pacman -Syu

Next, you probably want to change a root password and add a new regular user for yourself (remember to add yourself to the wheel group):

passwd
useradd -m -g users -G wheel -s /bin/bash luke
passwd luke

The Kirkwood install is very bare bones and it does not ship with sudo so you will probably want to install it:

pacman -S sudo

Configure it with visudo and append the following to the end of the file:

%wheel      ALL=(ALL) ALL

This will give your regular user and all the other potential future members of the wheel group access to sudo command. At this point it may be a good idea to log out and log back in to make sure the user account you just created works, and that you can use su and sudo to elevate your privileges. If everything works, you may want to disable the remote access to the root account like this:

passwd -l root

You will probably want to change the devices hostname. On Arch this is done via the hostnamectl command:

hostnamectl set-hostname mypogoplug

If you’re on a Windows network and you want to be able to use the hostname instead of the ip address when you ssh you will need to install samba and give it a netbios name:

pacman -S samba
cp /etc/samba/smb.conf.default /etc/samba/smb.conf

Modify the smb.conf file to include:

workgroup = MYWORKGROUP
netbios name = mypogoplug

Now start samba and set it to start on boot:

systemctl start samba
systemctl enable samba

You should now be able to ssh into your plug using mypogoplug rather than the IP address from Windows machines. If you have Apple machines on the network and you want to be able to access them using mypogoplug.local then you will need to install two additional packages: avahi and nss-mdns:

pacman -S avahi nss-mdns

Now open the /etc/nsswitch.conf file and change the following line:

hosts: files dns myhostname

into:

hosts: files mdns4_minimal [NOTFOUND=return] dns myhostname

Finaly, start the avahi-daemon and set it to be run on boot:

systemctl start avahi-daemon
systemctl enable avahi-daemon

At this point your device should be discoverable on the network and more or less ready to go for whatever purpose you may want to use it.

While there are dozens of random NPC generators, I figured I might as well write my own. It seemed like a fun and quick side project. How would one go about procedurally generating non player characters though?

First and foremost I figured it should be easy to modify and expand. Instead of hard coding values into the generator itself, I figured it should be stored as some sort of a structured list. I went with YAML because unlike many data serialization formats what claim to be “human readable” it actually is. Well, at least for me it is – your opinion may of course vary and it is not like YAML is without a lot of weird quirks. But I basically just needed a simple data format that could be easy to edit by hand, and that could be consumed by my code without doing a lot of parsing. Seeing how in Ruby can slurp a YAML file into an associative array in a single line of code, it was more or less perfect.

Moreover, I wanted my generator not to be “fully” random but rather use weighted probability scores for specific things. For example, it should be relatively rare to see a Rogue wearing a plate armor, but it would be common to see it on Warrior characters. How do you implement that? There is a few ways. For example you could find the discrete cumulative density function (CDF) of your list, generate a random number in the range between 0 and the sum of all weights, do a binary search to find this number… Actually, no. Scratch that. This is a solved problem and there is literally no reason to re-invent it other than as a classroom exercise maybe (or if you are worrying about performance). Instead of spending hours writing and debugging CDF code, we could just grab a gem such as this one and be done with it.

The basic idea was to let me write a weighted list like this in YAML (higher the number, the better likelihood the item gets picked):

race:
    Human     : 6
    Elf       : 6
    Dwarf     : 6
    Orc       : 6
    Goblin    : 5
    Halfling  : 4
    Hobgoblin : 3
    Kobold    : 2
    Ogre      : 2
    Troll     : 1
    
class:
    Fighter  : 4
    Soldier  : 3
    Cleric   : 1
    Bard     : 1
    Diplomat : 2
    Ranger   : 5
    Rogue    : 5
    Sage     : 1
    Scout    : 3
    Warrior  : 6

social:
    Commoner : 5
    Noble    : 2

Then in Ruby I could pull stuff out of it like this:

require 'yaml'
require 'pickup'

data = YAML.load_file('stuff.yml')
race = Pickup.new(data['race']).pick(1)
class = Pickup.new(data['class']).pick(1)

This was basically as complex as the code would get. As it is usually the case with this kind of projects the bulk of the work went into actually generating the data files that would yield not only a good deal of variety but also return both mundane and ordinary foot soldiers as well as funky and exceptional fun characters from time to time. It is more of a creative endeavor rather than programming.

What kind of weapons are appropriate for a rogue? What kind of armor should be worn by scouts? What color can Orc eyes be, and would this be any different for goblins? What kind of scale colors are most popular amongst the Kobolds? These were the sort of questions I had to answer while making this tool.

If you follow me on Twitter (as you should) you have probably seen me posting screenshots of the minions I was generating on the console:

Here are some more randomly-generated minions, now with personality traits and descriptions. pic.twitter.com/QERJtDXWP0

— Luke Maciak (@LukeMaciak) October 21, 2014

This is back when I still had “barbarian” as a class which I later decided against including. Why? Well, to me it seems like every other class (warrior, rogue, bard, cleric, etc..) is something you choose to be. Barbarian, on the other hand is something you are. It is more often than not used to describe a social caste or grouping of people rather than a profession / calling. So I removed it and replaced it with Fighter and Soldier to have 3 solid close combat classes. In my mind warriors fight out of conviction (they have a duty, seek glory, want justice, etc..), fighters do it because they like it (they are the brawler, trouble-maker types that start fights in taverns for shits and giggles) and soldiers do it strictly for money.

Creating plausibly sounding names proved to be a whole separate problem. I knew that when it came to elves and Dwarfs, I could just shamelessly crib from Tolkien if I wanted to because there are billions of good names for both of these races in the Middle Earth lore. But I didn’t just want to have gigantic copy pasted lists. So I opted for something slightly more clever. I grabbed some interesting names, broke them into (usually) two parts, and then randomly recombined them. For example, here is a sample of Orc name table:

Orc:
    given:
        Male:
            prefix:
                grish: 5
                gor: 5
                muz: 5
                maz: 5
                lag: 5
                lat: 5
                lar: 5
                lir: 5
                rad: 5
                shá: 5
                rag: 5
                rat: 5
                urf: 5
                goth: 5
                núr: 5
                nir: 5
                fár: 5

            postfix:
                nákh: 5
                bag: 5
                gash: 5
                gnash: 5
                bash: 5
                mash: 5
                gol: 5
                mol: 5
                duf: 5
                buf: 5
                rúf: 5
                muf: 5
                dúr: 5
                grat: 5
                gnat: 5
                thrak: 5
                lúk: 5
                múk: 5
                mog: 5
                rog: 5

This particular selection can yield names like Grishnákh, Gorbag and Muzgash (all of whom are named characters from Lord of the Rings) as well as dozens more or less plausibly sounding names.

Most races have gendered first names and last names dictated by social status. So for example a noble’s name may include the name of their estate, or name of their father, whereas the names of commoners are typically nicknames or trade profession related. Elves, Hobgoblins and Trolls ended up with gender neutral names just because of how alien they sounded and because I wanted to have at least one group which did not have a concept of gendered names.

Once I had basic data files created, I wrapped it up in a bit nicer interface and started generating minions by dozens. It was interesting just to read their short descriptions and try to imagine how they would look and what their personalities would be. At some point I even noticed emergent little micro-stories popping up every once in a while. For example, here are two randomly generated Orcs I got the other day:

Noble born warrior Ragma travels with her attendant Mizni seeking glory in combat 2 prove she is worthy of her title pic.twitter.com/FcUfYJuCi8

— Luke Maciak (@LukeMaciak) October 26, 2014

I found it interesting that they were both ambitious and feared losing face. It felt like they were connected somehow. Ragma was a noble born warrior while Mizni one was a commoner and a ranger. Possibly Ragma’s attendant and a guide? They were likely traveling companions: Ragma young, impetus, and irresponsible, but eager to make a name for herself. The older, wiser Mizni was likely appointed by her parents to keep the young warrior in check, and make sure she returns home safely from their adventures. They both driven by their ambition. Ragma wants to prove she can live up to the high standards of heroism set by her parents. Mizni wants to prove her value to the family by taking on a challenge of keeping the wild and irresponsible Ragma in check. You could literally write a short story about them, just based on this relationship.

This is the beauty of randomly generated content: sometimes a short little blurb can strike a chord with you and your imagination will immediately fill in the blanks creating interesting and meaningful relationships and scenarios. I figured it was worth sharing this little thing that I have done with others.

Minion Academy Screenshot

I set it up on Heroku cloud platform, and named it Minion Academy, mainly because I managed to snag that as a URL. So it is now up at minion.academy. When you visit the page you will get five randomly generated NPC’s and you can refresh the page for five new ones. It’s very basic, and still rather rough around the edges. There is still some work I want to do with it.

For example, I want to add more armor choices. Right now it’s basically just cloth, leather, chain or plate. I would like to expand it so that you could have a wide variety of different armor types for each of these categories. You might have also noticed there are no magic user types being generated right now. This is partly by design (I was initially trying to make a minion specific generator which kinda grew to cover all kind of NPC’s) but I’d like to add some wizards and sorcerers at some point.

If you notice a bug, I have the source code on Github so feel free to submit a bug report. As usual, let me know what you think in the comments.

For example, is this a good sign up form?

Facebook Signup Form as of Oct 2014.

I would argue that it is not great. In my opinion splitting the users name is absolutely pointless. Even, assuming your service needs to use the legal names of your customers (which 99.9% of web services do not “need” to do, they just choose to do so because of reasons) you really only need a single field. This is not a very popular opinion, and a lot of programmers are very, very defensive of their first name/last name split.

I get it, though. I too was taught the mantra of “always be normalizing” when it comes to a database scheme design. The software engineer in me wants (even needs) to have human identity split into two or more clearly labeled forms so that it can be properly sorted. But, asking for first and last name does not work for everyone. As soon as you normalize this way, you are automatically starting to exclude swaths of users whose names do not conform to the particular pattern you chose.

You probably heard of this little factoid: in some cultures you list your family name first, and your given name last. That alone should give you a pause, and make you re-consider using a two field strategy. Some people think that simply labeling the fields as “given” and “family” instead of “first” and “last” will do the trick. I also saw a developer claiming that his app is going to be primarily used by English speaking Americans so it does not matter. But that’s wrong too, because even in that narrow demographic you are going to have a number of people whose names do not fit into the first/last pattern. You want examples? How about Madonna, Eminem, Pink, Xzibit, Nelly, Sinbad, Rihanna, Kesha, Mr. T, Lady Gaga or “The Artist Formerly Known as Prince”. There is a strong history of performers adopting mononyms or stage names which either become their legal names, or at the very least are more publicly recognizable than their birth names.

The fact that I could rattle a dozen names of the top of my head, all of which belong to prominent and recognizable celebrities is a proof that this practice is very much part of western culture. Mononyms and funky stage names are as American as apple pie. So you can’t really use “culture” to defend the over-normalization of the name field, when your own culture has a large group of very prominent outliers.

People make a lot of assumptions as to how people’s names work, but all of them are false. Yes, all of them. The single, uniform field for name is not just something I pulled out of my ass for the purpose of this article. It is actually the best practice recommended by W3C.

Same goes for sex. Why does Facebook think it is necessary to ask its users what kind of genitals they have? I can see how this could be a valuable data point for a dating service, since people use those specifically to facilitate mutual mashing of genitals together. So it makes sense to let people sort and filter potential future romantic partners based on their sex and gender preferences in addition to other criteria. Facebook however, like most social and anti-social web apps in existence has virtually no business to ask this question.

Don’t even try to sell me on “demographics” and “advertising” argument because it is bullshit, at least with respect to Facebook since they track your behavior and browsing habits anyone. There is nothing your sex tells their advertisers that they could not get from analyzing your posts, likes and social graph interactions. In fact, the tracking data is more valuable and more accurate way to target advertising than an empty data point that designates you as “man” or “woman”.

Also, why is it a strict binary choice? I mean, unless you’re building something like Christian Mingle type service (where religious dogma only allows you to recognize an arbitrarily chosen set of genders and appropriate parings), why would you want to wantonly ignore biology? If you are going to ask this question (and you have no business doing so in the first place), why not ask it the right way?

Is the Facebook form asking for sex, or gender? Because I honestly can’t tell? This is an important question to ask because Facebook has weird “real name” policies that could result in the suspension of your account if their support staff determines you “lied” on this question. So what do you put down biological sex does not match the gender you identify with? What if you don’t identify neither as male nor as female?

I think Twitter does this right:

Twitter Signup Form as of Oct 2014.

A single field for “full name” and no unnecessary questions about sex and gender. This is how it should be.

My personal rule of thumb for designing web forms: make them Time Lord friendly. Whenever you want to add or normalize a field, think how the protagonist of BBC’s Doctor Who series would fill it out. Your form should allow one to use The Doctor as the single and only personal identifier.

• The Doctor does not have a first name
• The Doctor does not have a last name
• The Doctor does not have a middle name or middle initial
• The Doctor does not have a set of initials based on name
• The Doctor is not a given name
• The Doctor does not have a family name
• The Doctor does not use a honorific – it’s just The Doctor
• No, you can not abbreviate Doctor as Dr. or anything else
• The Doctor does not have a short name or nickname. You address him as Doctor
• You can’t use Doctor’s date of birth to calculate age because he is a time traveler
• The Doctor’s age won’t fit in your two-digit age field
• The Doctor’s does not have a fixed height, eye color, hair color, etc..
• The Doctor does not have a fixed ethnicity or skin color
• The Doctor does not have a fixed gender

If you keep these things in mind you can avoid common pitfalls of web form design and build signup forms that are not only intuitive but also maximally inclusive.

So I bought a replacement: Alienware Aurora-R4.

Alienware Aurora R4

I know, I know – Alienware is the hallmark of the gamer n00b and all that. There might have been a time when I cared about that, but I think I’m too old to worry about my street cred. If anyone wants to revoke my gamer card, they are welcome to it. And yes, I probably grossly overpaid for the brand name but after futzing with unreliable hardware on my own for several weeks now, I just wanted something off the shelf and with reliable warranty. Dell support can’t diagnose problems for shit, but if you diagnose for them, they will reliably keep replacing parts until the damn thing starts working. That alone gave me a safe and fuzzy feeling of assurance in the midst of frustration. This probably wasn’t my best purchasing decision, but I did take advantage of one of their big sales events and got pretty decent hardware setup without having to take all the money out of my bank (you know, just most of it):

• Quad Core 3.70GHz i7-4820K CPU with 3701 Mhz bus
• 16 GB of RAM (upgradable to 32)
• Two NVIDIA GeForce GTX 780 video cards
• 256 GB SSD (6Gb/s data rate) as main drive & 1TB 7200RPM HDD as secondary

This ain’t a Titan setup, but it’s close to it so I should be able to play all the new games on high without too much headache. The old machine was starting to get a bit log in the tooth in the recent days. Both the new Thief and Metro were choppy even on the lowest settings. I didn’t particularly care for those games but I was starting to get worried I might get locked out of new releases I do actually care about – like new Elder Scrolls games, or new Bethesda properties “fixed” by Obsidian. This shouldn’t be an issue anymore.

The former gaming rig valiantly survived 5 years of almost continuous usage. I never really shut it down, and I specifically prevented it from going into sleep mode so that I could download torrents or run batch jobs overnight, or VPN and remote desktop into it when I was at work. Hopefully this machine will do equally well, though I might try to ride it a little bit less hard to extend it’s life a bit.

I’m mostly posting this here for my own future reference so that I have a rough idea when I bought this machine when it decides to break at some point in the future.

Next post will be a review of a game that I totally did not need this rig for. In fact I played it on my MacBook while waiting for this machine to be delivered.

You might think you are fixing a long standing UI bug, or making the user interface more consistent and therefore user friendly, but it does not matter. At least one of your users probably worked the side-effects of said bug into the way they do things and it will appear broken to them afterwards.

Let me give you a few examples from personal experience. This is not a project I am personally involved in at the development side. For once I am actually sitting at the user-end and watching the fireworks. Let me set the stage for you: we have been using a third party time tracking tool for ages now. When we first deployed it, it was a self hosted application that we had to maintain ourselves. This involved periodically rebooting the server due to memory leaks, and applying the infrequent patches and upgrades. Prior to every upgrade we would first test it on a dummy instance, and would have the folks who used the tool extensively to do scheduling and processing time and expenses give it a once-over before we deployed it to production. If there were issues we would work with the vendor to iron them out prior to the deployment. It worked well.

Unfortunately about a year ago they discontinued support and licensing for the self-hosted version and we had to upgrade to their “state of the art” cloud based service. This was nice for me because it meant we no longer had to expend time and resources to maintain the tool internally. The end users were also happy because they would be getting all kinds of new bells and whistles to play with. The vendor promised the cloud version is developed and improved very aggressively based on user suggestions and that their new agile development process can deploy fixes and custom patches much faster than before. It sounded great on paper, but it turned out to be a disaster.

Our users after switching to the cloud platform.

The vendor likes to push out minor updates and patches every other Monday, and like clockwork this results in our ticketing system getting clogged up with timesheet software related issues. We verify all of these and tag-team grouping and compiling them into support requests who get forwarded to the vendor support team, and cc’d to our account manager. This is our third account manager since the switch and I suspect our company single-handedly got the last two fired by maintaining an unstoppable barrage of open tickets and constant demands for discounts and downtime compensation.

Most of the problems we are having stem from trivial “fixes” that make perfect sense if you are on the development team. For example, recently someone noticed that the box you use to specify how many hours you worked can accept negative values. There was no validation so the system wouldn’t even blink if you entered say negative five hours on a Monday. So they went in, added input validation, and just to be on the save side they fixed it in their database. And by fixed, I mean they took an absolute value of the relevant column, and then they changed the datatype to unsigned integer. Because if there were negative values there, they had to be in by a mistake, right? Because who in their right mind would use negative time? Well, it turns out it was my team. Somehow they figured out a way to use this bug to easily fudge time balances on the admin site. For example, if someone was supposed to work five hours on a Monday, but had an emergency and left three hours early, the admin would just go in and add -3 work hours to the timesheet with a comment. It allowed them to have both the record of what the person was supposed to do, and what actually happened. Needless to say, after the “fix” all our reports were wrong.

Ok, so you divide by zero here, put NULL in all these fields, put -1 here, and 1=1 in the date field, click through the segfault message and your report is ready.

More recently, they noticed that there were two ways for people to request time off in the system. You could create a time-off request ahead of time (which had to be approved by a supervisor) or you could submit it as a part your timesheet by putting in 8 hours as “personal day” or whatever. Someone on the vendor’s dev team decided to “streamline” the process and removed the ability to enter time off from the timesheet page. To them it made perfect sense to only have a single system pathway for entering time. Unfortunately my team relied on that functionality. We had a special use case for the hourly contractors which simply required them to record their downtime as “unpaid leave” (don’t ask me why – I did not come up with that). Before they could do that by simply filling out their time sheet. After the upgrade they had to go to the time off tab, and fill out a time of request for every partial day that week, then have that request approved by a supervisor before they could actually submit a timesheet. So their workflow went from clicking on a box and typing in a few numbers to going through 3-5 multi-stage dialog boxes and then waiting for an approval.

To the vendor’s credit, they are addressing most of these problems in a timely manner, and their rapid development cycle means we don’t have to wait long for the patches. They do however have serious issues with feature creep and each “fix” creates three new problems on average.

Pictured here: pair programming as implemented by our vendor.

Majority of these stem from the fact that our users are not using the software the way the developers intended to. They are using the application wrong… But whose fault is that? Should paying customers be punished or even chastised for becoming power users and employing the software in new, emergent ways rather than using it as you imagined they would? Every botched, incomplete or ill conceived UI element or behavior in your software is either an exploit or a power user “feature” in waiting.

I guess the point I’m trying to say is that once you deploy your software into production, and make it available to a non-trivial amount of users, it is no longer yours. From that point on, any “bug fix” can an will affect entire teams of people who rely on it. A shitty feature you’ve been campaigning to remove is probably someone’s favorite thing about your software. A forgotten validation rule is probably some teams “productivity crutch” and they are hopeless without it.

Full test coverage may help to limit the amount of “holes” your users may creatively take advantage of, but it only takes you so far. There is no way to automate testing for something you never anticipated users doing. You won’t even discover these emergent, colorful “power user tricks” by dog-fooding your app, because your team will use it as intended, rather than randomly flail around until they find a sequence of bugs that triggers an interesting side-effect and then make it the core of their workflow. This is something you can only find out if you work with genuine end users that treat your software like a magical, sentient black box that they are a little scared off.