Terminally Incoherent

I do talk a lot of trash about Microsoft on this blog, but since I work in a place which is a Windows shop I can’t really get away from it’s products. Today I dusted off my “C# Developer” hat, and started chugging away. Why .NET? Because it seemed like the right tool for the job. The task at hand was to figure out a way how someone could grab bunch of emails out of her Outlook, and then drag and drop them onto something so that:

1. The emails get saved as .msg files in an appropriate folder on the network share
2. Our internal intranet application becomes aware of these files, and display download links where applicable

So instead of forwarding emails to 4-5 people assigned to a given job, the scheduling person would simply put them in a central location on the server and appropriate people would see download links on their “project page” once they log into the web app. This would limit the “did you get that thing I sent you?” questions, and allow an administrator to add people to a project without the need of hunting down all the relevant emails and forwarding them to the new guy.

Communicating with the web application is not an issue - I can do that in any language. The tricky part the drag and drop functionality which is the whole point of this project. The whole exercise of saving the email as a .msg file and uploading it to the web app using a traditional HTML form was briefly considered for .5 seconds and then unanimously rejected because it would be a royal pain in the ass. On the other hand typing in a task number and then dragging/dropping relevant emails onto some sort of a form is kinda like forwarding them - only you just have to do it once.

I briefly considered Java, but the interaction between Outlook and the Java application looked like a really hairy business. On the other hand .NET seemed like a perfect choice. You’d think that Microsoft would have a relatively easy mechanism of scripting it’s flagship office product in their flagship programming environment. And it turns out that there is - it’s called MS Office Primary Interop Assemblies. These are bunch of COM objects which you can import into your Visual Studio project and they allow you to interact with Office applications at a somewhat intimate level. Unsurprisingly, they are dependent on the version of MS Office you are using (in my case Office 2003) making it hard to create a robust, catch-all solution. You can grab a distributable installer package for Office 2003 version from here.

I added the Microsoft Office 11 Core Library and Microsoft Outlook 11 Core Library as the references to my project. They automagically appeared under the COM tab of my reference dialog in Visual Studio after installing the redistributable package linked above.

Next on the menu was the Drag and Drop functionality. That was pretty easy. In fact, an article written by Tgueth on Code Project gives you a very good primer how to set things up. Just ignore his code, because it won’t work for what I’m trying to do. He was simply interested in dropping actual onto the form - so real files dragged over from Explorer, or Outlook attachments dragged from your email. The email themselves however are handled differently, and cannot be captured this way. I tried, and I failed. There is however an alternate solution using the Interop Assemblies I described above - and believe it or not, the code is actually much cleaner and more compact.

So I sat down and hacked up a quick prototype. Here is the relevant code:

namespace DragAndDrop
{
  public partial class Form1 : Form
  {
    private Microsoft.Office.Interop.Outlook.Application OL;
    private string my_dir = "C:\\";
 
    public Form1()
    {
      InitializeComponent();
      OL = new Microsoft.Office.Interop.Outlook.Application();
    }
 
    private void Form1_DragEnter(object sender, DragEventArgs e)
    {
      e.Effect = DragDropEffects.Copy;
    }
 
    private void Form1_DragDrop(object sender, DragEventArgs e)
    {
      for (int i = 1; i <= OL.ActiveExplorer().Selection.Count; i++)
      {
        Object temp = OL.ActiveExplorer().Selection[i];
 
        if (temp is Microsoft.Office.Interop.Outlook.MailItem)
        {
          Microsoft.Office.Interop.Outlook.MailItem mailitem = (temp as Microsoft.Office.Interop.Outlook.MailItem);
          string subject = mailitem.Subject.Replace(":", "");
          mailitem.SaveAs(my_dir + "\\" + subject + ".msg", Microsoft.Office.Interop.Outlook.OlSaveAsType.olMSG);
        }
      }
    }
  }
}

Let’s break it down, shall we? The first method Form1_DragEnter describes what is supposed to happen when you drag an object onto your form. We simply want to change the Effect property of the event to Copy:

e.Effect = DragDropEffects.Copy;

If you don’t do this, then your code will detect drop effects but that’s about it - it won’t know or care what was dropped. This will ensure that the target of the drop will hold on to a copy of whatever you dragged over when you drop it.

Next, we need to handle the items that were dropped in the Form1_DragDrop method. The following statement returns an array of items which are currently selected with your mouse (ie. the stuff you are dragging right now):

OL.ActiveExplorer().Selection

Since I want to allow users to drag bunch of emails at a time, I’m going to iterate over this array and then grab each item. Since potentially a user could drag over something that is not an email, I’m temporarily saving it as an Object:

Object temp = OL.ActiveExplorer().Selection[i];

If it turns out to be an Outlook email object we cast it back to Microsoft.Office.Interop.Outlook.MailItem and save it. Note that I use the subject line as the filename (an quickly sanitize it removing the illegal “:” character) and save it in the root of the C: drive for now.

The SaveAs method of the MailItem object takes two parameters: first one is the path where you want to save the item, and the other is the file type which must be a member of the OlSaveAsType Enumeration which is a part of the Interop library. I’m using olMSG since that will save as the default Outlook message format.

This code works under Office 2003 but there is one small issue. For every mail item you drop, Outlook will throw this nasty security warning message:

The good news is that the user has the option to disable it for up to 10 minutes. The bad news is that 10 minutes is the maximum amount of time you can choose from that drop down box. I do see why they have this warning there, I just suspect it will become really annoying unless I find out a way to disable it. I did not cross that bridge yet though. Any suggestions? So far the bare bones functionality is working and I can drag and drop email messages onto my little form, and have them saved perfectly along with attachments which is precisely what I wanted.

Next step is to figure out how to make that thing talk to a PHP + MySQL backend of our web application, but that should not be that difficult. All I need to do is to send a HTTPS POST request back to the server, and then have something on the other end process that data and dump it into a database. It doesn’t really need to be much more complicated than that.

I found the following quote from Scott Adam’s blog a bit depressing:

Let’s say you have a typical life and try to live it in the healthiest way. You might allocate your 24-hour weekday this way:

Sleep: 8 hours
Exercise: 1 hour
Work: 8 hours
Eating: 2 hours (leisurely)
Hygiene: 1 hour
Travel: 1 (Commute, errands)

That leaves you three hours for family time, sex, shopping, food preparation, chores, household repair, volunteering in the school, and so on. If you have a dentist appointment, or your talkative relative calls, or American Idol has a two-hour special, you’re tapped out.

In other words, in 5 out of 7 days in a week you spend over 88% of the time doing things you must do, and only 12% of the time doing things you actually want to do. This pattern is how you will spend 70% of your working days. Note that Scott cleverly allocates those precious 3 hours to social endeavors (such as family time, and sex) and necessary chores (shopping, personal errands, etc..). What about the so called “me time”? This schedule does not necessarily include the time for verging out in front of the TV, reading a good book, playing video games, writing a blog, catching up on your backed up RSS feeds (although most people do that at work these days), responding to emails and hacking away on that personal project of yours. If you are planning to do any, or all of that something else is got to give.

Scott claims that there are two ways to subvert this evil cycle, and neither one of them is very good. You either slowly kill yourself by not sleeping and/or exercising, or you take a pay cut and work less hours. For a while now I have been doing option #1, accruing an impressive sleep debt over the years. But I do believe there is a third path.

A wise man once told me that if you find a job you love, you will never have to work again. In fact I think he was channeling Confucius, but never mind that. I think this the key to the third path.

The problem naturally is finding a job that is both rewarding, exciting, challenging, interesting and fulfilling. If you wake up on a Monday morning and you actually don’t mind going to work that much, you have probably found that job. If you feel passionate about what you do and you take a great deal of satisfaction in your work, you are probably already there. If you get to tie in your hobbies, and your personal interest into your professional career you are threading the third path. If you get paid to contribute to your favorite open source project, or to nurse your personal pet project to life under protectorate of your company you are a lucky guy.

I think the third path is all about balance. It’s about taking that 8 hours of daily boredom and turning it into 8 hours of “fun”. It might sound silly, but I believe it is not impossible. And there are companies out there which seem to grok this concept. For example Google, which actually allows it’s developers to allocate portion of their work time to personal side projects. Of course I’m sure that no job is perfect, and that even working at Google has a dark side. I think that environment is just a part of the equation. The other important part is your attitude towards work.

Some people may find fulfillment and happiness wile launching some awesome startup based on some crazy idea they came up with. Others may find it working at a big, progressive company such as Google. Yet another may find personal fulfillment and serenity as a ditch digger.

Naturally, you still only get only 3 hours of personal time on a weekday. That part of equation doesn’t change. What changes is the setup:

Sleep: 8 hours
Exercise: 1 hour
Pursuing your dreams, achieving your goals and having fun: 8 hours
Eating: 2 hours (leisurely)
Hygiene: 1 hour
Travel: 1 (Commute, errands)

How does that look to you? Would you still say your life is slowly trickling through your fingers? Would you burn out as quickly? When you put things this way, the 3 hours of free time are merely just an icing on the cake. Easier said than done - I know. But I guess part of it is probably making choices, taking risks, and not settling for a job that merely “pays the bills”.

So how about you? Which path did you pick? Did you find that dream job yet? Are you looking? Or perhaps you had it and lost it? Share in the comments.

I finally got my hands on a fansubbed copy of the first installation of the much hyped and much anticipated cinematic re-make of the cult TV series Neon Genesis Evangelion. And no, this is not yet another movie that attempts to translate the original, infuriatingly abstract ending into something more approachable that would give fans a sense of closure. This is a complete re-telling of the story from the very first episode, all the way to the end.

Rebuild of Evangelion is actually planned as a tetralogy in which the 4^th and last part will be a brand new original ending which will differ from both the original, and the movie End of Evangelion.

Why to remake the cult TV series from 13 years ago now? I suppose money is a huge factor. Hollywood has proven time and time again that a glitzy remake of a time honored classic, or a cult title simply cannot fail at the box office. Especially if it is a super, special directors cut edition with never before seen footage. Take the Star Wars: Special Edition - you wouldn’t think that people would pay a ticket admission price to see the same old movie with few CGI shots of wompa rats and Banthas superimposed over the old footage. But they did.

So Rebuild of Evangelion is a little bit like that - same story, but with brand new CGI effects, and vastly improved animation and a new musical score. That and of course blatant product placement:

Naturally, this is not the only reason. There is also merchandising and tie-ins. But officially speaking, the reasons are a legion. You can read the official statement from Hideaki Anno (the mastermind behind this project) to see them for yourself. In short, he wants to tell the story from the beginning in a way that is more comprehensible, and more approachable to regular audience - and to sort of create the ultimate, directors cut, true version of the story with a definite ending where there previously was none. But mostly, I think it is about the money.

So how is the movie itself? In one word, unavailable in US. Or at least that’s what I summarized after reading the wikipedia entry and a cursory googe search. I could not find any information about US release schedule, or even if one is planned for the future. If you are more in the know than me, please post any and all available info in the comments. For me, this meant that I can proceed to procure a fan-subbed version of the show guilt free via the magical vehicle of bittorrent.

I’d love to give credit to the subber but the video came without an NFO and there are no credits in the subtitle track itself. The only clue I have about the identity of whoever did this was that the file was labeled with the [NF_ITK] tag which is I suppose either a scene group or a fansubbing group - but google didn’t really return any results for them either. The subs are serviceable but not perfect. A lot of the background chatter (especially during the Eva launch sequence) is simply missing. Fortunately I didn’t catch any “mass naked child events” type blunders which is a good thing. That’s as much I can say. I don’t actually know a lick of Japanese so validity of the sub remains to be determined. It seemed ok to me - and I recognized some of the memorable dialogs from the series (eg. the hedgehog’s dilemma and etc..).

As for the movie itself, I am somewhat underwhelmed. The plot is essentially a condensed version of the first 3 episodes which depict the attacks of the first 3 angels (Sachiel, Shamshel and Ramiel). While all the scenes were re-drawn and animated from scratch, with much more modern style of shading, and much more fluid movement, it seems that the team pretty-much re-used a lot of story boards from the series. Watching You Are (Not) Alone is very much like watching A New Hope: Special Edition. It’s the very same movie with a healthy dose of shiny new CGI and that one new really cool scene that was not in the original. The main difference is that while the Special Edition of Star Wars actually ran slightly longer than the original, the new Eva movie actually tries to condense events that originally spanned 3 hour episodes down into a 90 minute feature. I believe that some of the high school sequences depicting Shinji’s and Rei’s daily life ended up being cut. This means that we are seeing a bit less of our main character moping around and being miserable, but we also miss out on the little interactions that give his relationship with his classmates a bit more depth. It also slightly distorts your sense of time passage. The time between angel attacks seems much shorter now.

In fact, the whole thing felt a bit rushed. Since I’m familiar with the series, I could easily fill in the blanks, and I knew exactly what was going on at any given time. Perhaps this is due to the fansub quality, but I felt like we were not getting as much info as in the original. Perhaps, they intentionally decided to turn down the level of details and technobabble to meet Hideaki’s goal of increased approachability. If that’s the case, then I would say this was a wrong decision. Such a move would make the movie even less comprehensible to a new viewer. But it is hard to say without actually seeing an official translation. Remember - not a leak of Japanese here.

The new CGI is nice, but I’m not really someone who swoons over graphical fireworks. I saw Evangelion for the first time only few years ago so it was already a dated show. Still, I loved it despite the fact that it did not have flashy 3d effects which are so common in the contemporary anime. The combat is surely more dynamic and exciting, the angels have a new and improved look and you can observe the Tokyo-3 buildings rising out of the ground in painstaking detail. The original series had a tight budget, but this remake seems to be taking the “no expense spared” approach and it shows. Still, to me, visuals are secondary to the story itself. And this is where the movie is slightly underwhelming. Not that the story is bad in itself - after all it is the same store as before, which was excellent to begin with. It’s just that this is part of the problem - a brand new movie, with the same old story only in a more condensed mode. It’s a bit like watching that amateur Phantom Menace cut in which they remove all the stupid Jar Jar Brinks scenes - only this time, actually meaningful stuff gets cut or changed. There are notable exceptions though, which actually add to the story - some very ominous foreshadowing, and interesting tidbits here and there which almost make up for the loss of detail elsewhere.

What are these exceptions? Well, for one, Rei has nipples this time around. Yeah, I was surprised too. If I can remember correctly, the TV series had all the ladies sporting busts akin to those of a Barbie doll - ie. perfectly smooth featureless orbs. I guess Rei fans will love this upgrade especially since she is showing plenty of skin. But that’s largely irrelevant. There is more.

I spotted several very striking, and important departures from the source. If you don’t want to be spoiled, just scroll down to the very last paragraph of the post real quick and close your eyes not to see the screen shots. Mega spoilers lurk below.

First minor detail I noticed was that Shinji was actually at one point training in one of the unarmored simulation Eva bodies which were dormant and seemingly abandoned in the original. This is interesting, but not very significant. Yet it shows that they are fleshing out these little details, possibly in order to make the story more cohesive.

There are bigger surprises ahead though. For example, Misato not only knows what lies hidden in the Central Dogma but also has access to it. To convince Shinji to fight Ramiel, she takes him all the way down there, and shows him the crucified angel correctly identifying it as Lilith. In the original Misato did not know about it until it was revealed to her by Kanji near the end of the series. Furthermore, both of them assumed the angel on the cross was Adam not Lilith.

Furthermore, the new Lilith does not have the characteristic, symbolic mask with the 7 eyes. She is now wearing the plain, smooth gray mask similar to those seen on all the other angels. Perhaps this is done to establish her identity as one of the angels more firmly in the eyes of the audience? She is also not bleeding LCL. To that tune I do not recall Shinji ever saying that the LCL smells like blood in this version, but this once again might just be the fansubber dropping the ball. Still, this may suggest that they dropped the idea of LCL being the blood of Lilith.

Another huge departure from the original story is the scene in which Kaworu wakes up on the surface of the moon which seems to be stained with blood or something like that. He says something cryptic about “the third one” never changing, and expresses that he looks forward to meeting Shinji. In the original this character didn’t show up till the very end of the series and we initially did not know about his un-natural origins. Furthermore he seems to be standing over a body of yet another angel who looks strikingly similar to Lilith, and in fact is wearing a mask that looks very similar to the one Lilith wore in the original. Is that Adam by chance?

Overall, the movie by itself does not really live up to Hideaki Anno’s promises of being the ultimate anime of all times. It feels much more like watching Star Wars: Special Edition. But it is not all just a condensed carbon copy of the original with shiny new CGI and animated nipples either. The last act of the film seems to suggest that the creators are taking the story in a very different direction this time around. The next 3 movies that will soon follow You Are (Not) Alone are likely to become progressively more original leading up to a brand new and unexpected conclusion. There are also rumored new characters on the horizon that are yet to be introduced. So I suggest adopting a wait and see approach here. I will reserve my final judgment until I see the whole tetralogy in it’s entirety. While this first installment is not totally mind blowing, it seems like a good start. What remains is to hope that someone out there will at some point secure US distribution rights, and do a popper translation so we can all enjoy fully and support the whole project with our dollars without worrying about region codes, and the ever-present language barrier.

I was reading an article in a magazine last week and while I don’t exactly recall the magazine or the topic right now I distinctly remember mild annoyance at the editor’s decision to put list of URL’s at the end of the piece. Not that it’s a bad practice in itself but some of the listed links were actually pointing to Youtube videos. Youtube URL’s are not that long or difficult, but I don’t think they were designed to be used in printed media. For one, they use a cryptic case sensitive argument at the end which is a pain in the ass to type. I mean, look at it - the thing is ugly and awkward to type:

http://www.youtube.com/watch?v=oHg5SJYRHA0

It’s actually easier and quicker to simply google the videos in question by their title rather than even attempt to type the address by hand. It reminds me of the tricky exercise typing Microsoft’s CD product keys during activation. You hardly ever get them right on the first try. Not that I don’t appreciate the gesture but there is this dissonance between print media and electronic media. When you print URL’s you ought to keep their typability (is that a word?) in mind.

A lot of magazines do something different. They simply refer the reader to the online version of the article or specially prepared page on their website which contains the relevant links. This is an improvement, but you still usually end up with a fairly long URL’ which includes stuff like the publication date, the volume number, author’s name or all of the above. Sometimes the damn thing is to long, and they simply print a blurb to visit their site, and leave locating the right page as an exercise to the reader.

If I was distributing my content via printed media and had to cite online resources quite often, I’d probably look into URL shortening instead. I actually haven’t seen this being done on a bigger scale in magazines, but private people do it all the time. For example, Twitter users utilize services such as TinyURL, Snurl, is.gd or Twurl to post long addresses without exhausting the 140 character limit. So instead of a long ass Youtube link with a 10+ character alphanumeric, case sensitive argument you use something like this:

http://is.gd/AJ

Can you see the difference? First URL is extremely easy to mistype. The second one, almost impossible to get wrong.

Naturally, there is a downside to using a 3rd party URL shortening service - especially in print. The URL is no longer meaningful. In my first example, you at least know I’m sending you to a Youtube video. In the second one it’s not that clear. Furthermore, you are routing your visitor traffic through someone else’s service which may or may not be reliable or trustworthy. While you can get away with it in handouts for your class presentation or an informal documentation it is probably not recommended for official brochures, posters or other materials of that nature. It simply doesn’t look professional.

You want to use URL shortening though - just not the generic services open to anyone on the internet. What you ought to do is to roll your own. The added benefit is that you will get the first pickings on the really short 2 and 3 character URL’s which should last you for a while - which is usually not the case with services used by thousands of people.

How do you create your own URL shortening service? One way is to simply grab an off the shelf solution like Get Shorty. It is free, PHP + MySQL based donationware that does precisely what tinyurl and friends do, but on your own server. It actually gives you a choice on how you want to generate your shortened URL’s - it can do the semi-random 3-4 chacacter codes just like all the services out there, or it can prompt you for custom keywords - so you can actually construct your URL like so:

http://mydomain.com/my/keywords/here

This makes for nice readable URL’s that still retain your domain name (for the purpose of quality control, branding and etc..) but seamlessly redirect your users to relevant URL’s while giving them a hint of where they will be going as well.

Alternatively of course you could create your own system. It’s actually not that difficult to do this. First you will need a database. Let’s make it really easy and just use two fields - the ID and URL:

CREATE TABLE shorturl (
	id INT NOT NULL AUTO_INCREMENT, 
	url TEXT, 
	PRIMARY KEY(url));

If you didn’t notice, I just killed two birds with one stone here. The id is set to auto increment so it will generate unique values for us for free. Naturally this is tad limited as you will quickly run out of the nice 1, 2 and 3 character id’s but it’s simple and reliable.

Next, we need some .htaccess magic:

RewriteEngine on
RewriteBase /short/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /forward.php [L]

This should forward every link to a non-existent file in the directory /short/ to a script called forward.php. Then inside our script we do the following:

$id = end(explode("/", $_SERVER['REQUEST_URI']));
$result = mysql_query("SELECT id FROM shorturl where url='$id'");
$resultarray = mysql_fetch_array($result);
header("Location:".$resultarray['id']);

The REQUEST_URI gives me the address from which the user was redirected to my forwarding script. Whatever is behind the last “/” is my shortened id value. Once I grab that, I just need to look it up in the database, and then redirect you by sending your browser the “Location” header. Let’s say someone goes to the following URL:

http://mydomain.com/short/123

The script will look up 123 in the database, and then redirect this person to relevant URL that if finds.

Naturally, it will blow up in a very un-graceful way if the id is not found. I’m not really doing any error checking or validation here - I’m just illustrating the concept here. So don’t be copying and pasting this into any production code because it is likely wrong. Also, if it doesn’t work at all, let me know and post corrections because I admit I didn’t really test this. As I’m typing this it’s 2am so give me a break.

If you look at the get-shorty code you will note that they do it slightly differently, but I didn’t want to blatantly rip them off. There is clearly more than one way to do it, and you can probably vastly improve my .htaccess code up there.

I really think this is the way to go for print friendly URL’s. It’s easy to implement (I just did it and I’m half asleep) and it gives the reader easy to type shortcut that will take him directly to the resource without unnecessary searching or pit stops along the way.

Let me preface this post by saying that I nitpick because I love. It seems that what started as a random nitpick will turn into a whole series of posts wot the “Cylons don’t use” in the title. This one is about encryption. Yes, I have another bone to pick with the writers. This time it is about this:

screencap © galacticabbs.com

This is the super-secret note that Colonel Tigh (a very senior military officer, and the former leader of the New Caprica resistance movement) passed to Chief Tyrol (who also had experience in NC resistance movement) about an incredibly secretive meeting which was accidentally intercepted by Tyrol’s wife Cally. This sort of thing happened to me too - back in grade school. Since then, I have learned to use encryption.

Let’s review - both Tigh and Tyrol are Cylons (if I just spoiled you, then go back and watch S3 finale already and stop complaining) and they must keep this secret. If anyone would find out about who or what they really are, their lives vrypotwould be in danger. Therefore keeping their true identity secret should be their paramount concern - and no effort should be spared to cover their own tracks. But here we have an experienced leader who had tons and tons of experience operating behind enemy lines make a colossal, bone headed mistake that might have exposed them. It is just stupid!

I do think about encryption all the time because of my education and my job - I’m conditioned to notice these things. A lot of people never even consider it. For example a housewife cheating on her husband may not have been exposed to concepts such as cryptography and information security so she may never even think about using anything but unsecured notes like this one. But these folks, for one have a military training. I have never been in the military so I might be wrong about this but I would think that security and simple cryptography would be part of basic training at least for officers. I’d think that they ought to know about cryptography in case they find themselves behind enemy lines, or in a position where all their communications are intercepted. If I’m wrong, please correct me.

Even if they didn’t get this knowledge in basic training, you can’t tell me they haven’t developed strong, low tech methods of encryption during the New Caprica occupation. Let’s face it - if a silly note like that ended up in the hands of the occupying forces it could mean many deaths, or loss of valuable resources. They simply could not afford to openly communicate their attack plans or resistance movement secrets in plaintext.

What should they have used? There are many low-fi methods to obfuscate or encrypt hand written notes. A simple Caesar Cipher would probably be good enough for plausible deniability in front of Cally. Would she be curious enough to try to crack it? Perhaps, but it is likely that Tyrol could easily play the gibberish note as some sort of an entry code he needed on a routine repair job or simply shrug and act as if it was just that - gibberish.

For real security they should probably use One Time Pad encryption. The pad is a long random sequence of symbols that is used as a key to encrypt and decrypt your message. This method was used successfully by CIA and KGB in the past. How does a pad look? It can be something like this:

image © ranum.com

It is really a relatively secure method as long as you destroy the pads after each use, and you have a foolproof method of securely exchanging them. Our protagonists do have one, since they meet regularly. On each meeting they could exchange pads, which they would then use for written communication until the next meeting. The pads can be small and easy to conceal. A very nice disguise for a pad is a stack of papers with random stream of characters printed on one side, and unrelated hand written notes on the back. Plausible deniability - it’s just scrap paper - some gibberish spit out by some malfunctioning printer and you are just using it for personal notes.

When handled properly, it’s virtually unbreakable and in my honest opinion would be near prefect for this setup. But it seems that the BSG writers have never took a cryptology class. I’d say they ought to hire a consultant who could advise them on blunders like this one, but the difficult part is that an average person wouldn’t even think that this part of the screenplay would need to be looked at by a technologically clueful person.

So this is my BSG nitpick for today. I love that show, and this is why I hold it to a high standard and will ruthlessly pick on the little annoying little bits like this one. They simply destroy my enjoyment and prevent me from buying into the drama. I should be freaked out that Cally found the note and she will expose the final 5 but all I can think of is that C. Thigh is an idiot for leaving that note in plaintext.

Sigh… Sometimes I think I would enjoy TV and movies much more if I was just absolutely, technologically inept and clueless.

I’m on JavaScript kick lately. Or rather, I have been rediscovering it ever since I read the series of Steve Yegge’s blog posts on JS and Rhino and his predictions for the next big language. I really think he is right - JS will be the next big thing that will trump Ruby from it’s pedestal or at least become its viable competitor as the next big dynamic scripting behemoth. Why? I think a better question is: why not? It almost seems like this language is destined for greatness.

For one, JavaScript or rather ECMAScript is really a kick-ass language if you look at it objectively. When I say JavaScript most people automatically think about the client-side, browser dependent, inconsistent scripting language which only works half of the time. But this is not really the fault of the language but inconsistent implementation of the DOM model across different browsers.

Internally the language is really consistent, flexible, readable and fun to use. Not only does it have most of the cool features that are all the rage now (ducktyping, closures, variadic functions, prototypes, metaprogramming end etc..) but reads a lot like C. The syntax is full of the familiar and lovely brackets and semicolons that we all know and love (or hate) and all the control sequences and language constructs follow the familiar C patterns. Coding in JavaScript is really a lot like coding in Java only 5 times less verbose which is a good thing.

And when it comes to shuffling around HTML or XML data there is no other language that is both so good at it, and so well documented. We have been using it for that purpose decades now and nothing else compares. No other language can be used int the online so effectively and seamlessly.

Also, Javascrip has something that neither Python nor Ruby possess - a real spec. So it is really a good language - a Language with capital L. It is just burdened with a bad stigma of being a browser specific toy language.

This stigma is slowly dissipating as the Web 2.0 craze is raging on. I don’t care who you are right now - if you are doing any web development right now, you have to deal with Javascript whether you want it or not. You can avoid it, you can claim you are server side guy but if you want to be competitive right now your web app needs to use AJAX and if it is, then Javascript is becoming a big internal part of your code base. You can hide behind a framework, and have it generate AJAX code for you the way RoR does, but sooner or later you will have to do something complex and will end up hacking on js files. Whether you love it or hate it - you will be exposed to it at some point.

So your code is now roughly half Javascript and Half something else which is a bit messy, but that’s Web 2.0 for you. Does it have to be that way though? Of course not. You don’t need a dedicated client side language ad another dedicated server side language. You could really do all your scripting on both sides with Javascript. Yes, Server Side Javascript is not a joke or some pipe dream - it is reality. Believe it or not, but there is a lot of development going on in this area - there are tons of projects out there which use JS on the server side.

These projects range from very simple to very complex full featured solutions. For example, mod_js is an Apache module loosely based on Perl’s CGI.pm which implements ECMAScript on the server side. It is astonishingly simple, and compact but also very immature project.

If you want something more robust and mature there is Jaxer - a self contained Java based server which supports server side scripting in Javascript. The whole idea behind this environment is unifying your code base. With Jaxer you can use the very same language to query the database, dynamically rearrange the HTML output, and make asynchronous calls back to the server. Added bonus is that it is bundled with the very popular Aptana IDE (aka. the Eclipse for web designers) which means it is already near ubiquitous.

If you are allergic to Eclipse and it’s forks and you like NetBeans then Phobos is probably what you will want to sink your teeth into. It uses Glassfish server and uses Javascript as it’s default server side language. Same concept, different approach. The project seems to be mature, well supported and have a big community.

Want more? Theres Helma which is a Rails like framework which leverages server side Javascript and Rhino. The whole thing runs in the JVM and uses Jetty as the default, self contained web server - but you can also put it behind Apache or IIS if you want. How come you can go wrong with a combination like that? You get a MVC framework with a self contained server and set of useful tools and utilities that only requires you to have a basic JVM installed. You get to script it with the dynamic and flexible language that requires no compilation, and no SDK. You get the entirety of the Java API at your disposal courtesy of Rhino. Not only that, but you can simply drop in your custom precompiled Java libraries and classes and seamlessly integrate them into your project.

But there is more - recently I spotted another interesting project called Junction. Like Helma it is a Rails like MVC framework that uses Rhino but in addition, it also has a built in support for Google Gears. This means that apps built on the Junction framework get to benefit from the offline persistence features offered by the Google technology.

There is all this development going on out there. It’s not very visible yet - most of it is somewhat behind the scenes. But it’s like that distant rumbling before the storm. You know something big is coming. Just look around.

You want to develop rich client interfaces? There’s XUL and Adobe Air for you - both using Javascript at their core.

You want dynamic web UI framework? There is JQuery, Prototype, Dojo, Google Web Toolkit and YUI just to name few.

Want to use Java API? Use Rhino. Want to tap into the richness of Microsoft CLR and interact with the rest of .NET ecosystem? Theres JScript.NET. You want to do some native scripting on MacOS? There is Javascript OSA.

You want to add oflline persistence to your web applications? You need Google Gears which is also using Javascript.

The language is everywhere. Everything that is new, that is hot and that relates to the web is at least touching or benefiting from it. Every developer will see it, and work with or alongside it sooner or later. It’s ubiquitous.

The pieces are slowly falling in place and setting up the stage for the upcoming explosion. It’s no longer question of whether we will reach that critical mass, but when will it happen. We have the tools, we have the technology, we have the hook and the need (unifying the client and server side under one language) - now we just need to wait for the hype. And for that, we need branding. Steve Yegge talked about this during his Ted talk - we need to re-brand the language to make it more sexy. Everyone thingks that Javascript is a toy language, and ECMA sounds like a some sort of acne prescription so we need a new, cool name. We also have a good reason for re branding - the upcoming release of Javascrip2 standard which brings about many new features to the language such as classes, interfaces, namespaces, packages, destructuring assignments, structs, type declarations and more.

It is a bucket load of new features so why not start fresh, with a new sexy name and identity. I mean look at AJAX - the acronym really means “using the asynchronous calls in Javascript” but if you say it like that, people immediately fall asleep and run away. But AJAX is cool and sexy. Javascript can be to - and I predict it will be, sooner or later.

There are two storytelling crutches that often come into play to “hand wave” over some plot inconsistencies or explain something odd and unusual. One of them is called magic, and the other one is called technology. They are both used as explanations of improbable and impossible events which are supposed to help the viewers suspend their disbelief.

It is usually very easy to make viewers accept magical events. All you have to do is to make up some magical system and say “this is just how magic works in this mystical world”. After all, we don’t have magic in the real world, so we might as well go with it. As long as the rules governing this magical system seem logical, consistent within the story, and do not contradict themselves it is easy for us to accept it.

Technology tends to be used the same way. Unfortunately, unlike magic we are pretty good at science. A lot of viewers have scientific degrees and at the very lest, most people took a science class or two in school. So if you start bending and contradicting laws of physics, we can easily call you on it. When you introduce a futuristic technology it must not only be consistent, but also plausible. Of course if you are blatant enough about it, and make it obvious you don’t even pretend to be using “real science” you end up in the same realm as magic. For example Lightsabers are so far fetched that we just accept them even though it would be impossible to construct them. Same goes for various methods that allow spaceships to travel faster than light. We all know it’s impossible, but we agree on this convention because it makes the story more interesting. These devices have no scientific basis for them so they are really nothing more but techno-magic. But closer you show us something that looks and acts just like existing real world technology, we will sort-of expect it continue working within the same limitations and in the same way.

Having a degree in Computer Science and working in IT for a long time doesn’t help me suspend my disbelief when writers try to include technology in their stories without actually bothering to understand how it works in the real world. Usually any movie or show that features “hackers” or computer security experts and etc.. makes me roll my eyes so hard, and so often that it actually hurts. Science Fiction often doesn’t cause as much eye strain for me - often because it’s usually pretty good about portraying science and technology in plausible way. Still, every once in a while even hard SF or Space Operas manage to drop the ball and annoy me with inane or poorly researched and thought out “future tech”.

For example, let’s take one of my all time favorites - Battlestar Galactica. The Cylon resurrection system that is an important part of the story has been pissing me off for a while now. I don’t have a problem with the system itself but it is portrayed as having an inherent flaw that more than once was used as a plot device. It’s just that the flaw is ridiculous, and it is almost inconceivable that no one ever worked out a countermeasure for it. Any engineer or sysadmin would ask two questions during his first visit to a Cylon resurrection shop - “how do you do backups?” and “What is the emergency restore procedure to the when a unit fails away from the ship without having a chance to transmit the latest state?”. Sad part is that it seems that no one ever thought about that.

For those of you who haven’t watched the series let me explain.

In the BSG universe, when an individual Cylon dies his or her consciousness is instantaneously transferred into a vat grown spare body which is kept on a specialized, well protected Resurrection Ship. The transmission seems to be instant, and happens at the exact moment of death but this process seems to have a limited range. This is why every Cylon fleet takes at least one Resurrection Ship with them wherever it goes. So far, so good - I don’t have a problem with that - in fact it is a pretty good system. Personally, I’d have a resurrection chamber on each Base Star and turn it into a distributed system, but I guess growing new bodies requires space and resources so centralizing this function is not such a bad idea.

The show however came up with an interesting idea. Want to truly kill your Cylon enemies in a way that prevents them from ever coming back? It’s easy as pie. All you have to do is to destroy the local Resurrection Ship and then simply do your thing. Yep, it’s that easy. Imagine being a Cylon and having to deal with that. If you accidentally die 10 feed beyond the range of the closest R-ship then BOOM! Permadeath! Your near immortality is over.

The sysadmin part of me reels in horror after hearing this. Who designs systems like this? What we have here is a permanent, irrecoverable data loss happening every time an expendable and temporary shell for your AI expires beyond the range of your wireless network. Thats is a criminally stupid design flaw - and one which could be easily fixed! Didn’t these people hear about backups?

Let’s think about it - the transmission of consciousness is almost instant. We have seen it on the show many times. Old body dies, and the Cylon wakes up in a vat of goo almost instantly. This tells me that it takes only a split second to send given Cylon’s memories and personality back to the resurrection ship, and few more seconds to actually upload it to a spare body. Since they are able to do this at all I think it is indisputable that heir consciousness can be fully digitized. There is this notion out there that digitizing human consciousness will never be possible because there is actually more to it than a mapping of neural connections and distribution of electric charges. I don’t know whether or not this is true, but obviously Cylon psyche can be transmitted digitally without any problems. If you can digitize something, you can back it up to permanent storage.

In other words, if Cylons have technology to upload the transmitted consciousness to a vat grown body, then they absolutely must have a technology to temporarily or permanently store this data on some sort of media. I mean it is a prerequisite - before you can put the consciousness in a body, you have to be able to store it somewhere. In fact, I imagine that the Resurrection Ships must be buffering these transmissions in some way before they do the body uploads.

For example, what happens if a Base Star is destroyed? Now you have instantaneous transmission from hundreds if not thousand Cylons streaming down from the wreckage, probably on the same frequency. What happens if the vats are not ready? What if there is a temporary power outage in the vat room? What if the prepared bodies turn out to be defective and need to be discarded? What if the transmission comes in with potential errors caused by outside interference? It’s just simple engineering concerns - you need to have a buffer which will hold the upload until the body is ready. Otherwise you would often miss the window of opportunity and consciousness transfers would simply get lost in space for silly reasons. Not to mention the need to check for transmission errors. It would be a pity to see a Cylon emerge from a vat half-retarded because the post-death transmission included static noise that could have been easily removed in some pre-processing CRC check.

So, if they are processing and buffering the signal there is no reason why they couldn’t just dump it into storage for safe keeping. What is the difference between storing it in memory for 20 seconds and saving it in storage for 3 days? Well, main difference is that the latter is cheaper than the former on all counts - at least by current technology standards.

So why not do a daily backup? Or even hourly incremental backup for that matter? Or even a continuous rolling sync to the R-Ship storage since it seems the consciousness transfer is almost instant? If the Resurrection Ship kept the “last good copy” of each Cylons consciousness then instead of permanent death, a stranded Cylon would simply loose last few hours or days of his/her life. If you would make Resurrection Ships sync up their backup data on regular basis, you now have a highly efficient distributed system that offers you virtual immortality. Permadeath would only be possible if someone would manage to track down all the Resurrection Ships with the copy of your consciousness.

I’ll go even further - why not allow Cylons to make emergency backups to personal storage they could then stow away in a safe place. Drop a thumb drive in a safe deposit box on Caprica before going on an important mission and you can be restored even if your whole fleet was wiped out.

So yeah, I don’t buy the whole “Oh noes, where is the R-ship? If we die now we die FOR REALS!” I just refuse to believe that a race of sophisticated intelligent machines would neglect something as important and basic as backups. It is a small thing, but it bugs the hell out of me.

In the past if you missed an episode of your favorite show you had very limited choices. If you had Tivo or a similar service you could just pray and hope it was smart enough to record it. You could also hope it was available on some sort of on-demand service provided by your cable. Failing that, you could either wait for a re-run, or go online and download the episode from your favorite P2P network. Added bonus was that the friendly folks who were uploading the shows usually grabbed them from HD feed, and were kind enough to remove commercial breaks. I did this myself several times for this or that show. There is just no way to prevent it.

At the time I used to wonder why the networks didn’t simply put the episodes online, complete with commercial interruptions. Since they already circulate in the wild, why not turn around and profit from it. This rejection of adopting a viable stream of revenue out of fear that it will make it easier for pirates to “steal it” seemed silly to me. I mean, some groups out there are able to start seeding an episode 15-20 minutes after it was aired - all they need to do is to grab the file from their VDR, and cut out the commercials. Not releasing a digital copy of your show didn’t necessarily impede their work.

Conversely, if you stream the episodes from the official website, surround them with one or two advertising banners and include commercial spots embedded in your stream, you are capitalizing on several of the 8 generatives Kevin Kelly talked about in his essay Better than Free. Since this is an official release the fans of the show know immediately where to find it (they can bookmark the page, instead of searching P2P networks), and furthermore they know the content is authentic and trustworthy which is not always the case with the less than legal sources.

Well, lo and behold - this idea is starting to take ground. This Friday I missed the new Battlestar Galactica episode and to my surprise I noticed that all 3 episodes of the new season are available via SciFi Channels Rewind service. Yes, the quality is low, and the commercial breaks are a tad annoying but you know what - I don’t really care. I just want to catch up with so that I can participate in online discussions and the obligatory Monday lunch hour BSG discussion at work. And for that the grainy, slightly above Youtube quality compressed FLV is just fine. Hell, it’s in fact more convenient since all I need to do is to click a button and watch. I don’t need to wait for the download to finish, and I don’t need to worry that part 5 of 6 will be taken down before I am able to watch it as it is often the case on the TV-Links like link services.

I wanted to commend Scifi and all the other networks which choose to follow this route. I’m glad people are slowly begging to see the light. When you put episodes online like this we all win! You get an extra stream of revenue and I get to watch my show whenever I want. This is exactly how you should combat piracy - compete with them on terms they cannot match. The streaming show on your website is official, safe, permanent and readily available when needed - it is better than free.

Is this an indicative of change? I don’t think it’s just Scifi that is doing this. I think ABC was doing something very similar for Lost episodes, although I’m not sure if they just offer the last episode, or all of them. So there are networks out there that are beginning to understand how to make a buck in the digital world, and that is filling me with hope. Perhaps future is not as bleak as we all suspected.

The semester is almost over. I was convinced that I will actually have one regular class in May before the finals, but I was wrong. The regular schedule at MSU ends on the 5^th and then we go into the two week final examination period. Since I teach an evening class, I get to give the exam on May 13th which is I think the very last day of exams. So naturally I already have students clamoring that we should push the exam up.

One guy even told me he booked a trip on the 10^th and he doesn’t know what to do now. Sigh… I could understand this if the official final exam schedule was not posted on the university website on like the first day of the semester. But it was, and I could have told him the exact date and time of the final on Sept 1 if he asked. Sigh… I had professors who didn’t even tell us when the final was - they expected you to look it up and show up at the right time, or they failed you. I sometimes wonder how some of these folks actually survive in college.

So I’m not moving the exam up (I’ll work with my “special” cases on individual basis, and figure out what to do with them) but since I only have 1 regular lab period left I think my students will get to skip the LOGO exercise. I actually never did this lab before and it was supposed to be sort of a test run. But they won’t have enough time. will need to skip it.

Back in February I asked you what programming language should we teach first to CS majors. We pretty much established that it is usually a good idea to start with a lower level language like C or C++ and then bump them up to something more friendly like Java or Python as needed. What do you teach non CS majors though? It is generally a good idea to expose people to programming to show them it is not some sort of scary arcane magic.

This is where LOGO comes in. It is a is a functional programming language considered to be a distant cousin of Lisp. It doesn’t share Lisp’s peculiar syntax though. It is actually very, very simple - it uses one word which usually take a single argument. There are no brackets, semicolons or fancy control structures. It supports very simple loops and naturally recursion (after all it is based on lisp) - which is probably not something I would show to my 109 class. Simple iteration though is probably something that they could handle.

I was planning to have them use Tortue which is a Java based app with a built in interpreter and a canvas for drawing shapes. I actually found it via the CMPT 109 community on blackboard and it seems for this purpose. It doesn’t support all of LOGO but it does support a significant portion. In fact, it makes the language even simpler. For example, it does not support the short versions of the keywords, or the square bracket notation, which make code more compact, but may impact readability if you are a fresh user.

I like it because it requires no installation (which is something students cannot do in the lab) and is astonishingly simple. Students simply type the code in the left pane, hit run, and the results are drawn on the canvas. Yes, logo programming revolves mostly around drawing shapes. Let me show you a very simple code snippet:

REPEAT 4
	FORWARD 100
	LEFT 90
END REPEAT

This little snippet for example will draw a 100 pixel wide square on the screen. Tortue basically only supports drawing functions and very basic arithmetic. Logo itself is actually full featured and can be used to solve complex problems, but drawing on a canvas is really it’s favorite thing to do.

I’m not exactly sure what kind of program I would assign them. Probably something really simple that could be accomplished even without loops. I was thinking along the lines of tetris blocks since there is more than one way to draw them. You can either use stacked squares, rectangles, or just draw them with a single continuous line.

Perhaps I’ll manage to squeeze this exercise in next semester. What do you think about this approach? I’m not really expecting to teach these people any hard core programming. I just want to expose them to coding, debugging and solving problems. Or is logo to complex? I know some teachers use the 3D Alice framework to teach these concepts. It was being pimped at the same community portal where I found Tortue and I actually saw Alice files all the computers in the adjunct office so someone out there is using it. I looked at that toolkit, the first impression kinda to my first time using Macromedia Flash - I was overwhelmed by it’s complexity. I had to look at the tutorials to actually figure out what was one supposed to, so I can only imagine how my students would feel.

In comparison, simple textual commands like FORWARD 100 or LEFT 90 don’t seem that bad. Which reinforces my long standing opinion that CLI interfaces are actually easier to learn. While a well designed GUI can be self explanatory and can be learned by instinct, complex ones quickly become overwhelming. CLI on the other hand has to be learned gradually and because of it’s inherent modal nature, the user is only exposed to small portion of the system at once.

If you had a choice, would you teach Alice or Logo?

I just think Alice

My office has several network printers. Most of them are HP and have zero issues using them from under Linux. They worked right out of the box - all I had to do was to set them up as Windows/Samba shared printers via the KDE printer setup GUI. One that always gave me problems was the big, black and bulky Dell 3100cn color printer sitting in the corner which was not supported.

I finally figured out how to bend it to my will. If you digg around on the Dell website you will notice that it does offer Linux drivers for that machine. Only they are Red Hat drivers. No debs, despite the fact that Dell is officially distributing Ubuntu with their systems. Why is that? Who knows. The important part is that they are there.

These are Linux drivers, and that makes our life easier. If they work for Red Hat, they can be made to work for Ubuntu. The systems are not that different after all. Here is how we are going to do this.

First grab the Red Hat drivers from the Dell website:

wget ftp://ftp.us.dell.com/printer/Dell-Laser-Printer-3100cn-1.3-1.noarch.rpm

Next, let’s transform the rpm into a deb package using alien:

sudo alien Dell-Laser-Printer-3100cn-1.3-1.noarch.rpm

If all goes well, this should spit out a deb file into the same directory. The name is exactly the same, but for some reason alien likes to lower-case the whole thing. You should be able to install it bu doing the following:

sudo dpkg -i dell-laser-printer-3100cn_1.3-2_all.deb

This should do it, unless you run into some dependency issues (I had none) or some other problem. That was the hard part. It’s smooth sailing from here my brothers. Just pull up the KDE printer interface and let’s try to add our new printer:

Nothing out of unusual yet. By trial and error I learned we need to use the TCP/IP Network Printer option on the next screen:

On the next screen just put the IP address of your printer. You know what it is, right? In my office the printer is actually hooked up to a wireless printserver from Linksys which actually does have an IP address of it’s own. If you are using the built in networking logic this step might be different. Anyways, this is insignificant detail.

Finally you should be able to specify a printer by picking it from the driver list. A DELL entry should automagically appear in there, and it should have a 3100cn option for you to choose. If you don’t see it, you probably installed the package wrong. If you do see it, pick it, then keep clicking next until you are prompted for a printer name, enter it and you are done.

Easy as pie (that is if you consider pies to be easy)! Now I can finally print in color! Wohoo!

Btw, this worked on Gutsy. If you are running Hardy, you are on your own.