Terminally Incoherent

The other day I wrote about my little Drag and Drop application and mentioned I wanted it to send HTTPS POST requests to an existing PHP web application. I thought it would be a relatively trivial task, but it turned out not the be as easy as I thought. First let me show you the code, and then discuss the pitfalls.

Sending the request by itself is actually pretty easy. Observe:

// this is what we are sending
string post_data = "foo=bar&baz=oof";
 
// this is where we will send it
string uri = "https://someplace.example.com";
 
// create a request
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create(uri); request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
request.Method = "POST";
 
// turn our request string into a byte stream
byte[] postBytes = Encoding.ASCII.GetBytes(str);
 
// this is important - make sure you specify type this way
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postBytes.Length;
Stream requestStream = request.GetRequestStream();
 
// now send it
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Close();
 
// grab te response and print it out to the console along with the status code
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());
Console.WriteLine(response.StatusCode);

Piece of cake, right? Even if you don’t know C# you should be able to follow this. As a side note, can you see how freakishly verbose this shit really is? I mean, how come ruby can implement this like this:

Net::HTTP.post_form(URI.parse('https://somplace.example.com'), 
   {'foo'=>'bar', 'bas'=>'off'})

Or python for that matter:

data = urllib.urlencode({"foo" : "bar", "baz" : "oof"})
urllib.urlopen("https://somplace.example.com", data)

Is it really necessary for me to declare all that shit and go through all the motions to perform something that other languages do in a single line? This is one of these reasons why I no longer think that the traditional Java/C# approach of static typing, and overwhelming verbosity is the way to go. The functionality here is the same - and yet Ruby and Python totally win on expressiveness and readability hands down.

But I didn’t really start this post (just) to rag on the verbosity of C#. The problem with the code above is that it doesn’t work if your certificate is not valid. Why would I be posting to a web page with and invalid SSL certificate? Because I’m cheep and I didn’t feel like paying Verisign or one of the other jerk-offs for a cert to my test box so I self signed it. When I sent the request I got a lovely exception thrown at me:

System.Net.WebException The underlying connection was closed. Could not establish trust relationship with remote server.

I don’t know about you, but to me that exception looked like something that would be caused by a silly mistake in my code that was causing the POST to fail. So I kept searching, and tweaking and doing all kinds of weird things. Only after I googled the damn thing I found out that the default behavior after encountering an invalid SSL cert is to throw this very exception.

Fortunately, there is a quick and dirty workaround. You simply need to subclass the ICertificatePolicy class from the System.Security.Cryptography.X509Certificates namespace and rig it so that it always validates the certificate, no matter what:

using System.Security.Cryptography.X509Certificates;
using System.Net;
 
public class MyPolicy : ICertificatePolicy
{
  public bool CheckValidationResult(ServicePoint srvPoint, 
    X509Certificate certificate, WebRequest request, 
    int certificateProblem)
  {
    //Return True to force the certificate to be accepted.
    return true;
  }
}

Once you do that, you just need to toss the following line somewhere in your code, before you actually send the request. This will swap your default CertificatePolicy class for our fake one with the validation hack:

System.Net.ServicePointManager.CertificatePolicy 
    = new MyPolicy();

Note that the compiler may whine about this approach being an obsolete, but since this is a pretty ugly hack in itself, I paid no heed to it. There might be a better way to do this in .NET 3.5 but since it worked I let it be for now.

So there is how you do it. A more relevant exception message would be a lot of help in this circumstance. Perhaps something among the lines of “invalid certificate”? But I’m just thinking out loud here. If you ever run into this issue, here is how to solve it.

I do talk a lot of trash about Microsoft on this blog, but since I work in a place which is a Windows shop I can’t really get away from it’s products. Today I dusted off my “C# Developer” hat, and started chugging away. Why .NET? Because it seemed like the right tool for the job. The task at hand was to figure out a way how someone could grab bunch of emails out of her Outlook, and then drag and drop them onto something so that:

1. The emails get saved as .msg files in an appropriate folder on the network share
2. Our internal intranet application becomes aware of these files, and display download links where applicable

So instead of forwarding emails to 4-5 people assigned to a given job, the scheduling person would simply put them in a central location on the server and appropriate people would see download links on their “project page” once they log into the web app. This would limit the “did you get that thing I sent you?” questions, and allow an administrator to add people to a project without the need of hunting down all the relevant emails and forwarding them to the new guy.

Communicating with the web application is not an issue - I can do that in any language. The tricky part the drag and drop functionality which is the whole point of this project. The whole exercise of saving the email as a .msg file and uploading it to the web app using a traditional HTML form was briefly considered for .5 seconds and then unanimously rejected because it would be a royal pain in the ass. On the other hand typing in a task number and then dragging/dropping relevant emails onto some sort of a form is kinda like forwarding them - only you just have to do it once.

I briefly considered Java, but the interaction between Outlook and the Java application looked like a really hairy business. On the other hand .NET seemed like a perfect choice. You’d think that Microsoft would have a relatively easy mechanism of scripting it’s flagship office product in their flagship programming environment. And it turns out that there is - it’s called MS Office Primary Interop Assemblies. These are bunch of COM objects which you can import into your Visual Studio project and they allow you to interact with Office applications at a somewhat intimate level. Unsurprisingly, they are dependent on the version of MS Office you are using (in my case Office 2003) making it hard to create a robust, catch-all solution. You can grab a distributable installer package for Office 2003 version from here.

I added the Microsoft Office 11 Core Library and Microsoft Outlook 11 Core Library as the references to my project. They automagically appeared under the COM tab of my reference dialog in Visual Studio after installing the redistributable package linked above.

Next on the menu was the Drag and Drop functionality. That was pretty easy. In fact, an article written by Tgueth on Code Project gives you a very good primer how to set things up. Just ignore his code, because it won’t work for what I’m trying to do. He was simply interested in dropping actual onto the form - so real files dragged over from Explorer, or Outlook attachments dragged from your email. The email themselves however are handled differently, and cannot be captured this way. I tried, and I failed. There is however an alternate solution using the Interop Assemblies I described above - and believe it or not, the code is actually much cleaner and more compact.

So I sat down and hacked up a quick prototype. Here is the relevant code:

namespace DragAndDrop
{
  public partial class Form1 : Form
  {
    private Microsoft.Office.Interop.Outlook.Application OL;
    private string my_dir = "C:\\";
 
    public Form1()
    {
      InitializeComponent();
      OL = new Microsoft.Office.Interop.Outlook.Application();
    }
 
    private void Form1_DragEnter(object sender, DragEventArgs e)
    {
      e.Effect = DragDropEffects.Copy;
    }
 
    private void Form1_DragDrop(object sender, DragEventArgs e)
    {
      for (int i = 1; i <= OL.ActiveExplorer().Selection.Count; i++)
      {
        Object temp = OL.ActiveExplorer().Selection[i];
 
        if (temp is Microsoft.Office.Interop.Outlook.MailItem)
        {
          Microsoft.Office.Interop.Outlook.MailItem mailitem = (temp as Microsoft.Office.Interop.Outlook.MailItem);
          string subject = mailitem.Subject.Replace(":", "");
          mailitem.SaveAs(my_dir + "\\" + subject + ".msg", Microsoft.Office.Interop.Outlook.OlSaveAsType.olMSG);
        }
      }
    }
  }
}

Let’s break it down, shall we? The first method Form1_DragEnter describes what is supposed to happen when you drag an object onto your form. We simply want to change the Effect property of the event to Copy:

e.Effect = DragDropEffects.Copy;

If you don’t do this, then your code will detect drop effects but that’s about it - it won’t know or care what was dropped. This will ensure that the target of the drop will hold on to a copy of whatever you dragged over when you drop it.

Next, we need to handle the items that were dropped in the Form1_DragDrop method. The following statement returns an array of items which are currently selected with your mouse (ie. the stuff you are dragging right now):

OL.ActiveExplorer().Selection

Since I want to allow users to drag bunch of emails at a time, I’m going to iterate over this array and then grab each item. Since potentially a user could drag over something that is not an email, I’m temporarily saving it as an Object:

Object temp = OL.ActiveExplorer().Selection[i];

If it turns out to be an Outlook email object we cast it back to Microsoft.Office.Interop.Outlook.MailItem and save it. Note that I use the subject line as the filename (an quickly sanitize it removing the illegal “:” character) and save it in the root of the C: drive for now.

The SaveAs method of the MailItem object takes two parameters: first one is the path where you want to save the item, and the other is the file type which must be a member of the OlSaveAsType Enumeration which is a part of the Interop library. I’m using olMSG since that will save as the default Outlook message format.

This code works under Office 2003 but there is one small issue. For every mail item you drop, Outlook will throw this nasty security warning message:

The good news is that the user has the option to disable it for up to 10 minutes. The bad news is that 10 minutes is the maximum amount of time you can choose from that drop down box. I do see why they have this warning there, I just suspect it will become really annoying unless I find out a way to disable it. I did not cross that bridge yet though. Any suggestions? So far the bare bones functionality is working and I can drag and drop email messages onto my little form, and have them saved perfectly along with attachments which is precisely what I wanted.

Next step is to figure out how to make that thing talk to a PHP + MySQL backend of our web application, but that should not be that difficult. All I need to do is to send a HTTPS POST request back to the server, and then have something on the other end process that data and dump it into a database. It doesn’t really need to be much more complicated than that.

I was reading an article in a magazine last week and while I don’t exactly recall the magazine or the topic right now I distinctly remember mild annoyance at the editor’s decision to put list of URL’s at the end of the piece. Not that it’s a bad practice in itself but some of the listed links were actually pointing to Youtube videos. Youtube URL’s are not that long or difficult, but I don’t think they were designed to be used in printed media. For one, they use a cryptic case sensitive argument at the end which is a pain in the ass to type. I mean, look at it - the thing is ugly and awkward to type:

http://www.youtube.com/watch?v=oHg5SJYRHA0

It’s actually easier and quicker to simply google the videos in question by their title rather than even attempt to type the address by hand. It reminds me of the tricky exercise typing Microsoft’s CD product keys during activation. You hardly ever get them right on the first try. Not that I don’t appreciate the gesture but there is this dissonance between print media and electronic media. When you print URL’s you ought to keep their typability (is that a word?) in mind.

A lot of magazines do something different. They simply refer the reader to the online version of the article or specially prepared page on their website which contains the relevant links. This is an improvement, but you still usually end up with a fairly long URL’ which includes stuff like the publication date, the volume number, author’s name or all of the above. Sometimes the damn thing is to long, and they simply print a blurb to visit their site, and leave locating the right page as an exercise to the reader.

If I was distributing my content via printed media and had to cite online resources quite often, I’d probably look into URL shortening instead. I actually haven’t seen this being done on a bigger scale in magazines, but private people do it all the time. For example, Twitter users utilize services such as TinyURL, Snurl, is.gd or Twurl to post long addresses without exhausting the 140 character limit. So instead of a long ass Youtube link with a 10+ character alphanumeric, case sensitive argument you use something like this:

http://is.gd/AJ

Can you see the difference? First URL is extremely easy to mistype. The second one, almost impossible to get wrong.

Naturally, there is a downside to using a 3rd party URL shortening service - especially in print. The URL is no longer meaningful. In my first example, you at least know I’m sending you to a Youtube video. In the second one it’s not that clear. Furthermore, you are routing your visitor traffic through someone else’s service which may or may not be reliable or trustworthy. While you can get away with it in handouts for your class presentation or an informal documentation it is probably not recommended for official brochures, posters or other materials of that nature. It simply doesn’t look professional.

You want to use URL shortening though - just not the generic services open to anyone on the internet. What you ought to do is to roll your own. The added benefit is that you will get the first pickings on the really short 2 and 3 character URL’s which should last you for a while - which is usually not the case with services used by thousands of people.

How do you create your own URL shortening service? One way is to simply grab an off the shelf solution like Get Shorty. It is free, PHP + MySQL based donationware that does precisely what tinyurl and friends do, but on your own server. It actually gives you a choice on how you want to generate your shortened URL’s - it can do the semi-random 3-4 chacacter codes just like all the services out there, or it can prompt you for custom keywords - so you can actually construct your URL like so:

http://mydomain.com/my/keywords/here

This makes for nice readable URL’s that still retain your domain name (for the purpose of quality control, branding and etc..) but seamlessly redirect your users to relevant URL’s while giving them a hint of where they will be going as well.

Alternatively of course you could create your own system. It’s actually not that difficult to do this. First you will need a database. Let’s make it really easy and just use two fields - the ID and URL:

CREATE TABLE shorturl (
	id INT NOT NULL AUTO_INCREMENT, 
	url TEXT, 
	PRIMARY KEY(url));

If you didn’t notice, I just killed two birds with one stone here. The id is set to auto increment so it will generate unique values for us for free. Naturally this is tad limited as you will quickly run out of the nice 1, 2 and 3 character id’s but it’s simple and reliable.

Next, we need some .htaccess magic:

RewriteEngine on
RewriteBase /short/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /forward.php [L]

This should forward every link to a non-existent file in the directory /short/ to a script called forward.php. Then inside our script we do the following:

$id = end(explode("/", $_SERVER['REQUEST_URI']));
$result = mysql_query("SELECT id FROM shorturl where url='$id'");
$resultarray = mysql_fetch_array($result);
header("Location:".$resultarray['id']);

The REQUEST_URI gives me the address from which the user was redirected to my forwarding script. Whatever is behind the last “/” is my shortened id value. Once I grab that, I just need to look it up in the database, and then redirect you by sending your browser the “Location” header. Let’s say someone goes to the following URL:

http://mydomain.com/short/123

The script will look up 123 in the database, and then redirect this person to relevant URL that if finds.

Naturally, it will blow up in a very un-graceful way if the id is not found. I’m not really doing any error checking or validation here - I’m just illustrating the concept here. So don’t be copying and pasting this into any production code because it is likely wrong. Also, if it doesn’t work at all, let me know and post corrections because I admit I didn’t really test this. As I’m typing this it’s 2am so give me a break.

If you look at the get-shorty code you will note that they do it slightly differently, but I didn’t want to blatantly rip them off. There is clearly more than one way to do it, and you can probably vastly improve my .htaccess code up there.

I really think this is the way to go for print friendly URL’s. It’s easy to implement (I just did it and I’m half asleep) and it gives the reader easy to type shortcut that will take him directly to the resource without unnecessary searching or pit stops along the way.

The semester is almost over. I was convinced that I will actually have one regular class in May before the finals, but I was wrong. The regular schedule at MSU ends on the 5^th and then we go into the two week final examination period. Since I teach an evening class, I get to give the exam on May 13th which is I think the very last day of exams. So naturally I already have students clamoring that we should push the exam up.

One guy even told me he booked a trip on the 10^th and he doesn’t know what to do now. Sigh… I could understand this if the official final exam schedule was not posted on the university website on like the first day of the semester. But it was, and I could have told him the exact date and time of the final on Sept 1 if he asked. Sigh… I had professors who didn’t even tell us when the final was - they expected you to look it up and show up at the right time, or they failed you. I sometimes wonder how some of these folks actually survive in college.

So I’m not moving the exam up (I’ll work with my “special” cases on individual basis, and figure out what to do with them) but since I only have 1 regular lab period left I think my students will get to skip the LOGO exercise. I actually never did this lab before and it was supposed to be sort of a test run. But they won’t have enough time. will need to skip it.

Back in February I asked you what programming language should we teach first to CS majors. We pretty much established that it is usually a good idea to start with a lower level language like C or C++ and then bump them up to something more friendly like Java or Python as needed. What do you teach non CS majors though? It is generally a good idea to expose people to programming to show them it is not some sort of scary arcane magic.

This is where LOGO comes in. It is a is a functional programming language considered to be a distant cousin of Lisp. It doesn’t share Lisp’s peculiar syntax though. It is actually very, very simple - it uses one word which usually take a single argument. There are no brackets, semicolons or fancy control structures. It supports very simple loops and naturally recursion (after all it is based on lisp) - which is probably not something I would show to my 109 class. Simple iteration though is probably something that they could handle.

I was planning to have them use Tortue which is a Java based app with a built in interpreter and a canvas for drawing shapes. I actually found it via the CMPT 109 community on blackboard and it seems for this purpose. It doesn’t support all of LOGO but it does support a significant portion. In fact, it makes the language even simpler. For example, it does not support the short versions of the keywords, or the square bracket notation, which make code more compact, but may impact readability if you are a fresh user.

I like it because it requires no installation (which is something students cannot do in the lab) and is astonishingly simple. Students simply type the code in the left pane, hit run, and the results are drawn on the canvas. Yes, logo programming revolves mostly around drawing shapes. Let me show you a very simple code snippet:

REPEAT 4
	FORWARD 100
	LEFT 90
END REPEAT

This little snippet for example will draw a 100 pixel wide square on the screen. Tortue basically only supports drawing functions and very basic arithmetic. Logo itself is actually full featured and can be used to solve complex problems, but drawing on a canvas is really it’s favorite thing to do.

I’m not exactly sure what kind of program I would assign them. Probably something really simple that could be accomplished even without loops. I was thinking along the lines of tetris blocks since there is more than one way to draw them. You can either use stacked squares, rectangles, or just draw them with a single continuous line.

Perhaps I’ll manage to squeeze this exercise in next semester. What do you think about this approach? I’m not really expecting to teach these people any hard core programming. I just want to expose them to coding, debugging and solving problems. Or is logo to complex? I know some teachers use the 3D Alice framework to teach these concepts. It was being pimped at the same community portal where I found Tortue and I actually saw Alice files all the computers in the adjunct office so someone out there is using it. I looked at that toolkit, the first impression kinda to my first time using Macromedia Flash - I was overwhelmed by it’s complexity. I had to look at the tutorials to actually figure out what was one supposed to, so I can only imagine how my students would feel.

In comparison, simple textual commands like FORWARD 100 or LEFT 90 don’t seem that bad. Which reinforces my long standing opinion that CLI interfaces are actually easier to learn. While a well designed GUI can be self explanatory and can be learned by instinct, complex ones quickly become overwhelming. CLI on the other hand has to be learned gradually and because of it’s inherent modal nature, the user is only exposed to small portion of the system at once.

If you had a choice, would you teach Alice or Logo?

I just think Alice

The internal web application at my company tends to send out a lot of email notifications. Some of them are reminders about various deadlines. Recently I got a feature request to allow people to add these reminders as events to their Outlook calendar.

Outlook supports two calendaring formats vCalendar and iCalendar. I opted to go with the later one because it is an open standard and at least in theory should work with other calendaring applications. The apps which support iCalendar standard can export their events as .ics files which can be seamlessly exchanged between the applications. So my task was to generate such a file on the fly and either link to it, or attach it to the reminder emails.

You can actually serve the ics file directly from a php page by telling the browser you are sending it a text/Calendar mimetype. You do this by sending appropriate headers:

header("Content-Type: text/Calendar");
header("Content-Disposition: inline; filename=calendar.ics");

This will force the browser to interpret the input as a ics file and will show the user the dialog asking him to open the file in an external application. Then you simply need to output the appropriate data in the iCalendar format, which is relatively simple. Here is a very simple event you can copy and paste into your PHP file:

header("Content-Type: text/Calendar");
header("Content-Disposition: inline; filename=calendar.ics");
echo "BEGIN:VCALENDAR\n";
echo "VERSION:2.0\n";
echo "PRODID:-//Foobar Corporation//NONSGML Foobar//EN\n";
echo "METHOD:REQUEST\n"; // requied by Outlook
echo "BEGIN:VEVENT\n";
echo "UID:".date('Ymd').'T'.date('His')."-".rand()."-example.com\n"; // required by Outlok
echo "DTSTAMP:".date('Ymd').'T'.date('His')."\n"; // required by Outlook
echo "DTSTART:20080413T000000\n"; 
echo "SUMMARY:TEST\n";
echo "DESCRIPTION: this is just a test\n";
echo "END:VEVENT\n";
echo "END:VCALENDAR\n";

This will generate an event called TEST to happen on April 13 2008. Note the fields which I marked as required by Outlook. This is important. They are not required by the iCalendar spec so some examples out there skip them. If you do however you will get this lovely error message when you open the file in Outlook:

Microsoft Office Outlook cannot import vCalendar file. This error can appear if you have attempted to save a recurring Lunar appointment in iCalendar format. To avoid this error, set the appointment option to Gregorian instead of Lunar.

Gotta love Microsoft and their through, meaningful error messages. In this case the message is absolutely meaningless and the problem has nothing to do with “Lunar appointments” whatever the hell those are. You are simply missing the METHOD, DTSTAMP and UID fields.

I used REQUEST for the METHOD field but you can also try PUBLISH and few others. As far as I can tell the difference here is how the calendaring application will present the opened file to the user. If you use REQUEST it will give them a button to Accept or Reject the event. If you use PUBLISH they will be asked to save or cancel. Either way is acceptable by me.

The DTSTAMP field is, as the name suggest the date stamp when this event request was generated. As you can see I’m generating it on the fly.

The UID is supposed to be a unique identifier for this event, for some internal use. It is recommend that the right hand side of the UID be the host name or some unique identifier for the domain. I’m not exactly sure what the significance of this field is - I suspect it may be meaningful in certain situations, where the calendaring app is internally sorting the data. As you can see I use the date stamp on the right, followed by a randomly generated number, followed by host name which is pretty much what they advocate in the spec. It should yield UID’s that are sufficiently unique for each event. Some examples out there just use 0 or 1 here which doesn’t seem to be following the specification, but nevertheless it works just fine when you test it. However I figured it’s better to do it right, rather than run into weird issues much, much later.

I think the rest of the fields are pretty much self explanatory, with exception of PRODID. That’s the funny one. It is supposed to uniquely identify the product which has generated this event. In other words it is the place to pimp out your company and your software. And yet, all the online examples use the same 2 or 3 strings, showing blatantly that most people just copy and paste a working example into their production code and then blog about it without ever actually bothering to read the spec itself.

This is how I did it. I’m sure there are more elegant ways to accomplish this, but my email reminders will simply link to an appropriate php page which will fetch relevant data from the database and build appropriate ics file. I hope someone out there will find this writeup useful. And if I see “Foobar Corporation” in the future writeups, I will know you copied from me, and didn’t even bother reading the rest of the post.

I must confess that Rails makes me feel stupid every time I use it. The accepted truism about the framework is that it boosts your productivity like no other. Unfortunately people forget to tell you that there is second part to this statement that goes something like this: “once you learn to think the Rails way”. It really forces a certain mindset upon you, and deviating from it means that you are actually working against the framework, rather than having it do the work for you. It takes a little while to get used to it, and there are moments when you have a great scaffold thing going on with bunch of interacting tables/entities, but you sit there for 20 minutes trying to figure out how to make a simple pull down menu (aka select statement) which would let you choose the foreign key from the other table. You could do it the hard way, but it turns out that it is astonishingly simple:

<%= collection_select :foo, :bar_id, bar.find(:all), :id, :bar_name %>

This will create a select statement looking something like this:

<select name="foo[bar_id]">
    <option value="1" selected="selected">Bar 1</option>
    <option value="2">Bar 2</option>
    <option value="3">Bar 3</option>
</select>

It really took me some digging to figure that out - mainly to realize that what I needed was in a ActionView::Helpers::FormOptionsHelper class. A lot of the online tutorials simply gloss over little details like that - for example, the importance of helpers, which I now know are pretty damn convenient.

Then there is that whole Rails 2.x vs. Rails 1.x debacle. The two are not entirely compatibile, and there are significant differences in the way they work. Needless to say, when version 2.0 instantly invalidated every single Rails book on the market by removing the active scaffolding which everyone was using in the initial examples. I already got burned on it once, and now I was hit with it again, only from the other side. When I decided to install rails on Gutsy, I did what any reasonable Ubuntu would do:

sudo aptitude install ruby rails mongrel

Few minutes later I was all set up and ready to go. Or was I? I gently issued a command like this:

script/generate scaffold Foobar foo:string bar:string

I got hit by some cryptic error about unknown string identifier or something among those lines. WTF? It took me few minutes of useless googling, and cursing to realize I simply had the old version of rails installed. I could have went along and simply use the nice active scaffolding for my project, but I figured if I am to learn this damn framework, I should probably use the latest and the greatest version. How to install it on Gutsy though? The answer is - via gems.

First, get rid of the 1.x rails installation if you actually have it on your system:

sudo aptitude remove rails

Next, install the new rails:

sudo gem install rails --include-dependencies

That should do it. I think it’s possible to downgrade back to 1.x if you remove rails via gem and then install it back via apt bur I haven’t tried it.

Also, small caveat - you may or may not need to update your gems package to do that. You can do it by issuing a command:

sudo gem update --system

Be warned that it will actually break the gem command itself. If you try running it, you will get the following error:

/usr/bin/gem:23: uninitialized constant Gem::GemRunner (NameError)

Why is this? Take a look at this:

$ ls -l /usr/bin/ | grep gem
-rwxr-xr-x  1 root   root        701 2007-08-24 01:18 gem
-rwxr-xr-x  1 root   root        785 2008-04-01 11:25 gem1.8
-rwxr-xr-x  1 root   root       3201 2007-08-24 01:18 gemlock
-rwxr-xr-x  1 root   root       1778 2007-08-24 01:18 gem_mirror
-rwxr-xr-x  1 root   root        515 2007-08-24 01:18 gemri
-rwxr-xr-x  1 root   root         70 2007-08-24 01:18 gem_server
-rwxr-xr-x  1 root   root       1813 2007-08-24 01:18 gemwhich
-rwxr-xr-x  1 root   root       7947 2007-08-24 01:18 index_gem_repository

Apparently all the gem_* commands have been deprecated in the 1.x releases of rubygems. The version in gutsy repo is 0.9.4 which means it still uses them. The update command brings you to 1.1.0 release but unfortunately does not remove the old scripts from /usr/bin. So the original gem and gem_ commands are useless. Quick workaround here is:

sudo mv /usr/bin/gem /usr/bin/gem.old
sudo ln -s /usr/bin/gem1.8 /usr/bin/gem

You could probably remove the gem binary, but I simply renamed it, and then created a link to gem1.8 in it’s place. It works well enough, and if you need to do a downgrade later on, all the files are still intact.

My family has been decimated by the Sudoku bug. I’m actually physically tired of making the “Hey, the year 2005 called and they want their game back” jokes all the time. I’m not sure why is this happening now, but nevertheless everyone keeps solving the damn things. For added humor, my dad keeps forgetting the name of the game and keeps referring to it as seppuku. Whether I want it or not that damn puzzle has entered my household and is not going away anytime soon. Because of this I started looking at he puzzle more closely as an interesting problem to be cracked.

A while ago I mentioned an effective Sudoku solving algorithm - one that you can use with your pen and paper puzzle. For the sake of completeness, let me recount it here:

1. Start by writing numbers 1-9 in each empty cell of the puzzle (or get the nifty template with pre-filled numbers)
2. Pick one of the given (seed) numbers and eliminate it from the row, column and quadrant it is in (ie. cross it out from the 1-9 lists in these cells)
3. If you are lucky, few cells may end up having only one number on the list that was not crossed out - that’s the number that belongs in that cell. Take that number and eliminate it in a row, column and quadrant as above
4. If you can’t find a singleton number scan each crow, column and quadrant for a unique non repeating number (ie. one that appears only in a single cell in that row, column or quadrant. If you find one, that’s your number - put it in the cell where it appears and eliminate it from the row, column and quadrant

I believe that you can solve most of the easy, medium and hard sudokus this way. However, now that I actually did some digging, there seems to be a subset configurations in the “evil” range which actually requires trial and error strategies and can’t be solved by simply sticking to the algorithm above. Which of course doesn’t mean the puzzle is not solvable.

In fact, if you look at sudoku really hard it really boils down to the good old graph coloring problem. A standard 9×9 Sudoku grid translates to a graph with 81 vertices. Each vertex has exactly 20 edges connecting it to 20 neighboring vertices (8 more in the row, 8 more in the column and 4 more in the quadrant) and the chromatic number is 9. The bad news is that graph coloring is NP-hard. The good news is that sudoku’s don’t really scale upwards that much. So you can use an algorithm that performs acceptably with sudoku sized graphs (9 colors, 81 vertices, 1620 edges between them) and stick with it.

The fact that sudoku is a book case graph problem not surprising. A lion share of real life problems can be solved using nothing more but graph theory. So here is a word of advice for you younger whipper-snappers out there. Do use the Algorithms class to sleep off your drunken debaucheries - especially not on the day when they are covering Graphs. But don’t take just my word for it, listen to Steve Yegge’s recent rant:

Graphs are, like, really really important. More than you think. Even if you already think they’re important, it’s probably more than you think.

(…)

Whenever someone gives you a problem, think graphs. They are the most fundamental and flexible way of representing any kind of a relationship, so it’s about a 50-50 shot that any interesting design problem has a graph involved in it. Make absolutely sure you can’t think of a way to solve it using graphs before moving on to other solution types. This tip is important!

Big question: how many real world graph problems have I encountered at my at my job since I got my degree? I’ll give you three guesses.

The sad truth is that I haven’t touched these things since my algorithms class. Part of it is the nature of my work - which involves a lot of sysadmin stuff, in addition to hacking my PHP+MySQL monster into submission. The problems I face on the daily basis can mostly be solved in polynomial time and usually revolve around storing and fetching data from a database.

This state of things is regrettable but not unusual - a lot of people are in the same boat. Nevertheless, we are getting rusty and these graph problems are important, if for nothing else than for impressing interviewers when looking for another job.

Which tells me I need to dig out my Algorithms book and knock out a Sudoku solver. But which language should I use? I’m thinking Ruby just to get more feel for this language.

Few days ago I did that whole 3-value checkbox thing basing it on some script I found online. I went back and I re-implemented it using some JQuery magic. I’m not going to reiterate the whole setup here. I recommend that you check the linked post for details. You can find the JQuery-fied code below:

$(document).ready(function() 
{
   checked = '/path/to/checked.jpg';
   unchecked = '/path/to/unchecked.jpg';
   na = '/path/to/na.jpg';
   i = 0;
 
   // replace the checkboxes with images    
   $("form.funky_form > fieldset.funky_set :checkbox.funky_box").hide().each(function() {
 
      img = document.createElement('img');
      img.className = "funky_image";
 
      switch($(this).val())
      {
         case "0":
            img.src = unchecked;
            $(this).attr("checked", "false");
            break;
         case "1":
            img.src = checked;
            $(this).attr("checked", "true");
            break;
         case "2":
            img.src = na;
            $(this).attr("checked", "true");
      }
 
      // these will let us identify which image was clicked
      // and which checkbox does it belong to
      $(this).attr("id", "input" + i + "image" + i);
      $(img).attr("id", "" + i + "image" + i);
 
      i++;
 
      $(this).before(img);
   });
 
   // add onClick functionality to the new images
   $(".funky_image").click(function() {
 
      // select the checkbox corresponding to the clicked img
      t = $("#input" + $(this).attr("id"));
 
      switch(t.val())
      {
         case "0":
            $(this).attr("src", na);
            t.val(2).attr("checked", "true");
            break;
         case "1":
            $(this).attr("src", unchecked);
            t.val(0).removeAttr("checked");
            break;
         case "2":
            $(this).attr("src", checked);
            t.val(1).removeAttr("chekced");
      }             
   });
});

The major difference from the other version is that I’m much more discerning in which checkboxes get converted. The code will select only a box that is of the class funky_box and is inside a fieldset with a class of funky_set and inside a form with a class of funky_form. I’m mostly doing that to show power of JQuery - this is all specified in that very first select statement. I was trying to do something similar in the old code, but I was getting hung up on the silly DOM gotchas. JQuery makes this easy.

Also note the chaining functions. The first line after I declere i both hides all the selected check-boxes and begins the each block. Similarly, check out the second switch statement. In several places I set the value of t and change the checked attribute on the same time. It’s very expressive, and lends itself toward very compact and concise code.

Here is the HTML for the form to go with the code above (note the inclusion of the fieldset tag):

<form name="funky_form" method="POST">
      <fieldset class='funky_set'>
         <label for="c1">
            <input type="checkbox" name="c1" value="1" class="funky_box" checked>
            Some Important Task
         </label><br>
 
         <label for="c2">
            <input type="checkbox" name="c2" value="1" class="funky_box" checked>
            Some Important Task #2
         </label>
      </fieldset>
 
      <label for="c3">
         <input type="checkbox" name="c3" value="1" class="funky_box" checked>
         This is a checkbox with the funky_box class outside the funky_set
      </label><br><br>
 
   <input type="submit" value="Submit">
</form>

How readable is this version? To me it is actually better because it’s smaller, and more compact. To someone who never worked with JQuery it might be a bit confusing at first but I think you get used to the weirdness pretty quickly. I find the selection statements much more elegant than nested loops for example, even if they might The more code I can see on my screen the better. I’m not golfing though - all the methods and most variables have meaningful names (well, except stuff like t and i but you know how it goes).

Which version do you like better?

Here are two interesting quirks in the way JQuery works. Let’s say you want to check all the check-boxes on the page - how do you do it? It’s trivial:

$(":checkbox").attr("checked", "true");

Now quickly, how do you un-check all the check boxes on the page? If you said:

$(":checkbox").attr("checked", "false");

You are wrong. Despite the fact that if you use a more standard notation and say for example:

document.myform.mycheckbox.checked = false;

it will give you the desired result. The JQuery call however does not work. In fact, it does the exact opposite - it will set all the check-boxes on the page to their checked state. Apparently for the few funky attributes like “checked” and “selected” the second argument of the attr() method call can be any non-empty string. It makes sense because you actually don’t have to give these attributes an explicit value in your HTML code for them to work. So how do you un-check a box in JQuery? There are two ways. First one, and the more sure-fire one is to remove the checked attribute altogether:

$(":checkbox").removeAttr("checked");

Second one which has worked for me for some reason was to set the attribute to the empty string:

$(":checkbox").attr("checked", "");

Same goes for select attributes in combo boxes and list boxes. You need to remove them or set them to an empty string, or they won’t go away.

Quirk number two: axjax weirdness. Look at the code below:

$("a").hover(function() {// do something});
$("#sometrigger").click(function() { 
   $("#container").load("/some/page.html #links"); });

What am I doing here? I’m adding some hover functionality to all the links on the page. Then I add an on-click even to #sometrigger element. When it is clicked I want to load the contents of #links from /some/page.html into #container asynchronously. Where is the quirk? The hover functionality will not apply to the dynamically loaded links. It’s weird, but that’s just how it works. So if you want this functionality on all your links you will need to reapply them:

$("a").hover(function() {// do something});
$("#sometrigger").click(function() { 
   $("#container").load("/some/page.html #links", 
      $("#container > a").hover(function() {// do something});
         });
   );

It’s a little bit silly, but it kinda makes sense. You may not always want to apply the same events and formating to newly loaded html. But if you don’t expect it, it may throw you off. One you know about it however you learn to compensate for it. I’m being told that folks coming from the Prototype community are hitting this little quirk like a brick wall because their framework behaves in the opposite way (ie. the effects and behaviors are applied to the ajax loaded html).

I’m doing more and more work with JQuery lately, and as I mentioned several times before I’m totally loving Javascript again. It is really a very concise and terse framework so it doesn’t really take long until you start feeling like you know it inside out. But it is very powerful and learning to use it properly, and efficiently will probably take me a while. Last time when I was raving about it I did it without any neat code samples to show you how cool it is, so this post will have a few.

One nice thing about this framework is that it is all about closures. Most of the JQuery functions take a pointer to a callback function as an argument, and the selectors automatically loop through DOM instances they match forcing you to write very different code. For example, if I wanted to add an onClick event to all links on my page with regular javascript I would probably do something like this:

function addOnlclickToLinks()
{
    myLinks = document.links;
    for(var i=0;i < myLinks.length;i++) 
    {
       myLinks[i].onclick = return onClickFunction();
    }
}
 
function onClickFunction()
{
    // the on click code goes here
}
 
 window.onload = addOnClickToLinks

It’s readable, and understandable code, but it looks messy. JQuery saves you tons of typing and really makes the code more ellegant:

$(document).ready(function() {
    $("a").click(function() {
        // on click code goes here
    });
});

This is a totally new way to code, and it forces you to think in a much different way. After doing this for a while, you come to rely on this new way of coding to the point where it trips you up. For example, the other day I wanted to do something like this:

$("input.someclass").after(function() {
    if($(this).val() == foo)
    {
        // do somethign complex
    }
    else
    {
        // do something else
    }
 
    return something;
}());

What I want to do here is to generate element called something based on the value of the selected input - and the process may actually be quite complex, requiring several lines, or multiple function calls. The sample above doesn’t work because I’m using the $(this) selector in the wrong context. To the best of my knowledge, the way to achieve this sort of thing would be by doing:

$("input.someclass").each(function() {
    if($(this).val() == foo)
    {
        // do something complex
    }
    else
    {
        // do something else
    }
 
    $(this).after(something);
});

The each function actually iterates through all the selected elements, and runs the passed function in the context of each one. At least I think this is how you would do it.

Either way, it is really a much nicer way to program and the way you select DOM elements and interact with them is consistent across platforms. If you still haven’t tried JQuery, I highly recommend checking it out. Javascript is really a great language - it has a somewhat traditional syntax (braces and semicolons, ftw) but all the awesome features of a modern highly dynamic scripting language like Python or Ruby. The only thing that is holding it down these days is the inconsistency in which it is interpreted across browsers. JQuery fixes that issue, and draws on some of the cool dynamic features of the language to really make your life easier.