Terminally Incoherent

The other day I wrote about my little Drag and Drop application and mentioned I wanted it to send HTTPS POST requests to an existing PHP web application. I thought it would be a relatively trivial task, but it turned out not the be as easy as I thought. First let me show you the code, and then discuss the pitfalls.

Sending the request by itself is actually pretty easy. Observe:

// this is what we are sending
string post_data = "foo=bar&baz=oof";
 
// this is where we will send it
string uri = "https://someplace.example.com";
 
// create a request
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create(uri); request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
request.Method = "POST";
 
// turn our request string into a byte stream
byte[] postBytes = Encoding.ASCII.GetBytes(str);
 
// this is important - make sure you specify type this way
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postBytes.Length;
Stream requestStream = request.GetRequestStream();
 
// now send it
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Close();
 
// grab te response and print it out to the console along with the status code
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());
Console.WriteLine(response.StatusCode);

Piece of cake, right? Even if you don’t know C# you should be able to follow this. As a side note, can you see how freakishly verbose this shit really is? I mean, how come ruby can implement this like this:

Net::HTTP.post_form(URI.parse('https://somplace.example.com'), 
   {'foo'=>'bar', 'bas'=>'off'})

Or python for that matter:

data = urllib.urlencode({"foo" : "bar", "baz" : "oof"})
urllib.urlopen("https://somplace.example.com", data)

Is it really necessary for me to declare all that shit and go through all the motions to perform something that other languages do in a single line? This is one of these reasons why I no longer think that the traditional Java/C# approach of static typing, and overwhelming verbosity is the way to go. The functionality here is the same - and yet Ruby and Python totally win on expressiveness and readability hands down.

But I didn’t really start this post (just) to rag on the verbosity of C#. The problem with the code above is that it doesn’t work if your certificate is not valid. Why would I be posting to a web page with and invalid SSL certificate? Because I’m cheep and I didn’t feel like paying Verisign or one of the other jerk-offs for a cert to my test box so I self signed it. When I sent the request I got a lovely exception thrown at me:

System.Net.WebException The underlying connection was closed. Could not establish trust relationship with remote server.

I don’t know about you, but to me that exception looked like something that would be caused by a silly mistake in my code that was causing the POST to fail. So I kept searching, and tweaking and doing all kinds of weird things. Only after I googled the damn thing I found out that the default behavior after encountering an invalid SSL cert is to throw this very exception.

Fortunately, there is a quick and dirty workaround. You simply need to subclass the ICertificatePolicy class from the System.Security.Cryptography.X509Certificates namespace and rig it so that it always validates the certificate, no matter what:

using System.Security.Cryptography.X509Certificates;
using System.Net;
 
public class MyPolicy : ICertificatePolicy
{
  public bool CheckValidationResult(ServicePoint srvPoint, 
    X509Certificate certificate, WebRequest request, 
    int certificateProblem)
  {
    //Return True to force the certificate to be accepted.
    return true;
  }
}

Once you do that, you just need to toss the following line somewhere in your code, before you actually send the request. This will swap your default CertificatePolicy class for our fake one with the validation hack:

System.Net.ServicePointManager.CertificatePolicy 
    = new MyPolicy();

Note that the compiler may whine about this approach being an obsolete, but since this is a pretty ugly hack in itself, I paid no heed to it. There might be a better way to do this in .NET 3.5 but since it worked I let it be for now.

So there is how you do it. A more relevant exception message would be a lot of help in this circumstance. Perhaps something among the lines of “invalid certificate”? But I’m just thinking out loud here. If you ever run into this issue, here is how to solve it.

I said it before, and I’ll say it again - Linux is not for everyone. Convincing people to switch their primary OS is a little bit like trying to convince someone to change their religion, or political views. It’s possible, but not easy. You can’t really “convince” someone that your OS is superior in a discussion. The only way you can really accomplish something like that is by subtle influence - by showing, teaching, and leading by example. I never try to sell Linux to others - I simply use it and talk about the good and bad things that entails with a passion. I found that this approach is actually makes the right kind of people interested in the platform.

To that end however I know many people who like Linux, own dual boot systems, use it occasionally and are passionate about it but never make the plunge of making it their primary OS because they need application X or Y. Note that I didn’t say switch. Switching is the last thing I would want someone to do. I personally believe in being platform agnostic. Your personal computing ecosystem should be a healthy mix of different operating systems and you should be able to jump from Windows to Linux to MacOS without really skipping a beat. Each platform offers different things, and there is no reason to confine yourself to a single one. OS is not a religion. I naturally don’t mean that you ought to go and spend money on new systems so that you can experiment. What I mean, is that instead of throwing out or giving away your old computer when you upgrade to a new one, set it aside and use it to install that one OS you always wanted to try.

But I got side tracked. Let’s get back to the poor Linux enthusiasts who can’t live without that single Windows-only application they actually use every day. I often hear them saying things like “I would banish Windows from my hard drive long time ago if only there was a good Linux alternative for application X” or “if application X would actually run under Wine reliably”. Excuses, excuses, excuses.

I did some testing, and I found out that there is a reliable solution for most of these people. I will let you guess where I’m going with this. Let’s see if you were paying attention to my posts for the last few weeks. The answer here is Virtual Fucking Machines. And I use the f-word here for added emphasis and not to signify teledildonics or something silly like that. Virtualization will let you run most of the crucial Windows-only applications under Linux with no glitches and no problems. For example, for the last few days I have been hacking away in Visual Studio on a .NET application that is supposed to work with MS Outlook using Ubuntu as my primary OS. How did I do that? I was running Windows XP in a VirtualBox. So I was using a real Windows environment, and real Windows application without the need to actually boot into that OS. My machine has 2.4 GHz Intel Centrino Duo CPU and 2 GB of RAM which is actually pretty decent, but not entirely impressive by today standards, but I was able to comfortably run a Virtual Windows XP session. Note that I was running full KDE desktop with several applications open on the Linux side (namely Kontact, Firefox, Gvim and Komodo Edit). The performance was just fine, and the build times were actually pretty good. In fact, I don’t think I would get much more speed running windows natively because I would likely be bogged down with some bloated security suite.

In fact, I must say that Office 2003 inside the emulated session was actually much faster and responsive than Open Office running natively outside the VM. So I would say that roughly 80 to 90% of these killer windows apps that people cannot live without could be comfortably run this way. I haven’t really tested Photoshop (one of the most often mentioned apps in this category) but I don’t see why wouldn’t it run semi-decently in a VM. You may need to allocate more memory to the VM if you usually work with big images, but other than that I believe you should be fine.

There is of course a class of applications which won’t do well in a VM. These are performance intensive apps that usually like to talk directly to your video card such as games, 3d rendering apps and etc. Gaming is one of these things that will remain a Windows domain for a long time. If you are a gamer and you want to play the newest titles on the market, your gaming box needs to run Windows. There is just no way around it - performance is paramount, and it will always be worse under a VM. Not to mention the fact that you won’t have low level access to your system’s 3D acceleration hardware while running in a virtual sandbox. Same goes for rendering or doing 3d animation that is windows, or architecture specific. Some of the stuff you just can’t fake.

But apps which do not need low level hardware access, and are not total resource hogs that take over your whole machine in full screen mode should be happy to run in a virtual machine. So no more excuses people. Next time you have a chance, boot into your Linux partition, download VirtualBox or VMWare then install Windows inside it, and set up your work environment there. Then next time you need to use that killer windopws-only app of yours just fire up your VM and work from there. When you are done, shut it down and enjoy your life in a pristine Linux environment - just like you always wanted.

If the performance sucks, or you think the whole VM thing is a hell of an inconvenience then you can start booting back into your windows partition. I won’t blame you. This is just a friendly suggestion for those of you who would love to take the plunge into Linux - to live it and breathe it every day - but seem to be chained down to Windows by this one or two silly apps. This is something that could let you do your first step towards becoming a better Linux user, and let you learn the intimate inns and outs of the new OS while still having the old one at your fingertips when you need it.

Just don’t try virtualizing Vista yet - that’s a whole mess of issues in itself. XP will still be a perfectly serviceable OS for the next few more years. And by the time they end-of-life it and it is no longer safe to keep it around hopefully hardware and virtualization technology will catch up to the point where running Vista (without Areo) in a VM will be as pleasant and worry free as running XP is now.

I do talk a lot of trash about Microsoft on this blog, but since I work in a place which is a Windows shop I can’t really get away from it’s products. Today I dusted off my “C# Developer” hat, and started chugging away. Why .NET? Because it seemed like the right tool for the job. The task at hand was to figure out a way how someone could grab bunch of emails out of her Outlook, and then drag and drop them onto something so that:

1. The emails get saved as .msg files in an appropriate folder on the network share
2. Our internal intranet application becomes aware of these files, and display download links where applicable

So instead of forwarding emails to 4-5 people assigned to a given job, the scheduling person would simply put them in a central location on the server and appropriate people would see download links on their “project page” once they log into the web app. This would limit the “did you get that thing I sent you?” questions, and allow an administrator to add people to a project without the need of hunting down all the relevant emails and forwarding them to the new guy.

Communicating with the web application is not an issue - I can do that in any language. The tricky part the drag and drop functionality which is the whole point of this project. The whole exercise of saving the email as a .msg file and uploading it to the web app using a traditional HTML form was briefly considered for .5 seconds and then unanimously rejected because it would be a royal pain in the ass. On the other hand typing in a task number and then dragging/dropping relevant emails onto some sort of a form is kinda like forwarding them - only you just have to do it once.

I briefly considered Java, but the interaction between Outlook and the Java application looked like a really hairy business. On the other hand .NET seemed like a perfect choice. You’d think that Microsoft would have a relatively easy mechanism of scripting it’s flagship office product in their flagship programming environment. And it turns out that there is - it’s called MS Office Primary Interop Assemblies. These are bunch of COM objects which you can import into your Visual Studio project and they allow you to interact with Office applications at a somewhat intimate level. Unsurprisingly, they are dependent on the version of MS Office you are using (in my case Office 2003) making it hard to create a robust, catch-all solution. You can grab a distributable installer package for Office 2003 version from here.

I added the Microsoft Office 11 Core Library and Microsoft Outlook 11 Core Library as the references to my project. They automagically appeared under the COM tab of my reference dialog in Visual Studio after installing the redistributable package linked above.

Next on the menu was the Drag and Drop functionality. That was pretty easy. In fact, an article written by Tgueth on Code Project gives you a very good primer how to set things up. Just ignore his code, because it won’t work for what I’m trying to do. He was simply interested in dropping actual onto the form - so real files dragged over from Explorer, or Outlook attachments dragged from your email. The email themselves however are handled differently, and cannot be captured this way. I tried, and I failed. There is however an alternate solution using the Interop Assemblies I described above - and believe it or not, the code is actually much cleaner and more compact.

So I sat down and hacked up a quick prototype. Here is the relevant code:

namespace DragAndDrop
{
  public partial class Form1 : Form
  {
    private Microsoft.Office.Interop.Outlook.Application OL;
    private string my_dir = "C:\\";
 
    public Form1()
    {
      InitializeComponent();
      OL = new Microsoft.Office.Interop.Outlook.Application();
    }
 
    private void Form1_DragEnter(object sender, DragEventArgs e)
    {
      e.Effect = DragDropEffects.Copy;
    }
 
    private void Form1_DragDrop(object sender, DragEventArgs e)
    {
      for (int i = 1; i <= OL.ActiveExplorer().Selection.Count; i++)
      {
        Object temp = OL.ActiveExplorer().Selection[i];
 
        if (temp is Microsoft.Office.Interop.Outlook.MailItem)
        {
          Microsoft.Office.Interop.Outlook.MailItem mailitem = (temp as Microsoft.Office.Interop.Outlook.MailItem);
          string subject = mailitem.Subject.Replace(":", "");
          mailitem.SaveAs(my_dir + "\\" + subject + ".msg", Microsoft.Office.Interop.Outlook.OlSaveAsType.olMSG);
        }
      }
    }
  }
}

Let’s break it down, shall we? The first method Form1_DragEnter describes what is supposed to happen when you drag an object onto your form. We simply want to change the Effect property of the event to Copy:

e.Effect = DragDropEffects.Copy;

If you don’t do this, then your code will detect drop effects but that’s about it - it won’t know or care what was dropped. This will ensure that the target of the drop will hold on to a copy of whatever you dragged over when you drop it.

Next, we need to handle the items that were dropped in the Form1_DragDrop method. The following statement returns an array of items which are currently selected with your mouse (ie. the stuff you are dragging right now):

OL.ActiveExplorer().Selection

Since I want to allow users to drag bunch of emails at a time, I’m going to iterate over this array and then grab each item. Since potentially a user could drag over something that is not an email, I’m temporarily saving it as an Object:

Object temp = OL.ActiveExplorer().Selection[i];

If it turns out to be an Outlook email object we cast it back to Microsoft.Office.Interop.Outlook.MailItem and save it. Note that I use the subject line as the filename (an quickly sanitize it removing the illegal “:” character) and save it in the root of the C: drive for now.

The SaveAs method of the MailItem object takes two parameters: first one is the path where you want to save the item, and the other is the file type which must be a member of the OlSaveAsType Enumeration which is a part of the Interop library. I’m using olMSG since that will save as the default Outlook message format.

This code works under Office 2003 but there is one small issue. For every mail item you drop, Outlook will throw this nasty security warning message:

The good news is that the user has the option to disable it for up to 10 minutes. The bad news is that 10 minutes is the maximum amount of time you can choose from that drop down box. I do see why they have this warning there, I just suspect it will become really annoying unless I find out a way to disable it. I did not cross that bridge yet though. Any suggestions? So far the bare bones functionality is working and I can drag and drop email messages onto my little form, and have them saved perfectly along with attachments which is precisely what I wanted.

Next step is to figure out how to make that thing talk to a PHP + MySQL backend of our web application, but that should not be that difficult. All I need to do is to send a HTTPS POST request back to the server, and then have something on the other end process that data and dump it into a database. It doesn’t really need to be much more complicated than that.

I end up doing this each time I reinstall windows, and every time I forget how I did it, so I figured I’ll archive the process here. Perhaps it will help some of you. And I know, someone will say why don’t you use the PPM repository. Let’s just say i don’t want to. I want to grab the latest WWW::Mechanize package from CPAN and run with it.

Why am I posting it now? Because I needed to reinstall windows once again on my desktop, and now I need to get my blackboard scrips to work again.

This is really a multi step process. I’ll assume you have perl installed already. If not, you can get Activestate Perl which works pretty well on windows. Just grab the MSI package, install it and all the useful tools including perl, cpan and ppm will land in your path. From there follow these 3 easy steps:

Step 1: Get nmake

You will need nmake - the windows version of the make utility to compile most of the CPAN packages. How do you get nmake? There are several ways to do it, but probably easiest one is to grab the Microsoft version of the tool from their knowledge base. Once you download it, dump it somewhere in your path. It doesn’t really matter where it is, but I stuck it in the bin directory of my Perl install.

Step 2: Get YAML from CPAN

You will need YAML to build WWW::Mechanize. What is YAML? Sort answer is: do you care? Long answer: look it up. All you need to know is that you need it. So run cpan from your console and type in:

install YAML

This should cause some streaming text on the screen as the package is fetched and compiled. If it fails, make sure nmake is in your path, and that it is named nmake.exe and not something else.

Step 3: Install WWW::Mechanize

Final step is the easy one - just fetch and install the WWW::Mechanize package using the traditional method:

install WWW::Mechanize

Make sure you do step 1 and 2 before you try this. If you have tried this before you installed YAML the build will fail for some reason. To avoid that, just quit cpan, and run it again. This will clear the local cache and will re-fetch the package for a clean build.

So, there you have it. I know it’s a bit of a dry and uninteresting post for Monday morning. But this is more of a reminder to myself than anything else. I never remember where to grab nmake or what is that other package without which nothing ever builds on windows. Hopefully some of you will find it helpful.

I still have no solution to this problem. Will Sheldon had some useful suggestions for me yesterday, but messing around with boot.ini did not do anything. The /sos option did not display anything. I also tried stuff like /safeboot, /safeboot:minimal, /basevideo and etc. It seems that the problem happens before NT loader even gets to parse the ini file soo none of these instructions really do much.

Ben on the other hand seems to think it might be a power supply problem. It could be but I have full access to the hard drive when I boot off the CD. It just doesn’t boot. As far as I can tell my problem is somewhat unique. I have yet to find someone having exactly the same problem. There are a lot of documented issues out there that are somewhat similar (ie share the blinking dash thing) but most actually make sense and are some sort of easily solvable driver problems.

Either way, I figured out that I take a vid of what is happening so that you guys can see the what is this thing all about. This was taken at approximately 3am in a dark room with my Nikon Coolpix S50 so sorry for the quality. Also, don’t mind the mess on my desk.

The computer usually doesn’t sit on the floor like that. It’s actually tucked in the nifty little “computer shelf” of my desk but I moved it out when I was ripping out all the USB cables from the back hoping this will fix the issue.

And yes, that is a Think Geek binary clock on top of the monitor.

I wiped the drive clean and reinstalled Windows on it today. My machine boots once again. I already transfered my firefox profile from the backup drive so I’m 80% back to normal. Now I will need to slowly install all the little pieces of software that I will need. I’ll probably keep it light this time - this is bound to happen again. I just wonder when - I got 2 months out of it last time around. Let’s see how long can I go this time…