Terminally Incoherent

I was reading an article in a magazine last week and while I don’t exactly recall the magazine or the topic right now I distinctly remember mild annoyance at the editor’s decision to put list of URL’s at the end of the piece. Not that it’s a bad practice in itself but some of the listed links were actually pointing to Youtube videos. Youtube URL’s are not that long or difficult, but I don’t think they were designed to be used in printed media. For one, they use a cryptic case sensitive argument at the end which is a pain in the ass to type. I mean, look at it - the thing is ugly and awkward to type:

http://www.youtube.com/watch?v=oHg5SJYRHA0

It’s actually easier and quicker to simply google the videos in question by their title rather than even attempt to type the address by hand. It reminds me of the tricky exercise typing Microsoft’s CD product keys during activation. You hardly ever get them right on the first try. Not that I don’t appreciate the gesture but there is this dissonance between print media and electronic media. When you print URL’s you ought to keep their typability (is that a word?) in mind.

A lot of magazines do something different. They simply refer the reader to the online version of the article or specially prepared page on their website which contains the relevant links. This is an improvement, but you still usually end up with a fairly long URL’ which includes stuff like the publication date, the volume number, author’s name or all of the above. Sometimes the damn thing is to long, and they simply print a blurb to visit their site, and leave locating the right page as an exercise to the reader.

If I was distributing my content via printed media and had to cite online resources quite often, I’d probably look into URL shortening instead. I actually haven’t seen this being done on a bigger scale in magazines, but private people do it all the time. For example, Twitter users utilize services such as TinyURL, Snurl, is.gd or Twurl to post long addresses without exhausting the 140 character limit. So instead of a long ass Youtube link with a 10+ character alphanumeric, case sensitive argument you use something like this:

http://is.gd/AJ

Can you see the difference? First URL is extremely easy to mistype. The second one, almost impossible to get wrong.

Naturally, there is a downside to using a 3rd party URL shortening service - especially in print. The URL is no longer meaningful. In my first example, you at least know I’m sending you to a Youtube video. In the second one it’s not that clear. Furthermore, you are routing your visitor traffic through someone else’s service which may or may not be reliable or trustworthy. While you can get away with it in handouts for your class presentation or an informal documentation it is probably not recommended for official brochures, posters or other materials of that nature. It simply doesn’t look professional.

You want to use URL shortening though - just not the generic services open to anyone on the internet. What you ought to do is to roll your own. The added benefit is that you will get the first pickings on the really short 2 and 3 character URL’s which should last you for a while - which is usually not the case with services used by thousands of people.

How do you create your own URL shortening service? One way is to simply grab an off the shelf solution like Get Shorty. It is free, PHP + MySQL based donationware that does precisely what tinyurl and friends do, but on your own server. It actually gives you a choice on how you want to generate your shortened URL’s - it can do the semi-random 3-4 chacacter codes just like all the services out there, or it can prompt you for custom keywords - so you can actually construct your URL like so:

http://mydomain.com/my/keywords/here

This makes for nice readable URL’s that still retain your domain name (for the purpose of quality control, branding and etc..) but seamlessly redirect your users to relevant URL’s while giving them a hint of where they will be going as well.

Alternatively of course you could create your own system. It’s actually not that difficult to do this. First you will need a database. Let’s make it really easy and just use two fields - the ID and URL:

CREATE TABLE shorturl (
	id INT NOT NULL AUTO_INCREMENT, 
	url TEXT, 
	PRIMARY KEY(url));

If you didn’t notice, I just killed two birds with one stone here. The id is set to auto increment so it will generate unique values for us for free. Naturally this is tad limited as you will quickly run out of the nice 1, 2 and 3 character id’s but it’s simple and reliable.

Next, we need some .htaccess magic:

RewriteEngine on
RewriteBase /short/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /forward.php [L]

This should forward every link to a non-existent file in the directory /short/ to a script called forward.php. Then inside our script we do the following:

$id = end(explode("/", $_SERVER['REQUEST_URI']));
$result = mysql_query("SELECT id FROM shorturl where url='$id'");
$resultarray = mysql_fetch_array($result);
header("Location:".$resultarray['id']);

The REQUEST_URI gives me the address from which the user was redirected to my forwarding script. Whatever is behind the last “/” is my shortened id value. Once I grab that, I just need to look it up in the database, and then redirect you by sending your browser the “Location” header. Let’s say someone goes to the following URL:

http://mydomain.com/short/123

The script will look up 123 in the database, and then redirect this person to relevant URL that if finds.

Naturally, it will blow up in a very un-graceful way if the id is not found. I’m not really doing any error checking or validation here - I’m just illustrating the concept here. So don’t be copying and pasting this into any production code because it is likely wrong. Also, if it doesn’t work at all, let me know and post corrections because I admit I didn’t really test this. As I’m typing this it’s 2am so give me a break.

If you look at the get-shorty code you will note that they do it slightly differently, but I didn’t want to blatantly rip them off. There is clearly more than one way to do it, and you can probably vastly improve my .htaccess code up there.

I really think this is the way to go for print friendly URL’s. It’s easy to implement (I just did it and I’m half asleep) and it gives the reader easy to type shortcut that will take him directly to the resource without unnecessary searching or pit stops along the way.

The internal web application at my company tends to send out a lot of email notifications. Some of them are reminders about various deadlines. Recently I got a feature request to allow people to add these reminders as events to their Outlook calendar.

Outlook supports two calendaring formats vCalendar and iCalendar. I opted to go with the later one because it is an open standard and at least in theory should work with other calendaring applications. The apps which support iCalendar standard can export their events as .ics files which can be seamlessly exchanged between the applications. So my task was to generate such a file on the fly and either link to it, or attach it to the reminder emails.

You can actually serve the ics file directly from a php page by telling the browser you are sending it a text/Calendar mimetype. You do this by sending appropriate headers:

header("Content-Type: text/Calendar");
header("Content-Disposition: inline; filename=calendar.ics");

This will force the browser to interpret the input as a ics file and will show the user the dialog asking him to open the file in an external application. Then you simply need to output the appropriate data in the iCalendar format, which is relatively simple. Here is a very simple event you can copy and paste into your PHP file:

header("Content-Type: text/Calendar");
header("Content-Disposition: inline; filename=calendar.ics");
echo "BEGIN:VCALENDAR\n";
echo "VERSION:2.0\n";
echo "PRODID:-//Foobar Corporation//NONSGML Foobar//EN\n";
echo "METHOD:REQUEST\n"; // requied by Outlook
echo "BEGIN:VEVENT\n";
echo "UID:".date('Ymd').'T'.date('His')."-".rand()."-example.com\n"; // required by Outlok
echo "DTSTAMP:".date('Ymd').'T'.date('His')."\n"; // required by Outlook
echo "DTSTART:20080413T000000\n"; 
echo "SUMMARY:TEST\n";
echo "DESCRIPTION: this is just a test\n";
echo "END:VEVENT\n";
echo "END:VCALENDAR\n";

This will generate an event called TEST to happen on April 13 2008. Note the fields which I marked as required by Outlook. This is important. They are not required by the iCalendar spec so some examples out there skip them. If you do however you will get this lovely error message when you open the file in Outlook:

Microsoft Office Outlook cannot import vCalendar file. This error can appear if you have attempted to save a recurring Lunar appointment in iCalendar format. To avoid this error, set the appointment option to Gregorian instead of Lunar.

Gotta love Microsoft and their through, meaningful error messages. In this case the message is absolutely meaningless and the problem has nothing to do with “Lunar appointments” whatever the hell those are. You are simply missing the METHOD, DTSTAMP and UID fields.

I used REQUEST for the METHOD field but you can also try PUBLISH and few others. As far as I can tell the difference here is how the calendaring application will present the opened file to the user. If you use REQUEST it will give them a button to Accept or Reject the event. If you use PUBLISH they will be asked to save or cancel. Either way is acceptable by me.

The DTSTAMP field is, as the name suggest the date stamp when this event request was generated. As you can see I’m generating it on the fly.

The UID is supposed to be a unique identifier for this event, for some internal use. It is recommend that the right hand side of the UID be the host name or some unique identifier for the domain. I’m not exactly sure what the significance of this field is - I suspect it may be meaningful in certain situations, where the calendaring app is internally sorting the data. As you can see I use the date stamp on the right, followed by a randomly generated number, followed by host name which is pretty much what they advocate in the spec. It should yield UID’s that are sufficiently unique for each event. Some examples out there just use 0 or 1 here which doesn’t seem to be following the specification, but nevertheless it works just fine when you test it. However I figured it’s better to do it right, rather than run into weird issues much, much later.

I think the rest of the fields are pretty much self explanatory, with exception of PRODID. That’s the funny one. It is supposed to uniquely identify the product which has generated this event. In other words it is the place to pimp out your company and your software. And yet, all the online examples use the same 2 or 3 strings, showing blatantly that most people just copy and paste a working example into their production code and then blog about it without ever actually bothering to read the spec itself.

This is how I did it. I’m sure there are more elegant ways to accomplish this, but my email reminders will simply link to an appropriate php page which will fetch relevant data from the database and build appropriate ics file. I hope someone out there will find this writeup useful. And if I see “Foobar Corporation” in the future writeups, I will know you copied from me, and didn’t even bother reading the rest of the post.

If you ever played around with the native PHP mail function, you will probably know that it’s functionality is very basic. Plain text emails work great, but if you want to do HTML email, you need to figure out what mail headers to send with your message. And if you want to send an attachment you need to base64 encode it yourself.

Good news is that you only need to implement it once. Even better news is, that almost every PHP developer out there already implemented their own email class at one point or another. Therefore, you should not be wasting time on this - just go out there and steal some code.

If you are shopping around for a good PHP email functionality I recommend PHPMailer. Why do I like it? Let me count the ways:

1. Easy to use
2. No Setup involved - just drop the class files in your include path
3. Let’s you sen attachments, html emails and etc without any thinking
4. Works with PHP4
5. Widely used (included in projects like Mambo, Moodle, PostNuke, Xoops and etc..
6. Well documented
7. Distributed under GPL

How easy to use is it? Let me show you. Here is a simple script that will send an email with an attachment:

<?php
require("class.phpmailer.php");
 
$mail = new PHPMailer();
$mail->From     = "you@example.com";
$mail->FromName = "Your Full Name";
$mail->Host     = "smtp.example.com";
$mail->Mailer   = "smtp";
 
$mail->AddAddress("someone@example.com");
$mail->AddCC("supervisor@example.com");
$mail->AddBCC("secret@example.com");
$mail->Subject = "This is Your Subject";
$mail->Body    = "This is your message";
$mail->AddAttachment("~/files/something.zip", "new_name.zip");  // optional name
 
if(!$mail->Send())
{
     echo "There was an error sending the message";
     exit;
}
 
echo "<p>Message was sent successfully</p>";
?>

How is that for simple? It’s a piece of cake! I have been using it for a while now, and I haven’t had any problems with it. So go and steal it today.

Word clouds (or tag clouds if you will) are one of the interesting methods of visualizing textual data that got popular recently with the emergence of tagging, ad success folksonomy sites such as Flickr, del.ico.us and etc. One of the aspects of word clouds that interests me is the word frequency analysis. You can size the words in your cloud based on how often they appear in text, and get a clear visual indication of the key concepts and issues that are prevalent in it. For example, some people use it to analyze the presidential state of the union addresses.

Tag Crowd is a very good online tool to generate nice looking clouds from files, or copied and pasted text. But I decided to write one myself, just for shits and giggles (and to see how difficult it would be). I’m posting the complete Word Cloud class below. The code is loosely based on the example posted on lotsofcode.com.

<?php
 
class WordCloud
{
    var $words = array();
 
    function __construct($text)
    {
        $text = preg_replace('/\W/', ' ', $text);
 
        $words = split(' ', $text);        
        foreach ($words as $key => $value)
        {
                $this->addWord($value);
        }
 
    }
 
    function addWord($word, $value = 1)
    {
        $word = strtolower($word);
 
        if (array_key_exists($word, $this->words))
            $this->words[$word] += $value;
        else
            $this->words[$word] = $value;
    }
 
 
    function getSize($percent)
    {
        $size = "font-size: ";
 
        if ($percent >= 99)
            $size .= "4em;";
        else if ($percent >= 95)
            $size .= "3.8em;";
        else if ($percent >= 80)
            $size .= "3.5em;";
        else if ($percent >= 70)
            $size .= "3em;";
        else if ($percent >= 60)
            $size .= "2.8em;";
        else if ($percent >= 50)
            $size .= "2.5em;";
        else if ($percent >= 40)
            $size .= "2.3em;";
        else if ($percent >= 30)
            $size .= "2.1em;";
        else if ($percent >= 25)
            $size .= "2.0em;";
        else if ($percent >= 20)
            $size .= "1.8em;";
        else if ($percent >= 15)
            $size .= "1.6em;";
        else if ($percent >= 10)
            $size .= "1.3em;";
        else if ($percent >= 5)
            $size .= "1.0em;";
        else
            $size .= "0.8em;";
 
        return $size;
    }
 
    function showCloud($show_freq = false)
    {
        $this->max = max($this->words);
 
        foreach ($this->words as $word => $freq)
        {
            if(!empty($word))
            {
                $size = $this->getSize(($freq / $this->max) * 100);
                if($show_freq) $disp_freq = "($freq)"; else $disp_freq = "";
 
                $out .= "<span style='font-family: Tahoma; padding: 4px 4px 4px 4px; letter-spacing: 3px; $size'>
                            &nbsp; {$word}<sup>$disp_freq</sup> &nbsp; </span>";
            }
        }
 
        return $out;
    }
 
}
?>

Yes, I didn’t write it from scratch. In the time honored tradition, I stole the code, and adapted it to my needs. Why waste the precious wetware cycles in my brain, if someone already did most of the legwork.

The main difference is that my script takes unformatted text and then breaks it up into words. You can essentially pass it straight from the $_POST. The rest of the code pretty much follows the example. I also stripped the external stylesheet specification, and built it into the code, so you can just import the class at any place, without fearing that it will send HTML headers to early. I also made the sizing steps more gradual, and got rid of the color variation.

If you pass in a really long text you will have a big cloud as every word is represented. Feel free to add some conditioal statements on showCoud to prevent words with low frequency from being displayed. I’m to lazy at this point, and I want to get this post out.

Here is the usage:

<?php
 
$txt = "The text to be turned into a cloud";
$cloud = new WordCloud($txt);
echo $cloud->showCloud(ture);
 
?>

Passing a boolean true to the showCloud it will display the word frequency as a superscript to each word. It may or may not be an useful feature. The default is false, so if you just call the fucntion without arguments you will get no frequencies displayed.

Again, it’s not a perfect solution, but it is short, sweet and it works - at least for the most part.

I never wrote a wordpress plugin before, so when I set out to learn how they work, I was surprised to see how easy it is to create one. One would think that there would be lot’s of setup and a lot of digging in wordpress’ iinternals. But this is not the case. The Wordpress plugin API is surprisingly straightforward and clean. A simplest plugin you could think of would be this simple Hello World plugin:

<?php
/*
	Plugin Name: Helo World
*/ 
?>

What does this plugin do? Absolutely nothing, except for showing up in your plugin management screen:

click to enlarge

Worpress scans every php file in the wp-content/plugins directory and parses through the comments in those files to create the entries in the plugin manager. You write the comments, and they magically appear in those boxes. I will show you a full comment block in my next example.

Let’s write a very simple plugin - a word filter for your comments. What is a word filter? It’s a script that will replace occurrences of one word with another. You can use word filters to censor obscenities from your comments, or just mess with your readers by swapping words like male and female, straight and gay, or perhaps if your blog is popular enough to attract the First Poster change “first” to “I’m a looser”. In my example I will try to switch foo into bar and test into done.

For that purpose I set up two fake comments on my test blog:

If the plugin works, all the foos and test should change to bars and dones. I will show you the plugin first, and then explain the tricky bit(s):

<?php
/*
Plugin Name: Test Plugin
Plugin URI: http://example.com/
Description: This plugin is Awesome.
Author: Lukasz Grzegorz Maciak
Version: 0.1
Author URI: http://terminally-incoherent.com/
*/ 
 
// plugin function
function test_wordfilter($text) 
{
 
	$words = array ( "foo " => "bar ",
				" test" => " done");
 
	foreach ($words as $key => $value)
	{
		$text = str_replace($key, $value, $text);
	}
 
 
	return $text;
}
 
// wordpress hook
add_filter('comment_text', 'test_wordfilter');
 
?>

I think the function is pretty straightforward - take a string, create array, loop through array, replace every $key with a corresponding $value, return a string. If you don’t get it, tough. Figure it out.

The line you want to pay attention to is this one:

add_filter('comment_text', 'test_wordfilter');

This is a wordpress hook. There are two types of hooks - actions and filters. Actions are triggered by various events such as publishing a post. Filters on the other hand let you manipulate the way data is displayed on the screen.The function add_filter takes two parameters.

The first parameter is the name of the filter. In our case it is comment_text because we only want to use our word filter in the comments. If we wanted to also apply this filter to blog posts we would need to add another filter like so:

add_filter('the_content', 'test_wordfilter');

You can find a full list of available filters and actions in the Wordpress Plugin API Refference.

As you probably guessed, the second parameter is the name of the function you defined in the plugin. Note that your function name should be unique, not to collide with other plugins and native wordpress functions. Most filters will return some desired data and send it as a parameter to your function.

Note that this plugin includes a full comment block with all the attributes filled out. If you look at your plugin manager, here is how it will show up:

click to enlarge

Now all we have to do is to activate the plugin. If it works, it should tell you that activation was successful, and the entry should become green. If you have some errors in your script, you will get a brief error message. Usually it’s something among the lines of “unable to activate the plugin due to fatal error”. This makes debugging your plugins bit difficult, so it’s a good idea to use an editor which checks PHP syntax as you type, like for example Komodo Edit. This will cut down on the number of silly typos, and forgotten semicolons in your code.

Once the plugin is activated, pop over to our comments section we can see that it worked:

Note that this plugin is far from perfect. For example it changes the word foobar into barbar which might not exactly be what we wanted. It’s probably a good idea to change the str_replace fucntion to preg_replace and use regular expressions to ensure that only the words we want get replaced. But that’s something you can do on your own.

Now you have a fully functional plugin to play around with. How easy was that?