Archive for January, 2008

Sunday Posting Suspended

Sunday, January 20th, 2008

I’ve been running this blog on a 7 day schedule for quite a while now. I pretty much got it down to science, cranking out posts out on time. Recently though I noticed that this self imposed schedule started becoming tiresome. Weekdays are usually fine, but since my weekends tend to be hectic I’m sometime struggling to get these posts out. Obviously this schedule doesn’t work for me anymore. So I decided to decrease my posting frequency a bit to give me some space to breathe.

I’m officially cutting the Sunday posts for now. 6 days a week is still a tight schedule and I may need to drop it down to 5 at some point but not yet. I don’t think I will ever need to go below 5. Hopefully this slight drop in frequency will result in improved quality (ie. more proofreading for example).

Why am I posting this? Because if I don’t I will keep trying to make the 7 day schedule. If it’s out there in writing, I won’t feel guilty for skipping out on a day. )

tl;dr version: no more posts on Sunday

As an appeasement offering I leave you with the “Near Death Experiences” sketch by Picnicface:

This one has became like a local meme recently. I don’t think I will ever be able to watch any show or a movie dealing with near death experience with a straight face. mrgreen

Beyond the Red Line

Saturday, January 19th, 2008

I’m usually skeptical of total conversion mods. Most of them set very ambitious goals for themselves but never quite reach them. What you usually end up with is a half-assed, buggy mod with sub-par graphics and degraded game play. On the other hand, some turn out great. Beyond The Red Line is a total conversion of Freespace 2 set in the universe based on Battlestar Galactica series. I never played Freespace but I do like to play space sims from time to time, and I do love BSG so I decided to give it a whirl.

Beyond The Red Line

This is not a full review because I didn’t really get to play a lot. But I wanted to put a blurb about it here anyway. Besides, it is hard to review a product that is still in an early development stage. Right now only a demo is available on the website and it is a tad rough around the edges. The briefing UI and option panels are positively dreadful and unintuitive. Ship and weapon descriptions are read by the Microsoft text-to-speech engine and it sounds horrible. But I’m guessing these are little details that they can iron out later.

Fortunately the mission briefings and in-game chatter is all voice acted. This was the bit of the game that impressed me. The acting is really decent, and the voices are surprisingly clear. They follow BSG cannon closely using the familiar jargon - they even do that voice distortion thing when you are in the cockpit. I did notice a slight issue with volume though - sometimes it was really hard to hear the dialog over the effects, but I think that’s something I could just tweak myself via the options menu. The default volume settings are not prefect apparently. But it’s a demo.

The game puts you in the role of a rookie viper pilot on Batlestar Pegasus. The first mission is a tutorial where a slightly annoyed instructor puts you through the basic navigation and combat training. I think they did fudge around with Freespace controls a bit I can’t say for sure. What I can tell you though is that this game is more on the sim than the arcade side. You do have a pretty fine control over your ships acceleration, and thruster systems and the keyboard controls are all over the place. Initially I was a bit confused by the complexity but I guess it grows on you eventually.

I didn’t really get much past the tutorial mission so I can’t write much more. It does have a potential though and it looks like the crew is active and committed. I will be watching this one closely to see where it goes from here. I think that if anyone has a chance to become one of the few successful and worthwhile total conversions it is these guys. If you love BSG I recommend you grab the demo and check it out. You don’t actually need Freespace 2 to play - it is distributed as a standalone product.

Oh, and as most total conversions this one is free. )

DRM Software Industry Must be a Cash Cow

Friday, January 18th, 2008

I realized something today - we are in the wrong industry guys! We should all be writing DRM software! I mean, at least in theory. I would never do it because I find the idea of DRM morally reprehensible and intrinsically flawed. In fact, I think most self respecting programmers think the same way and stay away from that sector of the market. But it must be a fucking cash cow!

Ok, you don’t see it yet. Let me explain. Imagine doing highly abstract cryptography for people who are so technologically inept that they can’t even spell the word cryptography. Imagine working on products that no one actually expects to work. Let’s face it, even the big fat movie studio executive that just paid few mill to some shifty software company is expecting their DRM to actually prevent the final product from hitting usenet and torrent boards. And best yet - you don’t even have to do much quality assurance because your client doesn’t really give a fuck how this software will affect the machines of their clients. Even if you fuck up, and write something that actually can damage end-users optical drives (hi there Starforce!) you still get paid. It’s your client, not you will need to deal with the customer support, the bad PR, refunds and etc.. Hell, maybe they will even hire you back to write another DRM scheme for them.

What was the last big DRM thing? BioShock? Yes, it’s old news but I don’t recall anything more recent - I haven’t been paying much attention. That one however generated so much buzz it actually registered on my radar (few things do these days). Personally I haven’t used it, but I hear that the game has not only a built in rootkit, but also multi-step online activation process, and that it calls home all the time. In fact I hear that most people who bought it just downloaded a crack to get rid of that garbage.

If you were to slow, I will repeat it for you slowly:

In fact I hear that most people who bought it just downloaded a crack to get rid of that garbage.

Yes, DRM is such a pain that legit customers are cracking their own legally purchased copies (invariably breaking the DMCA) because the copy protection is such a pain. Can you see the irony here? The copy protection which was supposed to maintain the integrity of the package and prevent this sort of thing from happening is being easily removed by a widely available patch that appeared a week after the release of the game.

I guess we can’t forget about ACS and they lovely t-shirt I bought that has their super-sekrit encryption key printed on it. mrgreen DRM is really a joke - and not particularly funny one at that.

Remember Bob, Alice and Eve from your cryptography lessons? Bob and Alice always try to communicate, while Eve is listening. Most cryptographic problems involve securely passing information between Bob and Alice while protecting it from Eve. DRM poses a peculiar problem because it does not follow this model. When you work with DRM you want to send messages between Bob and Alice while protecting them from… Alice. After all, Alice can’t be trusted as she might share them with Eve. You can probably see why serious security researchers don’t actually bother working on one of these problems - it’s stupid, and unsolvable. If Alice can read and comprehend the message, she can pass it to Eve. Period. Entertainment industry calls this “The Analog Hole” while the rest of us refers to it as “The Reality”. The problem with this supposed hole is that it can’t be closed with software. That’s just how it works - you have to use hardware. Can you see where this is going?

Nah, you don’t see it. I didn’t see it at first either so let me tell you. Who do you blame when your DRM gets cracked? Anyone? Anyone? The hardware vendor of course. You thought I’m gonna say “the previous developer” but no - that’s who you blame at a real software shop. At a DRM shop you blame the hardware vendor for dropping the ball, and not making their shit impeccable, and impervious to everything including a voltmeter and a soldering iron attack. At some point the data must be analog, unless they figure out a way to directly stream content into a wetware DRM chip implanted into your head. So really, this is all a matter of where do you patch into the electronic system to recover the data.

Hardware folks know it, but they must play ball or they will be locked out of the content. What good is a next-gen DVD player if it can’t play any of the next-gen DVD’s? So you end up with a system that has two broken components: software that doesn’t work, and hardware that is intentionally slow, complex and expensive which doesn’t work either.

Since plugging the analog hole is an engineering task on par with building perpetum mobile, hardware people will always struggle with implementation. If you are behind the schedule, give the hardware folks a half assed incomplete spec to work from and then change it 3 or 4 times. Oh, and remember to revert to a previous spec at least once in that process to get them totally confused. Then you can blame them on delays. If the client asks why the spec is so shitty, or why you change it so often tell them details leaked out on the internet and you have to do this to keep implementation details secret. Sigh… I wish we all could play this game, but out in the real world developers are actually expected to deliver software which works, is on schedule and doesn’t mess up your system. Only DRM makers can churn out some piece of garbage that doesn’t really do anything beyond making your machine unstable, and still get paid.

But let’s get back to Bob and Alice again. There is a second part to this equation that few people talk about. Bob actually doesn’t send the message himself. He dictates the message to Eve who then encrypts it and hand delivers it to Alice. Confused? Think about it - I’m talking about the human element. How do you get a zero day scene release?

Ok, there is more than one way - I’ll grant you that. But more often than not you get a zero-day by having a supplier close to the source. Usually there are thousands of people involved a movie production, post production, publishing and distribution. They all have internet access and most of them probably have been known to download stuff without paying for it. Any one who touches the source can leak it and tracing such a leak is extremely difficult because copying digital data usually leaves no evidence. The only way you can work is backwards - if you nab the uploader you may or may not be able to work your way back to the supplier.

This is what I mean by Eve encrypting and delivering the message to Alice. Most movies get leaked onto the interwebs long before they get the DRM treatment. So you are really building software to protect something that is already available out there.

Let’s summarize:

  1. you build cryptography software for a client that doesn’t understand cryptography
  2. you are working on a problem that is known to be unsolvable
  3. your client does not expect your software to actually work
  4. stability of end-user’s machine is not an issue
  5. compatibility with hardware/software on end-users’s machine is not an issue
  6. ethics are not an issue - your client doesn’t care if you use a rootkit or a trojan
  7. support is mostly not an issue - at most you might just need to provide an un-installer for the rootkit
  8. if all else fails you can blame the hardware vendor for delays

All you are really expected to do is to cripple user experience to the point where they will just go and download illegal copy. So you make a shitty piece of software cobbled together any which way, make it do some hard-core math to facilitate your half-assed encryption, then charge the gullible but unreasonably wealthy client an arm and a leg and move on to the next victim. Pure profit.

Naturally, I bet the DRM industry does have some honest, hard working people who take pride in their work. They will probably come here and yell at me for talking shit. I’m not knocking you guys - I admit, cryptography is a fascinating subject. I’m sure that the software you build uses very cool ideas, and is actually very effective. I’m really happy that you get to work on those hard and challenging issues - I really am. In fact, I will think about all the hard work you did next time I’m watching (or playing) a pirate copy of the movie (or a game) that your software was supposed to protect. mrgreen

Starting a Rails 2.0 Project

Thursday, January 17th, 2008

Since Rails 2.0 fucked up just about every single online tutorial out there by removing the scaffold method, I decided to document a step by step process of starting a brand new project. I think it’s important because the approach changes. In the past you could start from designing database, and then work your way from there. This is no longer the case.

I’m using SITS as an example here. What is SITS? As far as you are concerned it is my little vaporware project that exist only in my feeble mind, and as a empty google code page. Oh, and it also exists as part of the monstrous 45,000 line php craziness I maintain at work. It used to be an individual PHP project at one point, then it got swallowed by the all purpose evaluation and report tracking monster and extracting it from there proved to amount almost to a total rewrite. I figured - why not use Rails and learn something along the way?

So let’s get started. First thing we want to do is to actually create a project. I’m assuming you have Ruby, Rails and MySQL installed. Don’t touch the database yet. Btw, I’m using windows because I don’t feel like switching machines for this - sue me.

Go go gadget RAILS:

C:\projects>rails sits
      create
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
      create  config/environments
      create  config/initializers
      create  db
      create  doc
      create  lib
 
      --- snip ---
 
      create  log/server.log
      create  log/production.log
      create  log/development.log
      create  log/test.log

It’s a whole lot of output so I snipped it. You will also note I’m using C:\projects as my directory. This is actually a junction that points somewhere else, but you don’t need to worry about this. I just got annoyed that the paths were so long they were causing my code box scroll sideways so I C:\projects to my real projects folder.

Anyways, on to configuration. \project\sits\database.yml is the database config file - and practically the only config file we will need to write for this project. I have set it up like so:

development:
  adapter: mysql
  database: sits_dev
  username: root
  password: [passwd]
 
test:
  adapter: mysql
  database: sits_test
  username: root
  password: [passwd]
 
production:
  adapter: mysql
  database: sits_prod
  username: root
  password: [passwd]

I’m using the root account right now because we will be using built in rails magic to add and drop tables and do all kinds of other fun things. Once the project is ready for production, I will probably go back and change this to a locked down user which only has select, update and delete privileges for the tables it needs. As a side note, when you edit this file make sure that the config lines for each environment are indented exactly two spaces. No more no less. I mean, unless you enjoy cryptic errors - cause you gonna get plenty if you fuck up the layout. It’s kinda like in python, but only for this file. Ruby itself doesn’t really care about indentation that much.

Let’s create the 3 databases:

C:\projects>rake db:create:all

Yup, rake is like fucking magic - you don’t even need to touch MySQL tools. I touched them though to verify the databases were created:

Databases Created by Rake

We have the databases now, now it’s time for scaffolding. What about models, you ask? What about controllers? Scaffolding does that for us. The bad part is that we need to roughly know what do we want to scaffold from the get-go. I kinda know what I’m doing since I did this once already in php and then fucked it up by integrating it into a monster that will one day devour my soul. So have an idea of what we should be designing. I will start with user table because it is stand-alone (ie. it does not belong_to or is not composed_of anything). Our user will have a username, a password (hashed of course), an email (naturally), account creation date and user level. The level will of course be on a scale from luser to admin, and I will use numeric values to represent them. Why? Because I used enums in the past and I ended up with contraptions like lpuser (low priv user), superuser, superadmin, and etc.. Numeric level system is easy to extend.

C:\projects\sits>ruby script\generate scaffold User username:string 
	password:string created_at:datetime level:integer
      exists  app/models/
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/users
      exists  app/views/layouts/
      exists  test/functional/
      exists  test/unit/
      create  app/views/users/index.html.erb
      create  app/views/users/show.html.erb
      create  app/views/users/new.html.erb
      create  app/views/users/edit.html.erb
      create  app/views/layouts/users.html.erb
      create  public/stylesheets/scaffold.css
  dependency  model
      exists    app/models/
      exists    test/unit/
      exists    test/fixtures/
      create    app/models/user.rb
      create    test/unit/user_test.rb
      create    test/fixtures/users.yml
      create    db/migrate
      create    db/migrate/001_create_users.rb
      create  app/controllers/users_controller.rb
      create  test/functional/users_controller_test.rb
      create  app/helpers/users_helper.rb
       route  map.resources :users

Can you see what I’m doing here? I’m specifying the fields and their data types as arguments for the scaffold script. These fields will be included in my views, and I will also get an automagical database migration file. I can go in and edit it adding extra fields that won’t show up in the views if I want to. To do so I just need to open the db\migrations\001_create_users.rb file:

class CreateUsers < ActiveRecord::Migration
 
  def self.up
    create_table :users do |t|
      t.string :username
      t.string :password
      t.datetime :created_at
      t.integer :level
      t.timestamps
    end
  end
 
  def self.down
    drop_table :users
  end
end

Note - I didn’t write that code. Rails did. Neat huh? All I needed to do was to specify bunch of field names and data types. What data types can you use in a migration (or when specifying scaffold)? I think the list is as follows:

Rails Migration Types

List is courtesy of the Rails Migration Cheatsheet. They have an expanded version of this table on their site - I just stole a piece of it for quick reference.

What now? We can use our old friend Mr. Rake:

C:\projects\sits>rake db:migrate
(in C:/projects/sits)
== 1 CreateUsers: migrating ===================================================
-- create_table(:users)
   -> 0.0630s
== 1 CreateUsers: migrated (0.0630s) ==========================================

In effect we should see two new tables in the _dev database like this:

Tables Created by Migration

Just to be sure, let’s look at the schema of the users table to make sure everything is correct:

Users: Database Schema

Yup, everything works! Now let’s launch this shit! I’m using Mongrel instead of WebRick because it is ♪Harder, Better, Faster, Stronger♪. Also it has a picture of a dog on it’s webpage so clearly it must be superior. Not even mentioning the Win32 service support that ma To install mongrel just do:

gem install mongrel

To start it, simply navigate to the project and run:

C:\projects\sits>mongrel_rails start
** Starting Mongrel listening at 0.0.0.0:3000
** Starting Rails with development environment...
** Rails loaded.
** Loading any Rails specific GemPlugins
** Signals ready.  INT => stop (no restart).
** Mongrel 1.1.2 available at 0.0.0.0:3000
** Use CTRL-C to stop.

I actually have no clue why the default mongrel output displays the ip as 0.0.0.0 instead of 127.0.0.1 but I assure you it works. We can test it by simply navigating to http://locahost:3000/Users

Scaffolded Project

All the CRUD actions are supported here so I can easily add a new user by simply hitting a link on the bottom of this page:

Create a New User

I don’t really like the creation date dialog - it probably would be smoother if the date got appended automatically at the moment of creation. But I think this is something I can easily change by editing either controller or the view. As it is right now the password is not getting encrypted so this is another bit that we need to fix But this is pretty much the point of scaffolding - it gives you a starting point, not really a complete solution.

Is this better than the old way? On one hand having the code generated for you is nice. On the other hand the auto-generated code might be a little bit confusing to a newbie. The dynamic scaffold gave you a blank slate you could fill out by coding from scratch. Now we get a pile of cryptic code that needs to be deciphered before we can move on. Perhaps we will learn better this way, but I’m not sure. I kinda like the from-scratch approach. But there is not much I can do about it other than downgrading to a previous version of Rails. P

That’s all I have for today. I mostly just wanted to document all the little things you need to do to start a scaffolded project from scratch. I might post more as I re-write SITS but I guess that really depends on whether or not I run into any interesting problems, or cool Rails thing I want to talk about.

What happened to scaffold in Rails 2.x?

Wednesday, January 16th, 2008

Rant time! This one may or may not ruffle some feathers, but frankly I don’t care. I’m probably late to this party and missed out on all the big flame wars seeing how 2.0 was released at the beginning of December. Still, this kinda pissed me off. I just wanted to ask the Rails team what exactly was so horribly wrong with the scaffold method that they had to rip it out in 2.x release?

Let me back track a second. Some of you probably don’t have a clue of what I’m talking about here so let me paint you a picture here. This is how just about every Rails book and online tutorial starts the same. First thing they teach you is to create a Rails app, create the database and then set up a model and a controller for some database table (let’s call it things).

Then, inside the things_controller.rb file you were supposed to do:

class ThingsController < ApplicationController
  scaffold :thing
end

This would generate an invisible dynamic scaffolding that would take care of basic CRUD operations. The idea was to start clean, and then slowly chip off the scaffolding by overloading it with your own content. For example, once you define index method in the controller, and and index view the listing is no longer handled by the scaffold.

At any point of development process you could have dropped this single line in any of your controllers to get bare-bone basic functionality on the fly. It had that wow factor, that was bringing all the boys to the yard, like the milkshake… Or something like that. My point is was that it was easy to use, did not clutter your codebase with auto-generated gunk and every fucking Rails tutorial on the planet was using it.

Do you know what happens when you try to use the scaffold line in the current 2.0.1 release of Rails? First you get an error message. Then a little guy jumps out of the closet and kicks you in the balls, steals your wallet and then jumps out the window. Or maybe that’s just happened to me. Either way, it’s not a pleasant experience.

At first I thought that maybe I just can’t spell scaffold. But no - it just no longer exists. I actually had to google it cause I had no fucking idea. They just riped it out and junked it. Yay backwards compatibility. I can’t wait to see what useful features you will remove in Rails 3.0 to fuck up my work flow! Wohoo!

I know that scaffolding was just a neat feature which really didn’t influence how a deployed rails application performs or behaves. I know it was not essential, and I know it was only useful in the very early stages of development. Blah, blah, blah. Still, why remove it? The official release post doesn’t even mention this change.

I looked through some mailing lists and forum posts, and I on every single one I saw pompous jackoffasaurs taking turns enlightening masses on how only idiots used scaffold method, that it was abused, that it was a “crutch” and how the new way actually forces retards to look at the code. Really? You are serious?! They removed it out of goodness of their hearts to save us heathens from our wretched scaffolding practices? Thank you there, Dijkstra - why don’t you write me a “considered harmful” essay while you are at it? Geezz…

So I’m asking - why was it removed? Perhaps someone can give me an objective, unbiased and non-elitist analysis on why it was not fit for 2.0. I just want to rationalize this. I want to find some sort of justification. And no, “cause u were doin it rong” just doesn’t cut it for me. Give me a benchmark showing that dynamic scaffolding is slow, tell me about security concerns, about compatibility migration issues. Tell me how how the “new way” improves the work flow, improves coding practices and is generally better. Give me something. I might buy it - who knows. I just couldn’t find any explanation like that anywhere out there. Everyone is simply viciously bashing n00bs and passive-aggressively flogging anyone even daring to ask about the scaffold method. evil

You can naturally still scaffold in 2.x but only via code generation. And when you do it, the script generates everything including the model, controller, views and database migrations. In fact, they dumbed it down to the point where it no longer even looks at database to dynamically detect the correct fields and data types.

The 2.0 way is to scaffold first, then create a database. The script generates a migration based on the list of fields you specify as arguments. Same fields are also used for the views. If you don’t specify any fields on the command line, you will get a half-assed views that do nothing. Nice thing is that if you do specify correct fields on the command line, you end up with a complete set of controllers, views, migrations and etc. All you need to do is to run the migration script, then fire up the server and you are good to go. It does make sense conceptually - after all in real world you put scaffolding up before you start building. Maybe this is a superior approach, but I just don’t see why we can’t have it both ways.

Needless to say, I’m a little bit disappointed with this change. I can only imagine how confusing it must be for new Rails users who are trying to follow a simple tutorial from a book. I was confused as hell for a minute there, and the whole thing left me a bit turned off to rails. It’s a great framework, and I’m still planning to use it on a few projects, but this whole scaffolding debacle really did a great job of dousing and subduing my enthusiasm. Maybe it’s for the better and it will allow me to look at RoR objectively instead of jumping on the “OMG this is FANTASTIC!” bandwagon.

Wantonly breaking backwards compatibility on a whim and without any justification is a cardinal sin of software development and frankly it scares me a bit that they chose to commit it. I’m not saying breaking backwards compatibility it’s always bad. Far from it - sometimes you need to ditch old code if it is holding you back. And that’s perfectly acceptable. But fucking up chapter one of just about every published Rails book, and every single online tutorial in one swooping move is not a good thing.

Since the begging of December people have been reading the now outdated tutorials, and scratching their heads trying to figure out why Rails is not working. It’s a good thing everyone loves Rails these days. I bet most people will forgive and forget, but I think they really shot themselves in the foot with this. P