@IceBrain: Well, what I was getting at is that migrating to another DB system will always be a hassle. Even if you do the right thing and use database abstraction libraries you will still have to worry about moving data over to the new system.

If you are lucky, you can just make a SQL dump of data on one end, and import it on the other – but that doesn’t always work since different DB systems often like to use slightly different SQL syntax.

So you either sit there and try to clean the SQL up until it becomes system agnostic enough to do an import or you start looking at migration tools which may or may not be free.

If you are lucky – it can be a walk in the park. If you are not, it can be a huge hassle. Either way it can be avoided if you pick MySQL or Postgre instead of SQL Express which constrains you to 2GB of data forcing you to switch to something else as soon as you start growing.

I don’t understand 3, why don’t we use database abstraction libraries, so we can move from SQLite to a “big” engine without much hassle? I’ve never dealt with huge databases, but unless you need the database specific functionalities it should be easy to switch when needed.

@Victoria: No worries! Oh, and you wouldn’t imagine how common this is.

Just the other day I talked with a guy who told me he used to be a COBOL programmer back in the day. He recently was able to visit the company where he got his very first job out of college. Apparently, the COBOL application he helped to develop is still in use right now. Apparently they have tried porting it to ASP few years ago but failed miserably. They are currently working on creating an ASP.NET port.

Funny thing is that the port – written in modern programming language, using best practices and running on very expensive new servers runs much slower than the old application which is a labyrinth of hacks and ugly kludges running on 15 year old hardware.

At my daytime job the main project is very similar to what you described. First it started as a medium size system with restricted functionality – sort of, this is what we got, take it or leave it. The client base grew, some large clients appeared and started to ask for additional functionality. And at some point the decision was not to make a new version of the product but to make patches and fixes for the certain clients. Then other clients learned about the additional functionality and asked for it too.

Now, 7+ years later it’s a code monster – it hasn’t been documented well (some things haven’t been documented at all), nobody knows its full capabilities, it is all hacks and tricks and hardcode. And guess what, there were at least two efforts to rewrite the damn thing on new platform (the old one is ColdFusion 5) – first .NET, then Java. Both were buried – the estimates were two years of development. I won’t even use the word ‘architecture’ near that thing – it’s crutches supporting crutches. Now there’s an ongoing project to move to ColdFusion 8 – and the preliminary results are that it would break in so many places….

Our biggest client decided they were up to recreating the functionality themselves. 1.5 years and they weren’t even close. So they bought us :)

The interface is… it feels insulting to other interfaces to call it that. No two pages are alike. It’s all frames and nested tables and if you try it with Opera it will ask you to download IE5 or newer. It falls to pieces in IE8 and our latest project was to add EmulateIE7 meta tag.

Sorry for the extensive whining :)

@Steve: Yeah, I’ve seen this sort of thing once. And as you said, the best approach here is to put these statuses in the database and then generate the pulldown menus dynamically – when the main one changes, just re-build the sub-status one from the DB and you are set.

@Chris Wellons: Good tip – building unit tests as you build the application definitely helps.

@jambarama: Ooh! Good idea with the spec document. It formalizes the whole process. :)

Good post, this has happened to me more times than I can count. “It is just a small VBA macro” becomes thousands of lines of unmaintainable kludge after kludge. What we started doing was this – every time we underwent a project we filled out a form. The form basically defined scope of the project, time allotted for completion, resources to be used, and a general spec. Changes to the scope needed a new form, new time allotted, and sometimes required new resources & spec. It was more overhead but it stopped the issues – small projects stayed small, large projects were build right from the start.

You nailed it. That’s exactly what I have seen happen with several projects at my current workplace and my last one. A small tool built for a single, simple task incrementally grows to become a big tool that not very good at a bunch of things. It also generates a growing list of issues, a list that never seems to be able to shorten despite any time it spent fixing issues.

Early modularization and security are probably the most important things. Hacking on security later won’t be very secure.

To add a tip: if it seems your project is growing like this, make sure you support it with unit testing (or some kind of testing) right away. Don’t put it off! Tests will make you a lot more confident in rewriting/refactoring code to be more maintainable. I would expect confidence to be proportional to how often and how quickly you rewrite/refactor.

Happens all the time. I am currently developinig with BMC’s Remedy AR Server, building custom help desk, ITIL-compliant stuff. More often than not, we have to look at what something MIGHT bloat into. For instance, one or two Statuses on an Order might have corresponding Substatused (a Status of Pending might have several different Substatuses). You could hardcode it so that the Substatus field becomes writable if the Status is set to Pending. Simple to code. But if the Client keeps requesting new or modified Statuses, and adding Substatuses to them, then you have a situation in which you keep adding hardcode for each Status that has a substatus.

The best approach is to table drive the Statuses and have related Substatuses (of course), but most people don’t think they need to data drive something finite like a list of Statuses.