Terminally Incoherent

I’m posting this a day late because it took me a whole morning to figure this one out. It appears that Comcast has completely blocked both inbound and outbound traffic on port 25 for my company. For a few years now we have been running a in-house authenticated SMTP server using IIS. It was running on port 587 and basically relayed emails to another server at an off-site location on port 25. Why was it set up this way is a topic for a whole other rant, but it worked well for us until now.

When I came in to work yesterday morning all was well. Few people around the office mentioned something about a slow day, noting their inbox was unusually empty. I didn’t really pay much attention to that chatter, until someone decided to email something to herself and it never came through. Then all hell broke loose.

You see, this problem was essentially hidden from regular users because they could connect to my in-house server on port 587 without any issues. So their emails were leaving their outbox as normal, and then queuing up on the server, never to be seen again. The server itself could not shoot them back failure notifications either, because it could not connect to it’s relay point on port 25. So people were emailing each other all morning without even realizing something was amiss. When they caught on, it was instant panic mode spreading throughout the organization like a fucking wildfire.

For several hours I was methodically checking, re-checking, restarting, and power-cycling every single device and service that had anything to do with email traffic. I was also calling the folks who maintain the off-site server every 5 minutes to see what was their progress. They were convinced the issue was on my side, and I was adamant that it was on their side. After much deliberation, we came to the conclusion that we were both wrong. The off-site server was accessible from everywhere but my location, but there was nothing here in the office which would prevent it from communicating on port 25.

We tested outbound and inbound traffic on their side and it was working just fine so that left only one conclusion - my ISP fucked us over and completely sealed off port 25. Once we realized that, the conclusion was as swift as it was simple. We simply switched the external server to listen on port 587, changed the outbound port in IIS and an avalanche of backed up email started streaming into people’s mailboxes.

Let me run that by you again in case you didn’t notice - once we figured out what the issue was, it took us 5 seconds to reconfigure our shit, and route around it. So if this supposed anti-spam measure is so easy to circumvent, then can someone explain to me how is it supposed to be stopping hard core spammers with their sprawling botnets out there? I’m pretty sure most of semi-modern spam-trojans can be remotely reconfigured to send out emails on alternate ports.

Port blocking has became pretty much an industry standard these days, but I still fail to see how it could ever be effective. What is stopping me from running an email server on port 80 or 443? Will they block these two ports as well? It is just a knee jerk reaction, that might be effective in a short term. It won’t work in the long run though - soon they will run out of ports to block, and regular customers won’t be able to use any kind of non-standard internet services for genuine purposes without bending over backwards.

This is just one of these wholesale, one-click-and-your-done spam solutions. Why do ISP’s do it? Because it’s easy! You block some important ports, and the amount of spam and genuine email routed through your network goes down. You boss is happy, your investors are happy, folks in the security business are clapping their hands marveling at the sudden drop in spam, forgetting it will be back to normal in a month or two as all the spammers will figure out the same thing I did just now.

The only people who are not happy about this are the customers, but Comcast does not really care about them that much anyway as it has blatantly demonstrated in the past with it’s bandwidth throttling, and lackluster tech support.

Also, Twitter > than regular tech support resources it seems:

Despite the fact that we were constrained to 140 characters per pop, talking asynchronously and multitasking, this was still way more pleasant than my experiences with Mr. Rooter and Mr. 125 Times. Not sure if that guy is an actually really affiliated with the company in any way, but he seems to be representing them well in the 140 character conversation universe.

One more reason to love Twitter and hate Comcast!

Remember my blinking underscore issue from February? It’s back again, and I still have no clue what is causing it.

Quick recap for new readers - I have a peculiar issue with my windows machine. Sometimes I shut it down, and it just won’t boot back up again. The damn thing goes through POST diagnostic at the beginning of the boot sequence and then just displays a blinking underscore/dash/cursor in the upper left corner of the screen. Apparently that indicates “boot failure” of some sort/ The data on the drive is still intact, and can be easily backed.

If you look through the archives here, you will see that I tried just about anything. Every time this happens I disconnect all the shit from this machine with exception of the mouse and keyboard hoping this is one of those odd USB related flukes. I then boot into recovery console, run fixboot and fixmbr commands, recreate my boot.ini with bootcfg /rebuild, run chkdsk and etc. Sadly going these motions usually doesn’t do squat. The only reliable way to get the machine working again is to format the drive and reinstall windows from scratch. But as it appears this is also only a temporary measure. The problem keeps coming back.

I’m beginning to suspect that my disk is faulty but I can’t really prove it (yet). I have ran chkdsk on this volume so many times I think I’m going to wear it out soon. At one occasion it told me the drive had a irrecoverable error on it and wouldn’t even attempt to scan it. Rebooting seemed to solve that issue, and all subsequent scans went through without major problems. On some occasions it said it fixed some errors but there didn’t seem to be any show stoppers in there. Anyone can recommend good hard drive diagnostic tools?

I’m thinking that perhaps something funky is going on in the boot sector, or perhaps the windows boot loader region which is just to subtle or to odd for chkdsk to worry about. This is the only thing that I can think of - I have literally no clue what else could be causing this issue. Then again, my drive is not that old. If I remember correctly I replaced it less than 2 years ago - in fact, it may still be on the warranty. I might still be able to return it back to Seagate and get a replacement. I will have to look into that.

In the meantime I ordered a new larger HD (I was running low on space anyway). This time I went with Western Digital. Once the drive ships (probably Friday although knowing my luck it will probably be Monday) I’m going to plug it into the machine, reinstall windows one last time and see how it goes. If I never have this issue again, then it means it was definitely a faulty drive. If it comes back after a month, I will know it must be something else. In either case I think I should just start putting together a new gaming rig for myself.

Worst of this is that I had some spare time yesterday and was planning to use it to finish KOTOR and then start on the final review post about this game. It seems that I will have to hold off with that until I reinstall windows again. It’s kinda sad that I will have to install the game for mere 20-30 minutes of gameplay I have left in order. But it has to be done. I need closure!

The other day this well known Excel issue trickled down to me. If you are to trigger happy applying formating to your excel cells, you eventually hit some sort of hard coded limit. Excel can only handle 4k of formating annotations. I suppose this is an inherent flaw of their binary file format, but then again, who knows. The bottom line is that this has been an issue in every single Excel version up to, and including Excel 2003. Microsoft has
a long standing knowledge base article which gives no suggestions on what to do once you hit that forbidden threshold. Their advice is essentially “don’t use so many different formats”. That’s about it.

The problem is that once you hit the magical number that oscillates somewhere in the 4k range Excel will block you from doing any new formating adjustments by popping up the “Too many different cell formats” error message. In theory, simplifying your worksheet will make the error go away, but to do that you will need to either remove or modify formating on some of the cells or cell ranges. But you can’t do that because of the error. So you end up in an endless loop. To solve the problem, you have to do the precise thing that it prevents you from doing.

The poor guy who fell victim to this bug went through 3 support drones who all told them the same thing - don’t use so much formating. Fortunately he CC’d some managers on the support emails, and eventually it ended up in my mailbox. Which was a good thing since I’m not as immersed in the Microsoft monoculture as the lowly tech support drones. The truth is that there is a solution to this problem, and it’s very easy. But Microsoft won’t even tell you about it because it involves using a competing product.

If you ever run into this issue do the following:

1. Open the document in Open Office
2. Save in the Microsoft XLS file format
3. ???
4. Profit

80% of the time this is enough to solve the issue, because Open Office organizes the formating information slightly differently due to the differences in implementation. For the most part it is a very exact port, and it faithfully mirrors Excel functionality, but there might be few minor glitches when you do this. However, it probably beats re-creating the document from scratch via copying and pasting.

The problem here is that none of the low level guys knew that, or even considered trying it. Living immersed in the monoculture has blinded them to alternative solutions. This is the price we pay for letting Microsoft run our lives.

I still have no solution to this problem. Will Sheldon had some useful suggestions for me yesterday, but messing around with boot.ini did not do anything. The /sos option did not display anything. I also tried stuff like /safeboot, /safeboot:minimal, /basevideo and etc. It seems that the problem happens before NT loader even gets to parse the ini file soo none of these instructions really do much.

Ben on the other hand seems to think it might be a power supply problem. It could be but I have full access to the hard drive when I boot off the CD. It just doesn’t boot. As far as I can tell my problem is somewhat unique. I have yet to find someone having exactly the same problem. There are a lot of documented issues out there that are somewhat similar (ie share the blinking dash thing) but most actually make sense and are some sort of easily solvable driver problems.

Either way, I figured out that I take a vid of what is happening so that you guys can see the what is this thing all about. This was taken at approximately 3am in a dark room with my Nikon Coolpix S50 so sorry for the quality. Also, don’t mind the mess on my desk.

The computer usually doesn’t sit on the floor like that. It’s actually tucked in the nifty little “computer shelf” of my desk but I moved it out when I was ripping out all the USB cables from the back hoping this will fix the issue.

And yes, that is a Think Geek binary clock on top of the monitor.

I wiped the drive clean and reinstalled Windows on it today. My machine boots once again. I already transfered my firefox profile from the backup drive so I’m 80% back to normal. Now I will need to slowly install all the little pieces of software that I will need. I’ll probably keep it light this time - this is bound to happen again. I just wonder when - I got 2 months out of it last time around. Let’s see how long can I go this time…

Remember that one time (at band camp) that my computer wouldn’t fucking start, because it was busing blinking a little tiny dash in the upper left corner of the screen? That was the end of December (just before Christmas). You might remember me struggling to find a reasonable solution and finally giving up and re-formating the drive and reinstalling windows.

So I have been running with a fresh Windows for close to 2 months now. Yesterday (actually, technically today at 3am) I rebooted the machine and it never came back up. All I see is that damned blinking dash. And once again I haven’t the faintest clue as to what is really causing this issue. This is really irritating, because now I know it’s not just some random glitch. It happened twice already, and it will probably happen again.

Btw, as far as I can tell, the data on the HD is fine - or at least it was last time. It just wouldn’t boot. I know this is not a corrupted MBR because running FIXBOOT and FIXMBR tools did absolutely nothing last time. Neither did installing windows on the same partition without actually formating the drive which means it is not a Windws problem either.

Could it be software related? It’s possible but it kinda seems unlikely at this point. I have no clue what could have caused it. Last time the machine bricked itself while applying Windows Media Player update. This time around it was rebooting to update McAfee. Then again, I don’t reboot that machine often - it runs continuously for weeks on end. The only time it reboots is when the software forces it too. I haven’t checked the uptime before it went down yesterday but I think it has been running for at least a week. I installed several apps in the last 2 weeks - Bitpim, MS Visual Studio, Flickr Uplodr, ActiveState Perl and some CPAN modules and probably few more things. It could have been any of these.

Or perhaps it is some insidious virus or trojan that went undetected. Perhaps it is sitting there camouflaged in one of my data files, just waiting to pounce and fuck up my boot sequence. It’s possible, but you’d think that a 2 month old piece of mallware would get snagged by the on access scanner made by any mainstream AV provider, no matter how shitty or bloated their software is.

I’m beginning to suspect this might be something like Ben described in one of the previous threads - a perfectly good hard drive with few faulty sectors in the boot area. But then again I ran the quick built in DELL IDE drive diagnostics and they passed. Of course it’s possible that these things gloss over this type of things and wouldn’t notice it. But this is a relatively new drive - I think it only has like a year or so. The data drive is much older, and it is working perfectly fine. Of course that one never was a bootable drive but still.

Perhaps it’s because I have 2 drives sitting virtually on top of each other in the cramped Dell case and the heat and vibration is doing it’s damage over time? I just don’t know. I’m at a loss here. Perhaps this is a sign from above that it is really about time to invest in a new gaming machine…