Terminally Incoherent

I apologize for not switching hosts once again. Dreamhost was actually improving lately. Several people told me that there was a very noticeable performance improvement evidenced by much faster load times of this blog. But, alas they are back to their old tricks again. Apparently things will be slow for a little while until the file server I’m on rebuilds its RAID drive. It should be fixed before breakfast I hope - but you never know with this company.

Anyways, go to sleep folks. Terminally Incoherent will attempt to resume being speedy and agile in the morning. Good night and sorry for the inconvenience.

Time for some shameless self promotion. I know that bunch of people read this blog via feed readers and rarely clicky the things on the sidebars. I also know that I rarely update the sidebar links to the other services I use. So this post will be a sort of “if you like this blog, you may also like this other stuff by me” kind of deal. I’m going to throw a few links out there and tell you what content you can expect there and you can decide if you want to check it out or subscribe.

First of, let me tell you why I just don’t put everything here. Over the last few years I sort of figured out what I what kind of stuff I want to post on this blog. My topic choices are still a rather wild scatter shot but I cut them down to the following list:

1. Programming and Web Development
2. Various IT and Technology stuff including security, software, and reviews
3. Linux fanboy stuff
4. Video Games and video game reviews
5. Book and Movie Reviews (leaning heavily towards SF/Fantasy genre)
6. RPG and Tabletop Games
7. Random humor/rants related to any of the above

So in broad terms this collection of topics can probably be defined as “geeky stuff” or “the stuff that definitely won’t get you laid”. That is sort of the direction where I’m taking this page in case you haven’t noticed. There is a wide array of things that I used to do here, but then stopped. For example I try to refrain from posting about politics and religion. When I had 2 regular readers who I knew in real life, and who shared my political and religious views this sort of stuff was fine. Now that I see more than a dozen of regulars from all parts of the world, and from across the political spectrum commenting on my posts I think I can do better. These sort of posts quickly get personal, and I sort of want the small but quite diverse community we have here to thrive and grow rather than bicker over politics.

Another thing I try to avoid here is one-link or one-picture posts. I try to put some thought and effort into every post (not that you could tell by looking at my atrocious spelling and grammar) and simply doing a “cheg this out guyz!” posts seems like cheating. Especially since these days I’m under my self imposed 5 posts per week limit. I try to do quality over quantity - it doesn’t always work out but that’s the idea. Same goes for blurting out one liners, or commenting on mundane shit that I’m dealing with at the moment.

I still have an urge to post that type of stuff and in most cases I have found different outlets for them. For example, I tend to post crazy links, silly pictures and interesting quotes up at /dev/random tumbleblog. Some of you probably know about it, some probably don’t. That’s where all the silly stuff and links/quotes which don’t warrant a full blown article go.

As a side note, have you noticed how Web 2.0 takes old concepts and transforms them into brand new things? In the past, link blogging had a more or less negative connotation - it was re-blogging, recycling and rehashing stuff that was not original. Web 2.0 took that, renamed it as tumble blogging and all of a sudden it is acceptable and cool to have a tumblr account and regurgitate content aggregated from various places. That said, I try to put fun stuff at /dev/random - you won’t see links to stories from the front page of slashdot or digg there but rather various visual memes, and random quotes that I found either funny, or insightful. Also, no LOLCats (cause they are stupid) and I try to keep it SFW.

Speaking of Web 2.0 and social media - my primary medium for blurting out stupid shit throughout the day is twitter. Some of you already follow me there, and there is a flash widget somewhere on the sidebar here but my profile is here in case you want to check it out. I don’t think I need to explain to you what Twitter is - unless of course you have been sitting under a rock for the last year or so.

I still do not have a place where I could openly wax poetic about politics - which of course may actually be a good thing. Perhaps I should just refrain from commenting on these things unless they are related to my area of interest (ie. silliness like software patents, IP laws and etc). If I ever set something up for purely political and jaded rants I will let you know but it will definitely be separate from this blog though.

Then there is of course my Flickr photostream which doesn’t get updated that often, but it’s there if you want to stalk me. And of course there is facebook and myspace linked in the sidebar but I really don’t use these to post any content of sorts. These are the tools for interacting with my technologically underdeveloped real life friends and family who do not blog, do not understand twitter and reject email as communication medium as a personal rule.

So this blog, /dev/random, twitter and flickr are the places where I’m most active and sort of encompass the full spectrum of my public online activities. Just to put it all together I created a friendfeed account (cause that’s the latest cool thing, I hear) and simply dumped feeds from various services I use into it creating a life-stream sort of thing. It is like a big feed of feeds - so for example if you don’t use Twitter but want to be updated when I tweet you can subscribe to it.

How does your online life break down? I know some of you use twitter - and have blogs. I tend to subscribe to the blogs of my frequent commenters and I follow them on twitter if they link to it. How does your online life split up into various services? Do you usually dump everything into a single blog/journal or do you try to compartmentalize it across various blogs and services?

Apparently some of you had the doubtful pleasure of seeing my directory index between 4 and 5:30 pm yesterday. Apparently I set their server on fire by (check this out) flushing the WP-Cache to see the small tweaks in the css style sheet. I made a small change, flushed the chache, refreshed the page, fixed it, flushed, refreshed and etc. So for like 20 minutes the site was mostly un-cached. Then the shit hit the fan and I got locked out of my own fucking blog only to find this in my mailbox:

Hello Lukasz,

I apologize for the interruption, but I have had to temporarily disable your site. Unfortunately, this was causing a huge load on the machine:

popcorn: 01:21 PM# uptime

13:21:58 up 78 days, 21:26, 8 users, load average: 96.58, 73.29, 53.76

After disabling this, the load dropped significantly:

popcorn: 01:22 PM# uptime

13:25:32 up 78 days, 21:30, 8 users, load average: 29.25, 51.65, 49.33

You probably need to enabling caching for this blog or disable any extraneous plugins before re-enabling it. You can check out more about it here: http://wiki.dreamhost.com/WordPress_Troubleshooting

Thanks!

I know that you get what you pay for, but I was not really happy about them just rendering my website inaccessible without:

1. Letting me know there is a problem, and allowing me to fix it
2. Telling me how to restore my website to usable condition again

Fortunately all they did was to rename index.php to disabled_index.php so I was able to quickly get it back online. I also tried to comply with their whiny demands and made Caching much more aggressive. Let me know if you notice any weirdness on the page - especially stuff like CAPTCHA’s not working properly.

Here is my reply to them - note that I was a tad pissed off when I wrote this. In retrospect perhaps I was a bit harsh on them. What do you think:

First, how to I enable my site back? I’m getting emails, my readers are not happy and I’m losing advertising revenue. It would be nice if I got some instructions on how to do it. I’m guessing I just rename disabled_index.php back to index.php? Thanks for telling me that though.

Second, all I’m running on my account is a basic Wordpress account with couple of plugins - chiefly WP-Cache, Akismet, and Bad Behaviour which aim to reduce the system load, and prevent spam. Two out of these were specifically listed on the troubleshooting site you linked to in your email. There is really not much else there.

You want to know what I was doing in the last few hours? I was tweaking my .css stylesheet and I cleared the cache few times to see the results. Could something as minor as that be the reason of such a huge spike? I will refrain from doing this in the future, but geez!

I haven’t added any plugins in the last 6 months. I haven’t changed the theme. I haven’t run any scripts. All I do on that account is blog. There is really nothing there that would be out of ordinary - just Wordpress installed via the one-click install from the panel. The only changes I really did to it in the past 6 months were small css tweaks.

Third, over the last year or so, I frequently witnessed the system load on popcorn to spike up way above 200. Some of my readers joke around that the site often takes up to 3 minutes to fully load during peak activity hours. Why is that? Is Wordpress really such a resource hog? Do I really get this many hits per second? I really doubt it. There is no way for me to control how much traffic do I get. I’m really not running some crazy scripts in there - it is just a fairly standard Wordpress install with a default theme. I can mess around with it a bit and try to switch off plugins but I really don’t think that will reduce the load of this website that much - I mean there is not much to disable there.

Really, I understand that it is hard to run a shared hosting company. I was always with you guys. I have been always very tolerant, and a loyal customer. I even recommended you to some of my friends despite the apparent flaws. I dealt with the regular downtimes, and I didn’t really say anything when you guys accidentally charged me over $200 for the next 3 years of hosting few months back. It happens. But disabling my website for undisclosed time period on a whim without giving me any prior notice and any possible chance to fix the issue before the website goes down is just pushing it. It’s unprofessional, and very disappointing. I’m hurt, I’m angry and I’m annoyed. Perhaps it is time for me to start looking for a new home.

If you see an activity spike on popcorn later today it might be because I’m backing up all my files and preparing to move to a new host.

So please tell me, what am I supposed to do here? What can you offer me? I’m not really interested in running a website composed from static HTML pages - I actually do want to use Wordpress and I do want to keep runing Akismet and other anti-spam plugins which keep it clean. I don’t really want to spend hours, hacking Wordpress and trying to figure out how to optimize the plugins that I use. I just don’t have time for that. And I definitely want more people visit my website - not less.

So what are my options? Will my blog get disabled every time there is a traffic spike? I mean, what if (God forbid) one of my posts ends up on a front page of Digg tomorrow? I’m not saying it will but it happened before, and I hope it will happen again. What then? Will popcorn spontaneously burst into flames?

Why should I stay with dreamhost?

I’m not a happy customer right now.

Was I out of line? I really think I will need to find a new host soon. I know I say it all the time, but stuff like this might just be the thing that will push me to finally do it. So this is once again time to pimp your host. Who do you use? What are their prices? What plans do they offer? How is their support like?

Update - it seems that I actually got a relatively knowledgeable and thoughtful response:

Hi Lukasz,

I’m the resident WordPress nerd around these parts and I can understand how upset you could be. Still, your site was causing undue strain on the server. I don’t think it has anythig to do with clearing out WP-Cache tho. I do notice that you are running Bad Behavior as an anti-spam plugin. Normally. that’s okay - but for whatever reason Bad Behavior and our system don’t seem to play nice with one another when a site is being hit by a spam attack. That very well could have been what was happening here - but as I wasn’t the one who disabled you, I can’t tell.

I run the WordPress blogs on our here (blog.dreamhost.com and dreamhoststatus.com) and I’d be happy to give you a few pointers to keep things lean and mean. First up, you’re doing the right thing by running WP-Cache. Unfortunately, the programmer isn’t actively updating that plugin anymore. The successor to that plugin is currently WP-Super-Cache. More info about it can be found here:

http://wordpress.org/extend/plugins/wp-super-cache/

As for spam fighting plugins, sticking with Akismet is good - but using WP-SpamFree in place of Bad Behavior is a bit kinder on CPU cycles. We use this as the one-two punch on our blog and the spam that Akismet has
to process has gone down considerably. Check it out when you have a moment:

http://wordpress.org/extend/plugins/wp-spamfree/

Turning that on and getting rid of Bad Behavior will actually take your overall memory usage down quite a bit, so please try that out and let me know how it goes.

Also, I’ve poked around your database and have noticed that the table for Top Post By Category has eaten up 111.6 MB of your database. That’s more than 3/4 of your total MySQL database size right there. If you are no
longer using that plugin, it might be a good idea to clear that category out. In fact, it’d probably be best to stop using it as that will help speed things up as far as page generation goes.

If you’re curious as to why you should stay with us, hopefully the observations I’ve offered you on how to make your site run a bit better in our environment will make a convincing argument. Most companies would leave the ball in your court when it comes to optimization - but I honestly want you to get all the traffic you can. Outside of the WP blogs here, I have my own sites (hosted here) and I know the feeling of folks outside of your immediate social circle checking out what you’re doing. It’s pretty nice.

Anyhow, try my suggestions out if you’d like and please go ahead and let me know if you have any further questions. I’ll be more than happy to help you out in whatever way possible if you do.

The nice thing about Dreamhost - one of their few redeeming factors is that their support staff is usually quick to respond and usually knowledgeable and helpful. Well, with the exception of that tool who renamed my index.php and didn’t tell me about it that is. My past interactions with them were very positive. Can’t hurt to try his suggestions for now. The top posts plugin is gonzo for now! I liked having it in the sidebar, but I did not realize how database intensive it was. Oh well, good riddance. I will try to switch over to wp-super-cache and wp-spamfree over the weekend which may or may not speed up the site a bit. We shall see.

Just wanted to drop a quick note here - I won’t be doing Saturday posts anymore. I already gave up on Sunday posting back in January and I mentioned I might go down to a much healthier 5 day schedule. So this is what I’m doing right now.

This doesn’t really mean anything negative. Rather, I noticed that over time I’ve been putting more and more thought, research and time into every post here. More often than not I’m actually trying to write something that’s either meaningful, interesting or worth discussing rather than just posting 3 paragraph rants apropos the newest hot issue found via digg or slashdot. More often than not I’m trying to post original material here rather than reiterating stuff that you can easily find elsewhere. I don’t know if this is gradual improvement in quality, but it does tend to be more time consuming. So the drop in posting frequency is good for my sanity, and will hopefully let me queue up posts, and perhaps even proofread all of them before they go live.

I don’t think I will drop below 5 days, but perhaps that will turn out to be necessary in the future. For now though, you can expect to see posts appearing here Mon-Fri usually between 11am and noon (although Google reader doesn’t pick it up until few hours later).

Comment notifications are not working. Or rather they are working but with like a 48 hour delay. So if there was a comment posted today, you may get it Wednesday or something like that. And there is nothing I can do about it. Or rather nothing short term - long term solution is to switch hosts naturally. Dreamhost doesn’t seem to be able to figure out how to run a fucking SMTP server.

They had the same exact problem like a month or two ago, and apparently they haven’t really fixed it cause it is back only worse this time. I swear, I can’t win with these guys. Do not host with them - seriously!

In the meantime I recommend using a service like Co.mments.com to keep track of the discussion threads on your favorite dreamhost hosted blogs. I routinely use it for blogs that do not offer email notifications at all. You don’t get notifications instantly the way it is with email, but it’s better than nothing.