WordPress: Vanishing Categories

Roughly a month or so ago, something weird happened to this website. It was one of those weird and a bit scary glitches that make you question your own sanity because they come out of nowhere and they have seemingly no reasonable explanation. I was busy typing away a new post when I noticed that all my tags and categories simply vanished.

The content, mind you was still there. All the posts and images were intact. They simply lost their category and tag associations. I have never actually seen anything like this before so my first thought was “database corruption”. I’m not sure how the DB could get corrupted, but WordPress is famously finicky about these sort of issues. It is not uncommon to see a badly written plugin touch one of the core WordPress tables in a bad way and make it freak out.

I promptly logged into the server and started running exploratory queries, but most of them came back looking very normal. The tag and category tables still had all of the entries in there, and posts were still correlated with them via foreign keys. The schemas of all the tables looked normal and I couldn’t detect any sign of plugin induced damage or even malicious tampering. All the information was in the database, but the UI refused to make the connections.

Luckily this is not the first time (and probably not the last time) I have seen WordPress go completely wonky. I have learned that running an active WordPress site without nightly backups is pretty much actively seeking out headaches. So after scratching my head for two hours, I decided to roll my VM back to the last night’s snapshot and see if that fixes the issue. Before that I decided to check how much disk space I have left.

It turned out that I had literally zero bytes.

Just on a lark I blew away the contents ~/temp and refreshed the site. The categories and tags have magically returned, but only partially. It appears that in order to render the tags and categories and associated pages WordPress needs to write a buch of temp files to disk. I have no clue why this happens, but I’m assuming it is an optimization strategy of some sort that is intended to limit the number of database reads per page view. However, if your disk is filled to the brim, it can’t do that. Therefore it fails silently and does the best to render the page without that additional information.

On one hand it is quite amazing that inability to perform some internal core caching does not bring the entire site down. On the other hand it seems wrong to me that such an operation is necessary. But despite using WordPress for many years now I have never actually felt compelled to peel the hood back and look at it’s database queries, so I guess I shouldn’t criticize something I don’t know all that much about.

Over the years I have gotten pretty good at cleaning up linux machines from accumulating temp file cruft. So it only took me a few minutes to identify the source of my disk bloat. It was the temp directory used by my WP-Cache plugin which ballooned up to few Gigabytes somehow. Apparently the old cache files were being discarded but never deleted. Blowing away the entire cache reduced by disk usage from 100% to 35%.

To prevent this sort of thing happening again I wrote a tiny guard-dog script that checks my disk usage on a weekly basis and prints out a nice report that is then emailed to me via a cron job:

# use colors if available
[ -f "$HOME/scripts/colors" ] && source $HOME/scripts/colors
command -v awk >/dev/null 2>&1 || { echo "awk not found. Please install it and try again"; exit 1; }
command -v du >/dev/null 2>&1 || { echo "du not found. Please install it and try again"; exit 1; }
command -v df >/dev/null 2>&1 || { echo "df not found. Please install it and try again"; exit 1; }
# grab the % usage of the primary partition (typically line 2, col 5 on df)
read USAGE <<< $( df -h | awk 'FNR == 2 { print $5 }' )
# Make red if usage is above 60
if [ ${USAGE%?} -lt 60 ]; then
echo -e "\nDisk Usage Report"
echo -e "-----------------\n"
echo -e "Disk usage: \t $Color_On$USAGE$Color_Off\n"
df -h
echo -e "\nLog file spot check:\n"
# Adding output to temp file so we can sort it later
# -sh provides human readable summary
# -BM sets the block size to Megabytes
du -shBM /tmp 2>/dev/null >> /tmp/du$$
du -shBM /var/log 2>/dev/null >> /tmp/du$$
du -shBM /srv/www/*/logs 2>/dev/null >> /tmp/du$$
du -shBM /srv/www/*/*/*/wp-content/cache 2>/dev/null >> /tmp/du$$
du -shBM /srv/www/*/*/*/wp-content/uploads 2>/dev/null >> /tmp/du$$
sort -nr /tmp/du$$
rm /tmp/du$$

The full version of the script is actually available here. The colors script I’m importing up top is also on Gighub if you want to check it out.

If you ever notice tags or categories vanishing from your blog, don’t panic. It probably just means your disk is full.

Posted in sysadmin notes | Tagged | 2 Comments

3D Printing and the future of Tabletop Gaming

3D printing has been around for quite a while now. It has always been something I considered kinda cool, but mostly inconvenient to use. You either had to build your own printer from scratch, or send your 3D model to a specialized company that would print it for you. Granted, the number of commercial 3D printing companies has dramatically increased in the recent years, and their services have become completely affordable for mere mortals. Places like My Mini Factory will even let you sell your printable designs CaffePress style for a small cut of the profits, which is amazing in itself.

What has completely blew me away though, is the fact that user-friendly, in-home 3D printing is now entirely possible and completely affordable. It turns out that you can buy a sleek, Wifi enable Makerbot for about the same price as a low-end Macbook.


Makerbot 3D printers are really affordable now.

No, Makerbot is not paying me to say this, though maybe they should. I’m not even trying to convince you to buy one. There are cheaper alternatives than this if you don’t mind doing some assembly on your own. In fact, I hear that while they are extremely sleek and user friendly on the surface, the Makerbots get clogged quite easily:

Still, it’s nothing short of amazing that you can get a device that will turn a 3D computer model into a physical object for about the same price as a current gen gaming console bundled with a few new games. It is even more amazing that you don’t even need to learn to use AutoCAD or a professional 3D modeling software like Blender because both Makerbot products as well as the above-mentioned on-demand printing services will happily accept files generated by super-accessible software like Sketchup.

This is actually really exciting for me because I’m into tabletop games and board games which often use small plastic, resin or metal models and dice. The fidelity of the commercially available printers isn’t high enough yet to allow “pirating”, say Warhammer miniatures, and it is not something I would advocate anyway. Trying to duplicate existing collectible miniatures seems like a waste of this marvelous technology. I’m more interested in creating original stuff: like models for games that don’t exist, because you haven’t written them yet.

In the past, if you wanted to create a board game or a tabletop battle system you had to find a publisher who would manufacture your models for you. The only other options were crowd funding, or keeping your game simple enough to allow paper tokens, or models from other game systems.

3D printing technology however opens the door for “indy” board and tabletop battle games. Think about it: you can design your custom models and test them out on you home Makerbot. Once you are happy with the design you publish them to an on-demand service. Players can then go and purchase the models they need, have them printed and shipped directly to their house.

I predict that in the next few years we might see an explosion of new amateur made games that rely on inexpensive 3D printing. It will be interesting to see how this affects both the printing market as well as the tabletop game industry. Could an independent, on-demand printed game become popular enough to threaten a market leader such as Games Workshop? I guess we will find out.

Posted in technology | Tagged | Leave a comment

Short form Blogging

You might have noticed my post output has been consistently dropping lately. This is partly due to the fact that over the last few years this blog morphed into a collection of essays and long-form reviews rather than a daily blog. In the past I used to re-blog interesting links or provide commentary on current issues in technology under the mistaken impression that my “expert opinion” was somehow relevant. In most cases it wasn’t. It was merely adding to the echo chamber effect of our little community without adding much to the discussion or providing any unique insight.

These days I feel that the things I post should be either original and creative (like my tabletop gaming posts), provide some value to the readers (like reviews or tutorials) or at the very least say something thought provoking. As it turns out it usually takes me a few thousand words and at least few hours of research to produce something I think is worth reading and interesting. The posts that I was really happy with, and even proud of were the ones that took me weeks to compile. For example I started working on the Fimir article was already in the making when I posted about the Duckbunny. These things take time, effort and attention… And when they don’t, when I just randomly throw something quick together over the weekend I’m usually not happy with it. I have a dozen posts in my drafts section right now, which were queued up for posting but never went live because I decided I hated them, and I would rather have nothing go up than post something rushed and unfinished.

On the other hand, I hate to leave the blog dormant for a while, because it’s disappointing for readers, bad for SEO, and it decreases my already mostly insignificant ad revenue. There is a conflict here. I do want to keep trying to raise the bar and keep pushing myself to write longer, more thoughtful essays. But I just can’t physically pump them out that fast and I’d much rather take time and make something good: either by being creative or by actually doing research on the topic. But at the same time I don’t want the site to seem abandoned while I work behind the scenes.

Which brings me to a question: what is a blog supposed to be anyway these days? What blogs do you read, and what is their format and post frequency?

Blogging has matured a lot ever since I started this site on a free Blogger account, long before it was acquired by Google. While the general format of a blog remains the same, blogs became many things to many people – from marketing to journalism and anything in between. It is quite interesting how this medium has evolved, and how it now has to share it’s niche with social media and collaborative experiments like Medium (where you partly cede ownership of content you produce exchange for exposure and curation). We did not have microbloging, tumbleblogs and curated colaborations back when it all started.

What should be the purpose of a modern blog then? Is it to inform or to analyze? Should posts be bite-sized and accessible, or exhaustive and deep? I guess you pick whatever works for you and you run with it… But that’s not much of an answer.

I know that personally I love reading thoughtful and well researched essays or long form reviews. At the same time I also do see value shorter posts that simply touch upon an interesting topic without going into much depth. These sort of things can be thought provoking in their own way, as they provide a launch-board for discussions, or can inspire you to write something of our own.

In the last few months I have posted a few shorter pieces and most of them went over well, even though each time I felt like I was cheating by not providing more thoughtful analysis. Perhaps I shouldn’t though. Perhaps I should just tag these sort of posts as what they are, “short musings”, and post more of these as I work on longer and more organized and researched essays. I think I can manage to write few hundred word blurb whenever I just don’t have a few thousand word essay in me. This may actually make it possible for me to squeeze out more than one post a week without feeling guilty about post quality.

Also, fellow Americans, have a happy 4th! I was planning to post a picture of Will Smith and Jeff Goldblum, like I do every year, but I’m kinda tired of that joke. Maybe next year.

Posted in meta | Tagged | 13 Comments