Terminally Incoherent

I recently stumbled upon this peculiar book titled “Man after Man” by Dougal Dixon. Just one look at the cover hooked me. I mean, look at this cover. Is it anything but intriguing?

Anthropology of the Future

“Anthropology of the Future” eh? I’m interested. Wouldn’t you be? I mean, I totally want to know how the human species could possibly evolve from the current state to Monkey-horse Darth Vader thing from the cover.

So I read the whole thing, and I must say it is interesting. It is not really a novel, as it does not have a cohesive story. It is written in the style of nature documentaries, or these classic Zoology/Dinosaur books we all have read as kids. As it is suggested on the cover, Dixon aims to predict where evolution will take the human species in the next 5 million years. He factors in various variable influences such as coming of another ice age, magnetic pole reversal of the planet and human meddling via genetic engineering. Even though it looks and feels like a nature book, it is a work of pure speculative fiction. Dixon makes a lot of assumptions early on – such as introduction of genetically engineered human sub-species adopted to specific environments and then tries to guess how might have they evolved when left to their own devices.

Vacumorph - a deep space adapted human

The entire book is a string of illustrated short stories that depict a “moment in life” of various descendants of the homo sapiens. The individual stories oscillate between dull and intriguing but in most cases give you a good overview of that particular species and its quirks with a little bit of personal touch. Dixon usually names his protagonists, and tries to write from their perspective giving the reader some insight to what these creatures would have had on their minds.

Hitek in his organic cradle

I must mention the artwork, which is pretty striking. As you can see, I included few of the images from the book in this post, but there are many more like it. Some of them are incredibly goofy looking, but still kinda interesting. Most chapters include at least one such image, many of them featuring a whole bunch. Dixon likes to show you close ups and cross-sections of various features such as modified limbs, skeletal changes or new organs. In fact, I would say these pictures are the main draw of the whole book. Without them it would be just a bunch of dry, mostly disconnected mini-stories. The illustrations however really bring the weirdness of Dixon’s imagined future to the forefront. They are also what keeps you reading the book. Whenever you hit a dull story, you can just flip a page or two, see a weird looking monstrosity, and you are instantly motivated to keep reading just to find out what that thing is all about.

Our very, very distant descendants...

Some of Dixon’s assumptions are more of the fiction side of science fiction. For example, he predicts that in a response to changing environmental conditions one variant of our desert dwelling descendants will develop a psychic water seeking ability. Or that two human descended species will develop symbiotic-telepathic link that will allow the agile and intelligent hunters to control slow, dim witted and large carriers. There is of course nothing wrong with that, but the tone look of the book, often makes people believe that it is more scientifically grounded than it is. Dixon is basically world building and letting his imagination run wild, and we are there for a ride.

That said, I was a bit disappointed by the ending. I was really looking forward to the Horsemonkey Darth Vader explanation, and when it finally came, I thought it was a bit of a cop-out. Maybe it is just me though.

It is definitely an interesting book, an if you can get your hands on a copy at a reasonable price I would recommend checking it out. That is if you are into this sort of thing. If you don’t think reading a fictional anthropology book about made-up descendants of homo sapiens is not up your alley avoid it. Otherwise, I think you will be interested. Sadly, it is now out of print, and the few copies which are available on Amazon go for $60 or more. The book is definitely not worth that much money.

Good news is that the book is available online – though probably not legally. I will not post the direct link to it due to copyright/lawyergram concerns but I can probably point you to a reddit thread where I found it.

As I mentioned last week, I’m working on a public key encryption tool. Something that requires very little setup, and no cumbersome key management that is usually necessary with established tools such as GPG or PGP. One of the things I did not want to do myself was the implementation of the actual encryption algorithms. While it is a great exercise to do this, there are just so many subtle ways to accidentally leak information that I decided not to bother with it. So I decided to use something that is already implemented and built into the language. For example, C# already has a built in implementation of RSA algorithm. You can generate a key pair like this:

// generate 2048 bit RSA key pair
RSACryptoServiceProvider RSA = new RSACryptoServiceProvider(2048));

Looks perfectly acceptable to me. Simple, quick, easy to apply. Once you have the object you can export the key and write it into a file, or you can squirrel it away from the user and hide it into a “key store” like so:

CspParameters csp = new CspParameters();
csp.KeyContainerName = "MyTPK";
RSACryptoServiceProvider RSA = new RSACryptoServiceProvider(2048, csp);

If you do the above, your key pair will be automatically saved under KeyContainerName. If a key was already saved under that name, it will be fetched and used to create the object instead giving you nice way to maintain persistence without actually writing a lot of code or reading/writing temp files. You can clear your key store at any time by calling RSA.Clear();.

Encryption is equally easy – it takes a byte array and returns a byte array so you can do something like this:

// read in a file as a byte array
byte[] plaintext = File.ReadAllBytes("/some/file/here");

byte[] ciphertext = RSAEncrypt(plaintext, RSA.ExportParameters(false), false);

The first argument is the plaintext, the second one is the encryption key, and the third one tells you whether or not to use padding. In this example I’m using my own public key (by exporting it from the RSA object) to encrypt the message. If you wanted to use some other key, you could import it like this:

string key = "public key in XML format";

RSACryptoServiceProvider temprsa = new RSACryptoServiceProvider(2048);
temprsa.FromXmlString(key);

Now you can use temprsa to encrypt with that particular key. You obtain the key by calling analogous ToXML() method on the other end of the equation.

This worked quite well on my test file, which was just the word “text” saved as a plain text file. But as soon as I tried to test it on any larger file it failed with an CryptographicException containing a very descriptive message which said “Bad data”.

I had no clue as to what could have been causing the problem – mostly because I never really looked very closely at how RSA works. I knew it was an asymmetric encryption algorithm and that as enough for me. If I looked at it more closely, I would have discovered that it has an interesting property: namely that the encrypted message must be smaller than the key. That’s just how it works. RSA was designed for exchanging small amounts of information – for example the symmetric encryption key swap which happens at the beginning of an SSL session. It was never intended to be used to encrypt large files.

In other words, it was useless for my purposes. Now I had a choice to make:

1. Abandon RSA altogether and look for a different public key algorithm
2. Figure out the maximum message size it supports, then encrypt my file piecemeal – one tiny part at a time
3. Use RSA for what it was designed: encrypting a symmetric key

Option #2 seemed work intensive an error prone. Not only that, but depending on how I implemented it, slicing and dicing the file could potentially leak information. So I picked the option #3.

This of course meant I would have to modify my concept a little bit. Instead of one key exchange I would have to have two. First I would need the users to exchange their public files, use them to encrypt a symmetric key and then use that… Or perhaps not. I could probably keep it as a single key exchange – at least as far as they user was concerned. The new procedure would look like this:

1. The recepient generates an RSA public key and gives it to the sender over unencrypted link
2. Sender generates a random symmetric (AES) key and uses it to encrypt a fle
3. Sender encrypts the AES key with the RSA key and then concatenates it to the encrypted file then sends the whole bundle to the recepient
4. Recipient extracts the RSA encrypted AES key and uses it to decrypt the rest of the file
5. ???
6. Profit

Here is how the code for the encryption end would look:

string key = "some xml here"; // this is given

AES = new AesCryptoServiceProvider();

AES.GenerateKey(); // 32 bytes
AES.GenerateIV();  // 16 bytes

// concatenate AES key and IV into a single byte array
byte [] buff = new byte[AES.Key.Length + AES.IV.Length]; //48 bytes
                        
Buffer.BlockCopy(AES.Key, 0, buff, 0, AES.Key.Length);
Buffer.BlockCopy(AES.IV, 0, buff, AES.Key.Length, AES.IV.Length);
                                    
// encrypt the AES key and iv, using the public key
RSACryptoServiceProvider temprsa = new RSACryptoServiceProvider(2048);
temprsa.FromXmlString(key);
                       
buff = temprsa.Encrypt(buff, temprsa.ExportParameters(false), false); // 256 bytes

// read in the plaintext file
byte[] data = File.ReadAllBytes(path);

// encrypt file with AES
AesCryptoServiceProvider AES = new AesCryptoServiceProvider();
ICryptoTransform encryptor = AES.CreateEncryptor();
data = encryptor.TransformFinalBlock(data, 0, data.Length);

// concatenate the encrypted key, on top of your ciphertext
byte[] output = new byte[buff.Length + data.Length];
            
Buffer.BlockCopy(buff, 0, output, 0, buff.Length);
Buffer.BlockCopy(data, 0, output, buff.Length, data.Length);

// now you write output into file, or do whatever

On the opposite end you do the reverse:

// read in the ciphertext
byte[] buff = readFile(path);

// create a bunch of empty arrays
byte[] key = new byte[32];
byte[] iv = new byte[16];
byte[] temp = new byte[256]; // will hold the encrypted key
byte[] encdata = new byte[buff.Length - 256]; // will hold the encrypted data

// copy the key and ciphertext data into the newly created arrays
Buffer.BlockCopy(buff, 0, temp, 0, temp.Length);
Buffer.BlockCopy(buff, 256, encdata, 0, encdata.Length);

// grab the RSA key from the key store
CspParameters cspParams = new CspParameters();
cspParams.KeyContainerName = "MyTPK";
RSA = new RSACryptoServiceProvider(2048, cspParams);

// decrypt
temp = RSA.Decrypt(temp, RSA.ExportParameters(true), false);

// copy the key and iv to separate arrays            
Buffer.BlockCopy(temp, 0, key, 0, key.Length);
Buffer.BlockCopy(temp, 32, iv, 0, iv.Length);

AesCryptoServiceProvider AES = new AesCryptoServiceProvider();

AES.Key = key;
AES.IV = iv;

// decrypt
ICryptoTransform decryptor = AES.CreateDecryptor();
byte [] data = decryptor.TransformFinalBlock(encdata, 0, encdata.Length);

// now you write data into a file or whatever

Granted, the method above is going to be impractical for large files – mostly due to the amount of in-memory copying I’m doing. Then again, I have never intended this tool to be used to exchange more than few MB of data.

Now that I did all this work, I believe there is an even simpler way of doing this. I probably could have used the RSAPCKS1KeyExchangeFormatter class in order to abstract much of my mucking around with the RSA keys into 2 or 3 lines of code. But I think that at the end of the day this is how it works.

Anyways, I will release the source code for the whole damn thing shortly. As usual, constructive criticism and tips on how to improve the tool are greatly appreciated.

I really don’t know how to review this book without spoiling its ending. There is just not much else there to talk about. The characters are flat, one dimensional and barely realized. They come and go as Clarke cumbersomely steamrolls his plot and ideas many centuries into the future. Half the time it doesn’t even seem like a novel – it reads almost like an RPG handbook which describes how the imagined futuristic setting is different from present times. You know that advice they give you in creative writing classes: show, not tell. Clarke ignores the living shit out of it in this novel. He takes pages upon pages to describe cultural shifts, technology, and the reasoning behind them. The whole thing is rather dry, a boring and almost mercifully short. It’s just not a very good book.

You know why I picked it up? Because the blurb on the back made me think about singularity. Here is a short non-spoilerish rundown on the plot:

Unimaginably advanced alien race invades Earth and takes it over almost overnight. Once in control, it dissolves world governments, brings about global peace, prosperity, and then social engineers the shit out of our race for several centuries toward some unknown end. Then all of a sudden, there is a breakthrough – they achieve the desired effect, and OMG: “the consequences will never be the same”.

That’s the long and short of it. Combine this, with the title of the novel (Childhood’s End) and you can see how I arrived at the conclusion that this might be a novel where Clarke inadvertently stumbled upon the idea of technological singularity long before Vinge and other gave it a name and popularized it. Well, not quite.

But it does have a few interesting ideas. The arrival of the alien Overlords is a proto-singularity event of sorts. It does have all the hallmarks of such an event. For one, it essentially brings about an end to academic research in the fields of science and technology. The Overlords are centuries ahead of us with their technology and they willingly share it with us. Science becomes less about trying to understand our universe through observation and tests, and more about catching up with the vast body of work compiled by a superior civilization. So understandably the interest in sciences and technology falters while humanities and arts temporarily flourish. But only for few generations since Overlord imposed global peace, medical advances and social reforms reduce the amount of suffering and anguish to the point where no one has anything interesting to write, sing or make movies about. In most cases however, singularity involves some sort of ascendancy.

Baseline humans are left behind, but new and vastly more intelligent AI based sentients forge ahead and continue carrying the torch of human progress. The Overlord situation seems less like an end of childhood and more like arrested development.

Naturally you start wondering where Clarke is going with this, and then he pulls out classic psionics. After centuries of Overlord machinations, social engineering and gentle prodding something happens. Something clicks in the human psyche and the minds of children all over the world start opening up and merging into a single shared super-consciousness. This new shared mind of the human race reaches out toward the void of space, only to discover an even greater super-consciousness lurking out there. An amalgamation of minds from all corners of the universe, which invites it to merge and join with it.

So hey, it is a singularity – but of a different kind. The much feared and admired Overloards turn out to be the retarded janitor of the universe. Their race seems to have fallen into an evolutionary trap that has left them without any psionic potential whatsoever. This is why the galactic Overmind uses them as guardians and sheppards that are sent out to keep an eye out on the fledgling psycho-sensitive races, and to gently guide them through the turbulent puberty (which usually involves throwing around nukes, creating black holes in your back yard and royally fucking up your home world’s environment) into adulthood.

It is a cool idea, and one that has been rippling throughout the science fiction novels ever since Clarke put it down on paper. One example which comes to mind, is the ending of the Rise of Endymion in which human race awakens to the ability to connect and travel through the Void Which Binds. But I’m sure there are more.

All in all, it is an interesting read – just not a very good novel.