When someone puts a file up on a server for download, how does the host or the end-user know, for sure, the file they are about to download (or are serving up) is the valid file? What if someone hacked into the server and replaced the file with a bogus file that contained malicious code? It’s happened before and it will happen again. Fortunately there is a way to avoid downloading invalid files – checking the md5 hash. The only problem is that this method only works if the host and user knows how to use md5 tools. In this tutorial you will learn how to add an md5 checksum to a file and how to run a check on a file you have downloaded.

What is md5 and checksum?

Before we continue with the actual steps, you might benefit from knowing exactly how the process of checksumming works. MD5 stands for Message Digest algorithm 5, which is a cryptographic 128 bit hash function and serves as a “fingerprint” for a digital file. A checksum is a fixed-size datum that is computed from a block of data. When it is crucial for a piece of data (such as a download) to be valid, the datum is compared to the original block the datum was computed from to check for a match. When an md5 checksum matches, the user/host can be certain the file is valid. When the md5 checksum does not match, a red flag should immediately go up and the original block of data should be discarded. If a file changes by so much as a byte, the checksum will fail.

For most users these tasks are handled from the command line. There are GUI tools available (such as GtkHASH) that can tackle the same tasks. But for the purposes of this tutorial we will stick with the command line tool.

Creating an md5 sum

For those who plan on hosting files for download, you will want to know how to create an md5 sum. This is very simple. Open up a terminal and change to the directory holding the file you want to work with. Say, for example, you want to create an md5 on the file /var/www/files/download.tgz. To do this you would change to the /var/www/files directory and issue the following command:

md5 download.tgz

The above command will output something like:

Related posts

• Verify the integrity and authenticity of files (5)
• Verify File Integrity Of Backups (4)
• Theoretical Cryptography Part I – MD5 (8)
• How To Download Files From The Internet The Secure Way (9)
• ClickCrypt 2.6 (0)

It is therefor recommended to verify the file integrity of backups regularly to make sure that they can still be used to recover data if the need should arise. Nothing’s worse than to realize that the backups are unusable.

File Check MD5 is a small portable software for Windows that has been designed for that specific purpose. It can be used to compute MD5 hashes of all files in a specific root folder including subfolders. All computed MD5 hashes will be stored in a text file in that root folder.

The Check Files option can then be used to check files on the backup storage with the MD5 hashes of the original files to ensure data integrity.

One interesting option of the software program is to test the md5 hashes that have been previously computed if the text file is located in the same folder as the executable. It would then only be a matter of adding the File Check MD5 software and the text file with the MD5 hashes to the root of the backup to be able to automatically verify the file integrity.

The only thing that is missing is the ability to select a different storage location for the MD5 hashes. This would make it possible to run the software on read only devices as well.

Related posts

• File Integrity Check (2)
• Youtube Video Search Is A Barebone Youtube Downloader (2)
• Xkcd Comic Wallpaper Changer (2)
• Wireless Networking Software Homedale (13)
• Windows XP: Default Internet Browser Per User Profile (0)

So for the first Part of this – hopefully ongoing – series, I decided to look at the MD5 hash algorithm. It’s one of the most commonly used cryptographic algorithms out there and I would claim that nearly everyone has a password somewhere that is stored with an MD5 or similar hash.

MD5 stands for Message-Digest Algorithm 5, and is – as already mentioned and you probably already knew – a hash algorithm.

The MD5 hash algorithm is in simple terms a deterministic function (or blackbox) that will calculate a 128-Bit hash value from a given string of well-nigh any length … yeah, I had to read this sentence over a few times, and it’s just rubbish. If I wanted to write something like that, I could’ve gone Wikipedia. So let’s crack this one open.

You feed the MD5-Box a string of any length you want. This “string” doesn’t have to be alphanumeric of course, any stream of bits and bytes is just fine, like the bitstream of a file, for instance. The output string has always a length of 128 bits and is usually noted as a string of 32 octets, like this one: “B5A8AD3A9CDD6A6953FCBE6975FDE734″ (try guessing what I typed in though).

One of the most important things about hashes is, that they are so-called one-way-functions, meaning, they only encrypt stuff, and can’t – and must not – be decrypted. So hashes are often used for storing passwords in a databases. The same plaintext will always be hashed to the same cipher text with MD5, so all you have to do to check if your password and the stored (hashed) password are identical is to compute the hash of the given password and compare it with the stored one.

There are several demands a good hash-function has to meet in order not to get cracked in the first two hours of its lifetime.
The first one is, that a minor change in the plaintext (like “ghacks” and “gHacks”) should have a big impact on the computed hash (”D1B81FBDEB51C3A850E37177A5A22498″ and “DB3E20DC88EF0B6CA6A8FD5DA448D323″). If the difference would be only minor, and I know the plaintext and hash of “ghacks” (which I do, of course), and have the hash of “gHacks” without the knowledge of its plaintext, I could easily guess it.

The second very important demand is that a hash-function produces a much smaller memory imprint than the original stream. If you hash an 11MB installer to verify its integrity and have to download another 10MB of hash file as well, it’s pretty useless. There are lots of other points to keep an eye on, but these will (and have to) suffice.

As I mentioned already, hash-functions such as MD5 are most commonly used to store passwords without actually storing them in plaintext, and to verify the integrity of files. When you put a file online, just compute the hash and publish it together (but separate) with the file. Ever user would be able to determine if the downloaded file has been tampered with by simply comparing the hash of the downloaded file with the one published on the website.

Now I’d like to say something about security and known (and partly successful) attacks against hashes and MD5 in particular.
Due to the reduction (a 2MB file gets reduced to a 32-octet hash), information gets lost. This gets perfectly clear, if you take a look at the numbers. There are only 2^128 possible hash values, but infinite possible plaintexts. So in a best-case-scenario, after hashing plaintext numbers (2^128)+1 you have at least two plaintexts getting mapped on one and the same hash value.

So the first attack tries to make use of this very fact. When the same hash value is calculated from two different plaintexts, it is called a collision. Depending on the scenario of the attack using collisions, the birthday paradox comes in handy as well, increasing the attackers chance of success.

That would mean that you do not attempt to break the encryption or guess the user’s password when trying to crack a password, but just try to create another password that leads to the very same hash value, granting you access to the account. Of course, knowledge of the hashed password is required, but without that information, most attacks on modern ciphers are more than just tricky.

Edit: please take a look at comments for more clarification on the types of attacks mentioned above.

The second attack is based on a brute-force attack, which is basically “try all possible keys/passwords”. Depending on the numbers this could take some time. Let’s say you’ve already acquired the target hash value and your machine is able to try 100 keys per ms. That would make 100.000 keys per second, and 6.000.000 keys per minute. 2^128 hash values. That’s 3.4E38. We’re talking “age of the universe in seconds”-numbers here.

But there’s more to it than meets the eye. There are several options to reduce the available possibilities. Can you reduce the amount of possible plaintexts maybe? Maybe the password only allows to be 8 alphanumeric letters long? Can you have a look at the used algorithm and find something that may help you further? Do you know part of the plaintext? Maybe a name of son/wife/pet? Then you could combine it with a dictionary-attack. Every bit of information helps reducing the number of possibilities further, which in the end leads to a situation like this:

The following is a description of an attack to crack the user passwords of windows accounts (up to XP), and implemented in a near-perfect way by ophcrack. If interested, do make sure to check this tutorial, it’s quite fascinating and yet unbelievably scary.

Windows saves hash values of the user passwords, but if a password is longer than 7 signs, it gets broken up into chunks of length <= 7. Then the chunks get converted to uppercase only. Microsoft used DES for creating the hashes, but there’s no difference regarding this kind of attack.

So the attacker knows pretty much about the plaintext and can reduce its possibilities by a great deal. Now a computer starts calculating all possible hash values for this particular range of plaintexts (up to 7 digits, uppercase, numbers and some special characters only) and stores them in a database. Once finished, the database is from about 0.7 to 4 GB in size and can be easily transported using a thumb drive or a DVD.

Now all the attacker needs is a few minutes alone with the target computer and it’s done. Again, check the tutorial mentioned above, it kinda blew my mind. 1.7 minutes was the average time in this experiment for cracking a password to your windows account. Ouch.

Since I read and heard all of the above some time ago, I started wondering about the benefits and risks of using MD5. Most security experts discourage the use of MD5 nowadays for its known vulnerability to collision attacks. It should be replaced by something like the SHA-1 or since it is kind of outdated as well the even newer SHA-512. But that doesn’t help against the attack last mentioned, apart from increasing the possible hash values to even greater dimensions.

After some time, I found this very helpful article about spicing up your hashs to be more secure. I have to say though, these tips are NOT increasing the security of your hash function in a mathematical way. Luckily, the real world’s not all about math, so I think they are an easy way to get some extra security.

Edit: Please keep in mind that the tutorial posted here is not a perfect implementation of salts. It’s – as always – a source for ideas, not a perfect solution. But I always like it more if it’s explained like that, easy and understandable and in a rather digestible way. Please correct me if I’m mistaken.

If you want to screw around with MD5 a bit, here’s a link to an applet where you can do just that (SHA-1 as well). Switch to MD5, enter some text and press “Text digest”. Try guessing my hash from above (reaaaal easy), if you like and post the answer in the comments. First to score gets a cookie ;)

Stay tuned for upcoming ramblings about encryption and stuff. Maybe AES will be next.

Related posts

• ClickCrypt 2.6 (0)
• Verify the integrity and authenticity of files (5)
• Verify File Integrity Of Backups (4)
• Take advantage of md5 checksums for download validity (6)
• Protect your Wireless Lan (9)

The hash has to be the same, if it is not either the download went wrong or someone tampered with the file in some way. This could be that a virus or trojan was added to it for instance.

HashTab is a Windows Shell extension that adds a File Hashes tab to the properties window when right-clicking a file. It does display the hashes of several algorithms including MD5 or CRC32 and even has a button to easily compare hashes. That’s better than manually trying to compare the 32 digit+ hashes.

Related posts

• Verify File Integrity Of Backups (4)
• Hash Tab Shell Extension (2)
• Check MD5 Hashes (0)
• ZoneScreen Finally Gets 64-bit Update (7)
• ZC DVD Creator Platinum [Free Full Version] (19)

Click ok and you are ready to use the program using the context menu. Right click a file that you want to encrypt and select Send to / Click Crypt.
A dialog appears and you have to enter a password twice for the file that you want to encrypt. The option at the bottom deletes the file after it is encrypted, so uncheck it if you want to keep that file. Hit ok and the file will be encrypted using your password and the algorithm you´ve chosen.
If you want to decrypt the file you simply doubleclick it and enter the password in the windows that appears. That means, if you send this file to someone else he needs clickcrypt installed on his system and of course the password.

Related posts

• Theoretical Cryptography Part I – MD5 (8)
• Steganos Locknote standalone text encrypter (7)
• Security and Privacy Complete (0)
• Perfect Keylogger lite (4)
• Astalavista Top 10 Freeware Tools (2)