Save disk space with UPX
Disk space is cheap, but there are still situations where you'd like to reclaim space. For me, one of them is getting the maximum of the free Dropbox account I use to synchronize my applications between work and home. Hey, don't say my sysadmin I do this, he'd get a heart attack from the sole thought of a virus among my tools collection.
Of course {zip/rar}ing everything is a no-go; nobody wants to have to open an archive before being able to launch an application. So what?
Enter UPX, the reference packer for executable files. UPX compresses executable files (mostly .exe and .dll under Windows), but contrarily to zip/rar/whatever, it keeps them executable, also preserving their properties (icon, version...). All this at the cost of a completely unnoticeable performance hit when starting the application. Sounds good? Let's compress everything executable in this Dropbox that is dangerously close to reaching its 2GB limit.
- Download UPX from SourceForge and extract it to <upxFolder>.
- Open a command prompt and browse to <upxFolder> (you can find the command prompt in Start > Accessories or run it via Windows+R, then "cmd". Use "cd FolderName" to navigate the filesystem).
- Run upx.exe to have a look at the options available. I'll use -9 but you can adjust this (for example if you feel a performance hit on an old machine --I don't--), and -v to have verbose output. Now, navigate to the folder of a program you want to compress and run your freshly-baked command:
"<upxFolder>\upx.exe" -9 -v *.exe *.dll
(replace <upxFolder> adequately, and keep the quotes around the path, or spaces will cause problems) - You'll see the files being compressed, the compression ratio, as well as the occasional "AlreadyPackedException: already packed by UPX", or "CantPackException: .NET files (win32/net) are not yet supported", or "IOException: read error: Invalid argument". These errors are all OK, UPX just notifies you that it left the file untouched.
Woot, you already gained 50MB by just compressing big ol'Inkscape.
Of course you now want to do this on ALL the executables in a folder. But you're lazy, and the perspective of running this line inside hundreds of subfolders leaves you bleak. Lucky you, I'm lazy too, so I dug Stack Overflow to assemble a script that will do the operation recursively.
- Paste this in a text file (of course adjusting the main line like we did before) you will rename to something like upxRecursive.bat
for /r /d %%x in (*) do (
pushd "%%x"
"<upxFolder>\upx.exe" -9 -v *.exe *.dll
popd
)
pause - Move this batch file inside your Dropbox folder, launch it and see the magic. This will take a long time on a big folder filled with executable files, but will seriously slim it down. In my case, I saved 400MB (from 1.7GB to 1.3GB), leaving me with plenty of space for more crap.
Final notes:
- UPX has been in development for a long time and is praised for its extreme stability and reliability, but could possibly leave you with some exotic executable files that won't run after compression. I personally never occurred to me, but accidents happen. In this case, use
"<upxFolder>\upx.exe" -d problematicFile.exeto get an uncompressed version. Note it won't be byte-identical, if you want to be able to get back to byte-identical versions, you should add the--exactswitch at compression time.
By the way, the folks at PortableApps.com pack all the software they release with UPX. Reassured about its reliability? - People familiar with the win32/pe format will know that .exe and .dll are not its only valid extensions. The full list is ".exe .dll .cpl .ocx .sys .scr .drv". However, I don't recommend compressing sensitive files like control panel extensions or driver files, and this is why the line I suggest targets only dll and exe files. Compressing VLC is OK because you can reinstall it if UPX butchers it. Compressing critical software is not.
- I'm just repeating my previous point, but don't want to see angry comments from people doing crazy things with this. Again, don't do this on all C: , it -will- wreck your system. Compress executables when you really need it, and inside folders containing only reinstallable applications if things go wrong.
- UPX is multi-platform. My article focuses heavily on Windows because this is the platform where my use case comes from, but you can of course adjust this idea/script for Linux (plus you will probably have access to a saner scripting language than batch).
- EDIT: as pointed by John T. Haller in the comments, PortableApps offers AppCompactor, a graphical frontend that relies among others on UPX.
Ronan is a geek and musician living in Montreal. He likes scaring wary sysadmins with 2GB folders full of false positives and writes about software, music and life at flying molehill.
Sorry, but this is the worst tip ever!
Read on http://www.jrsoftware.org/striprlc.php#execomp why exe compressors are bad:
“Upon startup of a compressed EXE/DLL, all of the code is decompressed from the disk image into memory in one pass, which can cause disk thrashing if the system is low on memory and is forced to access the swap file. In contrast, with uncompressed EXE/DLLs, the OS allocates memory for code pages on demand (i.e. when they are executed).
Multiple instances of a compressed EXE/DLL create multiple instances of the code in memory. If you have a compressed EXE that contains 1 MB of code (before compression) and the user starts 5 instances of it, approximately 4 MB of memory is wasted. Likewise, if you have a DLL that is 1 MB and it is used by 5 running applications, approximately 4 MB of memory is wasted. With uncompressed EXE/DLLs, code is only stored in memory once and is shared between instances.”
So while saving diskspace you’re hogging up your memory.. i wouldn’t recommend this tip to anyone!
Of course it is a trade-off.
– As said multiple times in the article, I don’t recommend doing this all the time, only in cases with heavy storage space constraints
– Upon startup: again, I’m repeating myself, but I didn’t feel any difference. Please do some benchmarks and see by yourself.
– Memory: dunno.
Rather than worrying about it manually or compiling scripts, you could just use the PortableApps.com AppCompactor and have it handle all that for you:
http://portableapps.com/apps/utilities/appcompactor
Regarding the issues with UPX compressing, portable apps (and apps within a DropBox) don’t share EXEs and DLLs anyway, they all use their own copies within their own directories. Any shared DLLs would be those used by Windows (ie on the machine itself) and would not be compressed. So that point doesn’t apply to this instance. It is true that a compressed EXE/DLL will consume more RAM, but that’s the tradeoff for smaller apps.
Thanks John, I added a link to AppCompactor in the article. I still like relying on a simple script for batch operations, but some people will sure prefer the GUI way.
WTF is this ‘article’ good for?
UPX has been around for nearly 15 years.
Grow up you wannabe admins.
To “Anonymous” it good for a F in WTF. Some of us haven’t heard of it, like me. And it’s nice info. As to being a wannabe admin – yep. So? I still make more than you. What about it?
Who is storing executable files in DropbBox in place of data files? Someone who is on the limit of disk space? Really?