20 December, 2015 (published)
20 December, 2015 (last modified)

Fighting the dll hell of Windows

  Categories: Bugs, Linux, Mac, MS Windows, Qt
  Tags:  , ,

hellAt first sight it looks sensible to put all functions your application needs inside your application. However, it would make your application very big and very difficult to manage. Whenever there is a function that you use - like printing something or saving a file - is being updated by the operating system you have to rebuild and relink your application. That is crazy. So thousands of mundane functions are supplied by the operating system and the only thing your application does is calling them. Software development kits supplied by the company that is responsible for the operating system in question will tell the developer how to use these system functions.


Dynamic link libraries

It becomes quite different if you use functions from a 3rd-party software supplier. Yo have the choice to link them statically or dynamically. When you link them statically they are literally being put inside your application. But suppose you localize your application and use different languages. Putting all the translations inside the application does not make sense. You do not want to rebuild and relink your application because you have corrected a minor mistake in the Spanish translation. So you would like to separate these functionalities. For this dynamic linking was invented. You put these functions inside a separate binary - called a dynamic link library on Windows and OSX and shared object on linux.

The principle of separating functionalities into a main application and supporting libraries is done quite often and not limited to binaries. The extremely good thing about putting functionality in separate dynamic link libraries is that it forces you to write your code in a modular way. So it makes your code easier to read, easier to debug and easier to maintain. However there is an immense complication: the dll hell.

Dll hell: system cannot find the dll

Dynamic link libraries are separate binaries that have to be located somewhere. In the first place on your own development computer as otherwise you cannot run or debug it.  In the second place if you deploy your application the needed dll's must be shipped with the application and deployed somewhere on the computer of your customer.

When the operating system is loading an application for execution and this executable tells the system to load a certain dll it will search several directory locations in a prescribed order only on the basis of the (short) file name of the dll. If it cannot find that dll it will issue a fatal warning and the loading of the executable is aborted. f you get the warning you can repair the warning, but if not you but your customer gets this warning you probably will loose this customer.

After several development cycles you will have different versions of your dll with updated and outdated functions. On Windows these improved dll's often have exactly the same name. Hold on for the next disaster

Dll hell: operating system loads wrong dll

In earlier time disk and physical memory was expensive and developers tried to minimize using resources by optimizing their sharing. In Windows this meant that everybody dumped his own dll's in the system32 directory. Even worse they added functionality to a system Windows dll and replaced the system dll with their own. This is a sure recipe of disaster and all kinds of screens of death. This situation has cleared up considerably: dumping a dll in a system directory is considered bad practice, and strongly discouraged if not forbidden by Windows. Unfortunately this bad practice is still common in Linux and OSX.

Nevertheless Windows still easily loads the wrong dll as the search path includes all paths included in the PATH environment variable and chances are that it wil find a dll there with exactly the same name. This most often occurs if you use a third-party dll of a popular software vendor.

Dll hell: prevent it by taking control

You cannot allow the system find a dll beyond your control. So yo must avoid this at all times. Her are some guide lines:

  • If the dll is developed by yourself put the version in (i) a C++ header file and (ii) put a function in your dll that returns its version (hard-coded in an implementation cpp file) if this version is equal to the version in the header. Otherwise the dll should throw an exception.
  • If your application loads this dll the first thing your application should do is to get the dll version and check whether or not this version is right for your application.
  • Copy all the non-system binaries your application depends on into the directory containing the application. Do not put them anywhere else. You can find the majority of these dll's by using DependencyWalker.
  • If you develop using the Microsoft Visual Compiler your customers will need the redistribution dll's that are supplied by Microsoft. These dll's are different for different compiler versions and differ for 32 and 64 bit. Microsoft advises to redistribute them as an executable that your customer should install. We prefer to copy only those redistribution dll's that your application needs to the application dir. You can (legally) rip them from the redistribution package.

Dll hell: test your solution

Windows is guaranteed to search for a dll first in the application directory. So if it finds it there you are safe. Do the following steps:

  • open a MS-DOS terminal Window at the file location of your application.
    Run first

    And then start the application inside the terminal. Windows will yell when it cannot find a dll. Usually the warning is intelligent enough to find out what dll is missing. If you would not have set PATH to an empty string Windows would probably find it somewhere else and you want to avoid this at all costs. Do not continue until your app runs.

  • Now rename one by one the dll's inside your application directory. For instance through using explorer. After any change run the application again inside the terminal window. Each time Windows should complain about not finding the dll you just renamed. If it does not complain and runs it apparently loaded the dll from somewhere else. Find the other dll and get it out of the way by (temporarily) renaming it or by renaming its containing folder.
  • Testing the VC redistribution dll's is trickier. If you have them installed on your computer you should get them out of your PATH temporarily or you should use a virtual machine.
  • You should perform the previous steps even if your are debugging inside your IDE as your IDE probably makes a ton of dll's visible to your application and I will bet some of them are of the wrong version.
  • As you now know all binary dependencies of your application you can from now on at each new build copy them with a script to the directory of your application.

Reward: deploying is a piece of cake

As you have all your necessary binaries in one folder you could just zip the whole folder and unzip at the customers computer. It will run right out of the box. This even does not require your customer to have Administrator rights. If your customer wants to "install" the program the content of the application directory should be put inside the install script and Administrator right will be required.



- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Write a reaction

By submitting a comment here you grant this site a perpetual license to reproduce your words and name/web site in attribution.