A client of mine recently rolled out their upgrade from McAfee 8.0 Enterprise to 8.7 Enterprise. On the Citrix server environment we had two major application failures that were the result of the McAfee upgrade uninstalling the MS XML 4.0 Parser on the servers. There were two applications on the production Citrix environment that required the MS XML 4.0 Parser and they both stopped working following the upgrade. Unfortunately, the issue was not caught in Lab/UAT testing and was only found after McAfee went out to production. To make matters worse, something went wrong with the deployment mechanism and the McAfee upgrade went to all servers in one blast whereas the server team usually pushes in 2-3 separate sets of server groups. All of that combined = FAIL
So McAfee is responsible for the outage, right? WRONG!
What the Mcafee installer was doing was proper behavior. It had incremented the shared component registry entry for MSXML4.dll when it was installed on the servers. The shared DLL component dependency counters are stored in HKLM\Software\Microsoft\Windows\Current Version\SharedDLLS. Here’s an example listing 5 installed apps that depend on MSXML4.dll on my workstation PC (see the DWORD 5 next to MSXML4.DLL).
Every time a program installs that depends on MSXML4.dll (or any other shared windows DLL), it’s supposed to read the existing counter value, increment it, and then write it back out. Upon uninstalling that program, the installer engine is supposed to read the counter, decrement it, and write it back out. However, if the value of the counter is 1 and a program is uninstalling that has a shared dependency on that DLL, it’s supposed to uninstall that component and remove the shared component counter from the registry. See, everyone is supposed to tidy up after themselves.
Windows Shared DLLs + Dumb Developers = FAIL
So in my client’s situation, they had two applications that apparently required MSXML 4.0 Parser, but those apps never bothered to increment the shared DLL counter in the registry when they were installed. This is usually due to a dumb developer that doesn’t realize that the component has to be registered with the installer engine in order to properly install and register the shared DLL component. So when McAfee Enterprise 8.0i went to uninstall, the installer engine noticed that the value of the shared component registry entry was 1and subsequently ran a removal process for the MSXML 4.0 Parser. This equals failure for the two apps that need MSXML.
So how do you know you’ve been affected?
1) If you’re lucky, the application will complain about not being able to find the MSXML4.dll, or not being able to instantiate a object for the MSXML4 DOM. If you’re not that lucky, then…
2) FileMon / ProcessMonitor from SysInternals will pinpoint the issue. Simply run it before launching your application and when you reach the place in code where the app is crashing or throwing an unexpected error, you’ll see messages in the log where it was attempting to locate MSXML4.DLL in the search path and failing.
How do you fix it?
Download MSXML 4.0 SP2 Parser from Microsoft and install it. You can do the install manually or push it with whatever ESD system you’re using. The MSI installs quite nicely with Citrix Installation Manager if you only have that.
Stay tuned for another entry from me regarding this issue with respect to application virtualization since it has some unique properties that make this particularly challenging…
Agree? Disagree? Let me know with a comment...