Wednesday, December 24, 2008

Cyberarmor viexpf2k.sys conflicts with MS's Driver Verifier

I had a BSOD (Blue Screen of Death) on my Windows XP laptop today. It freaked me out as I don't remember changing any system configurations previously, so I thought that it might be a virus. I've extracted the juicy part from the BSOD messages below:

Technical information:

*** STOP: 0x000000c9 (0x00000007, 0xA8655492, 0x8ADACF68, 0x00000000)

*** viexpf2k.sys - Address A8655492 base at A8655000, DateStamp 4372d07b

Beginning dump of physical memory

A little googling shows that "viexpf2k.sys" is either a malware, or a driver from CyberArmor. I have the Cyberarmor firewall (Corporate version) installed, so I investigated from this angle. But just in case, I also ran CureIt AntiVirus from DrWeb.

At first, I couldn't get into Safe Mode as it gets stuck during the driver loading phase. But CHKDSK pops up after a few hard reboots, and Safe Mode boots fine after the disk check.

For some reason, my laptop didn't generate any dump file (company policy, no direct control), so I could not confirm my suspicion with WinDbg. I had to do it the "hacker" way: I renamed "viexpf2k.sys" to "~viexpf2k.sys" in both the "C:\Program Files\CyberArmor" and "C:\WINDOWS\system32\drivers", and then my XP managed to boot normally, but without CyberArmor.

So it's confirmed that CyberArmor is the problem, but why? More googling shows stop code C9 to be DRIVER_VERIFIER_IOMANAGER_VIOLATION, and it hit me! I must have turned on Microsoft's Driver Verifier by mistake yesterday when I was researching on kernel dump debugging.

So I restored "viexpf2k.sys" and ran:

C:> verifier /reset

Voila! Every thing's back to normal after a reboot.

Tuesday, December 16, 2008

Performance tuning for Samba on Fedora 9

When I first set up Samba on my Fedora 9 server, the access speed was not very impressive to say the least.

So I googled a little, and I found out that I can get some speedups if my disk do not record the file access time as my application reads and writes to files very often. I added the noatime option to the entry for my share partition in "/etc/fstab" like:
UUID=????? /data ext3 defaults,noatime 1 1

I did get some speed increase after this, but it is still slower than a native Windows Share.

So I googled some more, and I found:

I added the following options to "/etc/samba/smb.conf":

dead time = 15

The speed increase is quite dramatic, but it is still slower than a native Windows Share. Still not sure if the "dead time" option helps, but I just don't have to time to test it out.

Monday, December 15, 2008

Symbol Server for WinDbg

Today, I used Microsoft's WinDbg debugger to check out a kernel memory dump to see if I can determine which is the last program in memory before my Windows 2003 server (Mystic Hare) crashed. I guessed that it may give me a clue about which is the offending program that is crashing the server.

I haven't used WinDbg in years, and I didn't have the symbol package for Windows 2003 with me. Downloading it is out of the question as my internet connection speed is slow and the package is a few hundred megabytes.

Lucky for me, WinDbg can now download symbols as needed if internet access is available, from what I read in

I created the "c:\websymbols" directory to download the symbols to.

1. Start a debugging session.

2. Set the debugger symbol path as:

Now, I can start the debugging process.

Unfortunately, the debugging didn't go far as the symbols for an external program's .sys file is not available, but it's another story.

Thursday, December 11, 2008

Editing fstab from "Repair Filesystem" prompt in Fedora 9

My Fedora 9 server is a laptop with a non-standard harddisk setup. In a mis-guided attempt to reduce the heat produced by the laptop, I removed the internal SATA II harddisk and put the Fedora's / root partition on a external SATA II harddisk connected by a SATA ExpressCard. As my laptop do not detect the main harddisk connected by the SATA ExpressCard, I created the /boot partition on a 1GB USB disk.

Here comes the problem, the USB disk is detected as either "/dev/sda1" or "/dev/sdb1" at different boot times. Thus I have the problem of deciding what to put in the fstab. You may ask when don't I use the UUID since these devices are controlled by udev anyway. It's because sometimes the UUID used to identify the USB disk do not work during boot up and Fedora drops me into the "Repair Filesystem" prompt.

Usually, I'll insert the Fedora DVD, boot into "Rescue" mode, chroot /mnt/sysimage/ and edit the fstab file. But when I rebooted and encountered the same problem last night, I got fed up of getting the DVD again.

I found the following forum while googling and the answer provided is exactly what I need:

In the "Repair Filesystem" prompt, login and remount the filesystem with:
> mount -o remount,rw /

I can modify the fstab file now.

Also, I found out that the system can boot up perfectly fine without mounting the /boot partition when I removed the entry from fstab. But I put the /boot entry back in case I forget to remount /boot when I upgrade the kernel in the future.

Monday, December 1, 2008

VBScript to backup Windows System, Application and Security event logs

To diagnose my crashing Windows 2003 server (Mystic Hare) mentioned previously, I needed to backup the server's System, Application and Security event logs every half hour. At first, I thought of just copying the *.evt files to another location, using the Windows Scheduler. But this might not be the best idea as the latest entries might not be in the event logs. So I clobbered together a VBScript from the following links:

The script will backup the event logs using standard WMI methods to "C:\scripts\" and append the current date and time to the saved event log files.

'********************** Begin Script *******************************

dtmThisSecond = PadDigits(Second(Now), 2)

dtmThisMinute = PadDigits(Minute(Now), 2)
dtmThisHour = PadDigits(Hour(Now), 2)
dtmThisDay = PadDigits(Day(Now), 2)
dtmThisMonth = PadDigits(Month(Now), 2)
dtmThisYear = Year(Now)

strBackupName = dtmThisYear & "-" & dtmThisMonth _
& "-" & dtmThisDay & "_" & dtmThisHour & "-" & dtmThisMinute & "-" & dtmThisSecond
strComputer = "."

Set objWMIService = GetObject("winmgmts:" _
& "{impersonationLevel=impersonate,(Backup, Security)}!\\" & _
strComputer & "\root\cimv2")

Set colLogFiles = objWMIService.ExecQuery _
("Select * from Win32_NTEventLogFile " _
& "Where LogFileName='Application'")
For Each objLogfile in colLogFiles
objLogFile.BackupEventLog("c:\scripts\" _
& strBackupName & _
' WScript.Echo "File saved: " & strBackupName & _
' "_application.evt"

Set colLogFiles = objWMIService.ExecQuery _
("Select * from Win32_NTEventLogFile " _
& "Where LogFileName='System'")
For Each objLogfile in colLogFiles
objLogFile.BackupEventLog("c:\scripts\" _
& strBackupName & _

Set colLogFiles = objWMIService.ExecQuery _
("Select * from Win32_NTEventLogFile " _
& "Where LogFileName='Security'")
For Each objLogfile in colLogFiles
objLogFile.BackupEventLog("c:\scripts\" _
& strBackupName & _

' Functions
' This function is used to pad date variables that contain only on digit.
Function PadDigits(n, totalDigits)
If totalDigits > len(n) then
PadDigits = String(totalDigits-len(n),"0") & n
PadDigits = n
End If
End Function

'************************ End Script *******************************

PS: The script was only tested on a Windows XP system.

Dummy write to Windows' System event log

I have a Windows 2003 server, let's call it Mystic Hare, which keeps crashing intermittently. The problem I have in diagnosing the problem, is that the last 3 hours of the Windows System logs are always empty.

So either:
1. The last 3 hours of the System log are wiped out during the crash; or
2. There is no event written to the System log during those times.

So I decided to write to the System event log with dummy entries as an experiment.

Googling brings up a lot of VBScripts, JavaScripts, C#, WSH scripts, etc. But I needed a simple app that I can just set in the Windows scheduler. Thank goodness I found what I need at:

The command I'm using is:
> eventcreate /L System /T Information /id 1 /d "For testing only"

This is what I see for the dummy System log entry in the Event Viewer"