Careless BugCheck

 from jonh lan

carelessly, a bugcheck occured when i was debugging Windows Server 2003 in VMWare.

Opened log file 'd:/bugcheck.log'
kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)
This is a very common bugcheck.  Usually the exception address pinpoints
the driver/function that caused the problem.  Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: c0000005, The exception code that was not handled
Arg2: 808201cc, The address that the exception occurred at
Arg3: fa0aeccc, Exception Record Address
Arg4: fa0ae9c8, Context Record Address

Debugging Details:
------------------

ERROR_CODE: (NTSTATUS) 0xc0000005 - "0x%08lx"

FAULTING_IP:
nt!KeRemoveQueue+1b9
808201cc 851b             test    [ebx],ebx

EXCEPTION_PARAMETER1:  fa0aeccc

CONTEXT:  fa0ae9c8 -- (.cxr fffffffffa0ae9c8)
.cxr fffffffffa0ae9c8
eax=00000000 ebx=00000000 ecx=81216020 edx=8179dd68 esi=00000000 edi=81602480
eip=808201cc esp=fa0aed94 ebp=fa0aedac iopl=0         nv up ei pl zr na po nc
cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010246
nt!KeRemoveQueue+0x1b9:
808201cc 851b             test    [ebx],ebx
.cxr
Resetting default scope

DEFAULT_BUCKET_ID:  INTEL_CPU_MICROCODE_ZERO

BUGCHECK_STR:  0x7E

CURRENT_IRQL:  0

MISALIGNED_IP:
nt!KeRemoveQueue+1b9
808201cc 851b             test    [ebx],ebx

LAST_CONTROL_TRANSFER:  from 80905d2c to 808201cc

STACK_TEXT: 
fa0aedac 80905d2c 000004bc 00000000 00000000 nt!KeRemoveQueue+0x1b9
fa0aeddc 80828499 f99c7005 00000000 00000000 nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
nt!KeRemoveQueue+1b9
808201cc 851b             test    [ebx],ebx

SYMBOL_STACK_INDEX:  0

FOLLOWUP_NAME:  MachineOwner

SYMBOL_NAME:  nt!KeRemoveQueue+1b9

IMAGE_NAME:  hardware

DEBUG_FLR_IMAGE_TIMESTAMP:  0

STACK_COMMAND:  .cxr fffffffffa0ae9c8 ; kb

MODULE_NAME:  hardware

FAILURE_BUCKET_ID:  IP_MISALIGNED

BUCKET_ID:  IP_MISALIGNED

Followup: MachineOwner
---------

kd> kv
ChildEBP RetAddr  Args to Child             
fa0ae3d4 80874ad9 00000003 c0000005 00000000 nt!RtlpBreakWithStatusInstruction (FPO: [1,0,0])
fa0ae420 808758f6 00000003 00000000 00000000 nt!KiBugCheckDebugBreak+0x19 (FPO: [Non-Fpo])
fa0ae7b8 80875d0e 0000007e c0000005 808201cc nt!KeBugCheck2+0x5b2 (FPO: [Non-Fpo])
fa0ae7d8 809966b0 0000007e c0000005 808201cc nt!KeBugCheckEx+0x1b (FPO: [Non-Fpo])
fa0ae7f4 80959b87 fa0ae81c 808270e9 fa0ae824 nt!PspUnhandledExceptionInSystemThread+0x1a (FPO: [Non-Fpo])
fa0aeddc 80828499 f99c7005 00000000 00000000 nt!PspSystemThreadStartup+0x56 (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

kd> dd fa0ae81c
fa0ae81c  fa0aeccc fa0ae9c8 fa0ae848 8081f64e
fa0ae82c  fa0aeccc fa0aedcc fa0ae9c8 fa0ae8e4
fa0ae83c  fa0aeca0 8081f662 fa0aedcc fa0ae8f8
fa0ae84c  8081f61f fa0aeccc fa0aedcc fa0ae9c8
fa0ae85c  fa0ae8e4 80827088 fa0aedd4 fa0aeccc
fa0ae86c  fa0aedcc 80838fdd fa0aeccc fa0aedcc
fa0ae87c  fa0ae9c8 fa0ae8e4 80827088 00000000
fa0ae88c  fa0aeccc fa0aed20 f8b0adf0 00000000

kd> .exr fa0aeccc
ExceptionAddress: 808201cc (nt!KeRemoveQueue+0x000001b9)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: 00000000
Attempt to read from address 00000000

kd> .cxr fa0ae9c8
eax=00000000 ebx=00000000 ecx=81216020 edx=8179dd68 esi=00000000 edi=81602480
eip=808201cc esp=fa0aed94 ebp=fa0aedac iopl=0         nv up ei pl zr na po nc
cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010246
nt!KeRemoveQueue+0x1b9:
808201cc 851b             test    [ebx],ebx

kd> kv
  *** Stack trace for last set context - .thread/.cxr resets it
ChildEBP RetAddr  Args to Child             
fa0aedac 80905d2c 000004bc 00000000 00000000 nt!KeRemoveQueue+0x1b9 (FPO: [Non-Fpo])
fa0aeddc 80828499 f99c7005 00000000 00000000 nt!PspSystemThreadStartup+0x2e (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

kd> u 80905d2c-e
nt!PspSystemThreadStartup+0x20:
80905d1e 864002           xchg    [eax+0x2],al
80905d21 0000             add     [eax],al
80905d23 037506           add     esi,[ebp+0x6]
80905d26 ff750c           push    dword ptr [ebp+0xc]
80905d29 ff5508           call    dword ptr [ebp+0x8]
80905d2c 834dfcff         or      dword ptr [ebp-0x4],0xffffffff
80905d30 6a01             push    0x1
80905d32 6a00             push    0x0

kd> dds esp
fa0aed94  f99c703d NDIS!ndisWorkerThread+0x2f
fa0aed98  f99c62dc NDIS!ndisWorkerQueue
fa0aed9c  00000000
fa0aeda0  00000000
fa0aeda4  00000000
fa0aeda8  816f98f8
fa0aedac  fa0aeddc
fa0aedb0  80905d2c nt!PspSystemThreadStartup+0x2e
fa0aedb4  000004bc
fa0aedb8  00000000
fa0aedbc  00000000
fa0aedc0  00000000
fa0aedc4  fa0aedb8
fa0aedc8  fa0ae81c
fa0aedcc  ffffffff
fa0aedd0  80827088 nt!_except_handler3
fa0aedd4  80828f58 nt!ObWatchHandles+0x5f4
fa0aedd8  00000000
fa0aeddc  00000000
fa0aede0  80828499 nt!KiThreadStartup+0x16
fa0aede4  f99c7005 NDIS!ndisWorkerThread
fa0aede8  00000000
fa0aedec  00000000
fa0aedf0  0000027f
fa0aedf4  00000000
fa0aedf8  00000000
fa0aedfc  00000000
fa0aee00  00000000
fa0aee04  00000000
fa0aee08  00001f80
fa0aee0c  00000000
fa0aee10  00000000
kd> uf f99c7005
NDIS!ndisWorkerThread:
f99c7005 8bff             mov     edi,edi
f99c7007 55               push    ebp
f99c7008 8bec             mov     ebp,esp
f99c700a 8b4508           mov     eax,[ebp+0x8]
f99c700d 56               push    esi
f99c700e 33f6             xor     esi,esi
f99c7010 3bc6             cmp     eax,esi
f99c7012 57               push    edi
f99c7013 0f8472600000     je      NDIS!ndisWorkerThread+0x10 (f99cd08b)

NDIS!ndisWorkerThread+0x10:
f99cd08b e8751fffff       call    NDIS!KeGetCurrentThread (f99bf005)
f99cd090 8bc8             mov     ecx,eax
f99cd092 a3b8629cf9       mov     [NDIS!ndisThreadObject (f99c62b8)],eax
f99cd097 ff15c0529cf9     call    dword ptr [NDIS!_imp_ObfReferenceObject (f99c52c0)]
f99cd09d e98e9fffff       jmp     NDIS!ndisWorkerThread+0x22 (f99c7030)

NDIS!ndisWorkerThread+0x22:
f99c7030 56               push    esi
f99c7031 56               push    esi
f99c7032 68dc629cf9       push    0xf99c62dc
f99c7037 ff15c4529cf9     call    dword ptr [NDIS!_imp__KeRemoveQueue (f99c52c4)]
f99c703d 8bf8             mov     edi,eax
f99c703f 57               push    edi
f99c7040 6805709cf9       push    0xf99c7005
f99c7045 56               push    esi
f99c7046 6aff             push    0xff
f99c7048 56               push    esi
f99c7049 68ff031f00       push    0x1f03ff
f99c704e 8d4508           lea     eax,[ebp+0x8]
f99c7051 50               push    eax
f99c7052 ff15c8529cf9     call    dword ptr [NDIS!_imp__PsCreateSystemThread (f99c52c8)]
f99c7058 85c0             test    eax,eax
f99c705a 0f8cef7c0000     jl      NDIS!ndisWorkerThread+0x59 (f99ced4f)

NDIS!ndisWorkerThread+0x4e:
f99c7060 ff7508           push    dword ptr [ebp+0x8]
f99c7063 ff15cc529cf9     call    dword ptr [NDIS!_imp__NtClose (f99c52cc)]
f99c7069 ebc5             jmp     NDIS!ndisWorkerThread+0x22 (f99c7030)

NDIS!ndisWorkerThread+0x59:
f99ced4f 56               push    esi
f99ced50 57               push    edi
f99ced51 ff15d0529cf9     call    dword ptr [NDIS!_imp__ExQueueWorkItem (f99c52d0)]
f99ced57 e9d482ffff       jmp     NDIS!ndisWorkerThread+0x22 (f99c7030)

NDIS!ndisWorkerThread+0x63:
f99c7019 8bf0             mov     esi,eax
f99c701b bf54639cf9       mov     edi,0xf99c6354
f99c7020 a5               movsd
f99c7021 a5               movsd
f99c7022 a5               movsd
f99c7023 a5               movsd
f99c7024 ff700c           push    dword ptr [eax+0xc]
f99c7027 ff5008           call    dword ptr [eax+0x8]
f99c702a 5f               pop     edi
f99c702b 5e               pop     esi
f99c702c 5d               pop     ebp
f99c702d c20400           ret     0x4

kd> db f99c52c4
f99c52c4  49 01 82 80 47 5d 90 80-58 55 90 80 5a 8b 82 80  I...G]..XU..Z...
f99c52d4  7a 54 89 80 c9 2a 92 80-a9 52 89 80 00 00 00 00  zT...*...R......
f99c52e4  5c 00 47 00 4c 00 4f 00-42 00 41 00 4c 00 3f 00  /.G.L.O.B.A.L.?.
f99c52f4  3f 00 5c 00 00 00 00 00-5c 00 44 00 45 00 56 00  ?./...../.D.E.V.
f99c5304  49 00 43 00 45 00 5c 00-00 00 00 00 82 6a 54 a9  I.C.E./......jT.
f99c5314  b0 fe d0 11 bd 26 00 aa-00 b7 b3 2a 6f 0a 7c 82  .....&.....*o.|.
f99c5324  b0 fe d0 11 bd 26 00 aa-00 b7 b3 2a bc eb 7d 9d  .....&.....*..}.
f99c5334  5d c8 d1 11 9e b4 00 60-08 c3 a1 9a 09 55 c3 dd  ]......`.....U..

kd> r
Last set context:
eax=00000000 ebx=00000000 ecx=81216020 edx=8179dd68 esi=00000000 edi=81602480
eip=808201cc esp=fa0aed94 ebp=fa0aedac iopl=0         nv up ei pl zr na po nc
cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010246
nt!KeRemoveQueue+0x1b9:
808201cc 851b             test    [ebx],ebx         ds:0023:00000000=????????

kd> ln 80820149
(80820149)   nt!KeRemoveQueue   |  (80820300)   nt!ExpWorkerThread
Exact matches:
    nt!KeRemoveQueue = <no type information>
kd> .logclose
Closing open log file d:/bugcheck.log

dump analysis shows that i made a stupid mistake, i typed 'bp' in kd to
set a breakpoint on address NDIS!_imp__KeRemoveQueue, so the debugger actually
wrote a __asm int 3 (opcode 0xcc) to the NDIS!_imp__KeRemoveQueue address, in this
case its memory content was originally 80820149, but the first byte was wrote as
0xcc!, so the808201cc comes. Ndis!NdisWorkerThread() will dequeue Ndis!NdisWorkerQueue
via indirectly invoking Ndis!_imp_KeRemoveQueue() (forwarded to nt!KeRemoveQueue()),
by comparing the stack
fa0aed94  f99c703d NDIS!ndisWorkerThread+0x2f
fa0aed98  f99c62dc NDIS!ndisWorkerQueue
with the disassembly
NDIS!ndisWorkerThread+0x22:
f99c7030 56               push    esi
f99c7031 56               push    esi
f99c7032 68dc629cf9       push    0xf99c62dc
f99c7037 ff15c4529cf9     call    dword ptr [NDIS!_imp__KeRemoveQueue (f99c52c4)]

to get the idea of the crash occur when __asm call instruction was executing...

so instead of use 'bp', 'ba' (memory access breakpoint) should be used to monitor
something like xxx!__imp_yyy.

very interesting, 'dd f99c52c4' showed the CORRECT but NON-REAL memory content,
the debugger hided the modification from u, so dont believe everything the debugger
show u :)