WinDbg: Access violation exception (0xC0000005) when running the !clrstack command


This exception occurs when you run the !clrstack command after the .loadby sos clr command.

0:003> .loadby sos clr
0:003> !clrstack
c0000005 Exception in C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.clrstack debugger extension.
      PC: 00007ff9`a0fec7e3  VA: 00000000`00000000  R/W: 0  Parameter: 00000000`00000000
(9ec.4b0): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
sos!GetCMDOption+0x63:
00007ff9`a35ac7e3 488b01          mov     rax,qword ptr [rcx] ds:00000000`00000000=????????????????
0:004> ub
sos!GetCMDOption+0x3f:
00007ff9`a35ac7bf 4889542420      mov     qword ptr [rsp+20h],rdx
00007ff9`a35ac7c4 488bf1          mov     rsi,rcx
00007ff9`a35ac7c7 4d85c0          test    r8,r8
00007ff9`a35ac7ca 743f            je      sos!GetCMDOption+0x8b (00007ff9`a35ac80b)
00007ff9`a35ac7cc 488d5a18        lea     rbx,[rdx+18h]
00007ff9`a35ac7d0 392dd2e20500    cmp     dword ptr [sos!ControlC (00007ff9`a360aaa8)],ebp
00007ff9`a35ac7d6 0f85fe000000    jne     sos!GetCMDOption+0x15a (00007ff9`a35ac8da)
00007ff9`a35ac7dc 488b0dfd6d0500  mov     rcx,qword ptr [sos!g_ExtControl (00007ff9`a36035e0)]
0:004> k
 # Child-SP          RetAddr           Call Site
00 0000000a`05aad2c0 00007ff9`a35973ac sos!GetCMDOption+0x63
01 0000000a`05aad3b0 00007ff9`a86f353b sos!ClrStack+0x21c
02 0000000a`05aad570 00007ff9`a86f3718 dbgeng!ExtensionInfo::CallA+0x233
03 0000000a`05aad630 00007ff9`a86f37f8 dbgeng!ExtensionInfo::Call+0x16c
04 0000000a`05aad830 00007ff9`a86f2689 dbgeng!ExtensionInfo::CallAny+0x78
05 0000000a`05aad870 00007ff9`a872a89b dbgeng!ParseBangCmd+0x4a9
06 0000000a`05aadd30 00007ff9`a872b6ab dbgeng!ProcessCommands+0xa8f
07 0000000a`05aade00 00007ff9`a8685fe8 dbgeng!ProcessCommandsAndCatch+0x8f
08 0000000a`05aade70 00007ff9`a868628f dbgeng!Execute+0x24c
09 0000000a`05aae340 00007ff7`0d0c5c72 dbgeng!DebugClient::ExecuteWide+0x83
0a 0000000a`05aae3a0 00007ff7`0d0c60d5 windbg!ProcessCommand+0x2b2
0b 0000000a`05aae7c0 00007ff7`0d0c7c17 windbg!ProcessEngineCommands+0x185
0c 0000000a`05aaf800 00007ff9`d6a316ad windbg!EngineLoop+0x3e3
0d 0000000a`05aaf840 00007ff9`d7244629 KERNEL32!BaseThreadInitThunk+0xd
0e 0000000a`05aaf870 00000000`00000000 ntdll!RtlUserThreadStart+0x1d

Let’s set a breakpoint for write access on the sos!g_ExtControl variable, to find where it was zeroed.

0:001> ba w8 sos!g_ExtControl
0:001> g
0:004> k
 # Child-SP          RetAddr           Call Site
00 0000008d`40d2d3f0 00007ff9`a7b1dc6e dbgeng!DebugClient::QueryInterface+0xb
01 0000008d`40d2d420 00007ff9`a7b471d0 sos!ExtQuery+0x2e
02 0000008d`40d2d450 00007ff9`a86f353b sos!ClrStack+0x40
...
0:004> k
 # Child-SP          RetAddr           Call Site
00 0000008d`40d2c080 00007ff9`a7b1dc6e dbgeng!DebugClient::QueryInterface+0xb
01 0000008d`40d2c0b0 00007ff9`a7b1d973 sos!ExtQuery+0x2e
02 0000008d`40d2c0e0 00007ff9`a86f2d25 sos!DebugExtensionInitialize+0x83
03 0000008d`40d2c120 00007ff9`a86f4182 dbgeng!ExtensionInfo::Load+0x48d
04 0000008d`40d2c3f0 00007ff9`a87312f6 dbgeng!ExtensionInfo::CheckAdd+0x6e
05 0000008d`40d2c430 00007ff9`a8731ca8 dbgeng!LoadSOSAndCheckVer+0x36
06 0000008d`40d2c690 00007ff9`a86f00e1 dbgeng!ProcessInfo::LoadClrDebugDllForExt+0x828
07 0000008d`40d2cec0 00007ff9`a7b61004 dbgeng!ExtIoctl+0xc6d
08 0000008d`40d2d410 00007ff9`a7b4722c sos!LoadClrDebugDll+0x24
09 0000008d`40d2d450 00007ff9`a86f353b sos!ClrStack+0x9c
...
0:004> k
 # Child-SP          RetAddr           Call Site
00 0000008d`40d2c0b0 00007ff9`a7b1d98d sos!ExtRelease+0x28
01 0000008d`40d2c0e0 00007ff9`a86f2d25 sos!DebugExtensionInitialize+0x9d
02 0000008d`40d2c120 00007ff9`a86f4182 dbgeng!ExtensionInfo::Load+0x48d
03 0000008d`40d2c3f0 00007ff9`a87312f6 dbgeng!ExtensionInfo::CheckAdd+0x6e
04 0000008d`40d2c430 00007ff9`a8731ca8 dbgeng!LoadSOSAndCheckVer+0x36
05 0000008d`40d2c690 00007ff9`a86f00e1 dbgeng!ProcessInfo::LoadClrDebugDllForExt+0x828
06 0000008d`40d2cec0 00007ff9`a7b61004 dbgeng!ExtIoctl+0xc6d
07 0000008d`40d2d410 00007ff9`a7b4722c sos!LoadClrDebugDll+0x24
08 0000008d`40d2d450 00007ff9`a86f353b sos!ClrStack+0x9c
...
0:004> g
(fe4.f70): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
sos!GetCMDOption+0x63:
00007ff9`a7b5c7e3 488b01          mov     rax,qword ptr [rcx] ds:00000000`00000000=????????????????
0:004> k
 # Child-SP          RetAddr           Call Site
00 0000008d`40d2d360 00007ff9`a7b473ac sos!GetCMDOption+0x63
01 0000008d`40d2d450 00007ff9`a86f353b sos!ClrStack+0x21c
...
sos!ExtQuery:
00007ff9`6a16dc40 48895c2408      mov     qword ptr [rsp+8],rbx
00007ff9`6a16dc45 4889742410      mov     qword ptr [rsp+10h],rsi
00007ff9`6a16dc4a 57              push    rdi
00007ff9`6a16dc4b 4883ec20        sub     rsp,20h
00007ff9`6a16dc4f 33f6            xor     esi,esi
00007ff9`6a16dc51 4c8d0588590900  lea     r8,[sos!g_ExtControl (00007ff9`6a2035e0)]
00007ff9`6a16dc58 488d1571760600  lea     rdx,[sos!GUID_d4366723_44df_4bed_8c7e_4c05424f4588 (00007ff9`6a1d52d0)]
00007ff9`6a16dc5f 4889357a590900  mov     qword ptr [sos!g_ExtControl (00007ff9`6a2035e0)],rsi
00007ff9`6a16dc66 488b01          mov     rax,qword ptr [rcx]
00007ff9`6a16dc69 488bf9          mov     rdi,rcx
00007ff9`6a16dc6c ff10            call    qword ptr [rax] ds:00007ff9`60e018d0={dbgeng!DebugClient::QueryInterface (00007ff9`60fd0240)}
sos!ExtRelease:
00007ff9`6a16ddc0 4053            push    rbx
00007ff9`6a16ddc2 4883ec20        sub     rsp,20h
00007ff9`6a16ddc6 488b0d13580900  mov     rcx,qword ptr [sos!g_ExtControl (00007ff9`6a2035e0)]
00007ff9`6a16ddcd 33db            xor     ebx,ebx
00007ff9`6a16ddcf 48891d02580900  mov     qword ptr [sos!g_ExtClient (00007ff9`6a2035d8)],rbx
00007ff9`6a16ddd6 4885c9          test    rcx,rcx
00007ff9`6a16ddd9 740d            je      sos!ExtRelease+0x28 (00007ff9`6a16dde8)
00007ff9`6a16dddb 488b01          mov     rax,qword ptr [rcx]
00007ff9`6a16ddde ff5010          call    qword ptr [rax+10h] ds:00007ff9`60e01358={dbgeng!DebugClient::Release (00007ff9`60fd0200)}

We can see that the IDebugControl2 interface was obtained twice and then released while it is still in use by the sos!ClrStack function. Also, we can see that the sos.dll is loaded the second time. To find out why, let's debug the dbgeng!LoadSOSAndCheckVer function. In the dbgeng!ExtensionInfo::Add method we can see that it’s calling the dbgeng!ExtensionInfo::FindByName method to find the extension by name in the dbgeng!ExtensionInfo::s_Chain linked list

0:003> k
 # Child-SP          RetAddr           Call Site
00 0000003a`a963c970 00007ff9`61044168 dbgeng!ExtensionInfo::Add
01 0000003a`a963c9d0 00007ff9`610812f6 dbgeng!ExtensionInfo::CheckAdd+0x54
02 0000003a`a963ca10 00007ff9`61081ca8 dbgeng!LoadSOSAndCheckVer+0x36
03 0000003a`a963cc70 00007ff9`610400e1 dbgeng!ProcessInfo::LoadClrDebugDllForExt+0x828
04 0000003a`a963d4a0 00007ff9`6a601004 dbgeng!ExtIoctl+0xc6d
05 0000003a`a963d9f0 00007ff9`6a5e722c sos!LoadClrDebugDll+0x24
06 0000003a`a963da30 00007ff9`6104353b sos!ClrStack+0x9c
...
0:003> r
rax=0000000000000036 rbx=0000003aa9505490 rcx=0000003aa963ca40
rdx=0000000000000000 rsi=0000000000000037 rdi=0000003aa963ca40
rip=00007ff961043dc5 rsp=0000003aa963c970 rbp=0000000000000000
 r8=0000000000000000  r9=0000003aa963c9f0 r10=0000000000000000
r11=0000003aa963c9e0 r12=0000000000000000 r13=0000000000000000
r14=0000003aa963c9f0 r15=0000003aa963cd01
iopl=0         nv up ei pl nz na pe nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
dbgeng!ExtensionInfo::Add+0x95:
00007ff9`61043dc5 e876feffff      call    dbgeng!ExtensionInfo::FindByName (00007ff9`61043c40)
0:003> du @rcx
0000003a`a963ca40  "C:\Windows\Microsoft.NET\Framewo"
0000003a`a963ca80  "rk64\v4.0.30319\SOS.dll"
0:003> u dbgeng!ExtensionInfo::FindByName
dbgeng!ExtensionInfo::FindByName:
00007ff9`61043c40 488bc4          mov     rax,rsp
00007ff9`61043c43 48895808        mov     qword ptr [rax+8],rbx
00007ff9`61043c47 48896810        mov     qword ptr [rax+10h],rbp
00007ff9`61043c4b 48897018        mov     qword ptr [rax+18h],rsi
00007ff9`61043c4f 48897820        mov     qword ptr [rax+20h],rdi
00007ff9`61043c53 4156            push    r14
00007ff9`61043c55 4883ec20        sub     rsp,20h
00007ff9`61043c59 4883cfff        or      rdi,0FFFFFFFFFFFFFFFFh
00007ff9`61043c5d 488bf2          mov     rsi,rdx
00007ff9`61043c60 488be9          mov     rbp,rcx
00007ff9`61043c63 4533f6          xor     r14d,r14d
00007ff9`61043c66 48ffc7          inc     rdi
00007ff9`61043c69 6644393479      cmp     word ptr [rcx+rdi*2],r14w
00007ff9`61043c6e 75f6            jne     dbgeng!ExtensionInfo::FindByName+0x26 (00007ff9`61043c66)
00007ff9`61043c70 488b1d91041c00  mov     rbx,qword ptr [dbgeng!ExtensionInfo::s_Chain (00007ff9`61204108)]
00007ff9`61043c77 4885db          test    rbx,rbx
0:003> !list -x "du poi(@$extret+8)" poi(dbgeng!ExtensionInfo::s_Chain)
0000003a`a94fb320  "C:\Windows\Microsoft.NET\Framewo"
0000003a`a94fb360  "rk64\v4.0.30319\sos"
0000003a`a94f8830  "dbghelp"
0000003a`a94f8560  "ext"
0000003a`a94f8290  "exts"
0000003a`a94f7fc0  "uext"
0000003a`a94f7cf0  "ntsdexts"

and when it doesn't, it calls the dbgeng!ExtensionInfo::Link method to add the new extension dll to the linked list.

0:003> r
rax=0000000000000000 rbx=0000003aa5f1e6b0 rcx=0000003aa5f1e6b0
rdx=0000003aa5f1e6d0 rsi=0000000000000037 rdi=0000003aa963ca40
rip=00007ff961043ed0 rsp=0000003aa963c970 rbp=0000000000000000
 r8=0000000000000000  r9=0000000000000000 r10=0000003aa95abd30
r11=0000003aa5f1e6d0 r12=0000000000000000 r13=0000000000000000
r14=0000003aa963c9f0 r15=0000003aa963cd01
iopl=0         nv up ei pl nz na pe nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
dbgeng!ExtensionInfo::Add+0x1a0:
00007ff9`61043ed0 e8e7110000      call    dbgeng!ExtensionInfo::Link (00007ff9`610450bc)
0:003> du @rdx
0000003a`a5f1e6d0  "C:\Windows\Microsoft.NET\Framewo"
0000003a`a5f1e710  "rk64\v4.0.30319\SOS.dll"
0:003> !list -x "du poi(@$extret+8)" poi(dbgeng!ExtensionInfo::s_Chain)
0000003a`a5f1e6d0  "C:\Windows\Microsoft.NET\Framewo"
0000003a`a5f1e710  "rk64\v4.0.30319\SOS.dll"
0000003a`a94fb320  "C:\Windows\Microsoft.NET\Framewo"
0000003a`a94fb360  "rk64\v4.0.30319\sos"
0000003a`a94f8830  "dbghelp"
0000003a`a94f8560  "ext"
0000003a`a94f8290  "exts"
0000003a`a94f7fc0  "uext"
0000003a`a94f7cf0  "ntsdexts"

Finally the dbgeng!ExtensionInfo::CheckAdd method calls the dbgeng!ExtensionInfo::Load to load the sos.dll:

0:003> u
dbgeng!ExtensionInfo::CheckAdd+0x69:
00007ff9`6104417d e816e7ffff      call    dbgeng!ExtensionInfo::Load (00007ff9`61042898)

To avoid the second load of the sos.dll and the exception you can use one of these commands before running the !clrstack command:

.loadby sos.dll clr
.cordll -ve -u -l

P.S. Version of the dbgeng.dll

0:004> lmvm dbgeng
    Image path: C:\Program Files (x86)\Windows Kits\8.1\Debuggers\x64\dbgeng.dll
    Image name: dbgeng.dll
    Browse all global symbols  functions  data
    Timestamp:        Thu Aug 22 14:10:43 2013 (5215F1B3)
    CheckSum:         004818CB
    ImageSize:        004AC000
    File version:     6.3.9600.16384
    Product version:  6.3.9600.16384
    File flags:       0 (Mask 3F)
    File OS:          40004 NT Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04b0
    CompanyName:      Microsoft Corporation
    ProductName:      Microsoft® Windows® Operating System
    InternalName:     DbgEng.Dll
    OriginalFilename: DbgEng.Dll
    ProductVersion:   6.3.9600.16384
    FileVersion:      6.3.9600.16384 (debuggers(dbg).130821-1623)
    FileDescription:  Windows Symbolic Debugger Engine
    LegalCopyright:   © Microsoft Corporation. All rights reserved.