Process Initiation

Overview

Taking a simple C program, we look at its initiation using NTSD. There are four segments to the program's life cycle:

1. Loading until the NTSD's entry breakpoint.
2. From the entry breakpoint to the start of the actual module code, during which it runs as an APC.
3. The running of the program, which consists of a compiler preamble, the code body, followed (perhaps) by a compiler postamble, until the hitting of NTSD's exit breakpoint.
4. Process teardown from the exit breakpoint until the removal of the thread as a schedulable entity.

 

Exhibits

 

• SimpleProgram.c
• Predawn
• Dawn
• Dawn to Dusk
• Twilight

 

The Details

 

Predawn

Process creation occurs on two levels: NT and Windows. CreateProcess is the Windows call which creates both a process and the initial thread in the process. This uses NT calls to create the process on the OS level, and talks to csrss to "register" the process with the Windows-subsystem server.

The CreateProcess routine is a #define for either CreateProcessA or CreateProcessW, depending on the operating system's overal option: ASCII or Unicode. This routine is visable to Win32 users as an export of Kernel32.dll, a dll which is built in nt/private/windows/base. The code for CreateProcess is in nt/private/windows/base/client/create.c.

Createprocess does several things, the goal of which is to leave us with an APC queue for the initial thread of the new process. To achieve this:

1. CreateProcess opens and verifies the exe file. Before loading the image, it is mapped by a call to LdrQueryImageFileExecutionOptions. This looks for registry key /Registry/Machine/Software/Microsoft/Windows NT/CurrentVersion/Image File Execution Options/ Using NtOpenKey, handing an Object Attributes block created by the macro InitializeObjectAttributes.
2. A section object is wrapped around this exe (NtCreateSection). The section object is queried for entry point and stack information. Then it creates a process object wrapped around this section object, NtCreateProcess, which creates stack and code segments, apparantly. BasePushProcessParameters is called on the process object to push arguments onto the stack of the newly created process (this is a trans-context operation). BaseCreateStack, nt/private/windows/base/client/support.c, is called in preparation for thread creation, to allocate stack space for the initial thread (each thread has it's own stack). BaseCreateStack manipulates the TEB to reflect the stack creation. It calls NtAllocateVirtualMemory to allocate and commit the stack and optionally a guard page.
3. BaseInitializeContext is called, nt/private/windows/base/client/i386/context.c, giving the starting address learned by the previous query to the section object (a long time ago!), to fill in a CONTEXT block. NtCreateThread is passed this block when the thread is being setup. The actual push onto the thread's stack in the thread's context might be in: KiInitializeContextThread in nt/private/ntos/ke/i386/thredini.c. Although we are in kernel mode, perhaps the memory for the new thread is mapped into the address space for the old process, since a full context switch does not seen to intervene.

Back in CreateProcess, windows, the process ends w/ a registration of the new process with csrss. (Some LPC magic here!)

Dawn

 

The queued APC is picked up by the kernel mode routine KiDeliverApc (nt/private/ntos/ke/apcsup.c). It individually calls out kernel and user mode APC's. The user mode part is handed off to KiUserApcDispatcher for handling (nt/private/ntos/rlt/i386/userdisp.asm). It's not clear what intervenes between KiDeliverApc and this routine, but this routine's documentation says that it is already in user mode (run "on return from kernel mode"). Indeed, this function is located in user memory, as part of the ntdll.dll. Furthermore, this function is passed CONTEXT, the continuation environment, as its argument.

Although APC's can do whatever their function pointer points to, this APC has been setup with certain specific function pointers to carry out its task of dll initialization. We are now in the context of the new process. At the bottom of the stack is the return to KiUserApcDispatcher and a pointer to KiUserApcDispatcher's important argument: the continuation environment built to start the users code. This CONTEXT structure (see nt/public/sdk/inc/nti386.h) is also in the user stack, put there by NtCreateThread.

1. KiUserApcDispatcher calls LdrpInitialize, IPL 0, user mode,
2. If Peb->BeingDebugged is true, a call to DbgBreakPoint in LdrpInitializeProcess stops the action before dll initialization, for debugging,
3. on return to KiUserApcDispatcher, it int 2e's to NtContinue passing on its own passed in argument (type CONTEXT see which should be a continuation onto the start of user-executable code.

Morning sickness

 

The CONTEXT record was created by CreateProcess, in the case of an Windows create process, and contains the start address according to the loaded exe, and also a thunking location according to the kernel32.dll visable when CreateProcess was run. In fact, the NtContinue thunks on through BaseProcessStartThunk, see nt/private/windows/base/client/context.c. The true starting address is in eax of the CONTEXT, BaseProcessStartThunk is found in eip.

This causes problems when the location of kernel32.dll does not agree in the two contexts:

1. Where CONTEXT was created: the caller of CreateProcess,
2. Where CONTEXT was used: the context of the newly created process.

There is no intrinsic reason why kernel32.dll should not move. This must be enforced outside of the Windows NT system.

Dawn to Dusk

 

The change from APC to normal thread is reflected in the return location on the stack. Now the Top of Stack is BaseProcessStart, which has an ExitThread if the user were to return at this point. However, this apparantly does not survive, as the stack trace at the exit break point indicates. The Top of this Stack as a return address inside the user's program, and it calls into system exit as a Kernel32 function. I do not think BaseProcessStart expects to regain control.

It is difficult to say whether the APC and the user code operate in the same thread, or are they two different threads. Although there is kernel mode operations which punctuate the two code runs, they are a single, contiguous run entity, sharing stack, and so forth.

 

Author

Burton Rosenberg
16 August 1998

 

---

 

Exhibits

• SimpleProgram.c

  int main(int argc, char * argv[]) {
      int i ;
      i = 1 ;
      _asm{ int 3 } ;
      i = 2 ;
  }
  

   

• Predawn

  Microsoft(R) Windows NT Debugger
  Version 4.00
  Copyright (C) Microsoft Corp. 1981-1998
  
  CommandLine: simpleprogram
  Symbol search path is: C:/WTSRV
  NTSD ModLoad: 00400000 00408000   Image@00400000
  NTSD ModLoad: 77f60000 77fbc000   ntdll.dll
  NTSD ModLoad: 77ef0000 77f57000   C:/WTSRV/system32/KERNEL32.dll
  eax=00000000 ebx=00000000 ecx=00000009 edx=00000000 esi=7ffdf000 edi=001406a0
  eip=77f764a8 esp=0012fb84 ebp=0012feac iopl=0         nv up ei pl nz na pe nc
  cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202
  *** WARNING: symbols checksum is wrong 0x000609b0 0x00062b31 for c:/wtsrv/symbols/dll/ntdll.dbg
  ntdll!DbgBreakPoint:
  77f764a8 cc               int     3
  0:000> kb
  ChildEBP RetAddr  Args to Child
  0012fb80 77f61c1d 00000000 7ffdf000 7ffdee0c ntdll!DbgBreakPoint
  0012feac 77f61187 0012ff30 77f60000 00000000 ntdll!LdrpInitializeProcess+0x825
  0012ff1c 77f767ff 0012ff30 77f60000 00000000 ntdll!LdrpInitialize+0x171
  00000000 00000003 0012ff30 77f60000 00000000 ntdll!KiUserApcDispatcher+0x7
  0:000>
  

   

• Dawn

  [break at return to KiUserApcDispatcher]
  
  0:000> bp 0x077f767ff
  0:000> g
  eax=ffffffff ebx=7ffdf000 ecx=00000101 edx=ffffffff esi=00000149 edi=0012ff30
  eip=77f767ff esp=0012ff30 ebp=00000000 iopl=0         nv up ei pl zr na po nc
  cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
  ntdll!KiUserApcDispatcher+0x7:
  77f767ff 6a01             push    0x1
  0:000> u
  ntdll!KiUserApcDispatcher+0x7:
  77f767ff 6a01             push    0x1
  77f76801 57               push    edi
  77f76802 e8950effff       call    ntdll!ZwContinue (77f6769c)
  77f76807 90               nop
  ntdll!KiUserCallbackDispatcher:
  77f76808 83c404           add     esp,0x4
  77f7680b 5a               pop     edx
  77f7680c 64a118000000     mov     eax,fs:[00000018]
  77f76812 8b4030           mov     eax,[eax+0x30]
  0:000>
  
  [Note: this is a CONTEXT block, argument to NtContinue]
  
  0:000> dd edi
  0012ff30  00010017 00000000 00000000 00000000
  0012ff40  00000000 0000c00b 00000000 811f22f0
  0012ff50  80139782 fd9cedb8 fd9cedb8 fd9cee34
  0012ff60  80139608 811f22f0 00120089 801430f4
  0012ff70  0000000c 00000000 e11e4b08 8017b59f
  0012ff80  00000000 e11e4b08 8019c76f 80ffc020
  0012ff90  8017b5e8 80ffc020 00000101 ffffffff
  0012ffa0  00000030 810f3020 80ffc09c 00000246
  0:000> dd
  0012ffb0  00000000 80ffc020 8019c837 00000000
  0012ffc0  00000038 00000023 00000023 0024068c
  0012ffd0  00000149 7ffdf000 77f694b3 00240000
  0012ffe0  00401010 002405c0 77ef53bc 0000001b
  0012fff0  00000200 0012fffc 00000023 00000000
  00130000  00000000 77fa5be0 77fa5718 77fa5bc8
  00130010  00000000 00000000 00000000 00000000
  00130020  00130040 00000000 00000000 00000000
  0:000>
  
  [ Heading to 0x0401010 via 0x077ef53bc ]
  
  0:000> ln 0x077ef53bc
  *** WARNING: symbols checksum is wrong 0x00063fce 0x000700ee for c:/wtsrv/symbols/dll/kernel32.dbg
  (77ef53bc)   KERNEL32!BaseProcessStartThunk   |  (77ef53c8)   KERNEL32!BaseSwitchStackThenTerminate
  0:000> u 0x077ef53bc
  KERNEL32!BaseProcessStartThunk:
  77ef53bc 33ed             xor     ebp,ebp
  77ef53be 50               push    eax
  77ef53bf 6a00             push    0x0
  77ef53c1 e991bf0100       jmp     KERNEL32!BaseProcessStart (77f11357)
  77ef53c6 8bc0             mov     eax,eax
  KERNEL32!BaseSwitchStackThenTerminate:
  77ef53c8 8b5c240c         mov     ebx,[esp+0xc]
  77ef53cc 8b442404         mov     eax,[esp+0x4]
  77ef53d0 8b642408         mov     esp,[esp+0x8]
  0:000>
  

   

• DawnToDusk

  0:000> bp 0x0401010
  0:000> g
  eax=00000000 ebx=7ffdf000 ecx=00000001 edx=ffffffff esi=00000149 edi=0024068c
  eip=00401010 esp=0012ffc4 ebp=0012fff0 iopl=0         nv up ei pl nz ac po nc
  cs=001b  ss=0023  ds=0023  es=0023  fs=0038  gs=0000             efl=00000216
  *** ERROR: Module load completed but symbols could not be loaded for image@00400000
  00401010 55               push    ebp
  0:000> kb
  *** WARNING: symbols checksum is wrong 0x00063fce 0x000700ee for c:/wtsrv/symbols/dll/kernel32.dbg
  ChildEBP RetAddr  Args to Child
  0012ffc0 77f11397 0024068c 00000149 7ffdf000 0x401010
  0012fff0 00000000 00401010 00000000 00000000 KERNEL32!BaseProcessStart+0x40
  0:000>
  
  [ user code, ExitThread waiting for us back at BaseProcessStart ]
  
  0:000> u 0x77f11397
  KERNEL32!BaseProcessStart+0x40:
  77f11397 50               push    eax
  77f11398 eb29             jmp     KERNEL32!BaseProcessStart+0x6c (77f113c3)
  77f1139a 8b45ec           mov     eax,[ebp-0x14]
  77f1139d 8b00             mov     eax,[eax]
  77f1139f 8b00             mov     eax,[eax]
  77f113a1 8945e4           mov     [ebp-0x1c],eax
  77f113a4 ff75ec           push    dword ptr [ebp-0x14]
  77f113a7 e8f71e0000       call   KERNEL32!UnhandledExceptionFilter (77f132a3)
  0:000> u 0x77f113c3
  KERNEL32!BaseProcessStart+0x6c:
  77f113c3 e8fa1b0000       call    KERNEL32!ExitThread (77f12fc2)
  77f113c8 c745fcffffffff   mov     dword ptr [ebp-0x4],0xffffffff
  77f113cf 8b45f0           mov     eax,[ebp-0x10]
  77f113d2 5f               pop     edi
  77f113d3 64a300000000     mov     fs:[00000000],eax
  77f113d9 5e               pop     esi
  77f113da 5b               pop     ebx
  77f113db 8be5             mov     esp,ebp
  0:000>
  
  [let it go to the exit breakpoint]
  
  0:000> g
  eax=00140500 ebx=00000000 ecx=7ffde000 edx=00000000 esi=00410200 edi=0024068c
  eip=77f68427 esp=0012fea0 ebp=0012ff64 iopl=0         nv up ei pl zr na po nc
  cs=001b  ss=0023  ds=0023  es=0023  fs=0038  gs=0000             efl=00000246
  ntdll!ZwTerminateProcess+0xb:
  77f68427 c20800           ret     0x8
  0:000> kb
  *** WARNING: symbols checksum is wrong 0x00063fce 0x000700ee for c:/wtsrv/symbols/dll/kernel32.dbg
  *** ERROR: Module load completed but symbols could not be loaded for image@00400000
  ChildEBP RetAddr  Args to Child
  0012fe9c 77f0f9b0 ffffffff 00410200 0024068c ntdll!ZwTerminateProcess+0xb
  0012ff64 00401270 00410200 00000149 0012ffc0 KERNEL32!ExitProcess+0x6d
  00410200 00000000 706d6973 7270656c 6172676f 0x401270
  0:000>
  
  

   

• Twilight