windows via C++笔记

80H

Chapter2：Working with Characters and Strings

1. Overview

       Buffer overrun errors have become a vector for security attacks against application.

2. ANSI and Unicode Character and String Data Types

       2.1. Declare a Unicode string:

       wchar_t szBuffer[100] = L"A String";

       2.2. WinNT.h(1)

       // Pointer to 8-bit character(s)

       typedef CHAR *PCHAR;

       typedef CHAR *PSTR;

       typedef CONST CHAR *PCSTR

       // Pointer to 16-bit character(s)

       typedef WCHAR *PWCHAR;

       typedef WCHAR *PWSTR;

       typedef CONST WCHAR *PCWSTR;

       2.2. WinNT.h(2)

       #ifdef UNICODE

              typedef WCHAR TCHAR, *PTCHAR, PTSTR;

              typedef CONST WCHAR *PCTSTR;

              #define __TEXT(quote) quote          // r_winnt

              #define __TEXT(quote) L##quote

       #else

              typedef CHAR TCHAR, *PTCHAR, PTSTR;

              typedef CONST CHAR *PCTSTR;

              #define __TEXT(quote) quote

       #endif

       #define   TEXT(quote) __TEXT(quote)

3. Unicode and ANSI Functions in Windows

       3.1. Windows exposes two versions of the same function

       3.2. In the ANSI version, simply allocate memory, perform the necessary string conversions, and call the Unicode version of the function

       3.3. Certain functions exist solely for backward compatibility with 16-bit Windows programs should be avoided because they supported only ANSI strings

       3.4. All COM interface methods requiring a string would accept only Unicode strings

       3.5. String values in your resources are always written as Unicode strings.

4. Unicode and ANSI Functions in the C Run-Time Library

       4.1. In Windows, only Unicode version of a function do the work; But in C run-time library both version of a function do the work.

       4.2. Make sure that both UNICODE(Windows) and _UNICODE(C run-time libary) are defined or neither is defined.

5. Secure String Functions in the C Run-Time Library

       5.1. memory corruption

       5.2. When include StrSafe.h

              String.h is also included

              Existing string manipulation functions of the C run-time library are flagged with   obsolete warning

       5.3. Each existing function has a corresponding new function that starts with the same name that ends with the _s (for secure) suffix, which accept an additional parameter for the buffer’s size in the character count, which is easily computed by using the _countof macro.

       5.4. When a memory corruption occurs, the secure version of string functions set the first character of string to ‘/0’. And all other bytes are filled by 0xfd.

       5.5. C run-time library’s function truncate the string when meet a memory corruption.Ex version can get more control of it.

       5.6. Two version of string comparing functions for two kinds of string.(whose return values are different from C run-time library's *cmp string comparison functions)

Chapter3：Kernel Object

1. What is a Kernel Object

       1.1. handle: An opaque process-relative value that can be used by any thread in your process.

       1.2. Usage Counting: If the object's usage count goes to 0, the kernel destroys the object.

       1.3. Security: Almost all functions that create kernel objects have a pointer to a SECURITY_ATTRIBUTES structure as an argument.

       1.4. The easiest way to determine whether an object is a kernel object (or a User objects) is to examine the function that creates the object. Almost all functions that create kernel objects have a parameter that allows you to specify security attribute information

2. A Process' Kernel Object Handle Table

      

Chapter13：Windows Memory Architecture

1. A Process’ Virtual Address Space

       Every process is given its very own 4GB virtual address space.32-bit pointer can have any value from 0x00000000 through 0xFFFFFFFF.

2. How a Virtual Address Space Is Partitioned

       NULL-Pointer Assignment; User-Mode; 64-KB Off-Limits; Kernel-Mode

3. Regions in an Address Space

       3.1. Calling VirtualAlloc to reserve region in virtual space

       3.2. Whenever you reserve a region of address space, the system ensures that the region begins on an allocation granularity boundary(64KB) and the size of the region is a multiple of the system’s page size(4KB).

       3.3. Sometimes the system reserves regions of address space on behalf of your process.For example process environment block(PEB) and thread environment blocks(TEBs).System is not subjected to the granularity boundary limitation.

4. Committing Physical Storage Within a Region

       4.1. VirtualAlloc cause a committing physical storage.

       4.2. Physical storage is always committed in pages.

       4.3. VirtualFree cause a decommitting the physical storage.

5. Physical Storage and the Paging File

       5.1. The file on the disk which contains the virtual memory is typically called a paging file.

       5.2. Simplified flowchart

 

       5.3. page fault:The data that the thread is attempting to access is not in RAM but is contained somewhere in the paging file.

       5.4. Thrashing:The operating system needs to copy pages of memory to the paging file and vice versa.

       5.5. The system use the image of the .exe file as the program’s reserved region of address space when load a program.The image is called a memory-mapped file.

6. Protection Attributes

       6.1. PAGE_WRITECOPY or PAGE_EXECUTE_WRITECOPY:3steps

       6.2. These two attributes are used by the operating system when it maps .exe and DLL file images. should not pass either of them to VirtualAlloc.

       6.3. PAGE_NOCACHE,PAGE_WRITECOMBINE for hardware device-driver developer.

       6.4. PAGE_GUARD, allows an application to receive a notification (via an exception) when a byte on a page has been written to.

7. Bringing It All Home

       7.1. VMMap(code in cap14)

       7.2. obtain the full pathname of the image using the PSAPI functions GetMappedFileName and ToolHelp.(mentioned at the end of cap4)

       7.3. ?It is possible for different committed blocks within a single region to be backed by different types of physical storage.

8. The Importance of Data Alignment

       8.1. IA-64 CPU cannot automatically fix up misaligned data accesses

              UNIT SetErrorMode(UINT fuErrorMode)

              SEM_NOALIGNMENTFAULTEXCEPT

              Use keyword __unaligned(not supported by x86) to add the additional CPU instructions

       8.2. Windows Reliability and Performance Monitor

Chapter 14: Exploring Virtual Memory

1. Overview

       1.1.VOID GetSystemInfo(LPSYSTEM_INFO psi);

       1.2. GetLogicalProcessorInformation(); more details about the processors

       1.3. SysInfo.cpp(haven’t read yet)

2. Virtual Memory Status

       2.1.VOID GlobalMemoryStatus(LPMEMORYSTATUS pmst);

       2.2. MEMORYSTATUSEX is for the very large memory(>4GB).

3. Memory Management on NUMA Machines

4. Determining the State of an Address Space

       4.1. Query certain information in current address space

              DWORD VirtualQuery(

                 LPCVOID pvAddress,

                 PMEMORY_BASIC_INFORMATION pmbi,

                 DWORD dwLength);

       4.2. Query memory information in other process

              DWORD VirtualQueryEx(

                 HANDLE hProcess,

                 LPCVOID pvAddress,

                 PMEMORY_BASIC_INFORMATION pmbi,

                 DWORD dwLength);

       4.3. VMQuery

Chapter 15: Using Virtual Memory in Your Own Applications

1.Reserving a Region in an Address Space

PVOID VirtualAlloc(

       PVOID pvAddress,

       SIZE_T dwSize,

       DWORD fdwAllocationType,

       DWORD fdwProtect);

       1.1. pvAddress:

              Most of time pass NULL for this parameter to tell system reserve the region wherever it        see fit.

              The system automatically rounds the address passed to this parameter to a multiple of    64KB (granularity boundary of the system).

       1.2. dwSize:

              the system must always reserve regions in multiples of the CPU’s page size(4,8,16KB)

       1.3. fdwAllocationType:

              To reserve a region of address space, pass the MEM_RESERVE identifier as the value.

       1.4. fdwProtect:

              The protection attribute associated with the region has no effect on the committed storage mapped to the region.

              system's internal record keeping behaves more efficiently when the region's protection   attribute matches the committed storage's protection attribute

              cannot specify either PAGE_WRITECOPY or PAGE_EXECUTE_WRITECOPY for this attribute.【13.6.2】

       1.5. Non-Uniform Memory Access (NUMA) machine

2. Committing Storage in a Reserved Region

       2.1. MEM_COMMIT identifier instead of the MEM_RESERVE identifier for the fdwAllocationType parameter

       2.2. pvAddress: rounds down to a multiple of granularity boundary

       2.3. dwSize: rounds up to the CPU’s page size

3. Reserving a Region and Committing Storage Simultaneously

       large-page support

4. When to Commit Physical Storage

       Later for these topics

5. Resetting the Contents of Physical Storage

       1.1. As the system looks for pages of RAM to satisfy recent load requests, the system will have to swap modified pages of RAM to the system's paging file.

       1.2. An application tell the system don't swap data page by calling VirtualAlloc, passing the MEM_RESET flag in the third parameter.

       1.3. Keep in mind a couple of additional things when reset storage:

       VirtualAlloc round down the base address is usually to a page boundary and round up the number of bytes to an integral number of pages. Rounding the base address and number of bytes this way would be very dangerous to do when resetting storage; therefore, VirtualAlloc rounds these values in the opposite direction when you pass MEM_RESET.

       MEM_RESET flag must always be used by itself and cannot be ORed with any other flags.

 

       VMAlloc;  AWE

Chapter 16: A Thread's Stack

1.Overview

       1.1. flow:

              By default, the system reserves 1 MB of address space and commits two pages of   storage

              After reserving this region,the system sets the thread's stack pointer register to point to   the end of the top page.

              The second page from the top is called the guard page

              Whenever the thread attempts to access the guard page, the system is notified to    commits another page below the guard page.【13.6.4】

              System does not apply the guard attribute to the bottom of stack and this page is always        reserved and never gets committed

              When system commits physical storage to the second page from the bottom, it raises an        EXCEPTION_STACK_OVERFLOW exception

       1.2. EXCEPTION_STACK_OVERFLOW: Summation sample at the end of this chapter demonstrates how to recover gracefully from stack overflows

       1.3. Access the reserved (uncommitted) bottom memory, the system raises an access violation exception

       1.4. The bottommost page of stack’s region is always reserved to protect other region of address space has committed next to the thread’s stack.

       1.5. underflow

2. The C/C++ Run-Time Library's Stack-Checking Function

       2.1. the system does not commit physical storage to lower area of the stack’s region until an attempt is made to access the memory address.

       2.2. If the first access to the stack is at an address that is below the guard page the thread will be accessing reserved memory and the system will raise an access violation.

       2.3. To avoid the problem above, the compiler automatically inserts a call to the stack-checking function.

Chapter 17: Memory-Mapped Files

1. Memory-Mapped Executables and DLLs

       1.1. Steps when load .exe and DLLs

              .exe:

              The system creates a new process kernel object.

              The system creates a private address space for this new process

              When reserve a region of address space for the .exe file,the base address is 0x00400000,       override this by using the linker’s /BASE option.

              DLLs:

              When reserve a region of address space for the DLLs, The desired location of this region      is specified inside the DLL file itself.( 0x10000000 for an x86 DLL and 0x00400000 for an        x64 DLL by default) .

              Override this by using the linker's /BASE option.

              All the standard system DLLs have different base addresses so that they won’t overlap.

              If the system is unable to reserve a region at the DLL's preferred base address, the system will then perform some relocations. It is unfortunate for two reasons:

       1) DLL must load at its preferred base address when using the linker's/FIXED switch.

       2)These relocations require additional storage from the system's paging file.they also      increase the amount of time needed to load the DLL.

       1.2. Static Data Is Not Shared by Multiple Instances of an Executable or a DLL

              System use PAGE_WRITECOPY or PAGE_EXECUTE_WRITECOPY to protect        memory from being corrupted by different instance of an application.

       1.3. Every .exe or DLL file image is composed of a collection of sections

              The compiler places all the code in a section called .text. The compiler also places all     the uninitialized data in a .bss section and all the initialized data in a .data section

              Use DumpBin(with the /headers switch) to show the list of sections in an .exe or DLL

       1.4. Create your own sections:

              #pragma data_seg("Shared")

              LONG g_lInstanceCount = 0;

              #pragma data_seg()

              This creates a new section called Shared and places all the initialized data variables in it.

       1.5. using the /SECTION switch on the linker's command line

              /SECTION:name,attributes

       1.6. embed linker switches inside source code using:

              #pragma comment(linker, "/SECTION:Shared,RWS")

              This line tells the compiler to embed the preceding string insede a special section     named “.derectve” and when the linker combines all the .obj modules together,the string        inside the “.derectve” section were passed to the linker as command-line arguments

2. Using Memory-Mapped Files

       2.1. Creating or Opening a File Kernel Object

       HANDLE CreateFile(

              PCSTR pszFileName,

              DWORD dwDesiredAccess,

              DWORD dwShareMode,

              PSECURITY_ATTRIBUTES psa,

              DWORD dwCreationDisposition,

              DWORD dwFlagsAndAttributes,

              HANDLE hTemplateFile);

       2.2. Creating a File-Mapping Kernel Object

       HANDLE CreateFileMapping(

              HANDLE hFile,

              PSECURITY_ATTRIBUTES psa,

              DWORD fdwProtect,

              DWORD dwMaximumSizeHigh,

              DWORD dwMaximumSizeLow,

              PCTSTR pszName);

       fdwProtect:When you create a file-mapping object, the system does not reserve a region.      However, when the system does map the storage to the process' address space, the system        must know what protection attribute to assign to the pages of physical storage.

       section attributes

       dwMaximumSizeHigh/Low:

       1)The main purpose of the CreateFileMapping function is to ensure that enough physical    storage is available for the file-mapping object.

       2)These two 32-bit parameters tell the system the maximum size of the file in bytes. 64-bit    value means that windows can process files as large as 16EB.

       3)Pass 0 for both parameters to create the file-mapping object reflects the current size of the        file.(read only)

       4)Choose a maximum file size that leaves some breathing room(append)

       5)If the file currently contains 0 bytes, you can’t pass two zeros to these parameters.

       pszName:0-terminated string for sharing the object with another process

       2.3. Mapping the File's Data into the Process' Address Space

       PVOID MapViewOfFile(

              HANDLE hFileMappingObject,

              DWORD dwDesiredAccess,

              DWORD dwFileOffsetHigh,

              DWORD dwFileOffsetLow,

              SIZE_T dwNumberOfBytesToMap);

       dwDesiredAccess:When specify the FILE_MAP_COPY flag, the system commits physical storage from the system's paging file. Any writes to any memory address within the      file's       mapped view, the system will copy the page of original data to this paging-file page, and then     map this copied page into your process' address space and change the new page’s protection        to PAGE_READWRITE.

       dwFileOffsetHigh/Low, dwNumberOfBytesToMap:A portion of a file that is mapped into     your process' address space is called a view. The offset in the file must be a multiple of the      system's allocation granularity

       Non-Uniform Memory Access (NUMA)

       2.4. Unmapping the File's Data from the Process' Address Space

       BOOL UnmapViewOfFile(PVOID pvBaseAddress);

       Whenever you call MapViewOfFile, the system always reserves a new region within your    process' address space—any previously reserved regions are not released.

       BOOL FlushViewOfFile(

              PVOID pvAddress,

              SIZE_T dwNumberOfBytesToFlush);

       1)Force the system to write a portion or all of the modified data back to the disk image.

       2) Pass the FILE_FLAG_WRITE_THROUGH flag to the CreateFile function to ensure a   memory-mapped file whose storage is over a network being update.

       3) If the view was originally mapped using the FILE_MAP_COPY flag, call this function     has nothing to update on the disk file and simply causes the pages in the paging file to be      freed—the data is lost.

       2.5. Closing the File-Mapping Object and the File Object

       CloseHandle(HANDL);

       The system increments the usage counts of the file object and the file-mapping object when   you call MapViewOfFile. So you can close file-mapping object and the file object at the       beginning of your code and eliminate potential resource leaks

       Sample

3. Processing a Big File Using Memory-Mapped Files

Mapping a view of the file many times.

       3.1. To get the total size of the file use the code below:

       DWORD dwFileSizeHigh;

       __int64 qwFileSize = GetFileSize(hFile, &dwFileSizeHigh);

       qwFileSize += (((__int64) dwFileSizeHigh) << 32);

4. Memory-Mapped Files and Coherence

       4.1. Windows does not guarantee that different file-mapping objects will be coherent. It guarantees only that multiple views of a single file-mapping object will be coherent.

       4.2. It is recommended that when you call CreateFile for files that will be memory mapped, you specify 0 as the value of the dwShareMode parameter.

5. Specifying the Base Address of a Memory-Mapped File

       5.1. PVOID MapViewOfFileEx(

              HANDLE hFileMappingObject,

              DWORD dwDesiredAccess,

              DWORD dwFileOffsetHigh,

              DWORD dwFileOffsetLow,

              SIZE_T dwNumberOfBytesToMap,

              PVOID pvBaseAddress);

       5.2. MapViewOfFileEx is useful when using memory-mapped files to share data whicth contains pointer. For example, If one process prepares the linked list in a memory-mapped file, the other process will map the file into a completely different location in its address space.

6. Implementation Details of Memory-Mapped Files

       6.1. When call MapViewOfFile() twice for the same file-mapping object, the two address returned by the function are most likely not be the same.

       6.2. The system reserves different address space region for each call to MapViewOfFile() but commits the same Physical Storage so the data is guaranteed to be coherent.

7. Using Memory-Mapped Files to Share Data Among Processes

       7.1. example: Starting application’s different instance share the same .exe file's image by the system using Memory-Mapped Files

       7.2. As with all kernel objects, you can use three techniques to share the objects with multiple processes: handle inheritance, naming, and handle duplication. See 【Chapter 3】 for a detailed explanation of all three techniques.

8. Memory-Mapped Files Backed by the Paging File

       8.1. No need to call the CreateFile() but simply call the  CreateFileMapping() and pass INVALID_HANDLE_VALUE as the hFile parameter.

       8.2. An interesting problem

       HANDLE hFile = CreateFile(...);

       HANDLE hMap = CreateFileMapping(hFile, ...);

       if (hMap == NULL)

          return(GetLastError());

       ...

9. Sparsely Committed Memory-Mapped Files

Chapter 18: Heaps

1. Overview

       1.1. Internally, a heap is a region of reserved address space. As you make more allocations from the heap, the heap manager commits more physical storage to the heap. Microsoft does not document the exact rules that the heap follows for committing and decommitting storage.

2. A Process' Default Heap

       2.1. To change the default region size

       /HEAP:reserve[,commit]

       2.2. Process' default heap is used by Windows functions so it is always serialized.

3. Reasons to Create Additional Heaps

       3.1. Protect one component’s memory to be corrupted by operations on another component. This kind of bug is tracked down and isolated difficult.

       3.2. To avoid memory becomes fragmented

       3.3. Allocate things close to each other to avoid page faults when traversing a component

       3.4. Avoiding thread synchronization overhead.

       3.5. Quick free the entire heap.

4. How to Create an Additional Heap

5. Miscellaneous Heap Functions

Chapter 19: DLL Basics

1. DLLs and a Process' Address Space

       1.1. Specify the /DLL switch to the link DLLs.

       1.2. Any objects created by code in the DLL's functions are owned by the calling thread or process.

       1.3. Be aware of using different version of malloc/free in DLLs and calling function. If DLLs include a function to malloc memory, embeding a function to free memory using the same version is a good ideal.

2. The Overall Picture

       2.1. #define MYLIBAPI extern "C" __declspec(dllexport)

       1) Use the extern "C" modifier to tells the compiler not to mangle the function names and    thereby make the function accessible to executable modules written in C, C++, or any other      programming language

       2) __declspec(dllexport): When the DLL is linked, the linker automatically produces export   section contains the list (in alphabetical order) of exported variables, functions, and class    symbols. The linker also places the relative virtual address (RVA), indicating where each    symbol can be found in the DLL module.

       3) Microsoft Visual Studio DumpBin.exe(with the -exports switch)

       2.2. #define MYLIBAPI extern "C" __declspec(dllimport)

       1) You do not have to use this modifier but the compiler can produce slightly more efficient   code if it knows ahead of time that the symbol will be imported from a DLL's .lib file.

       2) The linker embeds a special section called the import section in the resulting executable      module

Chapter 20: DLL Advanced Techniques

1.Overview

       1.1. Explicitly Loading the DLL Module

       HMODULE LoadLibrary(PCTSTR pszDLLPathName);

       HMODULE LoadLibraryEx(

              PCTSTR pszDLLPathName,

              HANDLE hFile /* must be NULL*/,

              DWORD dwFlags);

       1) The two functions’ return are not interchangeable.

       2) dwFlags:

              DONT_RESOLVE_DLL_REFERENCES: Map the file image without calling   DllMain and does not automatically load any additional DLLs. Avoid using this flag.

              LOAD_LIBRARY_AS_DATAFILE: Using when DLL that contains only resources or call LoadLibraryEx to load an .exe file.

              LOAD_LIBRARY_AS_DATAFILE_EXCLUSIVE: Forbidding any other application    from modifying the file.

              LOAD_LIBRARY_AS_IMAGE_RESOURCE: This is particularly handy when you     need to parse a DLL for exploring its portable executable (PE) sections.

              LOAD_WITH_ALTERED_SEARCH_PATH: Changes the search algorithm that    LoadLibraryEx uses to locate the specified DLL file

              LOAD_IGNORE_CODE_AUTHZ_LEVEL

       1.2. Explicitly Unloading the DLL Module

       BOOL FreeLibrary(HMODULE hInstDll);

       VOID FreeLibraryAndExitThread(

              HMODULE hInstDll,

              DWORD dwExitCode);

       1) In reality, the LoadLibrary and LoadLibraryEx functions increment a per-process usage count associated with the specified library. The FreeLibrary and       FreeLibraryAndExitThread functions decrement the library's per-process usage count

       2) The reason to creating the FreeLibraryAndExitThread function: The code that contains the call to ExitThread is no longer available after the FreeLibrary returns.

       1.3. Determine the full pathname of a DLL (or an .exe)

       DWORD GetModuleFileName(

              HMODULE hInstModule,

              PTSTR pszPathName,

              DWORD cchPath);

       1.4. Explicitly Linking to an Exported Symbol

       FARPROC GetProcAddress(

              HMODULE hInstDll,

              PCSTR pszSymbolName);

       1) pszSymbolName should only be ANSI string

       2) Don’t use the ordinal number of the symbol as value of the pszSymbolName parameter

       1.5. cast the function pointer returned by GetProcAddress into the right type that maps its signature: typedef void (CALLBACK *PFN_DUMPMODULE)(HMODULE hModule);

2. The DLL's Entry-Point Function

       2.1.