Inside CRT: Debug Heap Management -- Learn how heap management is done in a debug build[www.codeguru.com]

Inside CRT: Debug Heap Management
Learn how heap management is done in a debug build
Rating:

Marius Bancila (view profile) April 4, 2005

Environment:  Visual C++ 6.0

When you compile a debug build of your program with Visual Studio and run it in debugger, you can see that the memory allocated or deallocated has funny values, such as 0xCDCDCDCD or 0xDDDDDDDD. This is the result of the work Microsoft has put in to detect memory corruption and leaks in the Win32 platform. In this article, I will explain how memory allocation/deallocation is done via new/delete or malloc/free.

First, I will explain what all these values that you see, like CD, DD, and so forth, mean.

Value	Name	Description
0xCD	Clean Memory	Allocated memory via malloc or new but never written by the application.
0xDD	Dead Memory	Memory that has been released with delete or free. It is used to detect writing through dangling pointers.
0xFD	Fence Memory	Also known as "no mans land." This is used to wrap the allocated memory (like surrounding it with fences) and is used to detect indexing arrays out of bounds.
0xAB	(Allocated Block?)	Memory allocated by LocalAlloc().
0xBAADF00D	Bad Food	Memory allocated by LocalAlloc() with LMEM_FIXED, but not yet written to.
0xCC		When the code is compiled with the /GZ option, uninitialized variables are automatically assigned to this value (at byte level).

If you take a look at DBGHEAP.C, you can see how some of these values are defined:

static unsigned char _bNoMansLandFill = 0xFD;   /* fill no-man's land with this */
static unsigned char _bDeadLandFill   = 0xDD;   /* fill free objects with this */
static unsigned char _bCleanLandFill  = 0xCD;   /* fill new objects with this */ 

Before going any further, take a look at the memory management function that I will refer in this article.

Function	Description
malloc	C/C++ function that allocates a block of memory from the heap. The implementation of the C++ operator new is based on malloc.
_malloc_dbg	Debug version of malloc; only available in the debug versions of the run-time libraries. _malloc_dbg is a debug version of the malloc function. When _DEBUG is not defined, each call to _malloc_dbg is reduced to a call to malloc. Both malloc and _malloc_dbg allocate a block of memory in the base heap, but _malloc_dbg offers several debugging features: buffers on either side of the user portion of the block to test for leaks, a block type parameter to track specific allocation types, and filename/linenumber information to determine the origin of allocation requests.
free	C/C++ function that frees an allocated block. The implementation of C++ operator delete is based on free.
_free_dbg	Debug version of free; only available in the debug versions of the run-time libraries. The _free_dbg function is a debug version of the free function. When _DEBUG is not defined, each call to _free_dbg is reduced to a call to free. Both free and _free_dbg free a memory block in the base heap, but _free_dbg accommodates two debugging features: the ability to keep freed blocks in the heap's linked list to simulate low memory conditions and a block type parameter to free specific allocation types.
LocalAlloc GlobalAlloc	Win32 API to allocate the specified number of bytes from the heap. Windows memory management does not provide a separate local heap and global heap.
LocalFree GlobalFree	Win32 API free the specified local memory object and invalidates its handle.
HeapAlloc	Win32 API allocates a block of memory from a heap. The allocated memory is not movable.
HeapFree	Win32 API frees a memory block allocated from a heap by the HeapAlloc or HeapReAlloc function.

There are many other functions that deal with memory management. For a complete view please refer to MSDN.

Note: Because this article is about memory management in a debug build, all the references to malloc and free in the following are actually references to their debug versions, _malloc_dbg and _free_dbg.

Compile the following code and run it in the debugger, walking step by step into it to see how memory is allocated and deallocated.

int main(int argc, char* argv[])
{
   char *buffer = new char[12];

   delete [] buffer;

   return 0;
}

Here, 12 bytes are dynamically allocated, but the CRT allocates more than that by wrapping the allocated block with bookkeeping information. For each allocated block, the CRT keeps information in a structure called _CrtMemBlockHeader, which is declared in DBGINT.H:

#define nNoMansLandSize 4

typedef struct _CrtMemBlockHeader
{
        struct _CrtMemBlockHeader * pBlockHeaderNext;
        struct _CrtMemBlockHeader * pBlockHeaderPrev;
        char *                      szFileName;
        int                         nLine;
        size_t                      nDataSize;
        int                         nBlockUse;
        long                        lRequest;
        unsigned char               gap[nNoMansLandSize];
        /* followed by:
         *  unsigned char           data[nDataSize];
         *  unsigned char           anotherGap[nNoMansLandSize];
         */

} _CrtMemBlockHeader;

It stores the following information:

Field	Description
pBlockHeaderNext	A pointer to the next block allocated, but next means the previous allocated block because the list is seen as a stack, with the latest allocated block at the top.
pBlockHeaderPrev	A pointer to the previous block allocated; this means the block that was allocated after the current block.
szFileName	A pointer to the name of the file in which the call to malloc was made, if known.
nLine	The line in the source file indicated by szFileName at which the call to malloc was made, if known.
nDataSize	Number of bytes requested
nBlockUse	0 - Freed block, but not released back to the Win32 heap 1 - Normal block (allocated with new/malloc) 2 - CRT blocks, allocated by CRT for its own use
lRequest	Counter incremented with each allocation
gap	A zone of 4 bytes (in the current implementation) filled with 0xFD, fencing the data block, of nDataSize bytes. Another block filled with 0xFD of the same size follows the data.

Most of the work of heap block allocation and deallocation are made by HeapAlloc() and HeapFree(). When you request 12 bytes to be allocated on the heap, malloc() will call HeapAlloc(), requesting 36 more bytes.

blockSize = sizeof(_CrtMemBlockHeader) + nSize + nNoMansLandSize;

malloc requests space for the 12 bytes we need (nSize), plus 32 bytes for the _CrtMemBlockHeader structure and another nNoMansLandSize bytes (4 bytes) to fence the data zone and close the gap.

But, HeapAlloc() will allocate even more bytes: 8 bytes below the requested block (that is, at a lower address) and 32 above it (that is, at a bigger address). It also initializes the requested block to 0xBAADF00D (bad food).

Then, malloc() fills the _CrtMemBlockHeader block with information and initializes the data block with 0xCD and no mans land with 0xFD.

Here is a table that shows how memory looks after the call to HeapAlloc() and after malloc() returns. For a complete situation, see the last table. (Note: All values are in hex.)

Address	after HeapAlloc()	after malloc()
00320FD8 00320FDC 00320FE0 00320FE4 00320FE8 00320FEC 00320FF0 00320FF4 00320FF8 00320FFC 00321000 00321004 00321008 0032100C 00321010 00321014 00321018 0032101C 00321020 00321024 00321028 0032102C	09 00 09 01 E8 07 18 00 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA AB AB AB AB AB AB AB AB 00 00 00 00 00 00 00 00 79 00 09 00 EE 04 EE 00 40 05 32 00 40 05 32 00	09 00 09 01 E8 07 18 00 98 07 32 00 00 00 00 00 00 00 00 00 00 00 00 00 0C 00 00 00 01 00 00 00 2E 00 00 00 FD FD FD FD CD CD CD CD CD CD CD CD CD CD CD CD FD FD FD FD AB AB AB AB AB AB AB AB 00 00 00 00 00 00 00 00 79 00 09 00 EE 04 EE 00 40 05 32 00 40 05 32 00

Colors:

• Green: win32 bookkeeping info
• Blue: block size requested by malloc and filled with bad food
• Magenta: _CrtMemBlockHeader block
• Red: no mans land
• Black: requested data block

In this example, after the call to malloc() returns, buffer will point to memory address 0x00321000.

When you call delete/free, the CRT will set the block it requested from HeapAlloc() to 0xDD, indicating this is a free zone. Normally after this, free() will call HeapFree() to give back the block to the Win32 heap, in which case the block will be overwritten with 0xFEEEEEEE, to indicate Win32 heap free memory.

You can avoid this by using the CRTDBG_DELAY_FREE_MEM_DF flag to _CrtSetDbgFlag(). It prevents memory from actually being freed, as for simulating low-memory conditions. When this bit is on, freed blocks are kept in the debug heap's linked list but are marked as _FREE_BLOCK. This is useful if you want to detect dangling pointers errors, which can be done by verifying if the freed block is written with 0xDD pattern or something else. Use _CrtCheckMemory() to verify the heap.s integrity.

The next table shows how the memory looks during the free(), before HeapFree() is called and afterwards.

Address	Before HeapFree()	After HeapFree()
00320FD8 00320FDC 00320FE0 00320FE4 00320FE8 00320FEC 00320FF0 00320FF4 00320FF8 00320FFC 00321000 00321004 00321008 0032100C 00321010 00321014 00321018 0032101C 00321020 00321024 00321028 0032102C	09 00 09 01 5E 07 18 00 DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD AB AB AB AB AB AB AB AB 00 00 00 00 00 00 00 00 79 00 09 00 EE 04 EE 00 40 05 32 00 40 05 32 00	82 00 09 01 5E 04 18 00 E0 2B 32 00 78 01 32 00 EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE EE FE

Colors:

• Green: win32 bookkeeping info
• Blue: CRT block filled with dead memory
• Gray: memory given back to win32 heap

The two tables above are put in a single, more detailed, table below:

Address (hex)	Offset	HeapAlloc	malloc	Free before HeapFree	Free after HeapFree	Description
00320FD8	-40	01090009	01090009	01090009	01090082	Win32 Heap info
00320FDC	-36	001807E8	001807E8	0018075E	0018045E	Win32 Heap info
00320FE0	-32	BAADF00D	00320798	DDDDDDDD	00322BE0	pBlockHeaderNext
00320FE4	-28	BAADF00D	00000000	DDDDDDDD	00320178	pBlockHeaderPrev
00320FE8	-24	BAADF00D	00000000	DDDDDDDD	FEEEEEEE	szFileName
00320FEC	-20	BAADF00D	00000000	DDDDDDDD	FEEEEEEE	nLine
00320FF0	-16	BAADF00D	0000000C	DDDDDDDD	FEEEEEEE	nDataSize
00320FF4	-12	BAADF00D	00000001	DDDDDDDD	FEEEEEEE	nBlockUse
00320FF8	-8	BAADF00D	0000002E	DDDDDDDD	FEEEEEEE	lRequest
00320FFC	-4	BAADF00D	FDFDFDFD	DDDDDDDD	FEEEEEEE	gap (no mans land)
00321000	0	BAADF00D	CDCDCDCD	DDDDDDDD	FEEEEEEE	Data requested
00321004	+4	BAADF00D	CDCDCDCD	DDDDDDDD	FEEEEEEE	Data requested
00321008	+8	BAADF00D	CDCDCDCD	DDDDDDDD	FEEEEEEE	Data requested
0032100C	+12	BAADF00D	FDFDFDFD	DDDDDDDD	FEEEEEEE	No mans land
00321010	+16	ABABABAB	ABABABAB	ABABABAB	FEEEEEEE	Win32 Heap info
00321014	+20	ABABABAB	ABABABAB	ABABABAB	FEEEEEEE	Win32 Heap info
00321018	+24	00000000	00000000	00000000	FEEEEEEE	Win32 Heap info
0032101C	+28	00000000	00000000	00000000	FEEEEEEE	Win32 Heap info
00321020	+32	00090079	00090079	00090079	FEEEEEEE	Win32 Heap info
00321024	+36	00EE04EE	00EE04EE	00EE04EE	FEEEEEEE	Win32 Heap info
00321028	+40	00320540	00320540	00320540	FEEEEEEE	Win32 Heap info
0032102C	+44	00320540	00320540	00320540	FEEEEEEE	Win32 Heap info

About the Author
Marius Bancila is a software developer working for a major software company in the automotive field. He is mainly focused on building desktop applications with MFC. When he discovered the .NET framework quickly fell inlove. He considers that CodeGuru is the best place on internet to spend time on.