The Android ION memory allocator
February8, 2012
This article was contributed by Thomas M. Zeng
Back in December 2011, LWN reviewedthe list of Android kernel patches inthe linux-next staging directory. The merging of these drivers, one of which isa memory allocator called PMEM, holds the promise that the mainline kernelrelease can one day boot an Android user space. Since then, it has become clear that PMEM is considered obsolete andwill be replaced by the ION memory manager
. ION is a generalized memory manager that Google introduced in theAndroid 4.0 ICS (Ice Cream Sandwich) release to address the issue of fragmentedmemory management interfaces across different Android devices. There are atleast three, probably more, PMEM-like interfaces. On Android devices usingNVIDIA Tegra, there is "NVMAP"; on Android devices using TI OMAP,there is "CMEM"; and on Android devices usingQualcomm MSM, there is "PMEM" .All three SoC vendors are in the process of switching to ION.
This article takes a look at ION, summarizing its interfaces touser space and to kernel-space drivers. Besides being a memory pool manager,ION also enables its clients to share buffers, hence it treads the same groundas the DMAbuffer sharing framework from Linaro (DMABUF). This article will end with acomparison of the two buffer sharing schemes.
IONheaps
Like its PMEM-like predecessors, ION manages one or more memory pools,some of which are set aside at boot time to combat fragmentation or to servespecial hardware needs. GPUs, display controllers, and cameras are some of thehardware blocks that may have special memory requirements. ION presents itsmemory pools as ION heaps. Each type of Android device can be provisioned witha different set of ION heaps according to the memory requirements of thedevice. The provider of an ION heap must implement the following set ofcallbacks:
struct ion_heap_ops {
int (*allocate) (struct ion_heap *heap,
struct ion_buffer *buffer, unsigned long len,
unsigned long align, unsigned long flags);
void (*free) (struct ion_buffer *buffer);
int (*phys) (struct ion_heap *heap, struct ion_buffer *buffer,
ion_phys_addr_t *addr, size_t *len);
struct scatterlist *(*map_dma) (struct ion_heap *heap,
struct ion_buffer *buffer);
void (*unmap_dma) (struct ion_heap *heap,
struct ion_buffer *buffer);
void * (*map_kernel) (struct ion_heap *heap,
struct ion_buffer *buffer);
void (*unmap_kernel) (struct ion_heap *heap,
struct ion_buffer *buffer);
int (*map_user) (struct ion_heap *heap, struct ion_buffer *buffer,
struct vm_area_struct *vma);
};
Briefly, allocate() and free() obtain or release an ion_buffer object from the heap. A call to phys() will return the physical address and length of the buffer, butonly for physically-contiguous buffers. If the heap does not provide physicallycontiguous buffers, it does not have to provide this callback. Here ion_phys_addr_t is a typedef of unsigned long, and will, someday, be replaced by phys_addr_t in include/linux/types.h. The map_dma() andunmap_dma() callbacks cause the buffer to be prepared (or unprepared) for DMA.The map_kernel() and unmap_kernel() callbacks map (or unmap) the physical memory into the kernelvirtual address space. A call to map_user() will map the memory to user space. There is no unmap_user() because the mapping is represented as a file descriptor in userspace. The closing of that file descriptor will cause the memory to be unmappedfrom the calling process.
The default ION driver (which can be cloned from here) offers three heaps as listed below:
ION_HEAP_TYPE_SYSTEM: memory allocated via vmalloc_user().
ION_HEAP_TYPE_SYSTEM_CONTIG: memory allocated via kzalloc.
ION_HEAP_TYPE_CARVEOUT: carveout memory is physically contiguous and set aside at boot.
Developers may choose to add more ION heaps. For example, this NVIDIA patch was submitted to addION_HEAP_TYPE_IOMMU for hardware blocks equipped with an IOMMU.
Using ION fromuser space
Typically, user space device access libraries will use ION toallocate large contiguous media buffers. For example, the still camera librarymay allocate a capture buffer to be used by the camera device. Once the bufferis fully populated with video data, the library can pass the buffer to thekernel to be processed by a JPEG encoder hardware block.
A user space C/C++ program must have been granted access to the /dev/ion device before it can allocatememory from ION. A call to open("/dev/ion", O_RDONLY) returns a file descriptor as a handle representing an ION client.Yes, one can allocate writable memory with an O_RDONLY open. There can be no more than one client per user process. Toallocate a buffer, the client needs to fill in all the fields except the handle field in this data structure:
struct ion_allocation_data {
size_t len;
size_t align;
unsigned int flags;
struct ion_handle *handle;
}
The handle field is the output parameter, while the first three fieldsspecify the alignment, length and flags as input parameters. The flags field is a bit mask indicating one or more ION heaps to allocatefrom, with the fallback ordered according to which ION heap was first added viacalls toion_device_add_heap() during boot. In the default implementation, ION_HEAP_TYPE_CARVEOUT is added before ION_HEAP_TYPE_CONTIG. The flags of ION_HEAP_TYPE_CONTIG |ION_HEAP_TYPE_CARVEOUT indicate the intention toallocate from ION_HEAP_TYPE_CARVEOUT with fallback to ION_HEAP_TYPE_CONTIG.
User-space clients interact with ION using the ioctl() system call interface. Toallocate a buffer, the client makes this call:
int ioctl(int client_fd, ION_IOC_ALLOC, struct ion_allocation_data *allocation_data)
This call returns a buffer represented by ion_handle which is not a CPU-accessible buffer pointer. The handle can onlybe used to obtain a file descriptor for buffer sharing as follows:
int ioctl(int client_fd, ION_IOC_SHARE, struct ion_fd_data *fd_data);
Here client_fd is the file descriptor corresponding to /dev/ion, and fd_data is a data structure with an input handle field and an output fd field, as defined below:
struct ion_fd_data {
struct ion_handle *handle;
int fd;
}
The fd field is the file descriptor that can be passed around forsharing. On Android devices the BINDER IPC mechanism may be used to send fd to another process for sharing. To obtain the shared buffer, thesecond user process must obtain a client handle first via the open("/dev/ion", O_RDONLY)system call. ION tracks its user space clients by the PID of theprocess (specifically, the PID of the thread that is the "groupleader" in the process). Repeating the open("/dev/ion",O_RDONLY) call in the same process will get back another file descriptorcorresponding to the same client structure in the kernel.
To free the buffer, the second client needs to undo the effect of mmap() with a call to munmap(), and the first client needs toclose the file descriptor it obtained via ION_IOC_SHARE, andcall ION_IOC_FREE as follows:
int ioctl(int client_fd, ION_IOC_FREE, struct ion_handle_data *handle_data);
Here ion_handle_data holds the handle as shown below:
struct ion_handle_data {
struct ion_handle *handle;
}
The ION_IOC_FREE command causes the handle's reference counter to be decremented byone. When this reference counter reaches zero, the ion_handle object gets destroyed and the affected ION bookkeeping datastructure is updated.
User processes can also share ION buffers with a kernel driver, asexplained in the next section.
Sharing IONbuffers in the kernel
In the kernel, ION supports multiple clients, one for each driverthat uses the ION functionality. A kernel driver calls the following function toobtain an ION client handle:
struct ion_client *ion_client_create(struct ion_device *dev,
unsigned int heap_mask, const char *debug_name)
The first argument, dev, is theglobal ION device associated with /dev/ion; why aglobal device is needed, and why it must be passed as a parameter, is notentirely clear. The second argument, heap_mask,selects one or more ION heaps in the same way as the ion_allocation_data. The flags field was covered in the previous section. For smart phone usecases involving multimedia middleware, the user process typically allocates thebuffer from ION, obtains a file descriptor using the ION_IOC_SHARE command, then passes the filedesciptor to a kernel driver. The kernel driver calls ion_import_fd() which converts the file descriptor to an ion_handle object, as shown below:
struct ion_handle *ion_import_fd(struct ion_client *client, int fd_from_user);
The ion_handle object is the driver's client-local reference to the sharedbuffer. The ion_import_fd() call looks up the physical address of the buffer to see whetherthe client has obtained a handle to the same buffer before, and if it has, thiscall simply increments the reference counter of the existing handle.
Some hardware blocks can only operate on physically-contiguousbuffers with physical addresses, so affected drivers need to convert ion_handle to a physical buffer via thiscall:
int ion_phys(struct ion_client *client, struct ion_handle *handle,
ion_phys_addr_t *addr, size_t *len)
Needless to say, if the buffer is not physically contiguous, thiscall will fail.
When handling calls from a client, ION always validates the inputfile descriptor, client and handle arguments. For example, when importing afile descriptor, ION ensures the file descriptor was indeed created by an ION_IOC_SHARE command. When ion_phys() is called, ION validates whetherthe buffer handle belongs to the list of handles the client is allowed toaccess, and returns error if the handle is not on the list. This validationmechanism reduces the likelihood of unwanted accesses and inadvertent resourceleaks.
ION provides debug visibility through debugfs. It organizes debug informationunder /sys/kernel/debug/ion, with bookkeeping information in stored files associated withheaps and clients identified by symbolic names or PIDs.
Comparing IONand DMABUF
ION and DMABUF share some common concepts. The dma_buf concept
is similar to ion_buffer, while dma_buf_attachment serves a similarpurpose as ion_handle. Both ION and DMABUF use anonymous file descriptors as the objects thatcan be passed around to provide reference-counted access to shared buffers. Onthe other hand, ION focuses on allocating and freeing memory from provisionedmemory pools in a manner that can be shared and tracked, while DMABUF focusesmore on buffer importing, exporting and synchronization in a manner that isconsistent with buffer sharing solutions on non-ARM architectures.
Thefollowing table presents a feature comparison between ION and DMABUF:
The Android ION memory allocator
Posted Feb 9, 2012 9:31 UTC (Thu) by blackwood (subscriber,#44174) [Link]
The AndroidION memory allocator
PostedFeb 22, 2012 1:42 UTC (Wed) by zengtm (subscriber, #74989) [Link]
AndI agree that the persistent device mappings seem to be the more common smartphone use cases.
PostedFeb 11, 2012 16:24 UTC (Sat) by arnd (subscriber, #8866) [Link]
PostedFeb 22, 2012 1:44 UTC (Wed) by zengtm (subscriber, #74989) [Link]
PostedMay 19, 2013 6:54 UTC (Sun) by abai (guest, #91041) [Link]
The AndroidION memory allocator
PostedFeb 29, 2012 20:43 UTC (Wed) by cliveb (guest, #83236) [Link]
Howcan I get a physical address for the allocated memory from user space?
The AndroidION memory allocator
PostedMar 14, 2012 17:01 UTC (Wed) by zengtm (subscriber, #74989) [Link]
Youcannot get physical address from user space: not secure if you think about it.
The AndroidION memory allocator
PostedJan 4, 2013 13:29 UTC (Fri) by vkkashyap (guest, #88623) [Link]