The Android ION memory allocator

本文深入探讨了Android ION内存分配器的功能、接口及其与用户空间和内核驱动的交互方式,包括如何在用户空间中使用ION、内核中ION支持的多客户端和缓冲数据库、与DMA缓冲共享框架的比较,以及ION与DMABUF的整合前景。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

The Android ION memory allocator

February8, 2012

This article was contributed by Thomas M. Zeng

Back in December 2011, LWN reviewedthe list of Android kernel patches inthe linux-next staging directory. The merging of these drivers, one of which isa memory allocator called PMEM, holds the promise that the mainline kernelrelease can one day boot an Android user space. Since then, it has become clear that PMEM is considered obsolete andwill be replaced by the ION memory manager

. ION is a generalized memory manager that Google introduced in theAndroid 4.0 ICS (Ice Cream Sandwich) release to address the issue of fragmentedmemory management interfaces across different Android devices. There are atleast three, probably more, PMEM-like interfaces. On Android devices usingNVIDIA Tegra, there is "NVMAP"; on Android devices using TI OMAP,there is "CMEM"; and on Android devices usingQualcomm MSM, there is "PMEM" .All three SoC vendors are in the process of switching to ION.

This article takes a look at ION, summarizing its interfaces touser space and to kernel-space drivers. Besides being a memory pool manager,ION also enables its clients to share buffers, hence it treads the same groundas the DMAbuffer sharing framework from Linaro (DMABUF). This article will end with acomparison of the two buffer sharing schemes.

IONheaps

Like its PMEM-like predecessors, ION manages one or more memory pools,some of which are set aside at boot time to combat fragmentation or to servespecial hardware needs. GPUs, display controllers, and cameras are some of thehardware blocks that may have special memory requirements. ION presents itsmemory pools as ION heaps. Each type of Android device can be provisioned witha different set of ION heaps according to the memory requirements of thedevice. The provider of an ION heap must implement the following set ofcallbacks:

   struct ion_heap_ops {
        int (*allocate) (struct ion_heap *heap,
                        struct ion_buffer *buffer, unsigned long len,
                        unsigned long align, unsigned long flags);
        void (*free) (struct ion_buffer *buffer);
        int (*phys) (struct ion_heap *heap, struct ion_buffer *buffer,
                    ion_phys_addr_t *addr, size_t *len);
        struct scatterlist *(*map_dma) (struct ion_heap *heap,
                        struct ion_buffer *buffer);
        void (*unmap_dma) (struct ion_heap *heap, 
                 struct ion_buffer *buffer);
        void * (*map_kernel) (struct ion_heap *heap, 
                 struct ion_buffer *buffer);
        void (*unmap_kernel) (struct ion_heap *heap, 
                 struct ion_buffer *buffer);
        int (*map_user) (struct ion_heap *heap, struct ion_buffer *buffer,
                        struct vm_area_struct *vma);
   };

Briefly, allocate() and free() obtain or release an ion_buffer object from the heap. A call to phys() will return the physical address and length of the buffer, butonly for physically-contiguous buffers. If the heap does not provide physicallycontiguous buffers, it does not have to provide this callback. Here ion_phys_addr_t is a typedef of unsigned long, and will, someday, be replaced by phys_addr_t in include/linux/types.h. The map_dma() andunmap_dma() callbacks cause the buffer to be prepared (or unprepared) for DMA.The map_kernel() and unmap_kernel() callbacks map (or unmap) the physical memory into the kernelvirtual address space. A call to map_user() will map the memory to user space. There is no unmap_user() because the mapping is represented as a file descriptor in userspace. The closing of that file descriptor will cause the memory to be unmappedfrom the calling process.

The default ION driver (which can be cloned from here) offers three heaps as listed below:

   ION_HEAP_TYPE_SYSTEM:        memory allocated via vmalloc_user().
   ION_HEAP_TYPE_SYSTEM_CONTIG: memory allocated via kzalloc.
   ION_HEAP_TYPE_CARVEOUT:     carveout memory is physically contiguous and set aside at boot.

Developers may choose to add more ION heaps. For example, this NVIDIA patch was submitted to addION_HEAP_TYPE_IOMMU for hardware blocks equipped with an IOMMU.

Using ION fromuser space

Typically, user space device access libraries will use ION toallocate large contiguous media buffers. For example, the still camera librarymay allocate a capture buffer to be used by the camera device. Once the bufferis fully populated with video data, the library can pass the buffer to thekernel to be processed by a JPEG encoder hardware block.

A user space C/C++ program must have been granted access to the /dev/ion device before it can allocatememory from ION. A call to open("/dev/ion", O_RDONLY) returns a file descriptor as a handle representing an ION client.Yes, one can allocate writable memory with an O_RDONLY open. There can be no more than one client per user process. Toallocate a buffer, the client needs to fill in all the fields except the handle field in this data structure:

   struct ion_allocation_data {
        size_t len;
        size_t align;
        unsigned int flags;
        struct ion_handle *handle;
   }

The handle field is the output parameter, while the first three fieldsspecify the alignment, length and flags as input parameters. The flags field is a bit mask indicating one or more ION heaps to allocatefrom, with the fallback ordered according to which ION heap was first added viacalls toion_device_add_heap() during boot. In the default implementation, ION_HEAP_TYPE_CARVEOUT is added before ION_HEAP_TYPE_CONTIG. The flags of ION_HEAP_TYPE_CONTIG |ION_HEAP_TYPE_CARVEOUT indicate the intention toallocate from ION_HEAP_TYPE_CARVEOUT with fallback to ION_HEAP_TYPE_CONTIG.

User-space clients interact with ION using the ioctl() system call interface. Toallocate a buffer, the client makes this call:

   int ioctl(int client_fd, ION_IOC_ALLOC, struct ion_allocation_data *allocation_data)

This call returns a buffer represented by ion_handle which is not a CPU-accessible buffer pointer. The handle can onlybe used to obtain a file descriptor for buffer sharing as follows:

   int ioctl(int client_fd, ION_IOC_SHARE, struct ion_fd_data *fd_data);

Here client_fd is the file descriptor corresponding to /dev/ion, and fd_data is a data structure with an input handle field and an output fd field, as defined below:

   struct ion_fd_data {
        struct ion_handle *handle;
        int fd;
   }

The fd field is the file descriptor that can be passed around forsharing. On Android devices the BINDER IPC mechanism may be used to send fd to another process for sharing. To obtain the shared buffer, thesecond user process must obtain a client handle first via the open("/dev/ion", O_RDONLY)system call. ION tracks its user space clients by the PID of theprocess (specifically, the PID of the thread that is the "groupleader" in the process). Repeating the open("/dev/ion",O_RDONLY) call in the same process will get back another file descriptorcorresponding to the same client structure in the kernel.

To free the buffer, the second client needs to undo the effect of mmap() with a call to munmap(), and the first client needs toclose the file descriptor it obtained via ION_IOC_SHARE, andcall ION_IOC_FREE as follows:

     int ioctl(int client_fd, ION_IOC_FREE, struct ion_handle_data *handle_data);

Here ion_handle_data holds the handle as shown below:

     struct ion_handle_data {
             struct ion_handle *handle;
     }

The ION_IOC_FREE command causes the handle's reference counter to be decremented byone. When this reference counter reaches zero, the ion_handle object gets destroyed and the affected ION bookkeeping datastructure is updated.

User processes can also share ION buffers with a kernel driver, asexplained in the next section.

Sharing IONbuffers in the kernel

In the kernel, ION supports multiple clients, one for each driverthat uses the ION functionality. A kernel driver calls the following function toobtain an ION client handle:

   struct ion_client *ion_client_create(struct ion_device *dev, 
                   unsigned int heap_mask, const char *debug_name)

The first argument, dev, is theglobal ION device associated with /dev/ion; why aglobal device is needed, and why it must be passed as a parameter, is notentirely clear. The second argument, heap_mask,selects one or more ION heaps in the same way as the ion_allocation_data. The flags field was covered in the previous section. For smart phone usecases involving multimedia middleware, the user process typically allocates thebuffer from ION, obtains a file descriptor using the ION_IOC_SHARE command, then passes the filedesciptor to a kernel driver. The kernel driver calls ion_import_fd() which converts the file descriptor to an ion_handle object, as shown below:

    struct ion_handle *ion_import_fd(struct ion_client *client, int fd_from_user);

The ion_handle object is the driver's client-local reference to the sharedbuffer. The ion_import_fd() call looks up the physical address of the buffer to see whetherthe client has obtained a handle to the same buffer before, and if it has, thiscall simply increments the reference counter of the existing handle.

Some hardware blocks can only operate on physically-contiguousbuffers with physical addresses, so affected drivers need to convert ion_handle to a physical buffer via thiscall:

   int ion_phys(struct ion_client *client, struct ion_handle *handle,
               ion_phys_addr_t *addr, size_t *len)

Needless to say, if the buffer is not physically contiguous, thiscall will fail.

When handling calls from a client, ION always validates the inputfile descriptor, client and handle arguments. For example, when importing afile descriptor, ION ensures the file descriptor was indeed created by an ION_IOC_SHARE command. When ion_phys() is called, ION validates whetherthe buffer handle belongs to the list of handles the client is allowed toaccess, and returns error if the handle is not on the list. This validationmechanism reduces the likelihood of unwanted accesses and inadvertent resourceleaks.

ION provides debug visibility through debugfs. It organizes debug informationunder /sys/kernel/debug/ion, with bookkeeping information in stored files associated withheaps and clients identified by symbolic names or PIDs.

Comparing IONand DMABUF

ION and DMABUF share some common concepts. The dma_buf concept

 is similar to ion_buffer, while dma_buf_attachment serves a similarpurpose as ion_handle. Both ION and DMABUF use anonymous file descriptors as the objects thatcan be passed around to provide reference-counted access to shared buffers. Onthe other hand, ION focuses on allocating and freeing memory from provisionedmemory pools in a manner that can be shared and tracked, while DMABUF focusesmore on buffer importing, exporting and synchronization in a manner that isconsistent with buffer sharing solutions on non-ARM architectures.

Thefollowing table presents a feature comparison between ION and DMABUF:

Feature

ION

DMABUF

Memory Manager Role

ION replaces PMEM as the manager of provisioned memory pools. The list of ION heaps can be extended per device.

DMABUF is a buffer sharing framework, designed to integrate with the memory allocators in DMA mapping frameworks, like the work-in-progress DMA-contiguous allocator, also known as the Contiguous Memory Allocator(CMA). DMABUF exporters have the option to implement custom allocators.

User Space Access Control

ION offers the /dev/ion interface for user-space programs to allocate and share buffers. Any user program with ION access can cripple the system by depleting the ION heaps. Android checks user and group IDs to block unauthorized access to ION heaps.

DMABUF offers only kernel APIs. Access control is a function of the permissions on the devices using the DMABUF feature.

Global Client and Buffer Database

ION contains a device driver associated with /dev/ion. The device structure contains a database that tracks the allocated ION buffers, handles and file descriptors, all grouped by user clients and kernel clients. ION validates all client calls according to the rules of the database. For example, there is a rule that a client cannot have two handles to the same buffer.

The DMA debug facility implements a global hashtable,dma_entry_hash, to track DMA buffers, but only when the kernel was built with the CONFIG_DMA_API_DEBUG option.

Cross-architecture Usage

ION usage today is limited to architectures that run the Android kernel.

DMABUF usage is cross-architecture. The DMA mapping redesign preparation patchset modified the DMA mapping code in 9 architectures besides the ARM architecture.

Buffer Synchronization

ION considers buffer synchronization to be an orthogonal problem.

DMABUF provides a pair of APIs for synchronization. The buffer-user calls dma_buf_map_attachment() whenever it wants to use the buffer for DMA . Once the DMA for the current buffer-user is over, it signals 'end-of-DMA' to the exporter via a call to dma_buf_unmap_attachment() .

Delayed Buffer Allocation

ION allocates the physical memory before the buffer is shared.

DMABUF can defer the allocation until the first call todma_buf_map_attachment(). The exporter of DMA buffer has the opportunity to scan all client attachments, collate their buffer constraints, then choose the appropriate backing storage.

ION and DMABUF can be separately integrated into multimediaapplications written using the Video4Linux2 API. In the case of ION,these multimedia programs tend to use PMEM now on Android devices, so switchingto ION from PMEM should have a relatively small impact.

IntegratingDMABUF into Video4Linux2 is another story. It has taken ten patches to integratethe videobuf2 mechanismwith DMABUF; in fairness, many of these revisions were the result of changes toDMABUF as that interface stabilized. The effort should pay dividends in thelong run because the DMABUF-based sharing mechanism is designed with DMAmapping hooks for CMA and IOMMU. CMA and IOMMU hold the promise to reduce theamount of carveout memory that it takes to build an Android smart phone. In this email, Andrew Morton was urging thecompletion of the patch review process so that CMA can get through the 3.4merge window.

Even though ION and DMABUF serve similar purposes, the twoare not mutually exclusive. The Linaro Unified Memory Management team hasstarted to integrate CMA into ION. To reach thestate where a release of the mainline kernel can boot the Android user space,the /dev/ion interface touser space must obviously be preserved. In the kernel though, ION drivers maybe able to use some of the DMABUF APIs to hook into CMA and IOMMU to take advantageof the capabilities offered by those subsystems. Conversely, DMABUF might beable to leverage ION to present a unified interface to user space, especiallyto the Android user space. DMABUF may also benefit from adopting some of theION heap debugging features in order to become more developer friendly. Thusfar, many signs indicate that Linaro, Google, and the kernel community areworking together to bring the combined strength of ION and DMABUF to themainline kernel.


 

(Log in to postcomments)

The Android ION memory allocator

Posted Feb 9, 2012 9:31 UTC (Thu) by blackwood (subscriber,#44174) [Link]

Smallcorrection on the dma_buf attach/detach functions: They're just used toreconfigure the pipeline - the exporter is allowed to delay the allocation upto map time and needs to because only by then all devices will be attached.Similarly it's map/unmap that should enforce cache coherency andsynchronization and not attach/detach. We'll likely add an extension to allowstreaming dma with persistent device mappings.

Alsonote that map/unmap doesn't synchronize hw access in the sense of gl syncobjects. Imo that's an orthogonal issue to buffer sharing and dma bufferallocation. But something we might need to support in the kernel, too, becausesome SoC have hw-based semaphores and mailboxes to sync up different blockswithout the cpu being woken up.

The AndroidION memory allocator

PostedFeb 22, 2012 1:42 UTC (Wed) by zengtm (subscriber, #74989) [Link]

Inthe article it does say that map/unmap is the place to handle cache coherencyand buffer synchronization. Did I have wording to indicate attach/detach is forsynchronization?

AndI agree that the persistent device mappings seem to be the more common smartphone use cases. 

Integrationwith dma_buf

PostedFeb 11, 2012 16:24 UTC (Sat) by arnd (subscriber, #8866) [Link]

Itshould not be too hard to modify the ion code so it becomes a provider ofdma_buf file descriptors and integrates into that framework.

Whenwe discussed the design for dma_buf, it was intentionally made possible to havearbitrary subsystems provide interfaces that hand out dma_buf file descriptorsto user space, although the focus so far was on having drm provide descriptorsfor buffers that it already manages. If I understand the article correctly, thedesign of ion is that you always have to know in advance how to allocate yourbuffer through the ion ioctl but then can pass it into any driver using it.

Weshould probably look into unifying the in-kernel interfaces for the two,because they are already very similar. and make ION_IOC_SHARE return a dma_bufdescriptor that can be passed into any dma_buf enabled driver. Theion_import_fd function can then be replaced with the more abstract dma_bufequivalent.

Integrationwith dma_buf

PostedFeb 22, 2012 1:44 UTC (Wed) by zengtm (subscriber, #74989) [Link]

"thedesign of ion is that you always have to know in advance how to allocate yourbuffer": the answer is yes, based on the ION implementations that I havethe chance to study in the current Android 4.0.x release. 

Integrationwith dma_buf

PostedMay 19, 2013 6:54 UTC (Sun) by abai (guest, #91041) [Link]

Ifuser space change the ion memory content, how to flush cache to memery in userspace?
I see ION_IOC_SYNC only used for kernel space,it cannot flush user cache, right?

The AndroidION memory allocator

PostedFeb 29, 2012 20:43 UTC (Wed) by cliveb (guest, #83236) [Link]

Howcan I get a physical address for the allocated memory from user space?

The AndroidION memory allocator

PostedMar 14, 2012 17:01 UTC (Wed) by zengtm (subscriber, #74989) [Link]

Youcannot get physical address from user space: not secure if you think about it.

The AndroidION memory allocator

PostedJan 4, 2013 13:29 UTC (Fri) by vkkashyap (guest, #88623) [Link]

Thanksfor nice article.

-What type of security risks do you foresee?
- Any other reason(s) you see for limitingphysical addresses api to kernel space?

<think>我们正在处理用户关于Linux内核配置中DMAContiguousMemoryAllocator(CMA)的问题。用户希望了解如何启用或解决CMA缺失的问题。根据引用内容,我们可以总结以下几点关键信息:1.CMA(连续内存分配器)用于为设备驱动分配大块连续物理内存,当设备驱动不使用时,其他内核模块可以借用(要求申请可移动类型的页),当设备驱动需要时,再迁移已分配的页以形成连续内存块[^4]。2.引用[1]提到,在没有CMA或系统不允许阻塞的情况下,可以通过`__get_free_pages()`从DMA/普通区域(低端内存)分配内存,但这仅限于32位地址空间,且已经在低端内存区域由内核在启动时映射。3.引用[3]提到了一些解决方案,包括:汇总所有CMA需求并分配一个统一的CMA区域(对于单个CMA用户系统无效);调整`ARCH_FORCE_MAX_ORDER`配置参数以减少内存浪费(但会影响大页分配);对于较小的预留内存,CMA可以直接从伙伴分配器分配内存。因此,针对用户的问题,我们需要提供在内核配置中启用CMA以及解决相关问题的步骤和注意事项。###步骤指南####1.启用CMA在配置内核时,需要确保以下配置选项被启用:-`CONFIG_CMA`:启用连续内存分配器(CMA)功能。-`CONFIG_DMA_CMA`:如果用于DMA,则启用此选项(依赖`CONFIG_CMA`)。-其他与设备相关的CMA配置(如`CONFIG_CMA_SIZE_MBYTES`设置CMA区域大小)。**配置方法**:-使用`makemenuconfig`(或类似配置工具)进入内核配置界面。-导航至以下路径:```DeviceDrivers->GenericDriverOptions->ContiguousMemoryAllocator(CONFIG_CMA)[选中]DMAContiguousMemoryAllocator(CONFIG_DMA_CMA)[选中]```-在`CONFIG_CMA`下可以设置CMA区域的大小(例如`CONFIG_CMA_SIZE_MBYTES`,默认值可能为16MB,根据需求调整)。####2.解决CMA缺失问题如果系统配置中没有启用CMA,或者CMA区域不足,可以尝试以下方法:-**汇总CMA需求并分配统一区域**:在设备树(DeviceTree)或内核启动参数中指定一个足够大的CMA区域。例如,在内核启动参数中添加:```cma=128M```这将保留128MB的CMA区域。-**调整页块大小以减少浪费**:通过设置`ARCH_FORCE_MAX_ORDER`为较小值(如7),可以减小伙伴系统中最大可分配页块的大小,从而减少内存浪费。但注意这会影响大页分配(如透明大页THP)。在配置内核时,可以通过以下方式设置:```Processortypeandfeatures->Maximumzoneorder(ARCH_FORCE_MAX_ORDER)[设置为7]```-**对于小内存分配直接从伙伴系统获取**:如果设备驱动需要的内存块较小,可以考虑修改驱动代码,使其在CMA分配失败时回退到伙伴分配器(如使用`dma_alloc_coherent()`,它会尝试多种分配方式包括伙伴系统)。但此方法需要驱动支持,且不是通用解决方案。####3.验证CMA配置-系统启动后,检查`/proc/meminfo`:```cat/proc/meminfo|grepCma```输出应显示CMA区域的大小和已使用情况。-检查内核启动日志(`dmesg`)中是否有CMA初始化的信息,例如:```Reserved:128MB(CMA:128MB)```###注意事项-启用CMA可能会减少系统可用内存(因为CMA区域被预留),请根据实际需求设置大小。-调整`ARCH_FORCE_MAX_ORDER`可能影响系统性能,需谨慎评估。-如果设备驱动要求物理地址连续且较大,应确保CMA区域足够大,避免分配失败。###示例配置假设我们需要为某个DMA设备预留64MB的CMA区域,可以这样操作:1.内核配置中启用`CONFIG_CMA`和`CONFIG_DMA_CMA`,并设置`CONFIG_CMA_SIZE_MBYTES=64`。2.或者在内核启动参数中添加`cma=64M`。###引用说明根据引用内容,CMA的启用和配置涉及内核编译选项和启动参数[^1][^3][^4]。同时,引用[3]提供了几种解决CMA问题的思路,包括调整内核配置参数[^3]。§§相关问题§§1.如何在内核启动参数中设置CMA区域大小?2.如何验证CMA区域是否成功启用?3.调整`ARCH_FORCE_MAX_ORDER`对系统有哪些影响?4.设备驱动如何使用CMA分配的连续内存?</think>### Linux内核中启用DMA连续内存分配器(CMA)的配置指南 #### 1. **启用CMA内核配置选项** 在`make menuconfig`中配置: ```bash Device Drivers ---> Generic Driver Options ---> [*] Contiguous Memory Allocator (CMA) # 核心CMA支持 [*] DMA Contiguous Memory Allocator # DMA专用CMA (128) CMA size in MegaBytes # 设置CMA区域大小(单位MB) ``` **说明**: - `CONFIG_CMA`是基础支持,必须启用 - `CONFIG_DMA_CMA`提供DMA专用分配接口 - 大小建议设为设备DMA需求的1.5-2倍[^4] #### 2. **设备树(Device Tree)配置** 在设备树中为特定设备分配CMA区域: ```dts reserved-memory { #address-cells = <2>; #size-cells = <2>; ranges; mydevice_cma: linux,cma@90000000 { compatible = "shared-dma-pool"; reusable; size = <0x10000000>; // 256MB alignment = <0x2000>; linux,cma-default; }; }; mydevice { memory-region = <&mydevice_cma>; }; ``` #### 3. **解决CMA分配失败的方案** 当遇到"CMA not available"错误时: **方案1:调整内存区块大小** ```bash Processor type and features ---> (7) Maximum zone order (ARCH_FORCE_MAX_ORDER) # 默认值通常为11 ``` 减小此值可降低内存碎片但影响大页分配[^3] **方案2:预留统一CMA区域** 在内核启动参数添加: ```bash cma=256M@32M // 在32MB地址处保留256MB区域 ``` **方案3:伙伴分配器回退** 修改驱动代码,当CMA失败时回退到伙伴系统: ```c if (dma_set_coherent_mask(dev, DMA_BIT_MASK(64))) { dev_warn(dev, "CMA failed, using buddy allocator"); ptr = __get_free_pages(GFP_KERNEL, order); } ``` #### 4. **验证CMA状态** ```bash # 查看CMA区域 cat /proc/meminfo | grep Cma # CmaTotal: 262144 kB # CmaFree: 131072 kB # 检查设备DMA映射 dmesg | grep -i "dma: allocated" # [ 1.234567] mydevice dma: allocated 64MB for DMA ``` #### 5. **关键注意事项** 1. CMA仅分配**可移动页面**,确保内核配置启用: ```bash Memory Management options ---> [*] Allow for memory compaction [*] Contiguous Memory Allocator ``` 2. 32位系统需确保CMA在低端内存区(DMA zone): ```c dma_declare_contiguous(dev, size, PHYS_OFFSET, 0); ``` 3. 避免SWIOTLB冲突: ```bash Boot options ---> [ ] Software IO TLB (SWIOTLB) ``` > CMA通过"预留-借用-迁移"机制平衡设备独占和系统内存使用[^4]。配置时应综合考虑设备DMA需求(`dma_mask`)和系统内存压力,建议优先使用设备树声明专用区域[^1][^2]。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值