转自:https://anonymalias.github.io/2016/01/14/boost_ipc_pack/
Boost共享内存/内存映射文件构建容器的封装
文章目录
本人github主站:https://anonymalias.github.io
本人csdn主站:http://blog.youkuaiyun.com/anonymalias
Boost Interprocess组件提供了通过了托管内存段(managed_mapped_file & managed_shared_memory)在共享内存和内存映射文件上进行复杂数据结构构建的功能。
通过boost.ipc构造简单的POD纯数据结构, 非常的简单
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | #include <iostream> #include <cstdio> #include <boost/interprocess/managed_mapped_file.hpp> namespace bipc = ::boost::interprocess; typedef bipc::managed_mapped_file managed_mapped_file_t; typedef bipc::managed_mapped_file::segment_manager mapped_segment_manager_t; struct Msg { uint32_t id; char name[128]; uint32_t age; }; int main() { managed_mapped_file_t obj_mapped_file(bipc::open_or_create, "./msg_map.mmap", 1024*1024); Msg * p_msg = obj_mapped_file.find_or_construct<Msg>("msg_map")(); if(NULL == p_msg) { std::cerr<<"construct msg failed"<<std::endl; return -1; } std::cout<<p_msg->id<<std::endl; std::cout<<p_msg->name<<std::endl; std::cout<<p_msg->age<<std::endl; p_msg->id = p_msg->id + 1; strcpy(p_msg->name, "hello world!"); p_msg->age = p_msg->age + 1; return 0; } |
在项目过程中,更多的时候是需要在共享内存/内存映射文件上构建复杂的容器。我们知道一般
- 容器的分配器:容器的构造是直接在进程的地址空间(堆)上进行的,容器的分配器allocator都是默认构造的,所以其分配器可认为是无状态的(stateless)。
- boost.ipc的分配器:需要从具体的系统内核内存片段上进行内存的分配,而不是进程的内存资源,所以boost.ipc的分配器是有状态的(stateful), 言外之意,需要告知boost.ipc的分配器,在哪里进行内存分配,必须传入共享内存,或是内存映射文件。
所以要在boost.ipc上构造容器,需要将boost.ipc的分配器作为容器的allocator传入容器,让容器知道在ipc上进行内存分配和对象构造。Boost ipc的allocator内的get_segment_manager()可以返回allocator构造时传入的内存片段管理器。(注意:托管内存段(managed_mapped_file & managed_shared_memory)的内存片段管理器segment_manager 本质是一个allocator,在文章http://blog.youkuaiyun.com/anonymalias/article/details/50496563有详解)。
所以现在在boost.ipc上构建一个map,需要有两步操作(这里已mapped file举例):
- 构建mapped_file对象。
- 通过mapped_file对象,构建map,在构建map的时候需要传入mapped_file的分配器。
代码如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | #include <iostream> #include <boost/interprocess/managed_mapped_file.hpp> #include <boost/interprocess/containers/map.hpp> #include <boost/interprocess/containers/string.hpp> #include <boost/interprocess/allocators/node_allocator.hpp> #include <boost/container/string.hpp> namespace bipc = ::boost::interprocess; typedef bipc::managed_mapped_file managed_mapped_file_t; typedef bipc::managed_mapped_file::segment_manager mapped_segment_manager_t; struct Msg { uint32_t id; bipc::string name; uint32_t age; }; typedef std::pair<const uint32_t, Msg> pair_t; typedef bipc::node_allocator<pair_t, mapped_segment_manager_t> allocator_t; typedef std::less<uint32_t> less_t; typedef bipc::map<uint32_t, Msg, less_t, allocator_t> msg_map_t; typedef msg_map_t::iterator map_iter_t; int main() { managed_mapped_file_t obj_mapped_file(bipc::open_or_create, "./msg_map_boost.mmap", 1024*1024); msg_map_t *p_msg_map = obj_mapped_file.find_or_construct<msg_map_t>("msg_map")(less_t(), \ obj_mapped_file.get_segment_manager()); if(NULL == p_msg_map) { std::cerr<<"construct msg_map failed"<<std::endl; return -1; } for(int i = 0; i < 10; ++i) { map_iter_t itr = p_msg_map->find(i); if(itr == p_msg_map->end()) { std::cout<<"not find:"<<i<<" insert:"<<i<<std::endl; Msg msg = {i, "alias", 100+i}; p_msg_map->insert(std::pair<uint32_t, Msg>(i, msg)); } else { std::cout<<"find:"<<i<<" data:"<<itr->second.name<<" "<<\ itr->second.age<<std::endl; } } return 0; } |
为了代码的可读性,需要定义一堆的typedef来指定待构建map的allocator的构造传参是一个boost ipc的segment_manager。当map的value不是POD,而是嵌套其他容器是,你将会看到这样的定义:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | namespace bipc = ::boost::interprocess; typedef bipc::managed_mapped_file managed_mapped_file_t; typedef bipc::managed_mapped_file::segment_manager mapped_segment_manager_t; typedef bipc::node_allocator<float, mapped_segment_manager_t> vec_allocator_t; typedef boost::container::vector<float, vec_allocator_t> vector_t; struct Msg { Msg(const vec_allocator_t &vec_alloc) : score(vec_alloc){} uint32_t id; bipc::string name; uint32_t age; vector_t score; }; typedef std::pair<const uint32_t, Msg> pair_t; typedef bipc::node_allocator<pair_t, mapped_segment_manager_t> allocator_t; typedef std::less<uint32_t> less_t; typedef bipc::map<uint32_t, Msg, less_t, allocator_t> msg_map_t; typedef msg_map_t::iterator map_iter_t; |
程序员的天性(C++程序员。。。), tigerlan我的导师为了不愿意造轮子,对在boost ipc上进行容器构造的逻辑进行了封装:那我写这篇文章还有毛意义,又不是我造的轮子,其实使用过程中,还是发现一些问题,算是对自己的一个提升吧,写这篇文章的本意也是想让自己记录一下过程中遇到的问题和细节,已经和大家分享的过程。
1.首先是对boost ipc的托管内存段进行封装,如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | #include <boost/interprocess/managed_mapped_file.hpp> #include <boost/interprocess/managed_shared_memory.hpp> #include <boost/interprocess/allocators/allocator.hpp> #include <functional> #include <utility> #include <string> typedef boost::interprocess::managed_mapped_file mapped_file_t; typedef boost::interprocess::managed_shared_memory shared_memory_t; template <typename segment> class Segment { public: typedef segment segment_t ; typedef typename segment_t::segment_manager segment_manager_t ; public: Segment(const char * name, size_t size): m_name(name),m_size(size), m_segment(boost::interprocess::open_or_create, m_name.c_str(), m_size ) { } segment_t & operator () () { return m_segment; } /* ...省略 */ private: std::string m_name; size_t m_size; segment_t m_segment; }; typedef Segment< mapped_file_t > MapedFile; typedef Segment< shared_memory_t > SharedMemory; |
2.其次是对要构建的各个容器进行封装,这里举例map的封装:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | #include <boost/interprocess/allocators/node_allocator.hpp> #include <boost/interprocess/containers/map.hpp> #include <boost/interprocess/offset_ptr.hpp> #include <functional> #include <utility> #include "segment.hpp" namespace bip = boost::interprocess; template <typename key_t,typename data_t, typename segment_t> class SegmentMap { typedef typename segment_t::segment_manager_t segment_manager_t; typedef typename std::pair<const key_t, data_t> pair_t; typedef typename std::less<key_t> less_t; typedef typename bip::node_allocator <pair_t , segment_manager_t> allocator_t; public: typedef typename bip::map<key_t, data_t, less_t ,allocator_t> map_t; typedef typename map_t::iterator iterator; public: SegmentMap(const char * name, segment_t & segment ) : m_segment( segment ), m_allocator( m_segment().get_segment_manager() ) { m_segment_map = m_segment().template find_or_construct<map_t> (name) (less_t(),m_allocator); assert(m_segment_map != NULL); } SegmentMap(segment_t & segment) : m_segment( segment ), m_allocator( m_segment().get_segment_manager() ) { m_segment_map = m_segment().template find_or_construct< map_t > ( bip::anonymous_instance ) (less_t(), m_allocator); assert(m_segment_map != NULL); } ~SegmentMap(){}; map_t & operator () () { return *m_segment_map; } /* ...省略 */ private: segment_t & m_segment; allocator_t m_allocator; bip::offset_ptr<map_t> m_segment_map; }; |
3.在托管内存段上进行容器的构建如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | #include "iostream" #include "segment.hpp" #include "segment_map.hpp" #include "segment_list.hpp" typedef MapedFile _Memory; _Memory * g_memory = NULL; struct Company { typedef SegmentList<uint32_t, _Memory> leader_list_t; typedef SegmentList<uint32_t, _Memory> colleague_list_t; leader_list_t leader_list; colleague_list_t colleague_list; Company() : leader_list(*g_memory), colleague_list(*g_memory){} }; typedef SegmentMap<uint32_t, Company, _Memory> company_map_t; int main() { g_memory = new _Memory("test.mmap", 1024 * 1024); company_map_t * p_company_map = new company_map_t("test2", *g_memory); company_map_t::iterator itr; for(int i = 0; i < 10; ++i) { itr = (*p_company_map)().find(i); if(itr == (*p_company_map)().end()) { std::cout<<"not find:"<<i; Company &obj = (*p_company_map)[i]; obj.leader_list().push_back(i + 100); std::cout<<"insert:"<<i + 100<<std::endl; } else { std::cout<<"find:"<<i; for(Company::leader_list_t::iterator list_itr = itr->second.leader_list().begin(); list_itr != itr->second.leader_list().end(); ++list_itr) { std::cout<<" value:"<<*list_itr; } std::cout<<std::endl; } } } |
可以看出利用封装过后的Segment和SegmentList, SegmentMap,进行容器的构建是很方便的一件事情,需要注意的是map的value:struct Company的构造需要传入boost ipc的分配器,以便内部的list对象进行构造。所已需要Company的默认构造函数,无参,且参数初始化列表传入全局的已初始化的托管内存段对象,否则不能map的[]操作符。