Membase Key-Concept

Membase Server is a high-performance, highly-available, distributed, key-value database and caching system that is 100% compatible with memcached. It is optimized for storing the data behind highly interactive web applications and is a scale-out ("elastic") technology at its core.

• A Membase cluster is made up of one or more servers/nodes.  Within this cluster, an administrator can define a number of buckets that are used to provide the elastic data management services.
• There are a variety of ways to interract and administer a Membase cluster:
  • The Membase Web Console
  • The Membase Command Line Interface
  • The Membase REST API

Dynamic Scalability

• Attachments:1
• Added by Perry Krug, last edited by dasn on May 11, 2011  (view change)

Scaling out a Membase Server cluster horizontally is as simple as adding a Membase Server node and using the Membase Web Console to join a cluster. With Membase Server, clusters can grow dynamically without impacting the system.

Membase Server has a Cluster Manager that provides built-in cluster management and a simple REST interface. Client libraries and web applications that utilize the interface can determine the health and state of a Membase Server cluster. There is also a built-in proxy running on each server that handles the redirection of client traffic. This allows a Membase Server cluster to scale dynamically as servers are added to or removed from the cluster - while web applications continue to operate with no disruption.Smart clients coordinate directly with the cluster to achieve even better performance.

Definitions

• A node is an instance of Membase Server installed on a single host machine. The underlying architecture is less important to this discussion, but one can equate a Membase node with a server. A server can be running inside of [EC2], VMware, on dedicated hardware, etc.
• A cluster is a logical collection of one or more server nodes running Membase Server. Clustering is the foundation of Membase's horizontally scalable Cluster Manager. By clustering server nodes, production deployments of Membase Server have the ability to increase storage capacity and improve performance and system availability.

Membase Cluster Manager

Membase's Cluster Manager is a Membase Server component running on each server and provides the following services:

• Clustering (划分cluster)and cluster management
• Server node administration (startup and shutdown)
• Server node monitoring
• Statistics gathering and aggregation(信息汇总）
• Run-time logging (日志)
• Multi-tenancy(多租户？）
• Security for administrative access and access to cache resources (安全控制)

Client applications access these services via theadmin port (8091) and data ports (11211 or 11210).  The cluster communicates internally on ports

Memory Quotas

• Added by Perry Krug, last edited by Matt Ingenthron on Mar 13, 2011  (view change)

• 1 Managing Disk and Memory for Nodes in the Cluster
• 2 Server Quotas
• 3 Bucket Quotas

Managing Disk and Memory for Nodes in the Cluster

Membase Server automatically manages storing the working set between disk and memory resources for nodes in a cluster. This allows an installation to have a working set that is larger than the available RAM in the nodes participating in the cluster. To keep throughput high and latency low, Membase Server will always keep metadata about all items in memory.

When configuring a Membase Server, a memory quota is set. Membase Server will automatically migrate items from memory to disk when the configured memory quota is reached.If those items are later accessed, they will be moved back into system memory. For efficiency purposes, these operations are performed on a regular basis in the background.

At the moment, there is no ability define a quota for the on-disk persistent storage.(不能决定硬盘配额，只指定存储位置)  It is up to the administrator to appropriately monitor the disk utilization and take action(只能靠第三方来维护硬盘空间） (either deleting data from Membase or adding capacity by upgrading the nodes themselves or adding more nodes)

Server Quotas

Each server node has a memory quota that defines the amount of system memory that is available to that server node on the host system. The first node in a cluster sets a memory quota that is subsequently inherited by all servers joining the cluster. The maximum memory quota set on the first server node must be less than or equal to80% of the total physical RAM on that node. A server cannot join a cluster if it has less physical RAM than 1.25x the RAM quota (the same maximum allocation of 80% of physical RAM to the cluster).

Server nodes do not have disk quotas. System administrators are responsible for monitoring free disk space on individual server nodes.(Server 应该有负载均衡和数据添加不进去的处理，这里只是说硬盘空间无法智能维护) Each server node in a cluster has its own storage path - the location on disk where data will be stored. Storage paths do not need to be uniform across all server nodes in a cluster. If a server that was a standalone cluster joins another cluster, the storage path for that server remains unchanged.

Bucket Quotas (对用户来说，他所需要的数据位于一个bucket中) (bucket是虚拟的，代表用户的一个数据集)

Memory quota allocation is identical to that of the current Membase Memcached Server product. A fixed amount of memory per node is allocated for use by a bucket. Adding or removing nodes will change the size of the bucket.

Bucket 用于数据管理服务 

Membase Data Buckets

Membase Server provides data management services using data buckets (or just buckets), which are isolated virtual containers for data.A bucket is a logical grouping of physical resourceswithin a cluster of Membase Servers.They can be used by multiple client applications across a cluster. Buckets provide a secure mechanism for organizing, managing, and analyzing data storage resources.

Membase Server provides the following types of buckets that can be created and [managed]. Membase Server collects and reports on run-time [statistics] by bucket type.

Bucket Type	Description
Membase	Provides highly-available and dynamically reconfigurable distributed data storage, providing persistence and replication services.
Memcached	Provides a distributed (scale-out), in-memory, key-value cache. Memcached buckets are designed to be used alongside relational database technology.

Capabilities of Membase-type Data Buckets

Capability	Description
Persistence	Data objects can be persisted asynchronously(异步,不是和内存同步) to hard-disk resources from memory to provide protection from server restarts or minor failures. Persistence properties are set at the bucket level.
Replication	A configurable number of replica servers can receive copies of all data objects in the Membase-type bucket. If the host machine fails, a replica server can be promoted to be the host server, providing continuous (HA) cluster operations via fail-over.
Rebalancing	Rebalancing enables load distribution across resources and dynamic addition or removal of buckets and servers in the cluster.

--

Multitenancy with Membase

• Attachments:1
• Added by Perry Krug, last edited by James Phillips on Oct 11, 2010  (view change)

Smart clients discover changes in the cluster using the Membase Management REST API.Buckets can be used to isolate individual applications to provide multi-tenancy, or to isolate data types in the cache to enhance performance and visibility. 

Membase Server allows you to use and mix different types of buckets (Membase and Memcached) as appropriate in your environment.Buckets of different types still share the same resource pool and cluster resources.Quotas for RAM and disk usage are configurable per bucketso that resource usage can be managed across the cluster.