不看死不瞑目的文档 ：Solaris container - Container Example

Combining the Resource Management and Zones features available in Solaris 10 OS, a system administrator can create Solaris Containers tailored for a specific use. Building on the examples in the previous chapters, assume the administrator wants to run the following workloads on a single SMP system:

l        The production sales database

l        The production marketing database

l        A development environment for the marketing database with multiple developers working on application development

l        A development environment for the sales database

l        System management

The following issues prevent successful consolidation onto a single system:

l        The databases are managed by two different DBA organizations that each have their own (conflicting) standards

l        The database systems use different naming services

l        The development systems use the same usernames, file system paths and Oracle SIDs as the production environment

l        The database instances should be guarateed a certain minimum and maximum amount of CPU capacity at all times

l        The production systems should get preferential treatment over the development systems

l        The sales database is the most important workload

l        Developers should not be able to monopolize the CPU resources on the development systems

l        The marketing department is willing to pay for a maximum of two CPUs

The problem is that the sales and marketing databases cannot co-exist on a single system because of different database administration standards and the use of different naming services. This can be overcome by using a separate zone for each workload. The issue of the development environments sharing naming with production can also be overcome with zones. Each zone has its own namespace for users, file systems, network ports and naming services.

The guarantee for minimum and maximum CPU capacity can be ensured by using Dynamic Resource Pools and the Fair Share Scheduler. Resource Pools provide hard upper and lower bounds at the granularity of a CPU. By creating a pool for the sales production database, a pool for the marketing database, and a pool for all other workloads, the production databases can be given guaranteed CPU capacity.

The demand for preferential treatment of the production systems can be implemented using the Fair Share Scheduler by assigning the production zones more zone.cpu-shares than development zones. When contention for CPU resources arises, the production zones are given more CPU resources than the other zones.

To prevent a developer from monopolizing the CPU resources in a development zone, each developer is assigned to a project with a certain amount of project.cpu-shares. The Fair Share Scheduler is used inside the development zones to distribute CPU resources according to the shares.

This leads to the following design:

l        The resouce pool sales with a processor set large of at least two CPUs and no upper bound on CPUs

l        Bound to this pool is a single zone sales, thus allowing exclusive access to the CPUs in the large processor set for the sales production database only

l        Inside the zone there is a single project ora_sales, used to limit the amount of shared memory

l        The pool uses the default Time Sharing (TS) scheduler since there is no need to arbitrate between zones in the pool or between projects in the zone

l        A resource pool called marketing with a processor set small of at least one and at most two CPUs

l        Bound to this pool are two zones, mkt for the marketing production database and mkt_dev for the development database This pool uses the Fair Share Scheduler (FSS) to arbitrate between the two zones bound to the pool and between projects in those zones

l        Inside each zone is a single project ora_mkt to limit shared memory for the Oracle instance

l        Each developer is assigned a unique project in the development zone with a suitable amount of CPU shares

l        The default resource pool pool_default with a processor set with at least one CPU

l        Bound to this pool are the global zone and the sales_dev zone for the sales database developers

l        This pool uses the FSS scheduler to arbitrate between the two zones bound to the pool, and between projects in those zones

l        Each developer is assigned a unique project in the development zone with a suitable amount of CPU shares Container Construction

Creating the Pools

The pool configuration built in Chapter 6 almost matches the design, and could be used as a basis to create the required pools. However, the pools are created from scratch in order to show all relevant steps in a single location.

1. Enable the Resource Pools facility and save the default configuration to the /etc/pooladm.conf file. (The default configuration constists of a processor set pset_default with all CPUs and a single pool pool_default.)

global # pooladm -e global # pooladm -s

2. Create the sales resource pool with TS as the scheduler and the large processor set with at least two CPUs.

global # poolcfg -c ‘create pset large (uint pset.min=2; uint pset.max=65536)’ global # poolcfg -c ‘create pool sales (string pool.scheduler=”TS”)’ global # poolcfg -c ‘associate pool sales (pset large)’

3. Create the marketing resource pool with FSS as the scheduler and the small processor set with between one and two CPUs.

global # poolcfg -c ‘create pset small (uint pset.min=1; uint pset.max=2)’ global # poolcfg -c ‘create pool marketing (string pool.scheduler=”FSS”)’ global # poolcfg -c ‘associate pool marketing (pset small)’

4. Set the scheduler for the default pool to the Fair Share Scheduler and instantiate the pool configuration just created:

global # poolcfg -c 'modify pool pool_default (string pool.scheduler="FSS")' global # pooladm -c global # poolcfg -dc info system blondie string system.comment int system.version 1 boolean system.bind-default true int system.poold.pid 29072 pool marketing int pool.sys_id 5 boolean pool.active true boolean pool.default false string pool.scheduler FSS int pool.importance 1 string pool.comment pset small pool sales int pool.sys_id 6 boolean pool.active true boolean pool.default false string pool.scheduler TS int pool.importance 1 string pool.comment pset large pool pool_default int pool.sys_id 0 boolean pool.active true boolean pool.default true string pool.scheduler FSS int pool.importance 1 string pool.comment pset pset_default pset large int pset.sys_id 1 boolean pset.default false uint pset.min 2 uint pset.max 65536 string pset.units population uint pset.load 0 uint pset.size 2 string pset.comment cpu int cpu.sys_id 3 string cpu.comment string cpu.status on-line cpu int cpu.sys_id 2 string cpu.comment string cpu.status on-line pset small int pset.sys_id 2 boolean pset.default false uint pset.min 1 uint pset.max 2 string pset.units population uint pset.load 0 uint pset.size 2 string pset.comment cpu int cpu.sys_id 1 string cpu.comment string cpu.status on-line cpu int cpu.sys_id 0 string cpu.comment string cpu.status on-line pset pset_default int pset.sys_id -1 boolean pset.default true uint pset.min 1 uint pset.max 65536 string pset.units population uint pset.load 17 uint pset.size 2 string pset.comment cpu int cpu.sys_id 11 string cpu.comment string cpu.status on-line cpu int cpu.sys_id 10 string cpu.comment string cpu.status on-line

Binding Zones to Pools

Currently all zones are bound to the default pool because the pool property of the created zones has not been set, resulting in the zones being bound to the pool with the

pool.default attribute set to true. Setting the zone's pool property to the name of a resource pool binds that zone and all of its processes to that pool when the zone is booted. Note that since the sales zone is bound to a resource pool with the normal TS scheduler, the zone.cpu-shares resource control is no longer applicable and is therefore removed from the zone configuration.

 

global # zonecfg -z sales set pool=sales global # zonecfg -z sales remove rctl name=zone.cpu-shares global # zonecfg -z mkt set pool=marketing

 

To bind a running zone to a pool without rebooting the zone, the poolbind(1M) command can be used. This dynamically rebinds the zone and its processes to a pool until the next zone boot. To have this change persist accross zone reboots, the zone's property should be set as shown above.

 

global # poolbind -q `pgrep -z sales -x init` 28399 pool_default global # poolbind -p sales -i zoneid sales global # poolbind -q `pgrep -z sales -x init` 28399 sales global # poolbind -p marketing -i zoneid mkt global # poolbind -q `pgrep -z mkt -x init` 28545 marketing

 

The poolbind -q `pgrep -z sales -x init` command is used to ascertain to which zone the current pool is bound by querying the binding of the init(1M) process of that zone. As can been seen, the sales zone was bound to the pool pool_default and is now bound to the sales pool.

Creating Development Zones

The development environments for both databases get their own zones, enabing them to use the same user names, projects and file system paths as the production environments. The development zone for the sales database, sales_dev, is bound to the default pool and shares the pool with all processes of the global zone. To prevent the sales_dev zone from monopolizing CPU resources, its zone.cpu-shares is set to the same value as that of the global zone. This gives both zones equal access to CPU resources. When the Fair Share Scheduler is active in a resource pool, it only looks at processes in that pool. The amount of shares for the sales_dev zone is only relevant in relation to those of the global zone.

 

global # zonecfg -z sales_dev sales_dev: No such zone configured Use 'create' to begin configuring a new zone. zonecfg:sales_dev> create zonecfg:sales_dev> set zonepath=/export/zones/sales_dev zonecfg:sales_dev> set autoboot=true zonecfg:sales_dev> set pool=pool_default zonecfg:sales_dev> add rctl zonecfg:sales_dev:rctl> set name=zone.cpu-shares zonecfg:sales_dev:rctl> add value (priv=privileged,limit=1,action=none) zonecfg:sales_dev:rctl> end [...] global # chmod 700 /export/zones/sales_dev global # zoneadm -z sales_dev install [...] global # zoneadm -z sales_dev boot

 

The development environment of the marketing database uses the same pool as the zone for the marketing production database. The Fair Share Scheduler is used to give preferential access to the production zone. By setting the zone.cpu-shares of the mkt zone to 50, and the zone.cpu-shares of the mkt_dev zone to 10, the production database is granted five times as much CPU resources as the development database.

 

global # zonecfg -z mkt 'select rctl name=zone.cpu-shares; set value=(priv=privileged,limit=50,action=none);end' global # zonecfg -z mkt_dev mkt_dev: No such zone configured Use 'create' to begin configuring a new zone. zonecfg:mkt_dev> create zonecfg:mkt_dev> set zonepath=/export/zones/mkt_dev zonecfg:mkt_dev> set autoboot=true zonecfg:mkt_dev> set pool=marketing zonecfg:mkt_dev> add rctl zonecfg:mkt_dev:rctl> set name=zone.cpu-shares zonecfg:mkt_dev:rctl> add value (priv=privileged,limit=10,action=none) zonecfg:mkt_dev:rctl> end [...] global # chmod 700 /export/zones/mkt_dev global # zoneadm -z mkt_dev install [...] global # zoneadm -z mkt_dev boot

 

The pool bindings for the zones can be verified using the poolbind -q pid command on every zone's init(1M) process.

 

global # poolbind -q `pgrep -z sales_dev -x init` 6718 pool_default global # poolbind -q `pgrep -z sales -x init` 28399 sales global # poolbind -q `pgrep -z mkt -x init` 28545 marketing global # poolbind -q `pgrep -z mkt_dev -x init` 6579 marketing global # poolbind -q `pgrep -z global -x init` 1 pool_default

Creating Development Users and Projects

Once all zones are created, it is time to create users and projects inside the development zones and set the appropriate resource controls to implement the design. The Fair Share Scheduler is used to prevent the developers from consuming the CPU resources. In both zones three users and three projects are created.

 

global # zlogin mkt_dev mkt_dev # mkdir -p /export/home mkt_dev # groupadd dba mkt_dev # useradd -g dba -m -d /export/home/oracle -s /bin/bash oracle mkt_dev # projadd -U oracle ora_mkt mkt_dev # projmod -sK "project.max-shm-memory=(privileged,2G,deny)" ora_mkt mkt_dev # projmod -sK "project.cpu-shares=(privileged,100,none)" ora_mkt mkt_dev # useradd -m -d /export/home/dev1 -s /bin/bash dev1 mkt_dev # useradd -m -d /export/home/dev2 -s /bin/bash dev2 mkt_dev # projadd user.dev1 mkt_dev # projadd user.dev2 mkt_dev # projmod -sK "project.cpu-shares=(privileged,10,none)" user.dev1 mkt_dev # projmod -sK "project.cpu-shares=(privileged,10,none)" user.dev2 [Oracle installation omitted for brevity...]

 

global # zlogin sales_dev sales_dev # mkdir -p /export/home sales_dev # groupadd dba sales_dev # useradd -g dba -m -d /export/home/oracle -s /bin/bash oracle sales_dev # projadd -U oracle ora_sales sales_dev # projmod -sK "project.max-shm-memory=(privileged,2G,deny)" ora_sales sales_dev # projmod -sK "project.cpu-shares=(privileged,100,none)" ora_sales sales_dev # useradd -m -d /export/home/dev1 -s /bin/bash dev1 sales_dev # useradd -m -d /export/home/dev2 -s /bin/bash dev2 sales_dev # projadd user.dev1 sales_dev # projadd user.dev2 sales_dev # projmod -sK "project.cpu-shares=(privileged,10,none)" user.dev1 sales_dev # projmod -sK "project.cpu-shares=(privileged,10,none)" user.dev2 [Oracle installation omitted for brevity...]

Verifying the Configuration

The configuration just built can be verified using the following steps:

1.      Start the prstat -Z command in the global zone to observe the CPU utilization of the zones.

2.      Start the poolstat -r pset 5 command in the global zone to observe utlization in the resource pools.

3.      Create load using the nspin -n 4 command in the mkt zone as the user oracle in the ora_mkt project. Note the CPU consumption of the mkt zone peaks around 33% since the marketing resource pool to which the zone is bound consists of two CPUs. The other CPUs are idle.

4.      Add the same load in the mkt_dev zone. The combined CPU usage of the mkt and mkt_dev zones is approximately 33% since they share the same resource pool. The mkt zone receives approximately 27% and the mkt_dev zone about 6% because the mkt zone has five times more zone.cpu-shares than the mkt_dev zone.

5.      Add the same load in the sales zone. The sales zone receives 33% since it is bound to the sales pool, which also has two CPUs. The CPU consumption of the mkt and mkt_dev zones is not impacted by the CPU usage of the sales zone.

6.      Add load in the sales_dev zone. This zone is bound to the default pool. As a result, it is able to use all of the remaining CPU capacity since it is the only zone in that pool using CPU resources.

7.      Add the same load in the global zone. The global zone is also bound to the default pool, and has the same amount of zone.cpu-shares as the sales_dev zone. The CPU usage of both zones is therefore equal, and approximately 16 percent. The resulting prstat –Z command output looks as follows:

 

global # prstat -Z PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 1987 100 1144K 672K cpu3 20 0 0:04:17 8.4% nspin/1 [...] 2031 root 1144K 808K run 7 0 0:00:49 4.1% nspin/1 ZONEID NPROC SIZE RSS MEMORY TIME CPU ZONE 2 15 42M 30M 0.4% 0:17:17 33% sales 1 33 104M 69M 0.8% 0:23:00 27% mkt 0 65 388M 179M 2.2% 0:03:36 17% global 3 33 105M 71M 0.8% 0:06:27 16% sales_dev 4 33 103M 69M 0.8% 0:03:53 5.7% mkt_dev Total: 179 processes, 586 lwps, load averages: 19.81, 14.84, 8.04

8.      Add load in the sales_dev zone in the user.dev1 and user.dev2 projects. The total CPU usage of the sales_dev zone remains the same. However, in the zone the CPU should now be divided across the three projects according to the project.cpu-shares in the zone. Notice that a zone bound to a resource pool is only aware of the CPUs in the associated processor set. As a result, the sales_dev zone only knows about two CPUs, and the usage shown in the output of the prstat command is therefore based on two CPUs. That is why inside the zone the three projects seem to use 50%. (The other 50% is used by the global zone that is also bound to the same pool.) The user.dev1 and user.dev2 projects receive 10/120ths each of that 50% since they each have 10 project.cpu-shares and ora_sales has 100 project.cpu-shares.

 

sales_dev # prstat -J PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 2016 oracle 1144K 672K run 35 0 0:24:48 10% nspin/1 2088 dev1 1136K 704K run 1 0 0:00:03 1.4% nspin/1 [...] 2113 dev2 1144K 848K run 27 0 0:00:02 1.2% nspin/1 PROJID NPROC SIZE RSS MEMORY TIME CPU PROJECT 100 5 7456K 4864K 0.1% 1:39:06 40% ora_sales 101 5 7464K 5072K 0.1% 0:00:12 5.3% user.dev1 102 5 7464K 5072K 0.1% 0:00:08 5.0% user.dev2 1 5 11M 9336K 0.1% 0:00:00 0.2% user.root 0 26 93M 63M 0.8% 0:00:14 0.0% system 3 1 2904K 2064K 0.0% 0:00:00 0.0% default Total: 47 processes, 132 lwps, load averages: 14.43, 10.28, 8.84

 

This example illustrates some of the ways that Solaris Containers technologies can be used to facilitate consolidation. It should be noted that not all features must be used at the same time. Depending on the circumstances, some Solaris Container technologies such as Resource Management, Resource Pools and Zones, can be mixed and matched to meet the specific needs for a consolidation project. In some cases, just using the Fair Share Scheduler may be sufficient to meet requirements, while in other cases Solaris Zones can be the key technology to a succesful consolidation.