Updated 22 March 2010
Linux Logo

Enable Zone to Use Fixed Number of CPU's


Description:
  • This procedure describes how to create a pools and processor sets to set a fixed number of CPUs per zone.

Prerequisites:
  • Resource Pools must be enable.
  • Must be root.
Notes:
  • A fixed processor set is defined if the maximum and minimum number of processors are the same value in the set.
  • An advantage of fixed CPU's per zone is there is better level 1 and level 2 cache hit rates for each zone.
  • A disadvantage will be that zone that is idle would not share CPU resources with a zone that has a high demand.
  • Assigning all the processor in the global server to a single zone is not allowed.
References:

Step Action Description
1 Enable Pool
  • Use pgrep and pooladm to check to see if resource pool faclility is already running.
  • If not already running use pooladm -e to start resource pool facility.
# pooladm -e
global# pgrep -l poold
global# pooladm
pooladm: couldn't open pools state file: Facility is not active 
global#
global# pooladm -e
global#
global# pgrep -l poold
 1429 poold
global# pooladm
 
system titan
        string  system.comment
        int     system.version 1
[...]
2 Configure Pools Script

To make sure that a zone has a fixed number of processors assigned, the minimum and maximum values of the processor set must be the same value and should never be set to the total number of processors on the system.

  • Create a text file consisting of the parameters for creation of the new pools and processor sets. This text file contains three definition types.
    • defined a processor set,
    • defines a pool and
    • associates a pool to a processor set.
mypool temporary text file
# myscript - poolcfg script for fixed CPUs.
#
# Load using:
#       pooladm -x; pooladm -s; poolcfg -f myscript; pooladm -c
# 
# 14-Jan-2005   Brendan Gregg   Created this.
  
create pset work1-pset ( uint pset.min = 2; uint pset.max = 2 )
create pset work2-pset ( uint pset.min = 2; uint pset.max = 2 )
create pset work3-pset ( uint pset.min = 1; uint pset.max = 1 )
create pool work1-pool
create pool work2-pool
create pool work3-pool
create pool work4-pool
associate pool work1-pool ( pset work1-pset )
associate pool work2-pool ( pset work2-pset )
associate pool work3-pool ( pset work3-pset )

This text file consist of three entry types with key words:

  • create pset - creates a processor set definition using the keywords "create pset <set name>-pset" followed in parenthesis the unit of measure with the minimum and maximum number of processor separated by a semicolon surrounded by parenthesis. To make sure that a zone has a fixed number of processors assigned, the minimum and maximum values of the processors must be the same value. The name of the set should use the "-pset" convention. A processor set uses an integer unit of measure ("uint").
create pset <set name>-pset ( uint pset.min=<same value> uint pset.max=<same value> )
  • create pool - creates a resource pool using the keywords "create pool <pool name>-pool". The name should use the "-pool" convention.
create pool <pool name>-pool 
  • associate pool - associates a resource pool to a processor set using the keywords "associate pool <pool name>" followed by the processor set name in parenthesis using the keyword "pset".
assocaite pool <pool name>-pool ( pset <processor set name>-pset ) 
3 Configure Pools Configure and activate the new pool and processor set configuration using the pooladm and poolcfg in this order:
pooladm -x - flush current in-memory pool configuration
pooladm -s - create default /etc/pooladm.conf configuration file
poolcfg -f <pool file name> - process and save "pool file name" configuration in /etc/pooladm.conf
global# ls /etc/pooladm.conf
/etc/pooladm.conf: No such file or directory
global# pooladm -x
global# pooladm -s
global# ls /etc/pooladm.conf
/etc/pooladm.conf
global# cat /etc/pooladm.conf
<?xml version="1.0"?>
<!DOCTYPE system PUBLIC "-//Sun Microsystems Inc//DTD Resource Management 
                          All//EN" "file:///usr/share/lib/xml/dtd/rm_pool.dtd.1">
<!--
Configuration for pools facility. Do NOT edit this file by hand - 
                                        use poolcfg(1) or libpool(3POOL) instead.
-->
<system ref_id="dummy" name="lqwv20z" comment="" version="1" bind-default="true">
  <property name="system.poold.pid" type="int">28231</property>
  <pool name="pool_default" active="true" default="true" importance="1" 
                                        comment="" res="pset_-1" ref_id="pool_0">
    <property name="pool.sys_id" type="int">0</property>
  </pool>
  <res_comp type="pset" sys_id="-1" name="pset_default" default="true" min="1" 
                      max="65536" units="population" comment="" ref_id="pset_-1">
    <property name="pset.load" type="uint">0</property>
    <property name="pset.size" type="uint">2</property>
    <comp type="cpu" sys_id="1" comment="" ref_id="cpu_1">
      <property name="cpu.status" type="string">on-line</property>
    </comp>
    <comp type="cpu" sys_id="0" comment="" ref_id="cpu_0">
      <property name="cpu.status" type="string">on-line</property>
    </comp>
  </res_comp>
</system>
global# poolcfg -f mypool
global# cat /etc/pooladm.conf
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE system PUBLIC "-//Sun Microsystems Inc//DTD Resource Management 
                          All//EN" "file:///usr/share/lib/xml/dtd/rm_pool.dtd.1">
<!--
Configuration for pools facility. Do NOT edit this file by hand - 
                                        use poolcfg(1) or libpool(3POOL) instead.
-->
<system ref_id="dummy" name="lqwv20z" comment="" version="1" bind-default="true">
  <property name="system.poold.pid" type="int">28231</property>
  <pool name="pool_default" active="true" default="true" importance="1" 
                                        comment="" res="pset_-1" ref_id="pool_0">
    <property name="pool.sys_id" type="int">0</property>
  </pool>
  <res_comp type="pset" sys_id="-1" name="pset_default" default="true" min="1"
                      max="65536" units="population" comment="" ref_id="pset_-1">
    <property name="pset.load" type="uint">0</property>
    <property name="pset.size" type="uint">2</property>
    <comp type="cpu" sys_id="1" comment="" ref_id="cpu_1">
      <property name="cpu.status" type="string">on-line</property>
    </comp>
    <comp type="cpu" sys_id="0" comment="" ref_id="cpu_0">
      <property name="cpu.status" type="string">on-line</property>
    </comp>
  </res_comp>
  <res_comp ref_id="id_0" sys_id="-2" type="pset" name="work1-pset" min="2"
                                          max="2" units="population" comment="">
    <property name="pset.load" type="uint">0</property>
    <property name="pset.size" type="uint">0</property>
  </res_comp>
  <property name="system._next_id" type="uint">7</property>
  <res_comp ref_id="id_1" sys_id="-2" type="pset" name="work2-pset" min="2"
                                          max="2" units="population" comment="">
    <property name="pset.load" type="uint">0</property>
    <property name="pset.size" type="uint">0</property>
  </res_comp>
  <res_comp ref_id="id_2" sys_id="-2" type="pset" name="work3-pset" min="1"
                                          max="1" units="population" comment="">
    <property name="pset.load" type="uint">0</property>
    <property name="pset.size" type="uint">0</property>
  </res_comp>
  <pool ref_id="id_3" res="id_0" name="work1-pool" active="true"
                                                     importance="1" comment=""/>
  <pool ref_id="id_4" res="id_1" name="work2-pool" active="true"
                                                     importance="1" comment=""/>
  <pool ref_id="id_5" res="id_2" name="work3-pool" active="true"
                                                     importance="1" comment=""/>
  <pool ref_id="id_6" res="pset_-1" name="work4-pool" active="true"
                                                     importance="1" comment=""/>
</system>
global# pooladm -c
Notes:
  • Creating a temporary script configuration file to define a pool and processor set is the recommended procedure. But it may also be accomplished temporarily using poolcfg -dc
global# poolcfg -dc info
global# poolcfg -dc 'create pset work1-pset'
global# poolcfg -dc 'create pool work1-pool'
global# poolcfg -dc 'modify pset work1-pset ( uint pset.min = 2; uint pset.max = 2 )'
global# poolcfg -dc 'modify pool work1-pool ( string pool.scheduler = "FSS" )'
global# poolcfg -dc 'modify pool work1-pool ( int pool.importance = 10 )'
global# poolcfg -dc 'destroy pool work1-pool'

The "-d" option changes the memory configuration directly.

  • Use "poolcfg -dc info" to verify that the new pool set does exist.
  • Be careful when rerunning those four commands on a system that already has zones and pools configured. Existing zones may need to be rebooted or poolbind to attach their processor set.
4 Load Pools
  • Use "pooladm -c" to load processor set.
  • Check the processor sets by running "psrset" before and after to check if any processor sets have been automatically created.
global# pooladm -x
global# pooladm -s
global# poolcfg -f <pool file name>
global# pooladm -c
global# psrset  
user processor set 1: processors 0 1
user processor set 2: processors 2 3
user processor set 3: processor 512
global#
  • Use "pooladm" to check the current state in more detail.
global# pooladm
 
system titan
        string  system.comment 
        int     system.version 1
        boolean system.bind-default true
        int     system.poold.pid 247
 
        pool pool_default
                int     pool.sys_id 0
                boolean pool.active true
                boolean pool.default true
                int     pool.importance 1
                string  pool.comment 
                pset    pset_default
 
        pool work2-pool
                int     pool.sys_id 13
                boolean pool.active true
                boolean pool.default false
                int     pool.importance 1
                string  pool.comment 
                pset    work2-pset
 
        pool work1-pool
                int     pool.sys_id 12
                boolean pool.active true
                boolean pool.default false
                int     pool.importance 1
                string  pool.comment 
                pset    work1-pset
 
        pool work4-pool
                int     pool.sys_id 15
                boolean pool.active true
                boolean pool.default false
                int     pool.importance 1
                string  pool.comment 
                pset    pset_default
 
        pool work3-pool
                int     pool.sys_id 14
                boolean pool.active true
                boolean pool.default false
                int     pool.importance 1
                string  pool.comment 
                pset    work3-pset
 
        pset work1-pset
                int     pset.sys_id 1
                boolean pset.default false
                uint    pset.min 2
                uint    pset.max 2
                string  pset.units population
                uint    pset.load 0
                uint    pset.size 2
                string  pset.comment 
 
                cpu
                        int     cpu.sys_id 1
                        string  cpu.comment 
                        string  cpu.status on-line
 
                cpu
                        int     cpu.sys_id 0
                        string  cpu.comment 
                        string  cpu.status on-line
 
        pset work3-pset
                int     pset.sys_id 3
                boolean pset.default false
                uint    pset.min 1
                uint    pset.max 1
                string  pset.units population
                uint    pset.load 0
                uint    pset.size 1
                string  pset.comment 
 
                cpu
                        int     cpu.sys_id 512
                        string  cpu.comment 
                        string  cpu.status on-line
 
        pset work2-pset
                int     pset.sys_id 2
                boolean pset.default false
                uint    pset.min 2
                uint    pset.max 2
                string  pset.units population
                uint    pset.load 0
                uint    pset.size 2
                string  pset.comment 
 
                cpu
                        int     cpu.sys_id 3
                        string  cpu.comment 
                        string  cpu.status on-line
 
                cpu
                        int     cpu.sys_id 2
                        string  cpu.comment 
                        string  cpu.status on-line
 
        pset pset_default
                int     pset.sys_id -1
                boolean pset.default true
                uint    pset.min 1
                uint    pset.max 65536
                string  pset.units population
                uint    pset.load 5
                uint    pset.size 3
                string  pset.comment 
 
                cpu
                        int     cpu.sys_id 515
                        string  cpu.comment 
                        string  cpu.status on-line
 
                cpu
                        int     cpu.sys_id 514
                        string  cpu.comment 
                        string  cpu.status on-line
 
                cpu
                        int     cpu.sys_id 513
                        string  cpu.comment 
                        string  cpu.status on-line
global#
Note: If for some reason the pool set or processor set cannot be confured, and error message "pooladm: configuration at '/etc/pooladm.conf' cannot be instantiated on current system" will be generated. Update configuration and perform the sets above again. For example assigning all the processor in the global server to a single zone will generate this type of error.
5 Configure Zones Use /usr/sbin/zonecfg to permanently assoiciate the exiting pool or the processor set to an existing zone using the set pool option.
# zonecfg -z <zone name>
zonecfg:workzone1> set pool=<pool name>
zonecfg:workzone1> verify
zonecfg:workzone1> exit
 
global# zonecfg -z workzone1
zonecfg:workzone1> set pool=work1-pool
zonecfg:workzone1> verify
zonecfg:workzone1> exit
global#
global# zonecfg -z workzone2
zonecfg:workzone2> set pool=work2-pool
zonecfg:workzone2> verify
zonecfg:workzone2> exit
global# 
global# zonecfg -z workzone3
zonecfg:workzone3> set pool=work3-pool
zonecfg:workzone3> verify
zonecfg:workzone3> exit
global# 
global# zonecfg -z workzone4
zonecfg:workzone4> set pool=work4-pool
zonecfg:workzone4> verify
zonecfg:workzone4> exit
global# 

Verify the zone configuration has been updated by using zonecfg -z <zonename> info. This will display the updated configuration with the pool assignment displayed.

global# zonecfg -z workzone3 info
zonepath: /zones/workzone3
autoboot: true
pool: work3-pool
inherit-pkg-dir:
        dir: /lib
inherit-pkg-dir:
        dir: /platform
inherit-pkg-dir:
        dir: /sbin
inherit-pkg-dir:
        dir: /usr
inherit-pkg-dir:
        dir: /opt/sfw
net:
        address: 192.168.3.44/24
        physical: bge0
global# 
Note: You may also temporarily bind a pool and a processor set using poolbind command with the zoneid as the idtype.
global# poolbind -p work1-pool -i zoneid workzone1
6 Reboot Zones Use zoneadm with the reboot option to reboot the zone.
# zoneadm -z <zone name> reboot
 
global# zoneadm -z workzone1 reboot
global#
global# zonecfg -z workzone2 reboot
global#
global# zonecfg -z workzone3 reboot
global#
global# zonecfg -z workzone4 reboot
global# 

After the zone is running, log into the zone and verify that the number of processor are indeed enabled by using the "psrinfo -vp"

workzone1# psrinfo -vp
The physical processor has 1 virtual processor (0)
  x86 (AuthenticAMD family 15 model 5 step 1 clock 2193 MHz)
        AMD Opteron(tm) Processor 248
The physical processor has 1 virtual processor (1)
  x86 (AuthenticAMD family 15 model 5 step 1 clock 2193 MHz)
        AMD Opteron(tm) Processor 248
workzone1#
Click for more info