背景

Openstack已经成为主流IaaS云计算平台，国内多数云计算厂商提供的云计算服务底层都是使用的OpenStack。这里我们讲的“如何将实例部署到特定的计算节点上”是基于标准的Openstack平台。

通过Openstack我们可以申请以实例为核心的各类资源。通常情况下，我们只需要指定实例配置，如多少CPU、多大内存、挂多大数据盘、连接什么样的网络即可，Openstack会帮助我们将实例部署到指定的计算节点上。但是，需求往往是多样性的，有时候我们需要将实例部署到特定的计算节点上。例如Openstack资源池中的计算节点同时存在高性能的机架服务器和刀片服务器，我们希望将运行数据库的实例部署到机架服务器上，将运行中间件的实例部署到刀片服务器上。（这里的实例指的是虚拟主机实例，而非数据库实例或中间件实例）

Openstack的nova-scheduler服务可以帮我们实现这样个性化的需求。接下来我将介绍nova-scheduler如何帮助我们实现上述需求的工作原理，并且我会以两个示例分别用两种方法来实现“如何将实例部署到特定的计算节点上”。

原理（Nova Scheduler）

Openstack通过nova-scheduler来实现将实例部署到哪台计算节点上。Filter scheduler 是 nova-scheduler 默认的调度器，调度过程分为两步：

第一步：通过过滤器（filter）选择满足条件的计算节点（运行nova-compute）

第二步：通过权重计算（weighting）选择在最优（权重值最大）的计算节点上创建 Instance。

图 0‑1过滤

图0-2分析权重

 

Filters的调度策略有以下几种：

AggregateCoreFilter

AggregateDiskFilter

AggregateImagePropertiesIsolation

AggregateInstanceExtraSpecsFilter

AggregateIoOpsFilter

AggregateMultiTenancyIsolation

AggregateNumInstancesFilter

AggregateRamFilter

AggregateTypeAffinityFilter

AllHostsFilter

AvailabilityZoneFilter

ComputeCapabilitiesFilter

ComputeFilter

CoreFilter

NUMATopologyFilter

DifferentHostFilter

DiskFilter

GroupAffinityFilter

GroupAntiAffinityFilter

ImagePropertiesFilter

IsolatedHostsFilter

IoOpsFilter

JsonFilter

MetricsFilter

NumInstancesFilter

PciPassthroughFilter

RamFilter

RetryFilter

SameHostFilter

ServerGroupAffinityFilter

ServerGroupAntiAffinityFilter

SimpleCIDRAffinityFilter

TrustedFilter

TypeAffinityFilter

计算服务Filters的策略通过配置nova.conf配置文件中的配置项scheduler_available_filters完成。默认配置的是所有filters。配置如下：

scheduler_available_filters = nova.scheduler.filters.all_filters

这个配置项可以在nova.conf中多次出现，比如你可以通过Python实现自定义的filter（myFilter.MyFilter），只需要在配置文件中增加这个filter即可。

scheduler_available_filters = nova.scheduler.filters.all_filters scheduler_available_filters = myfilter.MyFilter

详细介绍请参见这里。

下面通过两个示例来实现将Openstack的示例部署到指定的主机中。

示例一：通过主机聚合的metadata实现将云主机部署到使用ssd的主机聚合中

假设我在公司的OpenStack环境中创建了一个主机聚合fast-io，这个聚合中，所有的计算节点都连接了SSD类型的本地磁盘。我想创建一个Flavor ssd.large，通过该Flavor申请的云主机全部部署到fast-io这个主机聚合中。

要完成这个任务，我们需要做6步工作：

1.    将AggregateInstanceExtraSpecsFilter添加到nova.conf文件的scheduler_default_filters配置项中，即：

scheduler_default_filters=AggregateInstanceExtraSpecsFilter,RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter

2.    创建一个主机聚合，命名为fast-io，可用域为nova

$ nova aggregate-create fast-io nova +----+---------+-------------------+-------+----------+ | Id | Name    | Availability Zone | Hosts | Metadata | +----+---------+-------------------+-------+----------+ | 1  | fast-io | nova              |       |          | +----+---------+-------------------+-------+----------+

3.    设置集群元数据ssd=true

$ nova aggregate-set-metadata 1 ssd=true +----+---------+-------------------+-------+-------------------+ | Id | Name    | Availability Zone | Hosts | Metadata          | +----+---------+-------------------+-------+-------------------+ | 1  | fast-io | nova              | []    | {u'ssd': u'true'} | +----+---------+-------------------+-------+-------------------+

 

4.    在主机聚合fast-io中增加两台主机node1和node2

$ nova aggregate-add-host 1 node1 +----+---------+-------------------+-----------+-------------------+ | Id | Name    | Availability Zone | Hosts      | Metadata          | +----+---------+-------------------+------------+-------------------+ | 1  | fast-io | nova              | [u'node1'] | {u'ssd': u'true'} | +----+---------+-------------------+------------+-------------------+   $ nova aggregate-add-host 1 node2 +----+---------+-------------------+---------------------+-------------------+ | Id | Name    | Availability Zone | Hosts                | Metadata          | +----+---------+-------------------+----------------------+-------------------+ | 1  | fast-io | nova              | [u'node1', u'node2'] | {u'ssd': u'true'} | +----+---------+-------------------+----------------------+-------------------+

5.    使用novaflavor-create命令创建flavor ssd.large，配置为4个vCPU、8GB内存和80GB root盘。

$ nova flavor-create ssd.large68192804 +----+-----------+-----------+------+-----------+------+-------+-------------+-----------+ | ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public | +----+-----------+-----------+------+-----------+------+-------+-------------+-----------+ | 6  | ssd.large | 8192      | 80   | 0         |      | 4     | 1.0         | True      | +----+-----------+-----------+------+-----------+------+-------+-------------+-----------+

6.    设置flavor ssd.large的元数据为ssd=true

$ nova flavor-key ssd.large set  aggregate_instance_extra_specs:ssd=true

完成以上6步工作后，当用户通过ssd.large这个flavor创建云主机时，nova-scheduler将会把这个云主机调度到元数据为ssd=true这个主机聚合的主机上。

示例二：通过配置flavor的Capability实现将云主机部署到厂商为intel的主机上

某些情况下，我们希望将申请的云主机与某些厂商的主机绑定。例如我想把使用flavor为vendor.intel的云主机全部部署到intel生产的主机上。

首先我们把compute-host-capabilities.json这个文件导入到元数据信息中。

其次我们创建一个flavor vendor.intel，并把元数据指定为cpu.info:vendor=intel。

此时申请flavor为vendor.intel的云主机会自动部署到intel生产的主机上。

compute-host-capabilities.json文件内容：

{

    "namespace":"OS::Compute::HostCapabilities",

    "display_name": "ComputeHost Capabilities",

    "description": "Hardwarecapabilities provided by the compute host. This provides the ability to finetune the hardware specification required when an instance is requested. TheComputeCapabilitiesFilter should be enabled in the Nova scheduler to use theseproperties. When enabled, this filter checks that the capabilities provided bythe compute host satisfy any extra specifications requested. Only hosts thatcan provide the requested capabilities will be eligible for hosting theinstance.",

    "visibility": "public",

    "protected": true,

    "resource_type_associations": [

        {

            "name":"OS::Nova::Flavor",

            "prefix":"capabilities:"

        },

        {

            "name":"OS::Nova::Aggregate",

            "prefix":"aggregate_instance_extra_specs:"

        }

    ],

    "properties": {

        "cpu_info:vendor": {

            "title":"Vendor",

            "description":"Specifies the CPU manufacturer.",

            "operators":["<or>"],

            "type":"string",

            "enum": [

                "Intel",

                "AMD"

            ]

        },

        "cpu_info:model": {

            "title":"Model",

            "description":"Specifies the CPU model. Use this property to ensure that your vm runs ona a specific cpu model.",

            "operators":["<or>"],

            "type":"string",

            "enum": [

                "Conroe",

                "Core2Duo",

                "Penryn",

                "Nehalem",

                "Westmere",

                "SandyBridge",

                "IvyBridge",

                "Haswell",

                "Broadwell",

                "Delhi",

                "Seoul",

                "Abu Dhabi",

                "Interlagos",

                "Kabini",

                "Valencia",

                "Zurich",

                "Budapest",

                "Barcelona",

                "Suzuka",

                "Shanghai",

                "Istanbul",

                "Lisbon",

                "Magny-Cours",

                "Valencia",

                "Cortex-A57",

                "Cortex-A53",

                "Cortex-A12",

                "Cortex-A17",

                "Cortex-A15",

                "Coretx-A7",

                "X-Gene"

            ]

        },

        "cpu_info:arch": {

            "title": "Architecture",

            "description":"Specifies the CPU architecture. Use this property to specify thearchitecture supported by the hypervisor.",

            "operators":["<or>"],

            "type":"string",

            "enum": [

                "x86",

                "x86_64",

                "i686",

                "ia64",

                "ARMv8-A",

                "ARMv7-A"

            ]

        },

        "cpu_info:topology:cores": {

            "title":"cores",

            "description":"Number of cores.",

            "type":"integer",

            "readonly": false,

            "default": 1

        },

        "cpu_info:topology:threads":{

            "title":"threads",

            "description":"Number of threads.",

            "type":"integer",

            "readonly": false,

            "default": 1

        },

        "cpu_info:topology:sockets":{

            "title":"sockets",

            "description":"Number of sockets.",

            "type":"integer",

            "readonly": false,

            "default": 1

        },

        "cpu_info:features": {

            "title":"Features",

            "description":"Specifies CPU flags/features. Using this property you can specify therequired set of instructions supported by a vm.",

            "operators":["<or>", "<all-in>"],

            "type":"array",

            "items": {

                "type":"string",

                "enum": [

                    "fpu",

                    "vme",

                    "de",

                    "pse",

                    "tsc",

                    "msr",

                    "pae",

                    "mce",

                    "cx8",

                    "apic",

                    "sep",

                    "mtrr",

                    "pge",

                    "mca",

                    "cmov",

                    "pat",

                    "pse36",

                    "pn",

                    "clflush",

                    "dts",

                    "acpi",

                    "mmx",

                    "fxsr",

                    "sse",

                    "sse2",

                    "ss",

                    "ht",

                    "tm",

                    "ia64",

                    "pbe",

                    "syscall",

                    "mp",

                    "nx",

                    "mmxext",

                    "fxsr_opt",

                    "pdpe1gb",

                    "rdtscp",

                    "lm",

                    "3dnowext",

                    "3dnow",

                    "arch_perfmon",

                    "pebs",

                    "bts",

                    "rep_good",

                    "nopl",

                    "xtopology",

                    "tsc_reliable",

                    "nonstop_tsc",

                    "extd_apicid",

                    "amd_dcm",

                    "aperfmperf",

                    "eagerfpu",

                    "nonstop_tsc_s3",

                    "pni",

                    "pclmulqdq",

                    "dtes64",

                    "monitor",

                    "ds_cpl",

                    "vmx",

                    "smx",

                    "est",

                    "tm2",

                    "ssse3",

                    "cid",

                    "fma",

                    "cx16",

                    "xtpr",

                    "pdcm",

                    "pcid",

                    "dca",

                    "sse4_1",

                    "sse4_2",

                    "x2apic",

                    "movbe",

                    "popcnt",

                   "tsc_deadline_timer",

                    "aes",

                    "xsave",

                    "avx",

                    "f16c",

                    "rdrand",

                    "hypervisor",

                    "rng",

                    "rng_en",

                    "ace",

                    "ace_en",

                    "ace2",

                    "ace2_en",

                    "phe",

                    "phe_en",

                    "pmm",

                    "pmm_en",

                    "lahf_lm",

                    "cmp_legacy",

                    "svm",

                    "extapic",

                    "cr8_legacy",

                    "abm",

                    "sse4a",

                    "misalignsse",

                    "3dnowprefetch",

                    "osvw",

                    "ibs",

                    "xop",

                    "skinit",

                    "wdt",

                    "lwp",

                    "fma4",

                    "tce",

                    "nodeid_msr",

                    "tbm",

                    "topoext",

                    "perfctr_core",

                    "perfctr_nb",

                    "bpext",

                    "perfctr_l2",

                    "mwaitx",

                    "ida",

                    "arat",

                    "cpb",

                    "epb",

                    "pln",

                    "pts",

                    "dtherm",

                    "hw_pstate",

                    "proc_feedback",

                    "hwp",

                    "hwp_notify",

                    "hwp_act_window",

                    "hwp_epp",

                    "hwp_pkg_req",

                    "intel_pt",

                    "tpr_shadow",

                    "vnmi",

                    "flexpriority",

                    "ept",

                    "vpid",

                    "npt",

                    "lbrv",

                    "svm_lock",

                    "nrip_save",

                    "tsc_scale",

                    "vmcb_clean",

                    "flushbyasid",

                    "decodeassists",

                    "pausefilter",

                    "pfthreshold",

                    "vmmcall",

                    "fsgsbase",

                    "tsc_adjust",

                    "bmi1",

                    "hle",

                    "avx2",

                    "smep",

                    "bmi2",

                    "erms",

                    "invpcid",

                    "rtm",

                    "cqm",

                    "mpx",

                    "avx512f",

                    "rdseed",

                    "adx",

                    "smap",

                    "pcommit",

                    "clflushopt",

                    "clwb",

                    "avx512pf",

                    "avx512er",

                    "avx512cd",

                    "sha_ni",

                    "xsaveopt",

                    "xsavec",

                    "xgetbv1",

                    "xsaves",

                    "cqm_llc",

                    "cqm_occup_llc",

                    "clzero"

                ]

            }

        }

    },

    "objects": []

}

参考资料

http://www.codexiu.cn/openstack/blog/14729/#OSC_h2_9

http://docs.openstack.org/kilo/config-reference/content/section_compute-scheduler.html