testing, certification, benchmarking, performance

[AmitKumarDas]

• [Openstack-operators] Measuring how fast is my Cloud with OpenStack

  - https://wiki.openstack.org/wiki/Rally

• How fast can be measured in a variety of ways:

  - How quickly a VM can be spawned and become available ?
  - How quickly it runs once it is available ? 
  - How many X work units / second can be achieved with N cores ?

  - Rally will help with the 1st case. 
  - For the second case, choose an appropriate benchmark for your intended workload.

  - Rally team is working on supporting point 2. 
  - We will allow you to run distributed loads in cloud like: 
    - IPerf, 
    - SPEC, 
    - HPCC and so on 
  - For now you can use https://github.com/openstack/shaker

• rally specs on vm workloads

https://review.openstack.org/#/q/status:open+project:openstack/rally+branch:master+topic:bp/vm-workloads-framework,n,z

• tempest

  - [openstack-dev] [QA][Tempest] Use tempest-config for tempest-cli-improvements
  - https://github.com/redhat-openstack/tempest/blob/master/tools/config_tempest.py
  - https://blueprints.launchpad.net/tempest/+spec/tempest-config-generator
  - https://etherpad.openstack.org/p/mitaka-qa-priorities
  - https://etherpad.openstack.org/p/tempest-cli-improvements
  - https://github.com/openstack/qa-specs/blob/master/specs/tempest/tempest-cli-improvements.rst