Date: Sat, 1 Aug 2015 02:53:18 +1000 Subject: Add talks --- src/talks/a-vision-for-the-future/index.html | 2 +- src/talks/ansible-cloud/index.html | 792 ++++++++++++++++++++++++++ src/talks/glean/index.html | 209 +++++++ src/talks/index/index.html | 74 +++ src/talks/lemmings/index.html | 331 +++++++++++ src/talks/now-what/index.html | 436 +++++++++++++++ src/talks/os-client-config/index.html | 242 ++++++++ src/talks/product-management/index.html | 454 +++++++++++++++ src/talks/tripleo-ansible/index.html | 794 +++++++++++++++++++++++++++ 9 files changed, 3333 insertions(+), 1 deletion(-) create mode 100644 src/talks/ansible-cloud/index.html create mode 100644 src/talks/glean/index.html create mode 100644 src/talks/index/index.html create mode 100644 src/talks/lemmings/index.html create mode 100644 src/talks/now-what/index.html create mode 100644 src/talks/os-client-config/index.html create mode 100644 src/talks/product-management/index.html create mode 100644 src/talks/tripleo-ansible/index.html (limited to 'src/talks') diff --git a/src/talks/a-vision-for-the-future/index.html b/src/talks/a-vision-for-the-future/index.html index 88fdbe5..abd3851 100644 --- a/src/talks/a-vision-for-the-future/index.html +++ b/src/talks/a-vision-for-the-future/index.html @@ -483,7 +483,7 @@ Creative Commons Attribution 4.0 International License .
- Source code available at http://git.inaugust.com/cgit/mordred-talks + Source code available at http://git.inaugust.com/cgit/inaugust.com diff --git a/src/talks/ansible-cloud/index.html b/src/talks/ansible-cloud/index.html new file mode 100644 index 0000000..37e66c6 --- /dev/null +++ b/src/talks/ansible-cloud/index.html @@ -0,0 +1,792 @@ + + + + + + + NoOps with Ansible and Puppet + + + + + + + + + + + + + + + + + + + + +

+ + +

+ +

+ +

+ NoOps with Ansible and Puppet +

Monty Taylor

http://inaugust.com/talks/ansible-cloud.html

twitter: @e_monty

+ +

Who am I?

Distinguished Technologist at HP
OpenStack Technical Committee
OpenStack Foundation Board of Directors
OpenStack Developer Infrastructure Core Team
Former Consultant for MySQL, Inc
Former Drizzle Core Developer

+ +

What are we going to talk about?

NoOps
Cloud Applications
Puppet
Ansible

+ +

NoOps

NoOps means developers can code and let a service deploy, manage and scale their code

I don't want to "do ops"

I want to change the system by landing commits

If I have to use my root access, it's a bug

+ +

Cloud Native

Ephemeral Compute
Data services
Design your applications to be resilient via scale out

+ +

Cloud Scale Out

Forget HA of one system
Forget long-lived systems
Shared-nothing for EVERYTHING

+ +

Cloud Scale Out is great for new applications

+ +

What about existing applications?

+ +

OpenStack Infra

+ +

Tooling, Automation and CI for OpenStack Project

+ +

2000 Developers

+ +

Gated Commits

Every commit is fully integration tested (twice) before landing

+ +

Each Test Runs on a Single Use Cloud Slave

This is that "cloud scale out" part

+ +

1.7 Million Test Jobs in the last 6 Months

+ +

15 Million Tests in December

+ +

18 Terabytes of Log Data in six months

+ +

We have no servers

It all runs across HP and Rackspace Public Clouds.

+ +

Architecture

+ +

It didn't start this way

+ +

Step One: Puppet

+ +

Overview

Open Source Config Management System
Written in Ruby
Models intended state
Wants to own entire system

+ +

Config Management is Great

Repeatable and consistent machines
Code Review
Collaboration from non-root users
Less repetition for me
Open Source Infrastructure
http://git.openstack.org/cgit/openstack-infra/system-config/
http://puppetdb.openstack.org/

+ +

Ruby DSL


+package { 'git':
+  ensure => 'present',
+}
+

+ +

Leaky Abstraction


+if !defined(Package['git']) {
+  package { 'git':
+    ensure => 'present',
+  }
+}
+

+ +

Three ways to run

puppet apply
puppetmaster + puppet agent daemons
puppetmaster + puppet agent non-daemon

+ +

Managing users and ssh keys


+define user::virtual::localuser(
+  $realname,
+  $groups     = [ 'sudo', 'admin', ],
+  $sshkeys    = '',
+  $key_id     = '',
+  $old_keys   = [],
+  $shell      = '/bin/bash',
+  $home       = "/home/${title}",
+  $managehome = true
+) {
+
+  group { $title:
+    ensure => present,
+  }
+

+ +

Managing users and ssh keys (cont.)


+  user { $title:
+    ensure     => present,
+    comment    => $realname,
+    gid        => $title,
+    groups     => $groups,
+    home       => $home,
+    managehome => $managehome,
+    membership => 'minimum',
+    shell      => $shell,
+    require    => Group[$title],
+  }
+

+ +

Managing users and ssh keys (cont.)


+  ssh_authorized_key { $key_id:
+    ensure  => present,
+    key     => $sshkeys,
+    user    => $title,
+    type    => 'ssh-rsa',
+  }
+
+  if ( $old_keys != [] ) {
+    ssh_authorized_key { $old_keys:
+      ensure => absent,
+      user   => $title,
+    }
+  }
+}
+

+ +

Our code is Open Source, right? What about passwords and keys ...

+ +

hiera

Simple YAML database, sits on puppetmaster
Use it for secret data
puppet code is still complete


+node default {
+  class { 'openstack_project::server':
+    sysadmins => hiera('sysadmins', []),
+  }
+}
+

Breaks ability to use simple puppet apply

+ +

Step Two: Ansible for Orchestration

+ +

About Ansible

Open Source System Management tool
Written in Python
Sequence of steps to perform
Works over SSH
Incremental Adoption

+ +

ad-hoc operation

+ansible '*' -m shell -p uptime
+

+ +

YAML Syntax


+- hosts: '*.slave.openstack.org'
+  tasks:
+    - shell: 'rm -rf ~jenkins/workspace/*{{ project }}*'
+

That's executed:

+ansible-playbook -f 10 /etc/ansible/clean_workspaces.yaml --extra-vars "project=$PROJECTNAME"
+

+ +

Ansible Organization

modules
plays
playbooks
roles

+ +

Use Ansible to Run Puppet!

+ +

puppet module

def main():
+    module = AnsibleModule(argument_spec=dict(
+        timeout=dict(default="30m"),
+        puppetmaster=dict(required=True),
+        show_diff=dict(default=False, aliases=['show-diff'], type='bool'),
+    ))
+    p = module.params
+
+    puppet_cmd = module.get_bin_path("puppet", False)
+    if not puppet_cmd:
+        module.fail_json(msg="Could not find puppet. Please ensure it is installed.")
+

+ +

puppet module (cont)


+    cmd = ("timeout -s 9 %(timeout)s %(puppet_cmd)s agent --onetime"
+           " --server %(puppetmaster)s"
+           " --ignorecache --no-daemonize --no-usecacheonfailure --no-splay"
+           " --detailed-exitcodes --verbose") % dict(
+               timeout=pipes.quote(p['timeout']), puppet_cmd=PUPPET_CMD,
+               puppetmaster=pipes.quote(p['puppetmaster']))
+    if p['show_diff']:
+        cmd += " --show-diff"
+    rc, stdout, stderr = module.run_command(cmd)
+

+ +

+ Please. Everyone. Marvel at the following logic +


+    if rc == 0:  # success
+        module.exit_json(rc=rc, changed=False, stdout=stdout)
+    elif rc == 1:
+        # rc==1 could be because it's disabled OR there was a compilation failure
+        disabled = "administratively disabled" in stdout
+        if disabled:
+            msg = "puppet is disabled"
+        else:
+            msg = "puppet compilation failed"
+        module.fail_json(rc=rc, disabled=disabled, msg=msg, stdout=stdout, stderr=stderr)
+    elif rc == 2:  # success with changes
+        module.exit_json(changed=True)
+    elif rc == 124:  # timeout
+        module.exit_json(rc=rc, msg="%s timed out" % cmd, stdout=stdout, stderr=stderr)
+    else:  # failure
+        module.fail_json(rc=rc, msg="%s failed" % (cmd), stdout=stdout, stderr=stderr)
+

+ +

puppet play


+- name: run puppet
+  puppet:
+    puppetmaster: "{{puppetmaster}}"
+

+ +

puppet role

roles/puppet/tasks/main.yml

+ +

remote puppet playbook


+- hosts: git0*
+  gather_facts: false
+  max_fail_percentage: 1
+  roles:
+    - { role: puppet, puppetmaster: puppetmaster.openstack.org }
+- hosts: review.openstack.org
+  gather_facts: false
+  roles:
+    - { role: puppet, puppetmaster: puppetmaster.openstack.org }
+- hosts: "!review.openstack.org:!git0*:!afs*"
+  gather_facts: false
+  roles:
+    - { role: puppet, puppetmaster: puppetmaster.openstack.org }
+

+ +

ansible inventory

List of servers to operate on
Optionally variables associated with each server
Optional groups of servers
Simple yaml file in /etc/ansible/hosts
Dynamic executable that returns JSON

+ +

Simple inventory

+review.openstack.org
+git01.openstack.org
+git02.openstack.org
+pypi.dfw.openstack.org
+pypi.iad.openstack.org
+
+[pypi]
+pypi.dfw.openstack.org
+pypi.iad.openstack.org
+
+[git]
+git01.openstack.org
+git02.openstack.org
+

+ +

ansible inventory from puppet certs


+import json
+import subprocess
+
+output = [
+    x.split()[1][1:-1] for x in subprocess.check_output(
+        ["puppet","cert","list","-a"]).split('\n')
+    if x.startswith('+')
+]
+
+data = {
+    '_meta': {'hostvars': dict()},
+    'ungrouped': output,
+}
+print json.dumps(data, sort_keys=True, indent=2)
+

+ +

Step Three: Ansible for Cloud Management

+ +

ansible and OpenStack

Ansible modules are just python
playbooks are lists of steps to take
Have plays/roles that provision servers
Infrastructure as code - for real!

+ +

Consider this data


+pypi.dfw.openstack.org:
+  image_name: Ubuntu 12.04.4
+  flavor_ram: 2048
+  region: DFW
+  cloud: rackspace
+  volumes:
+    - size: 200
+      mount: /srv
+pypi.region-b.geo-1.openstack.org:
+  image_name: Ubuntu 12.04.4
+  flavor_ram: 2048
+  region: region-b.geo-1
+  cloud: hp
+  volumes:
+    - size: 200
+      mount: /srv
+

+ +

Further consider this data


+pypi:
+  image_name: Ubuntu 12.04.4
+  flavor_ram: 2048
+  volumes:
+    - size: 200
+      mount: /srv
+  hosts:
+    pypi.dfw:
+      region: DFW
+    pypi.iad:
+      region: IAD
+    pypi.ord:
+      region: ORD
+    pypi.region-b.geo-1:
+      cloud: hp
+

+ +

Steps to launch a node

Create a compute instance
Wait for instance to exist
Create a floating IP
Attach floating IP to instance
Create one or more volumes
Attach volumes to instance
Wait for SSH to work
On host, format each volume
On host, mount each volume
On host, install config management software
On host, run config management software

+ +

Launch a node


+---
+- name: Launch Node
+  os_compute:
+      cloud: "{{ cloud }}"
+      region_name: "{{ region_name }}"
+      name: "{{ name }}"
+      image_name: "{{ image_name }}"
+      flavor_ram: "{{ flavor_ram }}"
+      flavor_include: "{{ flavor_include }}"
+      meta:
+          group: "{{ group }}"
+      key_name: "{{ launch_keypair }}"
+  register: node
+- name: Create volumes
+  os_volume:
+      cloud: "{{ cloud }}"
+      size: "{{ item.size }}"
+      display_name: "{{ item.display_name }}"
+  with_items: volumes
+- name: Attach volumes
+  os_compute_volume:
+      cloud: "{{ cloud }}"
+      server_id: "{{ node.id }}"
+      volume_name: "{{ item.display_name }}"
+  with_items: volumes
+  register: attached_volumes
+- debug: var=attached_volumes
+- name: Re-request server to get up to date metadata after the volume loop
+  os_compute_facts:
+      cloud: "{{ cloud }}"
+      name: "{{ name }}"
+  when: attached_volumes.changed
+- name: Wait for SSH to work
+  wait_for: host={{ node.openstack.interface_ip }} port=22
+  when: node.changed == True
+- name: Add SSH host key to known hosts
+  shell: ssh-keyscan "{{ node.openstack.interface_ip|quote }}" >> ~/.ssh/known_hosts
+  when: node.changed == True
+- name: Add all instance public IPs to host group
+  add_host:
+      name: "{{ node.openstack.interface_ip }}"
+      groups: "{{ provision_group }}"
+      openstack: "{{ node.openstack }}"
+  when: attached_volumes|length == 0
+- name: Add all instance public IPs to host and volumes group
+  add_host:
+      name: "{{ node.openstack.interface_ip }}"
+      groups: "{{ provision_group }},hasvolumes"
+      openstack: "{{ node.openstack }}"
+  when: attached_volumes|length != 0
+

+ +

Cloud based inventory

Just ask the cloud for the inventory
All of the meta-data the cloud knows is available
No need for puppetmaster now

+ +


+      "pypi.dfw.openstack.org": {
+        "ansible_ssh_host": "23.253.237.8",
+        "openstack": {
+          "HUMAN_ID": true,
+          "NAME_ATTR": "name",
+          "OS-DCF:diskConfig": "MANUAL",
+          "OS-EXT-STS:power_state": 1,
+          "OS-EXT-STS:task_state": null,
+          "OS-EXT-STS:vm_state": "active",
+          "accessIPv4": "23.253.237.8",
+          "accessIPv6": "2001:4800:7817:104:d256:7a33:5187:7e1b",
+          "addresses": {
+            "private": [
+              {
+                "addr": "10.208.195.50",
+                "version": 4
+              }
+            ],
+            "public": [
+              {
+                "addr": "23.253.237.8",
+                "version": 4
+              },
+              {
+                "addr": "2001:4800:7817:104:d256:7a33:5187:7e1b",
+                "version": 6
+              }
+            ]
+          },
+          "cloud": "rax",
+          "config_drive": "",
+          "created": "2014-09-05T15:32:14Z",
+          "flavor": {
+            "id": "performance1-4",
+            "links": [
+              {
+                "href": "https://dfw.servers.api.rackspacecloud.com/610275/flavors/performance1-4",
+                "rel": "bookmark"
+              }
+            ],
+            "name": "4 GB Performance"
+          },
+          "hostId": "adb603d4566efe0392756c76dab38ffcba22099368837c7973321e77",
+          "human_id": "pypidfwopenstackorg",
+          "id": "de672205-9245-46b6-b3df-489ccf9e0c17",
+          "image": {
+            "id": "928e709d-35f0-47eb-b296-d18e1b0a76b7",
+            "links": [
+              {
+                "href": "https://dfw.servers.api.rackspacecloud.com/610275/images/928e709d-35f0-47eb-b296-d18e1b0a76b7",
+                "rel": "bookmark"
+              }
+            ]
+          },
+          "interface_ip": "23.253.237.8",
+          "key_name": "launch-node-root",
+          "links": [
+            {
+              "href": "https://dfw.servers.api.rackspacecloud.com/v2/610275/servers/de672205-9245-46b6-b3df-489ccf9e0c17",
+              "rel": "self"
+            },
+            {
+              "href": "https://dfw.servers.api.rackspacecloud.com/610275/servers/de672205-9245-46b6-b3df-489ccf9e0c17",
+              "rel": "bookmark"
+            }
+          ],
+          "metadata": {},
+          "name": "pypi.dfw.openstack.org",
+          "networks": {
+            "private": [
+              "10.208.195.50"
+            ],
+            "public": [
+              "23.253.237.8",
+              "2001:4800:7817:104:d256:7a33:5187:7e1b"
+            ]
+          },
+          "progress": 100,
+          "region": "DFW",
+          "status": "ACTIVE",
+          "tenant_id": "610275",
+          "updated": "2014-09-05T15:32:49Z",
+          "user_id": "156284",
+          "volumes": [
+            {
+              "HUMAN_ID": false,
+              "NAME_ATTR": "name",
+              "attachments": [
+                {
+                  "device": "/dev/xvdb",
+                  "host_name": null,
+                  "id": "c6f5229c-1cc0-47c4-aab7-60db1f6cf8e8",
+                  "server_id": "de672205-9245-46b6-b3df-489ccf9e0c17",
+                  "volume_id": "c6f5229c-1cc0-47c4-aab7-60db1f6cf8e8"
+                }
+              ],
+              "availability_zone": "nova",
+              "bootable": "false",
+              "created_at": "2014-09-05T14:37:42.000000",
+              "device": "/dev/xvdb",
+              "display_description": null,
+              "display_name": "pypi.dfw.openstack.org/main01",
+              "human_id": null,
+              "id": "c6f5229c-1cc0-47c4-aab7-60db1f6cf8e8",
+              "metadata": {
+                "readonly": "False",
+                "storage-node": "1845027a-5e07-47a1-9572-3eea4716f726"
+              },
+              "os-vol-tenant-attr:tenant_id": "610275",
+              "size": 200,
+              "snapshot_id": null,
+              "source_volid": null,
+              "status": "in-use",
+              "volume_type": "SATA"
+            }
+          ]
+        }
+      },
+

+ +

Wait - what about secrets?

ansible can just pass secrets to puppet apply as parameters

+ +

Step Four: Just get rid of puppet ...

but that's another talk

+ +

Thank you!

http://inaugust.com/talks/ansible-cloud.html

Monty Taylor

twitter: @e_monty

http://inaugust.com/talks/ansible-cloud.html

+ +

+ + + + + + + + diff --git a/src/talks/glean/index.html b/src/talks/glean/index.html new file mode 100644 index 0000000..bbf943a --- /dev/null +++ b/src/talks/glean/index.html @@ -0,0 +1,209 @@ + + + + + + + glean: a minimal non-cloud-init cloud-init + + + + + + + + + + + + + + + + + + +

+ +

+ glean: a minimal non-cloud-init cloud-init +

Monty Taylor

http://inaugust.com/talks/glean.html

twitter: @e_monty

+ +

When you boot a VM, it needs boot time data or bootstrapping

+ +

I use ansible for orchestration

+ +

I don't need boot time fancy, + I JUST need ssh bootstrap

Ok. And maybe network

+ +

Things I must consume at boot

+ Network Configuration +
+ maybe SSH Keys +

+ +

Network information should be easy

There is this thing called DHCP

Dynamic Host Configuration Protocol

+ +

Some providers don't support DHCP - because bong

+ +

cloud-init

Handles many cases, but not all

Has lots of dependencies that conflict with gate depends

'frozen' pending 2.0 rewrite

+ +

glean

http://git.openstack.org/cgit/openstack-infra/glean

No dependencies

Handles future static network config in config drive

Optionally also reads ssh keys from config drive

Nothing else

+ +

network_info.json

Hopefully added in liberty

+services:
+- type: dns
+  address: 72.3.128.241
+- type: dns
+  address: 72.3.128.240
+networks:
+- network_id: 00000000-0000-0000-0000-000000000000
+  type: ipv4
+  netmask: 255.255.255.0
+  link: tapfafb5c05-a6
+  routes:
+  - netmask: 0.0.0.0
+    network: 0.0.0.0
+    gateway: 23.253.229.1
+  ip_address: 23.253.229.154
+  id: network0
+- network_id: 11111111-1111-1111-1111-111111111111
+  type: ipv4
+  netmask: 255.255.224.0
+  link: tape501e1cd-10
+  routes:
+  - netmask: 255.240.0.0
+    network: 10.176.0.0
+    gateway: 10.208.160.1
+  - netmask: 255.240.0.0
+    network: 10.208.0.0
+    gateway: 10.208.160.1
+  ip_address: 10.208.169.118
+  id: network1
+links:
+- ethernet_mac_address: BC:76:4E:01:62:86
+  mtu: 1500
+  type: null
+  id: tapfafb5c05-a6
+  vif_id: fafb5c05-a661-48ae-9810-46601c7e22d1
+- ethernet_mac_address: BC:76:4E:05:7B:06
+  mtu: 1500
+  type: null
+  id: tape501e1cd-10
+  vif_id: e501e1cd-10d0-4e63-b0c2-6542989ccbb2
+

+ +

Integrated with diskimage-builder

+disk-image-create -o debian.qcow2 debian-minimal vm simple-init
+

+ +

You know what has less depends than minimal python?

+ +

Rust

+ +

glean.rs

https://gitlab.com/mordred/glean.rs

+ +

Thank you!

http://inaugust.com/talks/glean.html

twitter: @e_monty

+ +

+ + +

+ + + + + + + + diff --git a/src/talks/index/index.html b/src/talks/index/index.html new file mode 100644 index 0000000..47b9774 --- /dev/null +++ b/src/talks/index/index.html @@ -0,0 +1,74 @@ + + + + + + +Monty Taylor Conference Talks + + + + + + + + + + + + + + + + + + + + + + +

Monty Taylor Conference Talks

+ +

os-client-config lightning talk

Using Ansible to Manage Cloud Applications

Deploying OpenStack using TripleO and Ansible

Lemmings, Think Different and Why Free Software is Important

Liberty, Product Management and OpenStack Technology

OpenStack works ... so now what?

glean: a minimal non-cloud-init cloud-init

OpenStack: a vision for the future + +

+ +

+ + + + + + diff --git a/src/talks/lemmings/index.html b/src/talks/lemmings/index.html new file mode 100644 index 0000000..bbaa982 --- /dev/null +++ b/src/talks/lemmings/index.html @@ -0,0 +1,331 @@ + + + + + + + Lemmings, Think Different and Why Free Software is Important + + + + + + + + + + + + + + + + + + +

+ +

+ Lemmings, Think Different and Why Free Software is Important +

Monty Taylor

http://inaugust.com/talks/lemmings.html

twitter: @e_monty

+ +

Mission

+ to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable. +

+ +

Borrowed from Ubuntu

Time Based Releases

Six Month Cadance

Design summits each cycle

Release codenames in alphabetical order

+ +

Different from Ubuntu

No BDFL

All decisions are democratic

+ +

Naming Process

+ Pick a name from names of cities or geographical features near the + summit location +

+ +

Grizzly

San Diego

Grizzly bear on the flag

+ +

Havana

Portland, OR

not Cuba

+ +

Icehouse

Hong Kong

Street in HK

+ +

Juno

Atlanta, GA

Town in Georgia

+ +

Kilo

Paris

location of the actual kilogramme

+ +

The L Word

Vancouver

Lemming?

Lemming Peak

+ +

The Marketing Community was not thrilled

+ +

Lemmings commit mass suicide by following each other off cliffs

+ +

Except they don't

+ +

White Wilderness

1958 Disney Documentary

Staged footage

Camera crew forced them off the cliff

Won an Oscar, of course

+ +

Definition of Irony

The actual meaning is the opposite of the literal meaning

+ +

Recursive Definition of Irony

The lemmings metaphor only works because people are lemmings in their adherence to the myth that lemmings are like lemmings.

+ +

This could never happen to us, right?

MySQL doesn't have transactions

MongoDB is Web Scale

PHP is insecure

Frequent password rotation is secure

sysvinit needs to be replaced

+ +

Results

We didn't pick Lemming - we picked Liberty

Liberty is a town in Saskatchewan

+ +

Four Freedoms

0 - to run the program as you wish, for any purpose

1 - to study how the program works, and change it so it does your computing as you wish

2 - to redistribute copies so you can help your neighbor

3 - to distribute copies of your modified version to others

+ +

GPL is anti-business

+ +

Except it isn't

+ +

Definition of Eating Crow

humiliation by admitting wrongness or having been proven wrong after taking a strong position

+ +

Enjoyable Definition of Eating Crow

Microsoft purchases Revolution Analytics

R language is GPL

+ +

Sun/MySQL

Sun bought MySQL for $1 Billion

Oracle bought Sun

Oracle now develops and ships MySQL under the GPL

+ +

Linux

anybody here heard of it?

+ +

Android is the market leading Mobile OS
+

+ +

Linux runs:

Google

Facebook

Twitter

They run MySQL too

+ +

Linux is not just for big companies - it's startup friendly too!

+ +

HP ships a Linux server every minute

business unit that does is a $28B business

+ +

Why is Free Software Important?

+ +

1984 - Apple's 1984 Super Bowl Commercial

1985 - Apple's Lemmings Super Bowl Commercial

1997 - Apple's Think Different Campaign

+ +

Don't be a lemming

+ +

Everybody in this room is wearing a uniform and don't kid yourself

Frank Zappa

+ +

"Think Different" isn't about allowing for individualism, it's about defining an alternate tribe

And don't kid yourself, it's about selling product

+ +

What if you're this poor fellow?

http://worstcats.tumblr.com

+ +

How much of your life is computerized?

How many of the four freedoms do you have with each of those things?

+ +

superfish
+
My laptop did not come with any software pre-installed that is designed to "improve my shopping experience"
+
If it had, I am empowered to remove it
+

+ +

Why are we here? I think many people assume, wrongly, that a company exists solely to make money. Money is an important part of a company's existence, if the company is any good. But a result is not a cause. We have to go deeper and find the real reason for our being.

David Packard

+ +

Free Software is a way that I can make the world a better place

+ +

Free Software is a way to reconnect with our humanity in a sea of white privilege

+ +

Free Software drives an unprecedented amount of innovation

+ +

Believe you can change the world

David Packard

+ +

Thank you!

http://inaugust.com/talks/lemmings.html

twitter: @e_monty

+ +

+ + +

+ + + + + + + + diff --git a/src/talks/now-what/index.html b/src/talks/now-what/index.html new file mode 100644 index 0000000..ca21fa8 --- /dev/null +++ b/src/talks/now-what/index.html @@ -0,0 +1,436 @@ + + + + + + + OpenStack works ... so now what? + + + + + + + + + + + + + + + + + + +

+ +

+ OpenStack works ... so now what? +

Monty Taylor

http://inaugust.com/talks/now-what.html

twitter: @e_monty

+ +

Who am I?

Distinguished Technologist

HP Cloud

+ +

Who am I?

Technical Committee

Foundation Board of Directors

Developer Infrastructure Core Team

+ +

What are we going to talk about?

OpenStack
My application
Your applications

+ +

As an application developer,
+ I want to deploy and run an application on the internet
+ so that my customers all over the world can consume it.

+ +

As an application developer,
+ I want to deploy the application across multiple clouds
+ so that my service survives issues in any one of them.

THIS WORKS

I'm doing it myself as we speak

+ + +

10-20k VMs per-day

3 (soon to be 6) clouds

Only using OpenStack APIs

+ +

OpenStack Infra

+ +

Tooling, Automation and CI for OpenStack Project

+ +

2000 Developers

+ +

Gated Commits

Every commit is fully integration tested (twice) before landing

+ +

Each Test Runs on a Single Use Cloud Slave

This is that "cloud scale out" part

+ +

1.7 Million Test Jobs in the last 6 Months

+ +

15 Million Tests in a month

+ +

18 Terabytes of Log Data in six months

+ +

We have no servers

It all runs across HP and Rackspace Public Clouds.

+ +

Architecture

+ +

Gerrit

+ Traditional 'Enterprise' Java Application
Single Nova VM, Cinder Volume
Scale out farm of git replicas

+ +

Fun Numbers

2500 changes every week
15000 change revisions every week
10,000 new changes every 42 days

+ +

nodepool

Cloud Native
+ Purpose built in Python
Keeps a pool of ready to go nodes
Multi-cloud
Fully elastic - responds to demand

+ +

OpenStack Works!

What next?

Make it easier

+ +

Basic things you want to do

+ +

Get (make/fetch/find) a base image

Upload it to each cloud

Boot a VM on one or more of the clouds

Ensure it's on the Internet

+ +

We've made this harder than it should be

+ +

Get a base image

OpenStack diskimage-builder

packer

Download pre-built image from Ubuntu/RedHat/SuSE

+ +

Problem: hypervisor image file format

+ Rackspace uses VHD +
+ HP uses qcow2 +
+ DreamHost uses RAW +

+ +

Problem: image API version

+ HP uses v1 +
+ vexxhost uses v2 +

Good news! We made a plan for this at + the summit

+ +

Problem: image task vs. PUT

+swift upload --object-name local-image-filename images image-name
+glance task-create
+   --type=import
+   --input='{"import_from": "images/image-name",
+             "image_properties" : {"name": "My Image Name"}}'
+

+glance image-create --name=image-name --file=local-image-filename
+

+ +

Problem: Ensure it's on the Internet

Cloud has externally routable IP from neutron (RunAbove, OVH)
Cloud has externally routable IP neutron AND supports optional private tenant networks (vexxhost)
Cloud has private tenant network provided by neutron and requires floating IP (HP, Dreamhost)
Cloud only has private tenant network provided by nova-network and requires floating-ip for external routing (auro)
Cloud has externally routable IP from neutron but no neutron APIs (Rackspace)

+ +

Maybe in code ...

+def get_server_external_ipv4(cloud, server):
+    if cloud.has_service('network'):
+        try:
+            server_ports = cloud.search_ports(
+                filters={'device_id': server.id})
+            ext_nets = cloud.search_networks(filters={'router:external': True})
+        except NeutronClientException as e:
+            pass  # fall through
+        else:
+            for net in ext_nets:
+                for port in server_ports:
+                    if net['id'] == port['network_id']:
+                        for ip in port['fixed_ips']:
+                            if _utils.is_ipv4(ip['ip_address']):
+                                return ip['ip_address']
+    ext_ip = get_server_ip(server, key_name='public')
+    if ext_ip is not None:
+        return ext_ip
+    for interfaces in server.addresses.values():
+        for interface in interfaces:
+            if _utils.is_ipv4(interface['addr']) and \
+                    _utils.is_globally_routable_ipv4(interface['addr']):
+                return interface['addr']
+    return None
+

+ +

I think we can do better than that

+ +

What am I doing about it?

+ +

os-client-config

http://git.openstack.org/cgit/openstack/os-client-config

A library to handle config information for openstack clients

Tracks differences in vendors that can't be discovered

In use in python-openstackclient and ansible

+ +

os-client-config

~/.config/openstack/clouds.yaml

+clouds:
+  hp-mordred:
+    profile: hp
+    auth:
+      username: mordred@inaugust.com
+      password: XXXXXXXXXXXXX
+      project_name: mordred@inaugust.com
+    region_name: region-b.geo-1
+  dreamhost:
+    profile: dreamhost
+    auth:
+      username: montay6
+      project_name: dhc2111978
+      password: XXXXXXXXXXXXX
+    region_name: RegionOne
+

+ +

shade

http://git.openstack.org/cgit/openstack-infra/shade

A library to wrap business logic around client libraries

+cloud.create_image('image-name', filename='image-filename.qcow2')
+cloud.create_server('my-server', image='immage-name', auto_ip=True)
+

In use in Infra Nodepool and ansible

+ +

ansible

Brand new modules, based on shade

Coming in 2.0 release

+- os_keypair:
+    cloud: hp-mordred
+    name: mordred
+    public_key_file: ~/.ssh/id_rsa.pub
+- os_image:
+    cloud: hp-mordred
+    name: Monty Ubuntu
+    file: ubuntu.vhd
+- os_server:
+    cloud: hp-mordred
+    name: my-server
+    flavor_ram: 1024
+    image: Monty Ubuntu
+

+ +

ansible

multi-cloud support

+- os_keypair:
+    cloud: "{{ item }"
+    name: mordred
+    public_key_file: ~/.ssh/id_rsa.pub
+    with-items:
+    - vexxhost
+    - rackspace
+    - mordred-hp
+    - ovh
+

+ +

What should we do about it?

+ Get back to basics +
+ shade existence is a bug +
+ Make some decisions about divergences in the basic levels +
+ Take a stand even if one of our product managers disagrees +
+ Ensure that simple things are simple +

+ + +

Thank you!

http://inaugust.com/talks/now-what.html

twitter: @e_monty

+ +

+ + +

+ + + + + + + + diff --git a/src/talks/os-client-config/index.html b/src/talks/os-client-config/index.html new file mode 100644 index 0000000..04ef2fd --- /dev/null +++ b/src/talks/os-client-config/index.html @@ -0,0 +1,242 @@ + + + + + + + os-client-config: Making OpenStack usability easier, starting with client configuration + + + + + + + + + + + + + + + + + + +

+ +

+ os-client-config +

Monty Taylor

http://inaugust.com/talks/os-client-config.html

twitter: @e_monty

+ +

I'm awash in cloud accounts

Infra has 5
- control plane and nodepool in 3 Rackspace regions
- control plane and nodepool in HP
two personal on HP and one on Rackspace
two internal HP accounts
That's 12 OpenStack cloud-regions

+ +

+ How do I connect to them? +

+ +

+ List Servers +

+openstack \
+        --auth-url=https://region-b.geo-1.identity.hpcloudsvc.com:35357/v2.0 \
+        --username=mordred@inaugust.com \
+        --password=XXXXXX \
+        --project-name=mordred@inaugust.com \
+        --region-name=region-b.geo-1 \
+    server list
+

+ +

+ Well that's silly +

+ +

+ Consistent Environment Variables +

+export OS_AUTH_URL=https://region-b.geo-1.identity.hpcloudsvc.com:35357/v2.0
+export OS_USERNAME=mordred@inaugust.com
+export OS_PASSWORD=XXXXXXXXXXXXX
+export OS_TENANT_NAME=mordred@inaugust.com
+export OS_REGION_NAME=region-b.geo-1
+

+ +

+ 12 shell script snippet files +

+ +

+ ENV processing is in command line tools, not client libraries +

+ +

+ ENV processing is just copied everywhere +

+$ egrep -r 'add_(option|argument).*os-username' openstack/python*client | wc -l
+13
+

+ +

Pre-existing knowledge

auth-url
glance API version
override settings
- URLs (swift URL in rackspace keystone catalog is wrong)
- service types (DNS in HP is hpext:dns)

+ +

I wrote a library (yay!)

os-client-config

+ +

Processes, in this order

config file: clouds.yaml
env vars
command line args

+ +

Provides vendor defaults

+CLOUD_DEFAULTS = dict(
+    hp=dict(
+        auth_url='https://region-b.geo-1.identity.hpcloudsvc.com:35357/v2.0',
+        dns_service_type='hpext:dns',
+    ),
+    rackspace=dict(
+        auth_url='https://identity.api.rackspacecloud.com/v2.0/',
+        database_service_type='rax:database',
+        image_api_version='2',
+    )
+)
+

Patches welcome for any cloud

+ +

clouds.yaml

+clouds:
+  mordred:
+    cloud: hp
+    username: mordred@inaugust.com
+    password: XXXXXXXXXX
+    project_name: mordred@inaugust.com
+    region_name: region-b.geo-1
+  monty:
+    cloud: https://region-b.geo-1.identity.hpcloudsvc.com:35357/v2.0
+    username: monty.taylor@hp.com
+    password: XXXXXXXXX
+    project_name: monty.taylor@hp.com-default-tenant
+    region_name: region-b.geo-1
+    dns_service_type: hpext:dns
+  rax:
+    cloud: rackspace
+    username: openstackci
+    password: XXXXXXX
+    project_id: 610275
+    region_name: DFW,ORD,IAD
+

+ +

Allows for named clouds

+openstack --cloud=mordred server list
+

+export OS_CLOUD=mordred
+openstack server list
+

+ + +

Where is it in use?

Shade library
python-openstackclient (patch in flight)
ansible (patch in flight)

+ +

Where is it?

http://git.openstack.org/stackforge/os-client-config

http://pypi.python.org/pypi/os-client-config

+ +

Where is it?

http://git.openstack.org/stackforge/os-client-config

http://pypi.python.org/pypi/os-client-config

+ +

+ + +

+ + + + + + + + diff --git a/src/talks/product-management/index.html b/src/talks/product-management/index.html new file mode 100644 index 0000000..58f8bd4 --- /dev/null +++ b/src/talks/product-management/index.html @@ -0,0 +1,454 @@ + + + + + + + Liberty, Product Management and OpenStack Technology + + + + + + + + + + + + + + + + + + +

+ +

+ Liberty, Product Management and OpenStack Technology +

Monty Taylor

http://inaugust.com/talks/product-management.html

twitter: @e_monty

+ +

Mission

+ to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable. +

+ +

Borrowed from Ubuntu

Time Based Releases

Design summits each cycle

Release codenames in alphabetical order

+ +

Different from Ubuntu

No BDFL

All decisions are democratic

+ +

"OpenStack needs product management"

+ +

Developers work on whatever they feel like, so there is no cohesion

not quite

+ +

OpenStack needs product management coordination

+ +

Product Management Working Group

Today 2:00pm - 3:30pm Room 212

+ +

What can the working group do?

+ Define problems +
+ Coordinate prorities +
+ Communicate problems clearly to tech community +

+ +

I've got 99 Problems ...

+ +

As an application developer,
+ I want to deploy and run an application on the internet
+ so that my customers all over the world can consume it.

+ +

As an application developer,
+ I want to deploy the application across multiple clouds
+ so that my service survives issues in any one of them.

+ +

THIS WORKS

I'm doing it myself as we speak

+ + +

10-20k VMs per-day

3 (soon to be 6) clouds

Only using OpenStack APIs

+ +

To do this, there are some basic steps

+ +

Get (make/fetch/find) a base image

Upload it to each cloud

Boot a VM on one or more of the clouds

Ensure it's on the Internet

+ +

Get a base image

OpenStack diskimage-builder

packer

Download pre-built image from Ubuntu/Fedora

+ +

NO

+ +

Problem: hypervisor image file format

+ Rackspace uses VHD +
+ HP uses qcow2 +
+ DreamHost uses RAW +

+ +

Upload it to each cloud

glance image-create

+ +

NO

+ +

Problem: image API version

+ HP uses v1 +
+ vexxhost uses v2 +

+ +

Problem: List API versions

Root of the Image API lists versions

keystone catalog only lists a versioned endpoint

+ +

Problem: API version discovery

Try one - if it doesn't work, try the other

maybe look at the end of the API endpoint for v1 or v2

+ +

Upload it to each cloud

glance image-create filename

+ +

NO

+ +

Problem: image task vs. PUT

+swift upload --object-name local-image-filename images image-name
+glance task-create
+   --type=import
+   --input='{"import_from": "images/image-name",
+             "image_properties" : {"name": "My Image Name"}}'
+

+glance image-create --name=image-name --file=local-image-filename
+

+ +

Boot a VM on one or more of the clouds

nova boot --image=image-name --flavor=something
+

+ +

NO

+ +

Problem: the image needs to get on the network

DHCP
Static Network Config in Config Drive
Vendor-specific agent doing file injection

+ +

nova boot --image=image-name --flavor=something
+

+ +

NO

+ +

Problem: Ensure it's on the Internet

My VM may have a public IP
My VM may need a floating IP from nova
My VM may need a floating IP from neutron

+ +

VM Network with nova-network

+        addresses:
+          private:
+          - addr: 10.181.5.77
+            version: 4
+          public:
+          - addr: 2001:4800:7810:512:be76:4eff:fe05:8325
+            version: 6
+          - addr: 98.129.169.30
+            version: 4
+

+ +

VM Network with nova-network

+        addresses:
+          mordred@inaugust.com-network:
+          - OS-EXT-IPS-MAC:mac_addr: fa:16:3e:a6:de:26
+            OS-EXT-IPS:type: fixed
+            addr: 10.0.0.6
+            version: 4
+          - OS-EXT-IPS-MAC:mac_addr: fa:16:3e:a6:de:26
+            OS-EXT-IPS:type: floating
+            addr: 15.126.239.219
+            version: 4
+

+ +

Boot a VM on one or more of the clouds

+nova boot --image=image-name --flavor=something --name=my-server
+nova floating-ip-create
+nova floating-ip-associate my-server {{ value_from_create }}
+

+ +

NO

+ +

Problem: My Internet server is behind a NAT

+ +

Problem: Security Groups

+
I was trying to spin up a web server, but all my ports were blocked
+
I'll be using ansible and puppet on this machine, I can configure iptables thanks
+

+ +

Boot a VM on one or more of the clouds

+nova secgroup-add-rule default tcp 80 80 0.0.0.0/0
+nova boot --image=image-name --flavor=something --name=my-server
+nova floating-ip-create
+nova floating-ip-associate my-server {{ value_from_create }}
+

+ +

Wow. So that's

+ Image Format +
+ Image API version +
+ Image upload mechanism +
+ Networking config / public private +
+ Networking config nova/neutron +
+ Floating IP? +
+ Floating IP nova/neutron +
+ DHCP or Static networking config +
+ Security Group config +

+ +

I think we can do better than that

+ +

What am I doing about it?

+ +

os-client-config

http://git.openstack.org/cgit/openstack/os-client-config

A library to handle config information for openstack clients

Tracks differences in vendors that can't be discovered

In use in python-openstackclient and ansible today

+ +

shade

http://git.openstack.org/cgit/openstack-infra/shade

A library to wrap business logic around client libraries

+cloud.create_server('my-server', auto_ip=True)
+

In use in Infra Nodepool and ansible today

+ +

Raising Issues

Product Management Working Group (today, 2pm, room 212)
DefCore (Wednesday 10:30 - 12:30, East Building Room 2/3)
Direct interaction with the teams and PTLs

+ +

What should we do about it?

+ Get back to basics +
+ shade existence is a bug +
+ Make some decisions about divergences in the basic levels +
+ Take a stand even if one of our product managers disagrees +
+ Ensure that simple things are simple +

+ + +

Thank you!

http://inaugust.com/talks/product-management.html

twitter: @e_monty

+ +

+ + +

+ + + + + + + + diff --git a/src/talks/tripleo-ansible/index.html b/src/talks/tripleo-ansible/index.html new file mode 100644 index 0000000..58814df --- /dev/null +++ b/src/talks/tripleo-ansible/index.html @@ -0,0 +1,794 @@ + + + + + + + TripleO and Ansible + + + + + + + + + + + + + + + + + + +

+ + +

+ +

+ TripleO and Ansible +

Monty Taylor

http://inaugust.com/talks/tripleo-ansible.html

twitter: @e_monty

+ +

Who am I?

Distinguished Technologist at HP
OpenStack Technical Committee
OpenStack Foundation Board of Directors
OpenStack Infra Core Team

+ +

What are we going to talk about?

TripleO
Ironic
Ansible

+ +

What is TripleO?

an idea
OpenStack On OpenStack
use Ironic-based OpenStack to Operate Openstack

+ + +

Velocity

Cloud Empowers developers
Cloud Enables Increased Velocity
Cloud Drives Agility

+ +

OpenStack is not a virtualization layer, + it's an abstraction layer.

+ +

What if your developers aren't writing Angry Birds?

+ +

What if your developers aren't developers?

+ +

If infrastructure is code ...

What if your developers are operators?

+ +

Why wouldn't you give your operators the same power as your + developers?

Don't you like them?

+ +

Ironic is a service capable of
+ managing and provisioning
+ physical machines.

+ +

Nova + Ironic

+ Same but different + +

+ +


+$ ironic node-create -d pxe_ipmitool \
+  -i ipmi_username=admin -i ipmi_password=fake -i ipmi_address=10.1.2.3 \
+  -p cpus=4 -p memory_mb=8192 -p local_gb=500 \
+  -e note='spare server'
++--------------+------------------------------------------------------------+
+| Property     | Value                                                      |
++--------------+------------------------------------------------------------+
+| chassis_uuid | None                                                       |
+| driver       | pxe_ipmitool                                               |
+| driver_info  | {u'ipmi_address': u'10.1.2.3', u'ipmi_username': u'admin', |
+|              | u'ipmi_password': u'fake'}                                 |
+| extra        | {u'note': u'spare server'}                                 |
+| properties   | {u'memory_mb': u'8192', u'local_gb': u'500', u'cpus': u'4'}|
+| uuid         | 7a1ce8d0-9679-4d87-8f54-b11f6e8adb8f                       |
++--------------+------------------------------------------------------------+
+


+$ tail -f /var/log/nova/n-cpu.log
+...
+2014-05-01 03:47:05.878 AUDIT nova.compute.resource_tracker [-]
+    Free ram (MB): 8192
+2014-05-01 03:47:05.878 AUDIT nova.compute.resource_tracker [-]
+    Free disk (GB): 500
+2014-05-01 03:47:05.878 AUDIT nova.compute.resource_tracker [-]
+    Free VCPUS: 4
+

+ +

Diversity is good

Cloud providers: HP, Rackspace, Dreamhost, Cloudwatt
Deployments: Public, Managed, Private
Compute drivers: virtual, bare metal, container
Ironic drivers: pxe/impi, ilo, HP OneView

+ +

What can you do?

anything - it's a cloud

+ +

What the heck has the TripleO team been doing then?

+ +

TripleO

Community developed
Opinionated
Avoid distro and config management religion
Be a usable real deployment we can gate on

+ +

Community Developed

Exist as part of the OpenStack project
Prove the story end to end
Subject to TC governance
Tighter feedback loop

+ +

Avoid distro and config management religion

rpm vs. deb - in the gate == rpm + deb
puppet vs. chef vs. salt vs. ansible == all of them
Choosing one excludes other folks from participating

+ +

Opinionated

Golden Images
Upgrade tied to HA
Target Continual Delivery
Drive fixes into OpenStack directly

+ +

Major Components

nova+ironic
heat
diskimage-builder
os-collect-config
os-apply-config
os-refresh-config

+ +

Lesson from os-*-config

+ +

Lesson from os-*-config

+ +

Lesson from os-*-config

If you ever think "oh, that's silly, it would be so much easier + if I just ..."
it will almost never actually be easier if you + just ...

+ +

The Deployment Story

disk-image-builder builds images and uploads to glance
Heat drives Nova/Ironic
Heat delivers metadata to os-collect-config
os-collect-config applies any in-instance changes needed

+ +

The Update Story

Heat magically just updates things +

+ +

BUT I ALREADY USE ?????

(screw you guys, I'm going home)

+ +

Yeah, I do to

(I use puppet and ansible myself)

+ +

That's fine - use them - it's a cloud!

This is supposed to be empowering, not enforcing

+ +

Whatever you want!

Heat to deploy and update images, os-*-*config for config
Heat to deploy images, ansible to update images, puppet for config
Ansible to deploy base image + packages, salt to update packages, chef for config
juju to deploy ... nah, I'm just kidding

+ +

The New Update Story

Ansible takes over for upgrades

+ +

Ansible for Orchestration

+ +

About Ansible

Open Source System Management tool
Written in Python
Sequence of steps to perform
Works over SSH
Incremental Adoption

+ +

ad-hoc operation

+ansible '*' -m shell -p uptime
+

+ +

YAML Syntax


+- hosts: '*.slave.openstack.org'
+  tasks:
+    - shell: 'rm -rf ~jenkins/workspace/*{{ project }}*'
+

That's executed:

+ansible-playbook -f 10 /etc/ansible/clean_workspaces.yaml --extra-vars "project=$PROJECTNAME"
+

+ +

Ansible Organization

modules
plays
playbooks
roles

+ +

Use Ansible to Run Puppet!

+ +

puppet module

def main():
+    module = AnsibleModule(argument_spec=dict(
+        timeout=dict(default="30m"),
+        puppetmaster=dict(required=True),
+        show_diff=dict(default=False, aliases=['show-diff'], type='bool'),
+    ))
+    p = module.params
+
+    puppet_cmd = module.get_bin_path("puppet", False)
+    if not puppet_cmd:
+        module.fail_json(msg="Could not find puppet. Please ensure it is installed.")
+

+ +

puppet module (cont)


+    cmd = ("timeout -s 9 %(timeout)s %(puppet_cmd)s agent --onetime"
+           " --server %(puppetmaster)s"
+           " --ignorecache --no-daemonize --no-usecacheonfailure --no-splay"
+           " --detailed-exitcodes --verbose") % dict(
+               timeout=pipes.quote(p['timeout']), puppet_cmd=PUPPET_CMD,
+               puppetmaster=pipes.quote(p['puppetmaster']))
+    if p['show_diff']:
+        cmd += " --show-diff"
+    rc, stdout, stderr = module.run_command(cmd)
+

+ +

+ Please. Everyone. Marvel at the following logic +


+    if rc == 0:  # success
+        module.exit_json(rc=rc, changed=False, stdout=stdout)
+    elif rc == 1:
+        # rc==1 could be because it's disabled OR there was a compilation failure
+        disabled = "administratively disabled" in stdout
+        if disabled:
+            msg = "puppet is disabled"
+        else:
+            msg = "puppet compilation failed"
+        module.fail_json(rc=rc, disabled=disabled, msg=msg, stdout=stdout, stderr=stderr)
+    elif rc == 2:  # success with changes
+        module.exit_json(changed=True)
+    elif rc == 124:  # timeout
+        module.exit_json(rc=rc, msg="%s timed out" % cmd, stdout=stdout, stderr=stderr)
+    else:  # failure
+        module.fail_json(rc=rc, msg="%s failed" % (cmd), stdout=stdout, stderr=stderr)
+

+ +

puppet play


+- name: run puppet
+  puppet:
+    puppetmaster: "{{puppetmaster}}"
+

+ +

puppet role

roles/remote_puppet/tasks/main.yml

+ +

remote puppet playbook


+- hosts: git0*
+  gather_facts: false
+  max_fail_percentage: 1
+  roles:
+    - { role: remote_puppet, puppetmaster: puppetmaster.openstack.org }
+- hosts: review.openstack.org
+  gather_facts: false
+  roles:
+    - { role: remote_puppet, puppetmaster: puppetmaster.openstack.org }
+- hosts: "!review.openstack.org:!git0*:!afs*"
+  gather_facts: false
+  roles:
+    - { role: remote_puppet, puppetmaster: puppetmaster.openstack.org }
+

+ +

ansible inventory

List of servers to operate on
Optionally variables associated with each server
Optional groups of servers
Simple file in /etc/ansible/hosts
Dynamic executable that returns JSON

+ +

Simple inventory

+review.openstack.org
+git01.openstack.org
+git02.openstack.org
+pypi.dfw.openstack.org
+pypi.iad.openstack.org
+
+[pypi]
+pypi.dfw.openstack.org
+pypi.iad.openstack.org
+
+[git]
+git01.openstack.org
+git02.openstack.org
+

+ +

ansible inventory from puppet certs


+import json
+import subprocess
+
+output = [
+    x.split()[1][1:-1] for x in subprocess.check_output(
+        ["puppet","cert","list","-a"]).split('\n')
+    if x.startswith('+')
+]
+
+data = {
+    '_meta': {'hostvars': dict()},
+    'ungrouped': output,
+}
+print json.dumps(data, sort_keys=True, indent=2)
+

+ +

Ansible for Cloud Management

+ +

ansible and OpenStack

Ansible modules are just python
playbooks are lists of steps to take
Have plays/roles that provision servers
Infrastructure as code - for real!

+ +

Consider this data


+pypi:
+  image_name: Ubuntu 12.04.4
+  flavor_ram: 2048
+  provision_group: ubuntu_hosts
+  volumes:
+    - size: 200
+      mount: /srv
+  hosts:
+    pypi.dfw:
+      region: DFW
+    pypi.iad:
+      region: IAD
+    pypi.ord:
+      region: ORD
+    pypi.region-b.geo-1:
+      cloud: hp
+

+ +

Steps to launch a node

Create a compute instance
Wait for instance to exist
Create a floating IP
Attach floating IP to instance
Create one or more volumes
Attach volumes to instance
Wait for SSH to work
On host, format each volume
On host, mount each volume
On host, install config management software
On host, run config management software

+ +

Launch a node


+---
+- name: Launch Node
+  os_compute:
+      cloud: "{{ cloud }}"
+      region_name: "{{ region_name }}"
+      name: "{{ name }}"
+      image_name: "{{ image_name }}"
+      flavor_ram: "{{ flavor_ram }}"
+      flavor_include: "{{ flavor_include }}"
+      meta:
+          group: "{{ group }}"
+      key_name: "{{ launch_keypair }}"
+  register: node
+- name: Create volumes
+  os_volume:
+      cloud: "{{ cloud }}"
+      size: "{{ item.size }}"
+      display_name: "{{ item.display_name }}"
+  with_items: volumes
+- name: Attach volumes
+  os_compute_volume:
+      cloud: "{{ cloud }}"
+      server_id: "{{ node.id }}"
+      volume_name: "{{ item.display_name }}"
+  with_items: volumes
+  register: attached_volumes
+- debug: var=attached_volumes
+- name: Re-request server to get up to date metadata after the volume loop
+  os_compute_facts:
+      cloud: "{{ cloud }}"
+      name: "{{ name }}"
+  when: attached_volumes.changed
+- name: Wait for SSH to work
+  wait_for: host={{ node.openstack.interface_ip }} port=22
+  when: node.changed == True
+- name: Add SSH host key to known hosts
+  shell: ssh-keyscan "{{ node.openstack.interface_ip|quote }}" >> ~/.ssh/known_hosts
+  when: node.changed == True
+- name: Add all instance public IPs to host group
+  add_host:
+      name: "{{ node.openstack.interface_ip }}"
+      groups: "{{ provision_group }}"
+      openstack: "{{ node.openstack }}"
+  when: attached_volumes|length == 0
+- name: Add all instance public IPs to host and volumes group
+  add_host:
+      name: "{{ node.openstack.interface_ip }}"
+      groups: "{{ provision_group }},hasvolumes"
+      openstack: "{{ node.openstack }}"
+  when: attached_volumes|length != 0
+

+ +

Cloud based inventory

Just ask the cloud for the inventory
All of the meta-data the cloud knows is available

+ +


+      "pypi.dfw.openstack.org": {
+        "ansible_ssh_host": "23.253.237.8",
+        "openstack": {
+          "HUMAN_ID": true,
+          "NAME_ATTR": "name",
+          "OS-DCF:diskConfig": "MANUAL",
+          "OS-EXT-STS:power_state": 1,
+          "OS-EXT-STS:task_state": null,
+          "OS-EXT-STS:vm_state": "active",
+          "accessIPv4": "23.253.237.8",
+          "accessIPv6": "2001:4800:7817:104:d256:7a33:5187:7e1b",
+          "addresses": {
+            "private": [
+              {
+                "addr": "10.208.195.50",
+                "version": 4
+              }
+            ],
+            "public": [
+              {
+                "addr": "23.253.237.8",
+                "version": 4
+              },
+              {
+                "addr": "2001:4800:7817:104:d256:7a33:5187:7e1b",
+                "version": 6
+              }
+            ]
+          },
+          "cloud": "rax",
+          "config_drive": "",
+          "created": "2014-09-05T15:32:14Z",
+          "flavor": {
+            "id": "performance1-4",
+            "links": [
+              {
+                "href": "https://dfw.servers.api.rackspacecloud.com/610275/flavors/performance1-4",
+                "rel": "bookmark"
+              }
+            ],
+            "name": "4 GB Performance"
+          },
+          "hostId": "adb603d4566efe0392756c76dab38ffcba22099368837c7973321e77",
+          "human_id": "pypidfwopenstackorg",
+          "id": "de672205-9245-46b6-b3df-489ccf9e0c17",
+          "image": {
+            "id": "928e709d-35f0-47eb-b296-d18e1b0a76b7",
+            "links": [
+              {
+                "href": "https://dfw.servers.api.rackspacecloud.com/610275/images/928e709d-35f0-47eb-b296-d18e1b0a76b7",
+                "rel": "bookmark"
+              }
+            ]
+          },
+          "interface_ip": "23.253.237.8",
+          "key_name": "launch-node-root",
+          "links": [
+            {
+              "href": "https://dfw.servers.api.rackspacecloud.com/v2/610275/servers/de672205-9245-46b6-b3df-489ccf9e0c17",
+              "rel": "self"
+            },
+            {
+              "href": "https://dfw.servers.api.rackspacecloud.com/610275/servers/de672205-9245-46b6-b3df-489ccf9e0c17",
+              "rel": "bookmark"
+            }
+          ],
+          "metadata": {},
+          "name": "pypi.dfw.openstack.org",
+          "networks": {
+            "private": [
+              "10.208.195.50"
+            ],
+            "public": [
+              "23.253.237.8",
+              "2001:4800:7817:104:d256:7a33:5187:7e1b"
+            ]
+          },
+          "progress": 100,
+          "region": "DFW",
+          "status": "ACTIVE",
+          "tenant_id": "610275",
+          "updated": "2014-09-05T15:32:49Z",
+          "user_id": "156284",
+          "volumes": [
+            {
+              "HUMAN_ID": false,
+              "NAME_ATTR": "name",
+              "attachments": [
+                {
+                  "device": "/dev/xvdb",
+                  "host_name": null,
+                  "id": "c6f5229c-1cc0-47c4-aab7-60db1f6cf8e8",
+                  "server_id": "de672205-9245-46b6-b3df-489ccf9e0c17",
+                  "volume_id": "c6f5229c-1cc0-47c4-aab7-60db1f6cf8e8"
+                }
+              ],
+              "availability_zone": "nova",
+              "bootable": "false",
+              "created_at": "2014-09-05T14:37:42.000000",
+              "device": "/dev/xvdb",
+              "display_description": null,
+              "display_name": "pypi.dfw.openstack.org/main01",
+              "human_id": null,
+              "id": "c6f5229c-1cc0-47c4-aab7-60db1f6cf8e8",
+              "metadata": {
+                "readonly": "False",
+                "storage-node": "1845027a-5e07-47a1-9572-3eea4716f726"
+              },
+              "os-vol-tenant-attr:tenant_id": "610275",
+              "size": 200,
+              "snapshot_id": null,
+              "source_volid": null,
+              "status": "in-use",
+              "volume_type": "SATA"
+            }
+          ]
+        }
+      },
+

+ + +

Thank you!

Monty Taylor

twitter: @e_monty

http://inaugust.com/talks/tripleo-ansible.html

+ +

+ + +

+ + + + + + + + -- cgit v1.2.3