From 63d06e732b6e5accee51587648882109a5b44079 Mon Sep 17 00:00:00 2001 From: Monty Taylor Date: Tue, 9 Oct 2018 01:52:18 +0200 Subject: Add initial work on tutorial --- src/zuulv3/tutorial.rst | 1611 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1611 insertions(+) create mode 100644 src/zuulv3/tutorial.rst (limited to 'src/zuulv3') diff --git a/src/zuulv3/tutorial.rst b/src/zuulv3/tutorial.rst new file mode 100644 index 0000000..7272594 --- /dev/null +++ b/src/zuulv3/tutorial.rst @@ -0,0 +1,1611 @@ +. display in 68x24 +.. display in 88x24 + +.. pygments yaml? (only file breaks (---) tinted) +.. slide on high level v3 changes +.. slide on nodepool + +.. transition:: dissolve + :duration: 0.4 + +Test Slide +========== +.. hidetitle:: + +.. ansi:: images/testslide.ans + +Preshow +======= +.. hidetitle:: + +.. ansi:: images/cursor.ans images/cursor2.ans + +Zuul +==== +.. hidetitle:: +.. ansi:: images/title.ans + +Overview +======== + +* Discussion of concepts +* Installation of software +* Configurting Zuul +* Writing jobs + +Please Ask Questions +==================== + +Pre-show +======== + +While we talk about other things ... + +* Install docker, docker-compose, git-review + +Debian/Ubuntu: + +:: + + apt-get install docker-compose git git-review + +Red Hat / SuSE + +:: + + yum install docker-compose git git-review + +* git clone https://git.zuul-ci.org/zuul +* cd zuul +* git review -d 608344 +* cd doc/source/admin/examples +* docker-compose up + +Output in docker-compose window +=============================== + +* All services running with debug logging to stdout +* Tons of information will have been output - including some errors +* Zuul connects to Gerrit before it's fully configured +* As it becomes configured, Zuul notices and becomes happy +* Once happy, it should stablize and become idle + +We'll come back to this +======================= + +It's going to do a bunch of network - and this is a conference. + +Red Hat +======= +.. hidetitle:: +.. container:: handout + i work for + +.. ansi:: images/redhat.ans + +OpenStack +========= +.. hidetitle:: +.. ansi:: images/openstack.ans + +OpenStack Infra +=============== + +:: + + "most insane CI infrastructure I've ever been a part of" + + -- Alex Gaynor + + "OpenStack Infra are like the SpaceX of CI" + + -- Emily Dunham + +Zuul +==== +.. hidetitle:: +.. ansi:: images/zuul.ans + + +Spoilers +======== + +* What Zuul does + + * multiple repositories + * integrated deliverable + * gated commits + * open tooling + * nobody is special + * testing like deployment + +OpenStack Is +============ + + * Federated + * Distributed + * Large + * Open + * Not Alone + +Federated +========= + + * Hundreds of involved companies + * No 'main' company + * "Decisions are made by those who show up" + * Union of priorities/use cases + +Impact of being Federated +========================= + + * No company can appoint people to positions in the project + * The project cannot fire anyone + * Variable background of contributors + * Heavy reliance on consensus + +Distributed +=========== + + * There is no office + * Contributor base is global + * Multitude of contributor backgrounds + +Impact of being Distributed +=========================== + + * Tooling must empower all contributors, regardless of background, + skill level or cultural context + * Heavy preference for text-based communication + * Cannot assume US-centric needs or solutions + +Large numbers of +================ + + * Contributors (\~2k in any given 6 month period) + * Changes + * Code Repositories (1955 as of this morning) + +Not Bragging About Scale +======================== + +OpenStack Scale Comparison +========================== + + * 2KJPH (2,000 jobs per hour) + * Build Nodes from 13 Regions of 5 Public and 2 Private OpenStack Clouds + * Rackspace, Internap, OVH, Vexxhost, CityCloud and Linaro, Limestone + * 10,000 changes merged per month + +OpenStack Scale Comparison +========================== + + * 2KJPH (2,000 jobs per hour) + * Build Nodes from 13 Regions of 5 Public and 2 Private OpenStack Clouds + * Rackspace, Internap, OVH, Vexxhost, CityCloud and Linaro, Limestone + * 10,000 changes merged per month + + * By comparison, our friends at the amazing project Ansible received + 13,000 changes and had merged 8,000 of them in its first 4 years. + +Four Opens +========== + + * Open Source + (we don't hold back Enterprise features, we don't cripple things) + * Open Design + (design process open to all, decisions are not made inside company doors) + * Open Development + (public source code, public code review, all code is reviewed and gated) + * Open Community + (lazy consensus, democratic leadership from participants, + public logged meetings in IRC, public archived mailing lists) + +We're Not Alone +=============== + + * Dependencies (libvirt/kvm/xen, mysql/pg, rabbit, + python/javascript, ceph/gluster, ansible/salt/puppet/chef, ovs/odl) + * Adjacencies (kubernetes, ansible, terraform, opnfv, spinnaker) + * Vendors (plugins, products, services, distros) + +Developer Process In a Nutshell +=============================== + + * Code Review - nobody has direct commit/push access + * 3rd-Party CI for vendors + * Gated Commits + +OpenStack Developer Workflow +============================ +.. container:: handout + + * Who has submitted a patch? + * Who wants to? + * (Who is here because the name of this talk is weird?) + +:: + + Hack Review Test + ========= ========== ========== + + push approve + +-------------+ +-------------+ + | | | | + +------+--+ +--v----+--+ +--v-------+ + | | | | | | + | $EDITOR | | Gerrit | | Zuul | + | | | | | | + +------^--+ +--+----^--+ +--+-------+ + | | | | + +-------------+ +-------------+ + clone merge + +Gerrit +====== +.. hidetitle:: +.. container:: handout + + explain patch upload, zuul runs, test results displayed in gerrit + this is all the interface to zuul users need to see + + switch to actual gertty screenshot + + also show zuul status page + + but zuul is doing a lot of work behind the scenes, and if you look + closer, this is what you see + +.. ansi:: images/color-gertty.ans + +Zuul Architecture +================= + +.. ansi:: images/architecture.ans + +Nodepool +======== + +* A separate program that works very closely with *Zuul* +* Builds images daily and uploads to clouds +* Creates and destroys (at least) a vm for every job + + (Remember that 2,000 jobs per hour number?) + +Zuul is not New +=============== + + * Has been in Production for OpenStack for Six Years + * Zuul v3 first release where not-OpenStack is first-class use case + * Zuul is now a top-level effort of OpenStack Foundation + +Not just for OpenStack +====================== + + * Zuul is in production for OpenStack (in OpenStack VMs) + + Also running at: + + * BMW (control plane in OpenShift) + * Easystack + * GoDaddy (control plane in Kubernetes) + * OpenContrail + * OpenLab + * Red Hat + * others ... + +Zuul in a nutshell +================== + + * Listens for code events + * Prepares appropriate job config and git repo states + * Allocates nodes for test jobs + * Pushes git repo states to nodes + * Runs user-defined Ansible playbooks + * Collects/reports results + * Potentially merges change + +All in Service of Gating +======================== + +Gating +====== + + Every change proposed for a repository is tested before it merges. + +Co-gating +========= + + Changes to a set of repositories merge monotonically such + that each change is tested with the current state of all + the other related repositories before it merges. + +Parallel Co-gating +================== + + Changes are serialized such that each change is tested + with all of the changes ahead of it to satisfy the + co-gating requirement while being able to run tests for + multiple changes simultaneously. + +Zuul Simulation +=============== +.. transition:: pan +.. container:: handout + + * todo + +.. ansi:: images/zsim-00.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-01.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-02.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-03.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-04.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-05.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-06.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-07.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-08.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-09.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-10.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-11.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-12.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-13.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-14.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-15.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-16.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-17.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-18.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-19.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-20.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-21.ans + +Zuul Simulation +=============== +.. transition:: cut +.. container:: handout + + * todo + +.. ansi:: images/zsim-22.ans + +Cross-Project Dependencies +========================== + +Testing or gating dependencies manually specified by developers + +.. container:: progressive + + * shade https://review.openstack.org/513913 + + Add unittest tips jobs + * os-client-config https://review.openstack.org/513915 + + Add shade-tox-tips jobs + + Depends-On: https://review.openstack.org/513913 + * os-client-config https://review.openstack.org/513751 + + Added nat_source flag for networks + + Depends-On: https://review.openstack.org/513915 + * shade https://review.openstack.org/51391 + + Add support for configured NAT source variable + + Depends-On: https://review.openstack.org/513751 + + +Live Configuration Changes +========================== + +.. container:: handout + + Zuul is a distributed system, with a distributed configuration. + +.. code:: yaml + + - tenant: + name: openstack + source: + gerrit: + config-repos: + - openstack-infra/project-config + project-repos: + - openstack/nova + - openstack/keystone + - openstack-infra/devstack-gate + +Zuul Startup +============ + +* Read config file + +Zuul Startup +============ + +* Read config file +* Ask mergers for branches of each repo + +.. ansi:: images/startup1.ans + +Zuul Startup +============ + +* Read config file +* Ask mergers for branches of each repo +* Ask mergers for .zuul.yaml for each branch + + of each repo + +.. ansi:: images/startup2.ans + +When .zuul.yaml Changes +======================= + +.. container:: progressive + + * Zuul looks for changes to .zuul.yaml + * Asks mergers for updated content + * Splices into configuration used for that change + * Works with cross-repo dependencies + + ("This change depends on a change to the job definition") + +How do you use this thing? +========================== +.. transition:: tilt +.. hidetitle:: +.. figlet:: Configuration + +Human Roles +=========== + +* Deployer +* Project Admin +* End User + +Deployer Config +=============== + +Zuul: Connection, Triggers, Reporters +Nodepool: Launcher and Builder Config + +Connection Plugins +================== + +Describes how Zuul connects to external systems + +* Gerrit +* Github +* git +* mqtt +* smtp +* sql + +Trigger Plugins +=============== + +Input to Zuul for use in causing jobs to run + +* Gerrit +* Github +* git +* zuul + +Reporter Plugins +================ + +Where Zuul should send information about jobs + +* Gerrit +* Github +* mqtt +* smtp +* sql + +Nodepool Launcher +================= + +Where build nodes should come from + +* OpenStack +* Static + +In works: + +* Kubernetes +* OpenShift +* ec2 +* Azure + +Nodepool Builder +================ + +Optionally periodically build and upload new base images + +* OpenStack + +main.yaml +========= + +* Most of Zuul's config comes from git repos +* main.yaml is static config +* what git repos to manage +* what config objects to load +* which repos to get config from + +Config Project vs. Untrusted Project +==================================== + +config project + +* special project containing project admin content +* has access to features that are normally restricted +* job changes are not applied speculatively + +untrusted project + +* most projects +* some actions (like executing code on localhost) are blocked +* job changes are applied speculatively + +Job Config +========== + +Pipelines +========= + +* A process definition that connects git repositories, jobs, and + reporting mechanisms. +* A context to fix a set of jobs to each project. +* Pipeline Managers are Plugins: dependent, independent, supercedent + +Check Pipeline +============== + +.. code:: yaml + + - pipeline: + name: check + manager: independent + source: gerrit + trigger: + gerrit: + - event: patchset-created + - event: change-restored + success: + gerrit: + verified: 1 + +Gate Pipeline +============= + +.. code:: yaml + + - pipeline: + name: gate + manager: dependent + source: gerrit + trigger: + gerrit: + - event: comment-added + approval: + - workflow: 1 + success: + gerrit: + verified: 2 + submit: true + +Jobs +==== + +* Jobs run on nodes from nodepool (static or dynamic) +* Metadata defined in Zuul's configuration +* Execution content in Ansible +* Jobs may be defined centrally or in the repo being tested +* Jobs have contextual variants that simplify configuration + +zuul-jobs +========= + +* Zuul jobs are all defined in git repositories +* Designed to be directly shared across zuul installations +* Standard library: https://git.zuul-ci.org/zuul-jobs +* Zuul installs should add ``zuul-jobs`` to their config +* As changes land in ``zuul-jobs`` - Zuul installs will get them automatically + +Job +=== + +.. code:: yaml + + - job: + name: base + parent: null + description: | + The base job for Zuul. + timeout: 1800 + nodeset: + nodes: + - name: primary + label: centos-7 + pre-run: playbooks/base/pre.yaml + post-run: + - playbooks/base/post-ssh.yaml + - playbooks/base/post-logs.yaml + secrets: + - site_logs + +Simple Job +========== + +.. code:: yaml + + - job: + name: tox + pre-run: playbooks/setup-tox.yaml + run: playbooks/tox.yaml + post-run: playbooks/fetch-tox-output.yaml + +Simple Job Inheritance +====================== + +.. code:: yaml + + - job: + name: tox-py36 + parent: tox + vars: + tox_envlist: py36 + +Inheritance Works Like An Onion +=============================== + + * pre-run playbooks run in order of inheritance + * run playbook of job runs + * post-run playbooks run in reverse order of inheritance + * If pre-run playbooks fail, job is re-tried + * All post-run playbooks run - as far as pre-run playbooks got + +Inheritance Example +=================== + +For tox-py36 job + + * base pre-run playbooks/base/pre.yaml + * tox pre-run playbooks/setup-tox.yaml + * tox run playbooks/tox.yaml + * tox post-run playbooks/fetch-tox-output.yaml + * base post-run playbooks/base/post-ssh.yaml + * base post-run playbooks/base/post-logs.yaml + +Simple Job Variant +================== + +.. code:: yaml + + - job: + name: tox-py27 + branches: stable/mitaka + nodeset: + - name: ubuntu-trusty + label: ubuntu-trusty + +Nodesets for Multi-node Jobs +============================ + +.. code:: yaml + + - nodeset: + name: ceph-cluster + nodes: + - name: controller + label: centos-7 + - name: compute1 + label: fedora-28 + - name: compute2 + label: fedora-28 + groups: + - name: ceph-osd + nodes: + - controller + - name: ceph-monitor + nodes: + - controller + - compute1 + - compute2 + +Multi-node Job +============== + +* nodesets are provided to Ansible for jobs in inventory + +.. code:: yaml + + - job: + name: ceph-multinode + nodeset: ceph-cluster + run: playbooks/install-ceph.yaml + +Multi-node Ceph Job Content +=========================== + +.. code:: yaml + + - hosts: all + roles: + - install-ceph + + - hosts: ceph-osd + roles: + - start-ceph-osd + + - hosts: ceph-monitor + roles: + - start-ceph-monitor + + - hosts: all + roles: + - do-something-interesting + +Projects +======== + +* Projects are git repositories +* Specify a set of jobs for each pipeline +* golang git repo naming as been adopted: + +:: + + zuul@ubuntu-xenial:~$ find /home/zuul/src -mindepth 3 -maxdepth 3 -type d + /home/zuul/src/git.openstack.org/openstack-infra/shade + /home/zuul/src/git.openstack.org/openstack/keystoneauth + /home/zuul/src/git.openstack.org/openstack/os-client-config + /home/zuul/src/github.com/ansible/ansible + +Project Config +============== + + * Specify a set of jobs for each pipeline + +.. code:: yaml + + - project: + check: + jobs: + - openstack-tox-py27 + - openstack-tox-py35 + - openstack-tox-docs + gate: + jobs: + - openstack-tox-py27 + - openstack-tox-py35 + - openstack-tox-docs + +Project with Local Variant +========================== + +.. code:: yaml + + - project: + check: + jobs: + - openstack-tox-py27 + - openstack-tox-py35 + - openstack-tox-py36: + voting: false + - openstack-tox-docs + gate: + jobs: + - openstack-tox-py27 + - openstack-tox-py35 + - openstack-tox-docs + +Project with More Local Variants +================================ + +.. code:: yaml + + - project: + check: + jobs: + - openstack-tox-py27 + - openstack-tox-py35 + - openstack-tox-py36: + voting: false + - openstack-tox-docs: + files: '^docs/.*$' + +Project with Many Local Variants +================================ + +.. code:: yaml + + - project: + check: + jobs: + - openstack-tox-py27: + nodeset: + - name: centos-7 + label: centos-7 + - openstack-tox-py27: + branches: stable/newton + nodeset: + - name: ubuntu-trusty + label: ubuntu-trusty + - openstack-tox-py35 + - openstack-tox-py36: + voting: false + - openstack-tox-docs: + files: '^docs/.*$' + +Project With Central and Local Config +===================================== + +.. code:: yaml + + # In git.openstack.org/openstack-infra/project-config: + - project: + name: openstack/nova + templates: + - openstack-tox-jobs + +.. code:: yaml + + # In git.openstack.org/openstack/nova/.zuul.yaml: + - project: + check: + - nova-placement-functional-devstack + +Project with Job Dependencies +============================= + +.. code:: yaml + + - project: + release: + jobs: + - build-artifacts + - upload-tarball: + dependencies: build-artifacts + - upload-pypi: + dependencies: build-artifacts + - notify-mirror: + dependencies: + - upload-tarball + - upload-pypi + +Playbooks +========= + +* Jobs run playbooks +* Playbooks may be defined centrally or in the repo being tested +* Playbooks can use roles from current or other Zuul repos or Galaxy +* Playbooks are not allowed to execute content on 'localhost' + +devstack-tempest Run Playbook +============================= + +.. code:: yaml + + # Changes that run through devstack-tempest are likely to have an impact on + # the devstack part of the job, so we keep devstack in the main play to + # avoid zuul retrying on legitimate failures. + - hosts: all + roles: + - run-devstack + + # We run tests only on one node, regardless how many nodes are in the system + - hosts: tempest + roles: + - setup-tempest-run-dir + - setup-tempest-data-dir + - acl-devstack-files + - run-tempest + +Simple Shell Playbook +===================== + +.. code:: yaml + + hosts: controller + roles: + - shell: | + cd {{ zuul.project.src_dir }} + ./run_tests.sh + + +Test Like Production +==================== + +If you use Ansible for deployment, your test and deployment processes +and playbooks are the same + +What if you don't use Ansible? +============================== + +OpenStack Infra Control Plane uses Puppet (for now) +=================================================== + +.. code:: yaml + + # In git.openstack.org/openstack-infra/project-config/roles/legacy-install-afs-with-puppet/tasks/main.yaml + - name: Install puppet + shell: ./install_puppet.sh + args: + chdir: "{{ ansible_user_dir }}/src/git.openstack.org/openstack-infra/system-config" + environment: + # Skip setting up pip, our images have already done this. + SETUP_PIP: "false" + become: yes + + - name: Copy manifest + copy: + src: manifest.pp + dest: "{{ ansible_user_dir }}/manifest.pp" + + - name: Run puppet + puppet: + manifest: "{{ ansible_user_dir }}/manifest.pp" + become: yes + +Secrets +======= + +* Inspired by Kubernetes Secrets API +* Projects can add named encrypted secrets to their .zuul.yaml file +* Jobs can request to use secrets by name +* Jobs using secrets are not reconfigured speculatively +* Secrets can only be used by the same project they are defined in +* Public key per project: + ``{{ zuul_url }}/{{ tenant }}/{{ project }}.pub`` + +:: + GET https://zuul.openstack.org/openstack-infra/shade.pub + +Secret Example (note, no admins had to enable this) +=================================================== + +.. code:: yaml + + # In git.openstack.org/openstack/loci/.zuul.yaml: + - secret: + name: loci_docker_login + data: + user: loci-username + password: !encrypted/pkcs1-oaep + - gUEX4eY3JAk/Xt7Evmf/hF7xr6HpNRXTibZjrKTbmI4QYHlzEBrBbHey27Pt/eYvKKeKw + hk8MDQ4rNX7ZK1v+CKTilUfOf4AkKYbe6JFDd4z+zIZ2PAA7ZedO5FY/OnqrG7nhLvQHE + 5nQrYwmxRp4O8eU5qG1dSrM9X+bzri8UnsI7URjqmEsIvlUqtybQKB9qQXT4d6mOeaKGE + 5h6Ydkb9Zdi4Qh+GpCGDYwHZKu1mBgVK5M1G6NFMy1DYz+4NJNkTRe9J+0TmWhQ/KZSqo + 4ck0x7Tb0Nr7hQzV8SxlwkaCTLDzvbiqmsJPLmzXY2jry6QsaRCpthS01vnj47itoZ/7p + taH9CoJ0Gl7AkaxsrDSVjWSjatTQpsy1ub2fuzWHH4ASJFCiu83Lb2xwYts++r8ZSn+mA + hbEs0GzPI6dIWg0u7aUsRWMOB4A+6t2IOJibVYwmwkG8TjHRXxVCLH5sY+i3MR+NicR9T + IZFdY/AyH6vt5uHLQDU35+5n91pUG3F2lyiY5aeMOvBL05p27GTMuixR5ZoHcvSoHHtCq + 7Wnk21iHqmv/UnEzqUfXZOque9YP386RBWkshrHd0x3OHUfBK/WrpivxvIGBzGwMr2qAj + /AhJsfDXKBBbhGOGk1u5oBLjeC4SRnAcIVh1+RWzR4/cAhOuy2EcbzxaGb6VTM= + +Secret Example +============== + +.. code:: yaml + + # In git.openstack.org/openstack/loci/.zuul.yaml: + - job: + name: publish-loci-cinder + parent: loci-cinder + post-run: playbooks/push + secrets: + - loci_docker_login + + # In git.openstack.org/openstack/loci/playbooks/push.yaml: + - hosts: all + tasks: + - include_vars: vars.yaml + + - name: Push project to DockerHub + block: + - command: docker login -u {{ loci_docker_login.user }} -p {{ loci_docker_login.password }} + no_log: True + - command: docker push openstackloci/{{ project }}:{{ branch }}-{{ item.name }} + with_items: "{{ distros }}" + +Important Links +=============== + +* https://zuul-ci.org/ +* https://git.zuul-ci.org/cgit/zuul +* https://zuul-ci.org/docs/zuul +* https://zuul-ci.org/docs/zuul-jobs/ +* https://docs.openstack.org/infra/openstack-zuul-jobs/ +* freenode:#zuul + +Coffee +====== + +I've been awake for many hours. Let's get a coffee? + +Installation of Software +======================== + +Ways to Install Zuul +==================== + +* Windmill: http://git.openstack.org/cgit/openstack/windmill +* Software Factory: https://softwarefactory-project.io/ +* Puppet: http://git.openstack.org/cgit/openstack-infra/puppet-zuul +* Containers: https://hub.docker.com/_/zuul/ + +Zuul Containers +=============== + +* Published on every commit +* Application/Process containers +* Built using tool 'pbrx' +* Config / Data should be bind-mounted in + +Container Philosophy +==================== + +* As minimal as possible +* OS inside of container does not matter + +zuul/zuul-executor +================== + +* In k8s, zuul-executor must be run privileged +* Uses bubblewrap for unprivileged sanboxing +* Restriction may be lifted in the future + +Release Management +================== + +* Zuul is a CI system +* C stands for "Continuous" +* It is run Continuously Delivered and Deployed upstream +* Releases are tagged from code run upstream +* There is no intent to have a 'stable' release +* 'stable' is a synonym for "old and buggy" + +zuul/zuul-scheduler +=================== + +* SPOF +* We're working on it +* Recommend running scheduler from tags + +Demo Installation using docker-compose +====================================== + +Remember this? + +* Install docker, docker-compose, git-review + +Debian/Ubuntu: + +:: + + apt-get install docker-compose git git-review + +Red Hat / SuSE + +:: + + yum install docker-compose git git-review + +* git clone https://git.zuul-ci.org/zuul +* cd zuul +* git review -d 608344 +* cd doc/source/admin/examples +* docker-compose up + +What's Running +============== + +* Zookeeper +* Gerrit +* Nodepool Launcher +* Zuul Scheduler +* Zuul Web Server +* Zuul Executor +* Apache HTTPD +* A container to use as a 'static' build node + +How they're connected +===================== + +* End Users talk to Gerrit and Apache HTTPD +* Zuul Scheduler talks to Gerrit +* Nodepool Launcher, Zuul Scheduler, Zuul Web talk to Zookeeper +* Zuul Executor talks to Zuul Scheduler (using Gearman) + +Initial provided config +======================= + +* docker-compose has plumbed in basic config ``etc_zuul/zuul.conf`` + and ``etc_zuul/main.yaml`` +* Gerrit Connection named "gerrit" +* Zuul user for that connection +* Git connection named "zuul-ci.org" for ``zuul-jobs`` standard library + +Initial tenant +============== + +* Zuul is (always) multi-tenant +* Example config contains a tenant called ``example-tenant`` +* Three projects in the ``example-tenant`` tenant: + ``zuul-config``, ``test1``, ``test2`` +* Three projects are also in gerrit ready to use + +zuul.conf +========= + +:: + + [gearman] + server=scheduler + + [gearman_server] + start=true + + [zookeeper] + hosts=zk + + [scheduler] + tenant_config=/etc/zuul/main.yaml + + [web] + listen_address=0.0.0.0 + + [executor] + private_key_file=/var/ssh/nodepool + default_username=root + +zuul.conf part 2 +================ + +:: + + [connection "gerrit"] + name=gerrit + driver=gerrit + server=gerrit + sshkey=/var/ssh/zuul + user=zuul + password=secret + baseurl=http://gerrit:8080 + auth_type=basic + + [connection "zuul-ci.org"] + name=zuul-ci + driver=git + baseurl=https://git.zuul-ci.org/ + +main.yaml +========= + +:: + + - tenant: + name: example-tenant + source: + gerrit: + config-projects: + - zuul-config + untrusted-projects: + - test1 + - test2 + zuul-ci.org: + untrusted-projects: + - zuul-jobs: + include: + - job + +Gerrit Account +============== + +* Need a user account to interact with Gerrit +* Gerrit is configured in dev mode - no passwords required +* Visit http://localhost:8080 +* Click "Become" +* Click "New Account" +* Click "Register" +* Enter Full Name +* Click "Save Changes" +* Enter username in Username field (match your local laptop user) +* Copy ``~/.ssh/id_rsa.pub`` contents into SSH Key field +* Click Continue + +Config Repo +=========== + +* ``zuul-config`` is a trusted ``config-repo`` +* Security and functionality of system depend on this repo +* Limit its contents to minimum required + +Config Files vs. Directories +============================ + +* Zuul reads config from: + ``.zuul.yaml``, ``zuul.yaml``, ``zuul.d`` or ``.zuul.d`` +* For projects with substantial zuul config, like ``zuul-config`` + ``zuul.d`` directory is likely best. +* The directories are read run-parts style. +* Recommended practice is splitting by type of object + +Setting up Gating +================= + +* We want to have changes to ``zuul-config`` be gated +* We need to define pipelines: ``check`` and ``gate`` +* Need to attach ``zuul-config`` to them +* Start with builtin ``noop`` job (always return success) +* Use regex to attach all projects to ``check`` and ``gate`` + +Pipeline Definitions +==================== + +* Zuul has no built-in workflow definitions, let's add ``check`` and ``gate`` + +check pipeline +============== + +:: + + - pipeline: + name: check + description: | + Newly uploaded patchsets enter this pipeline to receive an + initial +/-1 Verified vote. + manager: independent + require: + gerrit: + open: True + current-patchset: True + trigger: + gerrit: + - event: patchset-created + - event: change-restored + success: + gerrit: + Verified: 1 + failure: + gerrit: + Verified: -1 + +gate pipeline +============= + +:: + - pipeline: + name: gate + description: | + Changes that have been approved are enqueued in order in this + pipeline, and if they pass tests, will be merged. + manager: dependent + post-review: True + require: + gerrit: + open: True + current-patchset: True + approval: + - Workflow: 1 + trigger: + gerrit: + - event: comment-added + approval: + - Workflow: 1 + start: + gerrit: + Verified: 0 + success: + gerrit: + Verified: 2 + submit: true + failure: + gerrit: + Verified: -2 + +Add the pipeline definitions +============================ + +.. code-block:: bash + + git clone http://localhost:8080/zuul-config + cd zuul-config + mkdir zuul.d + cp ../examples/zuul-config/zuul.d/pipelines.yaml . + +Shared Project Pipeline Definition +================================== + +In ``examples/zuul-config/zuul.d/projects.yaml`` + +.. code-block:: yaml + + - project: + name: ^.*$ + check: + jobs: [] + gate: + jobs: [] + + - project: + name: zuul-config + check: + jobs: + - noop + gate: + jobs: + - noop + +Attach the projects to the pipelines +==================================== + +.. code-block:: bash + + cp ../examples/zuul-config/zuul.d/projects.yaml . + +Commit the changes and push up for review +========================================= + +.. code-block:: bash + + git add zuul.d + git commit + git review + +Force merging bootstrap config +============================== + +* Zuul is running with no config, so it won't do anything +* For this change (and this change only) we will bypass gating + +Reviewing normally +================== + +* visit http://localhost:8080/#/c/zuul-config/+/1001/ +* click reply +* vote +2 Code Review +1 Approved + +Verified +2 is Missing +====================== + +Verified +2 is what we have zuul configured to do. + +:: + success: + gerrit: + Verified: 2 + submit: true + + +Bypassing Gating +================ + +* visit http://localhost:8080/ +* click 'switch account' +* click 'admin' +* visit http://localhost:8080/#/c/zuul-config/+/1001/ +* click reply +* vote +2 Verified (normal users do not see this) +* click submit (normal users do not see this) +* click 'switch account' +* click your username + +Base Job +======== + +* Every Zuul installation must define a ``base`` job +* Push git repos to build node +* Publish logs/artifacts +* Any local specific setup +* Goes in config repo - because it impacts EVERY job + +Add Base Job to zuul-config +=========================== + +:: + + cp ../examples/zuul-config/zuul.d/jobs.yaml . + git add jobs.yaml + git commit + git review + +Then go to http://localhost:8080/#/c/zuul-config/+/1002/ and approve it + +Zuul should merge the patch +=========================== + +zuul-config is configured to use the ``noop`` job + +Zuul tests syntax automatically +=============================== + +* Edit jobs.yaml +* Change ``parent: null`` to ``parent: broken`` +* git commit ; git review +* Check out the review in gerrit ... there should be errors! + +Questions +========= + +.. ansi:: images/questions.ans + +Presentty +========= +.. hidetitle:: +.. transition:: pan +.. figlet:: Presentty + +* Console presentations written in reStructuredText +* Cross-fade, pan, tilt, cut transitions +* Figlet, cowsay! +* https://pypi.python.org/pypi/presentty -- cgit v1.2.3