From a768a041b8fdd482771a9548052f028cdb7c8b30 Mon Sep 17 00:00:00 2001 From: Monty Taylor Date: Thu, 25 Aug 2016 10:30:48 -0500 Subject: Add zuul talk --- src/talks/zuul.hbs | 212 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 src/talks/zuul.hbs (limited to 'src/talks') diff --git a/src/talks/zuul.hbs b/src/talks/zuul.hbs new file mode 100644 index 0000000..4494ba6 --- /dev/null +++ b/src/talks/zuul.hbs @@ -0,0 +1,212 @@ + + + + + + + Zuul + + + + +

+

What is Zuul?

+

Multi-cloud, scalable, elastic CI/CD engine
Validation of speculative future states
Test things like you deploy them
Single-use VM build nodes - safely run tests that need root
Fully support Bare Metal, VMs and Containers
Multi-node builds
Multi-repo projects
Native support for gating configuration

+

+ +

+

Terminology

+

Periodic: jobs run in response to a timer
Post: jobs run after a change
Check: job run when someone proposes a change
Gate: jobs run between change approval and landing

+

+ +

+

Why - the original OpenStack use case

+

Fully automated gated commits
Full end-to-end integration tests from scratch for every commit
Massive scale

+

+ +

+

OpenStack Scale by the numbers

+

1 KJPH (kilo-jobs / hour)
2500 arbitrary developers
11727 Jobs
Merge 10k Changes / 42 days

+

ansible has _received_ 13171 PRs (changes), has + merged 8190 of them and has 37788 commits in its entire lifetime

+

+ +

+

Multi Repository Speculative Execution

+

Zuul constructs speculative states as-if a change were merged
Tests future states without landing those changes first
The as-if spans multiple repos
In the Gate pipeline, speculative changes are put into a + virtual serial queue, then tested in parallel as-if each change + combination in front of them had landed

+

+ +

+

Zuul Animation

+ http://docs.openstack.org/infra/publications/zuul/#(18) +

+ +

+

Not Specific to OpenStack

+

"Gate" and "Check" are merely configurations
50+ OpenStack Vendors use Zuul for "3rd Party CI" of drivers
HP uses zuul for both OpenStack and non-OpenStack products
Wikimedia uses zuul

+

+ +

+

Status Pages

+

+ https://integration.wikimedia.org/zuul/

+

+ http://status.openstack.org/zuul/

+

+ +

+

Pluggable

+

Triggers
Reporters
Node Providers
Execution content (ansible)

+

+ +

+

Zuul v2

+

In production for OpenStack for 4 years
What most people run
Triggers: Gerrit, Periodic
Reporters: Gerrit, Email, MySQL
Node Providers: OpenStack, Long-lived non-managed servers
Jobs executed by Jenkins

+

+ +

+

Zuul v2.5

+

In use only by OpenStack (on purpose)
Replaced Jenkins with Ansible
Jobs still written using JJB - playbooks generated on the fly

+

+ http://logs.openstack.org/09/352209/7/check/gate-networking-ovn-python35/0fae86c/_zuul_ansible/ +

+

+ +

+

Why Replace Jenkins?

+

+ +

+

Not for lack of trying

+

OpenStack started on Jenkins (actually, on Hudson, remember that?)
We funded the Jenkins Jclouds Plugin
We did deep dev in the Gerrit Trigger Plugin
Maintain the SCP artifact plugin (added console log support)
Added 0mq notification plugin
Added Gearman Worker plugin - allowed us to grow to 8 Masters/1000 slaves
We wrote Jenkins Job Builder

+

+ +

+

Jenkins Problems

+

Security

+

don't run WebUI on the internet
ssh slave plugin - it's possible for a slave to run arbitrary + code on the master

+

Stability

+

almost every Jenkins upgrade has broken us

+

Scalability

+

Jenkins has global mutexes, especially in plugins
Extra large cloud server could handle ~100 concurrent jobs
We ran 8 Jenkins Masters with slaves sharded across them

+

Overkill

+

we only used it as a remote shell execution engine

+

+ +

+

We know a better engine for remote execution

+

+ +

+

Zuul V3

+

Intended for broad use
triggers: gerrit, periodic, github + (? bitbucket, stash, fedmsg, email)
reporters: gerrit, email, github (? bitbucket, stash, resultsdb)
node providers: pre-existing servers, dynamic cloud slaves (OpenStack, AWS, GCE), k8s clusters
jobs written in and executed with ansible
in-repo config
Multi-node build clusters as first class resource
Multi-Tenant

+

+ +

+

Focus

+

So far

+

OpenStack, and the hard problems that brings
Extra-hard is handled. So is simple - but zuul is complex to run + if you only have the simple use cases

+

Zuul v3

+

Get it ready for Ansible project
Making it truly suitable for not-OpenStack Infra to run
Making the easy tasks simple
Making zuul the thing everyone WANTS to use

+

+ +

+

For More Information

+ +

http://docs.openstack.org/infra/zuul/
http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html
freenode:#zuul
https://post-office.corp.redhat.com/mailman/listinfo/zuul-discuss
http://docs.openstack.org/infra/publications/zuul/#(1)

+

+ + + -- cgit v1.2.3