summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorMonty Taylor <mordred@inaugust.com>2016-08-25 10:30:48 -0500
committerMonty Taylor <mordred@inaugust.com>2016-08-25 10:53:58 -0500
commita768a041b8fdd482771a9548052f028cdb7c8b30 (patch)
tree72b95dbeccf456c02dd3e8f77d6317f35b2418a1
parente35631d87e9a27d5ae0406cafafcac564217e65f (diff)
Add zuul talk
Diffstat
-rw-r--r--src/talks/zuul.hbs212
1 files changed, 212 insertions, 0 deletions
diff --git a/src/talks/zuul.hbs b/src/talks/zuul.hbs
new file mode 100644
index 0000000..4494ba6
--- /dev/null
+++ b/src/talks/zuul.hbs
@@ -0,0 +1,212 @@
1<!doctype html>
2<html lang="en">
3
4 <head>
5 <meta charset="utf-8">
6
7 <title>Zuul</title>
8
9 </head>
10 <body>
11
12 <section id="what-is-zuul" class="slide level2">
13 <h1>What is Zuul?</h1>
14 <ul>
15 <li>Multi-cloud, scalable, elastic CI/CD engine</li>
16 <li>Validation of speculative future states</li>
17 <li>Test things like you deploy them</li>
18 <li>Single-use VM build nodes - safely run tests that need root</li>
19 <li>Fully support Bare Metal, VMs and Containers</li>
20 <li>Multi-node builds</li>
21 <li>Multi-repo projects</li>
22 <li>Native support for gating configuration</li>
23 </ul>
24 </section>
25
26 <section id="terminoloy" class="slide level2">
27 <h1>Terminology</h1>
28 <ul>
29 <li>Periodic: jobs run in response to a timer</li>
30 <li>Post: jobs run after a change</li>
31 <li>Check: job run when someone proposes a change</li>
32 <li>Gate: jobs run between change approval and landing</li>
33 </ul>
34 </section>
35
36 <section id="why" class="slide level2">
37 <h1>Why - the original OpenStack use case</h1>
38 <ul>
39 <li>Fully automated gated commits</li>
40 <li>Full end-to-end integration tests from scratch for every commit</li>
41 <li>Massive scale</li>
42 </ul>
43 </section>
44
45 <section id="openstack-scale" class="slide level2">
46 <h1> OpenStack Scale by the numbers</h1>
47 <ul>
48 <li>1 KJPH (kilo-jobs / hour)</li>
49 <li>2500 arbitrary developers</li>
50 <li>11727 Jobs</li>
51 <li>Merge 10k Changes / 42 days</li>
52 </ul>
53 <p class='fragment'>ansible has _received_ 13171 PRs (changes), has
54 merged 8190 of them and has 37788 commits in its entire lifetime</p>
55 </section>
56
57 <section id="speculative-execution" class="slide level2">
58 <h1>Multi Repository Speculative Execution</h1>
59 <ul>
60 <li>Zuul constructs speculative states as-if a change were merged</li>
61 <li>Tests future states without landing those changes first</li>
62 <li>The as-if spans multiple repos</li>
63 <li>In the Gate pipeline, speculative changes are put into a
64 virtual serial queue, then tested in parallel as-if each change
65 combination in front of them had landed</li>
66 </ul>
67 </section>
68
69 <section id="animation" class="slide level2">
70 <h1>Zuul Animation</h1>
71 <a href='http://docs.openstack.org/infra/publications/zuul/#(18)'>http://docs.openstack.org/infra/publications/zuul/#(18)</a>
72 </section>
73
74 <section id="not-specific" class="slide level2">
75 <h1>Not Specific to OpenStack</h1>
76 <ul>
77 <li>"Gate" and "Check" are merely configurations</li>
78 <li>50+ OpenStack Vendors use Zuul for "3rd Party CI" of drivers</li>
79 <li>HP uses zuul for both OpenStack and non-OpenStack products</li>
80 <li>Wikimedia uses zuul</li>
81 </ul>
82 </section>
83
84 <section id="status-pages" class="slide level2">
85 <h1>Status Pages</h1>
86 <p><a href="https://integration.wikimedia.org/zuul/">
87 https://integration.wikimedia.org/zuul/</a></p>
88 <p><a href="http://status.openstack.org/zuul/">
89 http://status.openstack.org/zuul/</a></p>
90 </section>
91
92 <section id="pluggable" class="slide level2">
93 <h1>Pluggable</h1>
94 <ul>
95 <li>Triggers</li>
96 <li>Reporters</li>
97 <li>Node Providers</li>
98 <li>Execution content (ansible)</li>
99 </ul>
100 </section>
101
102 <section id="zuul-v2" class="slide level2">
103 <h1>Zuul v2</h1>
104 <ul>
105 <li>In production for OpenStack for 4 years</li>
106 <li>What most people run</li>
107 <li>Triggers: Gerrit, Periodic</li>
108 <li>Reporters: Gerrit, Email, MySQL</li>
109 <li>Node Providers: OpenStack, Long-lived non-managed servers</li>
110 <li>Jobs executed by Jenkins</li>
111 </ul>
112 </section>
113
114 <section id="zuul-v2.5" class="slide level2">
115 <h1>Zuul v2.5</h1>
116 <ul>
117 <li>In use only by OpenStack (on purpose)</li>
118 <li>Replaced Jenkins with Ansible</li>
119 <li>Jobs still written using JJB - playbooks generated on the fly</li>
120 </ul>
121 <p><a href='http://logs.openstack.org/09/352209/7/check/gate-networking-ovn-python35/0fae86c/_zuul_ansible/'>
122 http://logs.openstack.org/09/352209/7/check/gate-networking-ovn-python35/0fae86c/_zuul_ansible/
123 </a></p>
124 </section>
125
126 <section id="replace-jenkins" class="slide level2">
127 <h1>Why Replace Jenkins?</h1>
128 </section>
129
130 <section id="no-lack-of-trying" class="slide level2">
131 <h1>Not for lack of trying</h1>
132 <ul>
133 <li>OpenStack started on Jenkins (actually, on Hudson, remember that?)</li>
134 <li>We funded the Jenkins Jclouds Plugin</li>
135 <li>We did deep dev in the Gerrit Trigger Plugin</li>
136 <li>Maintain the SCP artifact plugin (added console log support)</li>
137 <li>Added 0mq notification plugin</li>
138 <li>Added Gearman Worker plugin - allowed us to grow to 8 Masters/1000 slaves</li>
139 <li>We wrote Jenkins Job Builder</li>
140 </ul>
141 </section>
142
143 <section id="problems" class="slide level2">
144 <h1>Jenkins Problems</h1>
145 <p>Security</p>
146 <ul>
147 <li>don't run WebUI on the internet</li>
148 <li> ssh slave plugin - it's possible for a slave to run arbitrary
149 code on the master</li>
150 </ul>
151 <p>Stability</p>
152 <ul><li>almost every Jenkins upgrade has broken us</li></ul>
153 <p>Scalability</p>
154 <ul>
155 <li>Jenkins has global mutexes, especially in plugins</li>
156 <li>Extra large cloud server could handle ~100 concurrent jobs</li>
157 <li>We ran 8 Jenkins Masters with slaves sharded across them</li>
158 </ul>
159 <p>Overkill</p>
160 <ul><li>we only used it as a remote shell execution engine</li></ul>
161 </section>
162
163 <section id="better-engine" class="slide level2">
164 <h1>We know a better engine for remote execution</h1>
165 </section>
166
167 <section id="zuul-v3" class="slide level2">
168 <h1>Zuul V3</h1>
169 <ul>
170 <li>Intended for broad use</li>
171 <li>triggers: gerrit, periodic, github
172 (? bitbucket, stash, fedmsg, email)</li>
173 <li>reporters: gerrit, email, github (? bitbucket, stash, resultsdb) </li>
174 <li>node providers: pre-existing servers, dynamic cloud slaves (OpenStack, AWS, GCE), k8s clusters</li>
175 <li>jobs written in and executed with ansible</li>
176 <li>in-repo config</li>
177 <li>Multi-node build clusters as first class resource</li>
178 <li>Multi-Tenant</li>
179 </ul>
180 </section>
181
182 <section id="focus" class="slide level2">
183 <h1>Focus</h1>
184 <p>So far</p>
185 <ul>
186 <li>OpenStack, and the hard problems that brings</li>
187 <li>Extra-hard is handled. So is simple - but zuul is complex to run
188 if you only have the simple use cases</li>
189 </ul>
190 <p>Zuul v3</p>
191 <ul>
192 <li>Get it ready for Ansible project</li>
193 <li>Making it truly suitable for not-OpenStack Infra to run</li>
194 <li>Making the easy tasks simple</li>
195 <li>Making zuul the thing everyone WANTS to use</li>
196 </ul>
197 </section>
198
199 <section id="focus" class="slide level2">
200 <h1>For More Information</h1>
201
202 <ul>
203 <li>http://docs.openstack.org/infra/zuul/</li>
204 <li>http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html</li>
205 <li>freenode:#zuul</li>
206 <li>https://post-office.corp.redhat.com/mailman/listinfo/zuul-discuss</li>
207 <li>http://docs.openstack.org/infra/publications/zuul/#(1)</li>
208 </ul>
209 </section>
210
211 </body>
212</html>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-17 15:01:32 +0000