diff options
Diffstat (limited to 'src')
-rw-r--r-- | src/talks/zuul.hbs | 212 |
1 files changed, 212 insertions, 0 deletions
diff --git a/src/talks/zuul.hbs b/src/talks/zuul.hbs new file mode 100644 index 0000000..4494ba6 --- /dev/null +++ b/src/talks/zuul.hbs | |||
@@ -0,0 +1,212 @@ | |||
1 | <!doctype html> | ||
2 | <html lang="en"> | ||
3 | |||
4 | <head> | ||
5 | <meta charset="utf-8"> | ||
6 | |||
7 | <title>Zuul</title> | ||
8 | |||
9 | </head> | ||
10 | <body> | ||
11 | |||
12 | <section id="what-is-zuul" class="slide level2"> | ||
13 | <h1>What is Zuul?</h1> | ||
14 | <ul> | ||
15 | <li>Multi-cloud, scalable, elastic CI/CD engine</li> | ||
16 | <li>Validation of speculative future states</li> | ||
17 | <li>Test things like you deploy them</li> | ||
18 | <li>Single-use VM build nodes - safely run tests that need root</li> | ||
19 | <li>Fully support Bare Metal, VMs and Containers</li> | ||
20 | <li>Multi-node builds</li> | ||
21 | <li>Multi-repo projects</li> | ||
22 | <li>Native support for gating configuration</li> | ||
23 | </ul> | ||
24 | </section> | ||
25 | |||
26 | <section id="terminoloy" class="slide level2"> | ||
27 | <h1>Terminology</h1> | ||
28 | <ul> | ||
29 | <li>Periodic: jobs run in response to a timer</li> | ||
30 | <li>Post: jobs run after a change</li> | ||
31 | <li>Check: job run when someone proposes a change</li> | ||
32 | <li>Gate: jobs run between change approval and landing</li> | ||
33 | </ul> | ||
34 | </section> | ||
35 | |||
36 | <section id="why" class="slide level2"> | ||
37 | <h1>Why - the original OpenStack use case</h1> | ||
38 | <ul> | ||
39 | <li>Fully automated gated commits</li> | ||
40 | <li>Full end-to-end integration tests from scratch for every commit</li> | ||
41 | <li>Massive scale</li> | ||
42 | </ul> | ||
43 | </section> | ||
44 | |||
45 | <section id="openstack-scale" class="slide level2"> | ||
46 | <h1> OpenStack Scale by the numbers</h1> | ||
47 | <ul> | ||
48 | <li>1 KJPH (kilo-jobs / hour)</li> | ||
49 | <li>2500 arbitrary developers</li> | ||
50 | <li>11727 Jobs</li> | ||
51 | <li>Merge 10k Changes / 42 days</li> | ||
52 | </ul> | ||
53 | <p class='fragment'>ansible has _received_ 13171 PRs (changes), has | ||
54 | merged 8190 of them and has 37788 commits in its entire lifetime</p> | ||
55 | </section> | ||
56 | |||
57 | <section id="speculative-execution" class="slide level2"> | ||
58 | <h1>Multi Repository Speculative Execution</h1> | ||
59 | <ul> | ||
60 | <li>Zuul constructs speculative states as-if a change were merged</li> | ||
61 | <li>Tests future states without landing those changes first</li> | ||
62 | <li>The as-if spans multiple repos</li> | ||
63 | <li>In the Gate pipeline, speculative changes are put into a | ||
64 | virtual serial queue, then tested in parallel as-if each change | ||
65 | combination in front of them had landed</li> | ||
66 | </ul> | ||
67 | </section> | ||
68 | |||
69 | <section id="animation" class="slide level2"> | ||
70 | <h1>Zuul Animation</h1> | ||
71 | <a href='http://docs.openstack.org/infra/publications/zuul/#(18)'>http://docs.openstack.org/infra/publications/zuul/#(18)</a> | ||
72 | </section> | ||
73 | |||
74 | <section id="not-specific" class="slide level2"> | ||
75 | <h1>Not Specific to OpenStack</h1> | ||
76 | <ul> | ||
77 | <li>"Gate" and "Check" are merely configurations</li> | ||
78 | <li>50+ OpenStack Vendors use Zuul for "3rd Party CI" of drivers</li> | ||
79 | <li>HP uses zuul for both OpenStack and non-OpenStack products</li> | ||
80 | <li>Wikimedia uses zuul</li> | ||
81 | </ul> | ||
82 | </section> | ||
83 | |||
84 | <section id="status-pages" class="slide level2"> | ||
85 | <h1>Status Pages</h1> | ||
86 | <p><a href="https://integration.wikimedia.org/zuul/"> | ||
87 | https://integration.wikimedia.org/zuul/</a></p> | ||
88 | <p><a href="http://status.openstack.org/zuul/"> | ||
89 | http://status.openstack.org/zuul/</a></p> | ||
90 | </section> | ||
91 | |||
92 | <section id="pluggable" class="slide level2"> | ||
93 | <h1>Pluggable</h1> | ||
94 | <ul> | ||
95 | <li>Triggers</li> | ||
96 | <li>Reporters</li> | ||
97 | <li>Node Providers</li> | ||
98 | <li>Execution content (ansible)</li> | ||
99 | </ul> | ||
100 | </section> | ||
101 | |||
102 | <section id="zuul-v2" class="slide level2"> | ||
103 | <h1>Zuul v2</h1> | ||
104 | <ul> | ||
105 | <li>In production for OpenStack for 4 years</li> | ||
106 | <li>What most people run</li> | ||
107 | <li>Triggers: Gerrit, Periodic</li> | ||
108 | <li>Reporters: Gerrit, Email, MySQL</li> | ||
109 | <li>Node Providers: OpenStack, Long-lived non-managed servers</li> | ||
110 | <li>Jobs executed by Jenkins</li> | ||
111 | </ul> | ||
112 | </section> | ||
113 | |||
114 | <section id="zuul-v2.5" class="slide level2"> | ||
115 | <h1>Zuul v2.5</h1> | ||
116 | <ul> | ||
117 | <li>In use only by OpenStack (on purpose)</li> | ||
118 | <li>Replaced Jenkins with Ansible</li> | ||
119 | <li>Jobs still written using JJB - playbooks generated on the fly</li> | ||
120 | </ul> | ||
121 | <p><a href='http://logs.openstack.org/09/352209/7/check/gate-networking-ovn-python35/0fae86c/_zuul_ansible/'> | ||
122 | http://logs.openstack.org/09/352209/7/check/gate-networking-ovn-python35/0fae86c/_zuul_ansible/ | ||
123 | </a></p> | ||
124 | </section> | ||
125 | |||
126 | <section id="replace-jenkins" class="slide level2"> | ||
127 | <h1>Why Replace Jenkins?</h1> | ||
128 | </section> | ||
129 | |||
130 | <section id="no-lack-of-trying" class="slide level2"> | ||
131 | <h1>Not for lack of trying</h1> | ||
132 | <ul> | ||
133 | <li>OpenStack started on Jenkins (actually, on Hudson, remember that?)</li> | ||
134 | <li>We funded the Jenkins Jclouds Plugin</li> | ||
135 | <li>We did deep dev in the Gerrit Trigger Plugin</li> | ||
136 | <li>Maintain the SCP artifact plugin (added console log support)</li> | ||
137 | <li>Added 0mq notification plugin</li> | ||
138 | <li>Added Gearman Worker plugin - allowed us to grow to 8 Masters/1000 slaves</li> | ||
139 | <li>We wrote Jenkins Job Builder</li> | ||
140 | </ul> | ||
141 | </section> | ||
142 | |||
143 | <section id="problems" class="slide level2"> | ||
144 | <h1>Jenkins Problems</h1> | ||
145 | <p>Security</p> | ||
146 | <ul> | ||
147 | <li>don't run WebUI on the internet</li> | ||
148 | <li> ssh slave plugin - it's possible for a slave to run arbitrary | ||
149 | code on the master</li> | ||
150 | </ul> | ||
151 | <p>Stability</p> | ||
152 | <ul><li>almost every Jenkins upgrade has broken us</li></ul> | ||
153 | <p>Scalability</p> | ||
154 | <ul> | ||
155 | <li>Jenkins has global mutexes, especially in plugins</li> | ||
156 | <li>Extra large cloud server could handle ~100 concurrent jobs</li> | ||
157 | <li>We ran 8 Jenkins Masters with slaves sharded across them</li> | ||
158 | </ul> | ||
159 | <p>Overkill</p> | ||
160 | <ul><li>we only used it as a remote shell execution engine</li></ul> | ||
161 | </section> | ||
162 | |||
163 | <section id="better-engine" class="slide level2"> | ||
164 | <h1>We know a better engine for remote execution</h1> | ||
165 | </section> | ||
166 | |||
167 | <section id="zuul-v3" class="slide level2"> | ||
168 | <h1>Zuul V3</h1> | ||
169 | <ul> | ||
170 | <li>Intended for broad use</li> | ||
171 | <li>triggers: gerrit, periodic, github | ||
172 | (? bitbucket, stash, fedmsg, email)</li> | ||
173 | <li>reporters: gerrit, email, github (? bitbucket, stash, resultsdb) </li> | ||
174 | <li>node providers: pre-existing servers, dynamic cloud slaves (OpenStack, AWS, GCE), k8s clusters</li> | ||
175 | <li>jobs written in and executed with ansible</li> | ||
176 | <li>in-repo config</li> | ||
177 | <li>Multi-node build clusters as first class resource</li> | ||
178 | <li>Multi-Tenant</li> | ||
179 | </ul> | ||
180 | </section> | ||
181 | |||
182 | <section id="focus" class="slide level2"> | ||
183 | <h1>Focus</h1> | ||
184 | <p>So far</p> | ||
185 | <ul> | ||
186 | <li>OpenStack, and the hard problems that brings</li> | ||
187 | <li>Extra-hard is handled. So is simple - but zuul is complex to run | ||
188 | if you only have the simple use cases</li> | ||
189 | </ul> | ||
190 | <p>Zuul v3</p> | ||
191 | <ul> | ||
192 | <li>Get it ready for Ansible project</li> | ||
193 | <li>Making it truly suitable for not-OpenStack Infra to run</li> | ||
194 | <li>Making the easy tasks simple</li> | ||
195 | <li>Making zuul the thing everyone WANTS to use</li> | ||
196 | </ul> | ||
197 | </section> | ||
198 | |||
199 | <section id="focus" class="slide level2"> | ||
200 | <h1>For More Information</h1> | ||
201 | |||
202 | <ul> | ||
203 | <li>http://docs.openstack.org/infra/zuul/</li> | ||
204 | <li>http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html</li> | ||
205 | <li>freenode:#zuul</li> | ||
206 | <li>https://post-office.corp.redhat.com/mailman/listinfo/zuul-discuss</li> | ||
207 | <li>http://docs.openstack.org/infra/publications/zuul/#(1)</li> | ||
208 | </ul> | ||
209 | </section> | ||
210 | |||
211 | </body> | ||
212 | </html> | ||