summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorMonty Taylor <mordred@inaugust.com>2017-03-02 08:38:07 -0800
committerMonty Taylor <mordred@inaugust.com>2017-03-02 08:38:07 -0800
commitb59afe1ad31624cc1d691ba62bd5670fbab443d1 (patch)
treeb1ff116349350418afd0b67c3e892656a755a95c
parentcd58078109ea297f1d683e740b32d9abf162e931 (diff)
Add talk for openinfraday
Diffstat
-rw-r--r--src/talks/zero-to-2500.hbs302
1 files changed, 302 insertions, 0 deletions
diff --git a/src/talks/zero-to-2500.hbs b/src/talks/zero-to-2500.hbs
new file mode 100644
index 0000000..85130d6
--- /dev/null
+++ b/src/talks/zero-to-2500.hbs
@@ -0,0 +1,302 @@
1<!doctype html>
2<html lang="en">
3
4 <head>
5 <meta charset="utf-8">
6
7 <title>From Zero to 2500: Managing Development for OpenStack Completely in
8 the Open</title>
9
10 </head>
11 <body>
12
13 <section id="who-am-i-redhat" class="slide level2">
14 <h1>Who am I?</h1>
15 <img style="float:right; margin:24pt" src="/images/Logo_RH_CMYK_Default.jpg" />
16 <p> Office of Technology </p>
17 <p> Zuul </p>
18 <p> Ansible </p>
19 </section>
20
21 <section id="who-am-i-openstack" class="slide level2">
22 <h1>Who am I?</h1>
23 <img style="float:right; margin-right:24pt; width:300px; height: auto" src="/images/openstack-cloud-software-vertical-large.png" />
24 <p>Technical Committee</p>
25 <p>Developer Infrastructure Core Team</p>
26 </section>
27
28 <section class="slide level2">
29 <img src="/images/openstack-software-diagram.png"
30 style="width:800px; height: auto"
31 alt="OpenStack" />
32 </section>
33
34 <section class="slide level2">
35 <h1>The Four Opens</h1>
36 <ul>
37 <li>Open Source<br/>
38 <small>we don't hold back "Enterprise" features, we don't cripple things</small></li>
39 <li>Open Design<br/>
40 <small>design process open to all, decisions are not made inside company doors</small></li>
41 <li>Open Development<br/>
42 <small>public source code, public code review, all code is reviewed and gated</small></li>
43 <li>Open Community<br/>
44 <small>lazy consensus, democratic leadership from participants,
45 public logged meetings in IRC, public archived mailing lists</small></li>
46 </section>
47
48 <section id="openstack-infra" data-transition='zoom'>
49 <h1>OpenStack Infra</h1>
50 <p>Tooling, Automation and CI for OpenStack Project</p>
51 </section>
52
53 <section id="why" class="slide level2">
54 <h1>Why?</h1>
55 <h2> The original OpenStack use case</h2>
56 <ul>
57 <li>Fully automated gated commits</li>
58 <li>Full end-to-end integration tests from scratch for every commit</li>
59 <li>Massive scale</li>
60 </ul>
61 </section>
62
63 <section id="kjph" class="slide level2" data-transition='zoom'>
64 <h1> OpenStack Scale by the numbers</h1>
65 <ul>
66 <li>2 KJPH (kilo-jobs / hour) (1/3 the total Travis job rate)</li>
67 <li>2376 arbitrary developers</li>
68 <li>1474 git repositories</li>
69 <li>11727 Jobs</li>
70 <li>Merge 10k Changes / Month</li>
71 </ul>
72 <p class='fragment'>ansible has <em>received</em> 13171 PRs (changes),
73 has merged 8190 of them and has 37788 commits in its entire lifetime</p>
74 </section>
75
76 <section id='runs-the-same' class="slide level2">
77 <h1>Infra operates the same way as OpenStack</h1>
78 </section>
79
80 <section class="slide level2">
81 <h1>How do we do this?</h1>
82 </section>
83
84 <section class="slide level2">
85 <h1>Control plane</h1>
86 <h3>http://git.openstack.org/cgit/openstack-infra/system-config</h3>
87 <ul>
88 <li>All server config management in git</li>
89 <li>Puppet manages the servers: puppet apply</li>
90 <li>Ansible runs puppet: ansible puppet module</li>
91 <li>Ansible OpenStack Dynamic Inventory</li>
92 <li>Only thing not public are keys and secrets</li>
93 </ul>
94 </section>
95
96 <section class="slide level2">
97 <h1>It wasn't always this way!</h1>
98 <p class='fragment'>Let me take you on a walk down memory lane ...</p>
99 </section>
100
101 <section class="slide level2">
102 <h1>We started with 4 cloud servers in Rackspace</h1>
103 <ul>
104 <li>Hudson Master (https://launchpad.net/~hudson-openstack)</li>
105 <li>Nova Build Node</li>
106 <li>Swift Build Node</li>
107 <li>The other server (now known as old-wiki)</li>
108 </ul>
109 <p class='fragment'>old-wiki is still running! (On Ubuntu 10.04)</p>
110 <p class='fragment'>I didn't even have access to the cloud account!</p>
111 </section>
112
113 <section class="slide level2">
114 <h1>The Setup</h1>
115 <ul>
116 <li>Hudson jobs ran Tarmac, which tested and merged Launchpad Merge Requests</li>
117 <li>Hudson ran Tarmac in a loop, published the build results</li>
118 <li>One Job per project</li>
119 <li>Three of us with direct Hudson Admin permissions</li>
120 </ul>
121 </section>
122
123 <section class="slide level2">
124 <h1>This state persisted for the first year and first three OpenStack
125 releases</h1>
126 </section>
127
128 <section class="slide level2">
129 <h1>Project Proliferation</h1>
130 <p>Each project got a node and a job. Configured by hand. By me.</p>
131 <p class='fragment'>It got annoying</p>
132 </section>
133
134 <section class="slide level2">
135 <h1>Config Management!</h1>
136 <small>Please remember we're talking 2011 here</small>
137 </section>
138
139 <section class="slide level2">
140 <h1>Puppet vs. Chef and git vs. bzr and humans pushing things</h1>
141 </section>
142
143 <section class="slide level2">
144 <h1>We were so excited about sharing Ops best practices!</h1>
145 <p class='small'>We were so sad</p>
146 </section>
147
148 <section class="slide level2">
149 <h1>Brief Rant - I do not want to write Apache configs in Puppet DSL</h1>
150 </section>
151
152 <section class="slide level2">
153 <h1>So we introduced Puppet</h1>
154 <p><a href='http://git.openstack.org/cgit/openstack-infra/system-config/tree/?id=99540d91a75d2b021db01d815e46bd585f9235cd'>http://git.openstack.org/cgit/openstack-infra/system-config/tree/?id=99540d91a75d2b021db01d815e46bd585f9235cd</a></p>
155 </section>
156
157 <section class="slide level2">
158 <h1>Open Development</h1>
159 <p>Our developers wanted to collaborate on test jobs.</p>
160 <p>Giving hundreds of people access to directly edit test jobs == sadness</p>
161 <p>Did I mention our test jobs implement captive gating?</p>
162 </section>
163
164 <section class="slide level2">
165 <h1>Jenkins Job Builder</h1>
166 <p>YAML encoding of Jenkins Job definitions with templating</p>
167 <p>Allowed jobs to go through code review before being applied!</p>
168 <h4>Andreas Jaegar is OpenStack's all-time contributions leader. He works
169 on docs and test jobs</h4>
170 </section>
171
172 <section class="slide level2">
173 <h1>Introduction of Puppetmaster</h1>
174 <ul>
175 <li>In cron job, config repo updated on puppetmaster</li>
176 <li>puppet agent ran on each node</li>
177 <li>Landing a commit == config running on hosts (eventually)</li>
178 <li>What to do with passwords and keys?</li>
179 </ul>
180 </section>
181
182 <section class="slide level2">
183 <h1>Introduction of Hiera for Secrets</h1>
184 <p>Hiera let us store YAML files with only secrets. Reference secrets
185 by name in puppet manifests</p>
186 <p><a href='http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp'>http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp</a></p>
187 </section>
188
189 <section class="slide level2">
190 <h1>Project Creation</h1>
191 <ul>
192 <li>Create replica repos on git farm (and on github I suppose)</li>
193 <li>Create repo in Gerrit</li>
194 <li>Push contents</li>
195 </ul>
196 <p>Too much clicking!</p>
197 </section>
198
199 <section class="slide level2">
200 <h1>jeepyb - Gerrit Project Builder</h1>
201 <p>Lesson: Don't let Monty name things</p>
202 </section>
203
204 <section class="slide level2">
205 <h1>Ansible to run Puppet</h1>
206 </section>
207
208 <section class="slide level2">
209 <h1>Back up: Salt to run Puppet</h1>
210 </section>
211
212 <section class="slide level2">
213 <h1>Ansible to run Puppet</h1>
214 <ul>
215 <li>puppet ansible module</li>
216 <li>openstack-infra/ansible-role-puppet</li>
217 <li>Role copies subset of hiera secrets to node before puppet</li>
218 <li>Moved from puppetmaster to puppet apply</li>
219 </section>
220
221 <section class="slide level2">
222 <h1>Remaining manual human tasks</h1>
223 <ul>
224 <li>Adding new secrets to hiera</li>
225 <li>Launching new servers</li>
226 </ul>
227 </section>
228
229 <section class="slide level2">
230 <h1>Ansible Role Cloud Launcher</h1>
231 <h2>http://git.openstack.org/cgit/openstack/ansible-role-cloud-launcher</h2>
232 <pre>
233profiles:
234 - name: admin-clouds
235 flavors:
236 - name: aoclcompany.xlarge
237 ram: 128
238 vcpus: 1
239 - name: ops
240 images:
241 - name: ubuntu-trusty
242 filename: /home/ubuntu/trusty-server-cloudimg-amd64-disk1.img
243 - name: bootstrap-keypair
244 keypairs:
245 - name: bootstrap-key
246 public_key_file: /home/ubuntu/.ssh/id_rsa.pub
247clouds:
248 - name: awesomecloud
249 profiles:
250 - admin-clouds
251 - bootstrap-keypair
252 - name: yaycloud-ops
253 oscc_cloud: yaycloud-opsuser
254 profiles:
255 - bootstrap-keypair
256 - ops
257 </pre>
258 </section>
259
260 <section class="slide level2">
261 <h1>Problems depending on services</h1>
262 <p>Even when the service is Open Source, it can stop being</p>
263 </section>
264
265 <section class="slide level2">
266 <h1>Transifex</h1>
267 <p><em>WAS</em> an Open Source translations system.</p>
268 </section>
269
270 <section class="slide level2">
271 <h1>Zanata!</h1>
272 <p>We run Zanata ourselves now. (Thanks Lyz!)</p>
273 </section>
274
275 <section class="slide level2">
276 <h1>Remaining external service dependencies</h1>
277 <ul>
278 <li>Rackspace Public Cloud</li>
279 <li>Launchpad</li>
280 </ul>
281 </section>
282
283 <section class="slide level2">
284 <h1>Launchpad OpenID -> openstackid</h1>
285 </section>
286
287 <section class="slide level2">
288 <h1>Launchpad Bugs -> storyboard</h1>
289 </section>
290
291 <section class="slide level2">
292 <h1>The Multi-cloud OpenStack Story</h1>
293 <ul>
294 <li>Our build nodes already span 12 different OpenStack Public Clouds</li>
295 <li>Work starting on spreading the Control Plane out</li>
296 <li>Starting with Vexxhost</li>
297 </ul>
298 </section>
299
300
301 </body>
302</html>
generated by cgit v1.2.3 (git 2.25.1) at 2025-11-02 00:21:15 +0000