summaryrefslogtreecommitdiff
path: root/src
diff options
context:
space:
mode:
authorMonty Taylor <mordred@inaugust.com>2018-12-05 14:22:40 (GMT)
committerMonty Taylor <mordred@inaugust.com>2018-12-05 14:22:40 (GMT)
commit6c866e5f77b35fd573ae12e2de3d37eaa155f32c (patch)
tree4ccde4d905b90d1b21c1cf48e4fbbe6678a6f41d /src
parent6dddf880243376b9e3f3ef73be31ab4548b10cc0 (diff)
Added a slightly larger overview talk
Diffstat (limited to 'src')
-rw-r--r--src/zuulv3/overview.rst1258
1 files changed, 1258 insertions, 0 deletions
diff --git a/src/zuulv3/overview.rst b/src/zuulv3/overview.rst
new file mode 100644
index 0000000..5e6b281
--- /dev/null
+++ b/src/zuulv3/overview.rst
@@ -0,0 +1,1258 @@
1. display in 68x24
2.. display in 88x24
3
4.. pygments yaml? (only file breaks (---) tinted)
5.. slide on high level v3 changes
6.. slide on nodepool
7
8.. transition:: dissolve
9 :duration: 0.4
10
11Test Slide
12==========
13.. hidetitle::
14
15.. ansi:: images/testslide.ans
16
17Preshow
18=======
19.. hidetitle::
20
21.. ansi:: images/cursor.ans images/cursor2.ans
22
23Zuul
24====
25.. hidetitle::
26.. ansi:: images/title.ans
27
28Monty Taylor
29============
30
31* @e_monty
32* freenode:mordred
33* mordred@inaugust.com
34
35Red Hat
36=======
37.. hidetitle::
38.. container:: handout
39 i work for
40
41.. ansi:: images/redhat.ans
42
43Ansible
44=======
45.. hidetitle::
46.. ansi:: images/ansible.ans
47
48OpenDev
49=======
50
51::
52
53 "most insane CI infrastructure I've ever been a part of"
54
55 -- Alex Gaynor
56
57 "OpenStack Infra are like the SpaceX of CI"
58
59 -- Emily Dunham
60
61Zuul
62====
63.. hidetitle::
64.. ansi:: images/zuul.ans
65
66
67What Zuul Does
68==============
69
70 * gated changes
71 * one or more git repositories
72 * integrated deliverable
73 * testing like deployment
74
75Underlying Philosophy
76=====================
77
78 * All changes flow through code review
79 * Changes only land if they pass all tests
80 * Computers are cheaper than humans
81
82Ramifications of Philosophy
83===========================
84
85 * No direct push access for anyone
86 * Software should be installable from source
87 * Testing should be automated and repeatable
88 * Developers write tests with their patches
89 * Code always works
90
91Getting to Gating
92=================
93
94No Tests / Manual Tests
95=======================
96
97 * No test automation exists or ...
98 * Developer runs test suite before pushing code
99 * Prone to developer skipping tests for "trivial" changes
100 * Doesn't scale organizationally
101
102Periodic Testing
103================
104
105 * Developers push changes directly to shared branch
106 * CI system runs tests from time to time - report if things still work
107 * "Who broke the build?"
108 * Leads to hacks like NVIE model
109
110Post-Merge Testing
111==================
112
113 * Developers push changes directly to shared branch
114 * CI system is triggered by push - reports if push broke something
115 * Frequently batched / rolled up
116 * Easier to diagnose which change broke things
117 * Reactive - the bad changes are already in
118
119Pre-Review Testing
120==================
121
122 * Changes are pushed to code review (Gerrit Change, GitHub PR, etc)
123 * CI system is triggered by code review change creation
124 * Test results inform review decisions
125 * Proactive - testing code before it lands
126 * Reviewers can get bored waiting for tests
127 * Only tests code as written, not potential result of merging code
128
129Gating
130======
131
132 * Changes are pushed to code review
133 * CI system is triggered by code review approval
134 * CI system merges code IFF tests pass
135 * Proactive - testing code before it lands
136 * Future state resulting from merge of code is tested
137 * Reviewers can fire-and-forget safely
138
139Mix and Match
140=============
141
142 * Zuul supports all of those modes
143 * Zuul users frequently combine them
144 * Run pre-review (check) and gating (gate) on each change
145 * Post-merge/post-tag for release/publication automation
146 * Periodic for catching bitrot
147
148Multi-repository integration
149============================
150
151 * Multiple source repositories are needed for deliverable
152 * Future state to be tested is the future state of all involved repos
153
154To test proposed future state
155=============================
156
157 * Get tip of each project. Merge appropriate change. Test.
158 * Changes must be serialized, otherwise state under test is invalid.
159 * Integrated deliverable repos share serialized queue
160
161Speculative Execution
162=====================
163
164 * Correct parallel processing of serialized future states
165 * Create virtual serial queue of changes for each deliverable
166 * Assume each change will pass its tests
167 * Test successive changes with previous changes applied to starting state
168
169Nearest Non-Failing Change
170==========================
171
172(aka 'The Jim Blair Algorithm')
173
174 * If a change fails, move it aside
175 * Cancel all test jobs behind it in the queue
176 * Reparent queue items on the nearest non-failing change
177 * Restart tests with new state
178
179Zuul Simulation
180===============
181.. transition:: pan
182.. container:: handout
183
184 * todo
185
186.. ansi:: images/zsim-00.ans
187
188Zuul Simulation
189===============
190.. transition:: cut
191.. container:: handout
192
193 * todo
194
195.. ansi:: images/zsim-01.ans
196
197Zuul Simulation
198===============
199.. transition:: cut
200.. container:: handout
201
202 * todo
203
204.. ansi:: images/zsim-02.ans
205
206Zuul Simulation
207===============
208.. transition:: cut
209.. container:: handout
210
211 * todo
212
213.. ansi:: images/zsim-03.ans
214
215Zuul Simulation
216===============
217.. transition:: cut
218.. container:: handout
219
220 * todo
221
222.. ansi:: images/zsim-04.ans
223
224Zuul Simulation
225===============
226.. transition:: cut
227.. container:: handout
228
229 * todo
230
231.. ansi:: images/zsim-05.ans
232
233Zuul Simulation
234===============
235.. transition:: cut
236.. container:: handout
237
238 * todo
239
240.. ansi:: images/zsim-06.ans
241
242Zuul Simulation
243===============
244.. transition:: cut
245.. container:: handout
246
247 * todo
248
249.. ansi:: images/zsim-07.ans
250
251Zuul Simulation
252===============
253.. transition:: cut
254.. container:: handout
255
256 * todo
257
258.. ansi:: images/zsim-08.ans
259
260Zuul Simulation
261===============
262.. transition:: cut
263.. container:: handout
264
265 * todo
266
267.. ansi:: images/zsim-09.ans
268
269Zuul Simulation
270===============
271.. transition:: cut
272.. container:: handout
273
274 * todo
275
276.. ansi:: images/zsim-10.ans
277
278Zuul Simulation
279===============
280.. transition:: cut
281.. container:: handout
282
283 * todo
284
285.. ansi:: images/zsim-11.ans
286
287Zuul Simulation
288===============
289.. transition:: cut
290.. container:: handout
291
292 * todo
293
294.. ansi:: images/zsim-12.ans
295
296Zuul Simulation
297===============
298.. transition:: cut
299.. container:: handout
300
301 * todo
302
303.. ansi:: images/zsim-13.ans
304
305Zuul Simulation
306===============
307.. transition:: cut
308.. container:: handout
309
310 * todo
311
312.. ansi:: images/zsim-14.ans
313
314Zuul Simulation
315===============
316.. transition:: cut
317.. container:: handout
318
319 * todo
320
321.. ansi:: images/zsim-15.ans
322
323Zuul Simulation
324===============
325.. transition:: cut
326.. container:: handout
327
328 * todo
329
330.. ansi:: images/zsim-16.ans
331
332Zuul Simulation
333===============
334.. transition:: cut
335.. container:: handout
336
337 * todo
338
339.. ansi:: images/zsim-17.ans
340
341Zuul Simulation
342===============
343.. transition:: cut
344.. container:: handout
345
346 * todo
347
348.. ansi:: images/zsim-18.ans
349
350Zuul Simulation
351===============
352.. transition:: cut
353.. container:: handout
354
355 * todo
356
357.. ansi:: images/zsim-19.ans
358
359Zuul Simulation
360===============
361.. transition:: cut
362.. container:: handout
363
364 * todo
365
366.. ansi:: images/zsim-20.ans
367
368Zuul Simulation
369===============
370.. transition:: cut
371.. container:: handout
372
373 * todo
374
375.. ansi:: images/zsim-21.ans
376
377Zuul Simulation
378===============
379.. transition:: cut
380.. container:: handout
381
382 * todo
383
384.. ansi:: images/zsim-22.ans
385
386
387Cross-Project Dependencies
388==========================
389
390Testing or gating dependencies manually specified by developers
391
392.. container:: progressive
393
394 * nodepool https://review.openstack.org/612168
395
396 Make functional src jobs actually install from source
397 * openstacksdk https://review.openstack.org/612186
398
399 Don't start task managers passed in to Connection
400
401 Depends-On: https://review.openstack.org/612168
402 * openstacksdk https://review.openstack.org/604521
403
404 Add support for per-service rate limits
405
406 (git parent is 612186)
407 * nodepool https://review.openstack.org/612169
408
409 Consume rate limiting task manager from openstacksdk
410
411 Depends-On: https://review.openstack.org/604521
412
413 (nodepool-functional-py35-src should pass, but
414 nodepool-functional-py35 should not fail until openstacksdk release)
415
416Lock Step Changes
417=================
418
419 * Circular Dependencies are not supported on purpose
420 * Rolling upgrades across interdependent services
421 * HOWEVER - many valid use cases - support will be coming
422
423Live Configuration Changes
424==========================
425
426.. container:: handout
427
428 Zuul is a distributed system, with a distributed configuration.
429
430.. code:: yaml
431
432 - tenant:
433 name: openstack
434 source:
435 gerrit:
436 config-repos:
437 - opendev/project-config
438 project-repos:
439 - opendev/zuul-jobs
440 - zuul/zuul
441 - zuul/nodepool
442 - ansible/ansible
443 - openstack/openstacksdk
444
445Zuul Startup
446============
447
448* Read config file
449
450Zuul Startup
451============
452
453* Read config file
454* Ask mergers for branches of each repo
455
456.. ansi:: images/startup1.ans
457
458Zuul Startup
459============
460
461* Read config file
462* Ask mergers for branches of each repo
463* Ask mergers for .zuul.yaml for each branch
464
465 of each repo
466
467.. ansi:: images/startup2.ans
468
469When .zuul.yaml Changes
470=======================
471
472.. container:: progressive
473
474 * Zuul looks for changes to .zuul.yaml
475 * Asks mergers for updated content
476 * Splices into configuration used for that change
477 * Works with cross-repo dependencies
478
479 ("This change depends on a change to the job definition")
480
481Zuul Architecture
482=================
483
484.. ansi:: images/architecture.ans
485
486
487Nodepool
488========
489
490 * A separate program that works very closely with *Zuul*
491 * Creates and destroys zero or more node resources
492 * Resources can include VMs, Containers, COE contexts or Bare Metals
493 * Static driver for allocating pre-existing nodes to jobs
494 * Optionally periodically builds images and uploads to clouds
495
496Nodepool Launcher
497=================
498
499 Where build nodes should come from
500
501 * OpenStack
502 * Static
503 * Kubernetes
504
505 In review:
506
507 * OpenShift
508 * AWS
509
510 In work / coming soon:
511
512 * Azure
513 * GCE
514 * Mac Stadium
515
516What about test/job content?
517============================
518
519 * Written in Ansible
520 * Ansible is excellent at running one or more tasks in one or more places
521 * The answer to "how do I" is almost always "Ansible"
522
523What Zuul Does
524==============
525
526 * Listens for code events
527 * Prepares appropriate job config and git repo states
528 * Allocates nodes for test jobs
529 * Pushes git repo states to nodes
530 * Runs user-defined Ansible playbooks
531 * Collects/reports results
532 * Potentially merges change
533
534Jobs
535====
536
537* Jobs run on nodes from nodepool (static or dynamic)
538* Metadata defined in Zuul's configuration
539* Execution content in Ansible
540* Jobs may be defined centrally or in the repo being tested
541* Jobs have contextual variants that simplify configuration
542
543Job
544===
545
546.. code:: yaml
547
548 - job:
549 name: base
550 parent: null
551 description: |
552 The base job for Zuul.
553 timeout: 1800
554 nodeset:
555 nodes:
556 - name: primary
557 label: centos-7
558 pre-run: playbooks/base/pre.yaml
559 post-run:
560 - playbooks/base/post-ssh.yaml
561 - playbooks/base/post-logs.yaml
562 secrets:
563 - site_logs
564
565Simple Job
566==========
567
568.. code:: yaml
569
570 - job:
571 name: tox
572 pre-run: playbooks/setup-tox.yaml
573 run: playbooks/tox.yaml
574 post-run: playbooks/fetch-tox-output.yaml
575
576Simple Job Inheritance
577======================
578
579.. code:: yaml
580
581 - job:
582 name: tox-py36
583 parent: tox
584 vars:
585 tox_envlist: py36
586
587Inheritance Works Like An Onion
588===============================
589
590 * pre-run playbooks run in order of inheritance
591 * run playbook of job runs
592 * post-run playbooks run in reverse order of inheritance
593 * If pre-run playbooks fail, job is re-tried
594 * All post-run playbooks run - as far as pre-run playbooks got
595
596Inheritance Example
597===================
598
599For tox-py36 job
600
601 * base pre-run playbooks/base/pre.yaml
602 * tox pre-run playbooks/setup-tox.yaml
603 * tox run playbooks/tox.yaml
604 * tox post-run playbooks/fetch-tox-output.yaml
605 * base post-run playbooks/base/post-ssh.yaml
606 * base post-run playbooks/base/post-logs.yaml
607
608Simple Job Variant
609==================
610
611.. code:: yaml
612
613 - job:
614 name: tox-py27
615 branches: stable/mitaka
616 nodeset:
617 - name: ubuntu-trusty
618 label: ubuntu-trusty
619
620Nodesets for Multi-node Jobs
621============================
622
623.. code:: yaml
624
625 - nodeset:
626 name: ceph-cluster
627 nodes:
628 - name: controller
629 label: centos-7
630 - name: compute1
631 label: fedora-28
632 - name: compute2
633 label: fedora-28
634 groups:
635 - name: ceph-osd
636 nodes:
637 - controller
638 - name: ceph-monitor
639 nodes:
640 - controller
641 - compute1
642 - compute2
643
644Multi-node Job
645==============
646
647* nodesets are provided to Ansible for jobs in inventory
648
649.. code:: yaml
650
651 - job:
652 name: ceph-multinode
653 nodeset: ceph-cluster
654 run: playbooks/install-ceph.yaml
655
656
657Multi-node Ceph Job Content
658===========================
659
660.. code:: yaml
661
662 - hosts: all
663 roles:
664 - install-ceph
665
666 - hosts: ceph-osd
667 roles:
668 - start-ceph-osd
669
670 - hosts: ceph-monitor
671 roles:
672 - start-ceph-monitor
673
674 - hosts: all
675 roles:
676 - do-something-interesting
677
678Project With Central and Local Config
679=====================================
680
681.. code:: yaml
682
683 # In git.openstack.org/openstack-infra/project-config:
684 - project:
685 name: openstack/nova
686 templates:
687 - openstack-tox-jobs
688
689.. code:: yaml
690
691 # In git.openstack.org/openstack/nova/.zuul.yaml:
692 - project:
693 check:
694 - nova-placement-functional-devstack
695
696Project with Job Dependencies
697=============================
698
699.. code:: yaml
700
701 - project:
702 release:
703 jobs:
704 - build-artifacts
705 - upload-tarball:
706 dependencies: build-artifacts
707 - upload-pypi:
708 dependencies: build-artifacts
709 - notify-mirror:
710 dependencies:
711 - upload-tarball
712 - upload-pypi
713
714Secrets
715=======
716
717* Inspired by Kubernetes Secrets API
718* Projects can add named encrypted secrets to their .zuul.yaml file
719* Jobs can request to use secrets by name
720* Jobs using secrets are not reconfigured speculatively
721* Secrets can only be used by the same project they are defined in
722* Public key per project:
723 ``{{ zuul_url }}/{{ tenant }}/{{ project }}.pub``
724
725::
726 GET https://zuul.openstack.org/openstack-infra/shade.pub
727
728Secret Example (note, no admins had to enable this)
729===================================================
730
731.. code:: yaml
732
733 # In git.openstack.org/openstack/loci/.zuul.yaml:
734 - secret:
735 name: loci_docker_login
736 data:
737 user: loci-username
738 password: !encrypted/pkcs1-oaep
739 - gUEX4eY3JAk/Xt7Evmf/hF7xr6HpNRXTibZjrKTbmI4QYHlzEBrBbHey27Pt/eYvKKeKw
740 hk8MDQ4rNX7ZK1v+CKTilUfOf4AkKYbe6JFDd4z+zIZ2PAA7ZedO5FY/OnqrG7nhLvQHE
741 5nQrYwmxRp4O8eU5qG1dSrM9X+bzri8UnsI7URjqmEsIvlUqtybQKB9qQXT4d6mOeaKGE
742 5h6Ydkb9Zdi4Qh+GpCGDYwHZKu1mBgVK5M1G6NFMy1DYz+4NJNkTRe9J+0TmWhQ/KZSqo
743 4ck0x7Tb0Nr7hQzV8SxlwkaCTLDzvbiqmsJPLmzXY2jry6QsaRCpthS01vnj47itoZ/7p
744 taH9CoJ0Gl7AkaxsrDSVjWSjatTQpsy1ub2fuzWHH4ASJFCiu83Lb2xwYts++r8ZSn+mA
745 hbEs0GzPI6dIWg0u7aUsRWMOB4A+6t2IOJibVYwmwkG8TjHRXxVCLH5sY+i3MR+NicR9T
746 IZFdY/AyH6vt5uHLQDU35+5n91pUG3F2lyiY5aeMOvBL05p27GTMuixR5ZoHcvSoHHtCq
747 7Wnk21iHqmv/UnEzqUfXZOque9YP386RBWkshrHd0x3OHUfBK/WrpivxvIGBzGwMr2qAj
748 /AhJsfDXKBBbhGOGk1u5oBLjeC4SRnAcIVh1+RWzR4/cAhOuy2EcbzxaGb6VTM=
749
750Secret Example
751==============
752
753.. code:: yaml
754
755 # In git.openstack.org/openstack/loci/.zuul.yaml:
756 - job:
757 name: publish-loci-cinder
758 parent: loci-cinder
759 post-run: playbooks/push
760 secrets:
761 - loci_docker_login
762
763 # In git.openstack.org/openstack/loci/playbooks/push.yaml:
764 - hosts: all
765 tasks:
766 - include_vars: vars.yaml
767
768 - name: Push project to DockerHub
769 block:
770 - command: docker login -u {{ loci_docker_login.user }} -p {{ loci_docker_login.password }}
771 no_log: True
772 - command: docker push openstackloci/{{ project }}:{{ branch }}-{{ item.name }}
773 with_items: "{{ distros }}"
774
775OpenDev - Largest Known Zuul
776====================================
777
778 * 2KJPH (2,000 jobs per hour)
779 * Build Nodes from 16 Regions of 5 Public and 3 Private OpenStack Clouds
780 * Rackspace, Internap, OVH, Vexxhost, CityCloud
781 * Linaro (ARM), Limestone, Packethost
782 * 10,000 changes merged per month
783
784Zuul is not New
785===============
786
787 * Has been in Production for OpenStack for Six Years
788 * Zuul is now a top-level effort of OpenStack Foundation
789 * Zuul v3 first release where not-OpenStack is first-class use case
790
791Not just for OpenStack
792======================
793
794 * BMW (control plane in OpenShift)
795 * GoDaddy (control plane in Kubernetes)
796 * Le Bon Coin
797 * GoodMoney
798 * Easystack
799 * TungstenFabric
800 * OpenLab
801 * Red Hat
802 * others ...
803
804Code Review Systems
805===================
806
807 * Gerrit
808 * GitHub (Public and Enterprise)
809
810 In work / coming soon:
811
812 * GitLab
813 * Bitbucket
814
815Support for non-git
816===================
817
818.. container:: progressive
819
820 * Nope
821 * helix4git may work for perforce, but is untested
822
823Installation of Software
824========================
825
826Ways to Install Zuul
827====================
828
829* Containers: https://hub.docker.com/_/zuul/
830* Windmill: http://git.openstack.org/cgit/openstack/windmill
831* Software Factory: https://softwarefactory-project.io/
832* Puppet: http://git.openstack.org/cgit/openstack-infra/puppet-zuul
833
834Zuul Containers
835===============
836
837* Published on every commit
838* Application/Process containers
839* Config / Data should be bind-mounted in
840
841zuul/zuul-executor
842==================
843
844* In k8s, zuul-executor must be run privileged
845* Uses bubblewrap for unprivileged sanboxing
846* Restriction may be lifted in the future
847
848Release Management
849==================
850
851* Zuul is a CI system
852* C stands for "Continuous"
853* It is run Continuously Delivered and Deployed upstream
854* Releases are tagged from code run upstream
855* There is no intent to have a 'stable' release
856* 'stable' is a synonym for "old and buggy"
857
858zuul/zuul-scheduler
859===================
860
861* SPOF
862* We're working on it
863* Recommend running scheduler from tags
864
865Quick Start
866===========
867
868https://zuul-ci.org/docs/zuul/admin/quick-start.html
869
870Important Links
871===============
872
873* https://zuul-ci.org/
874* https://git.zuul-ci.org/cgit/zuul
875* https://zuul-ci.org/docs/zuul
876* https://zuul-ci.org/docs/zuul-jobs/
877* freenode:#zuul
878
879Questions
880=========
881
882.. ansi:: images/questions.ans
883
884Quick Start Prereq
885==================
886
887* Install docker, docker-compose, git-review
888
889Debian/Ubuntu:
890
891::
892
893 sudo apt-get install docker-compose git git-review
894
895RHEL / CentOS / Fedora:
896
897::
898
899 sudo yum install docker docker-compose git git-review
900
901OpenSuse:
902
903::
904
905 sudo zypper install docker docker-compose git git-review
906
907RHEL / CentOS / Fedora / OpenSuse
908
909::
910
911 sudo systemctl enable docker.service
912 sudo systemctl start docker.service
913
914Actual Quick Start
915==================
916
917* git clone https://git.zuul-ci.org/zuul
918* cd zuul
919* cd doc/source/admin/examples
920* docker-compose up
921
922What's Running
923==============
924
925* Zookeeper
926* Gerrit
927* Nodepool Launcher
928* Zuul Scheduler
929* Zuul Web Server
930* Zuul Executor
931* Apache HTTPD
932* A container to use as a 'static' build node
933
934How they're connected
935=====================
936
937* End Users talk to Gerrit and Apache HTTPD
938* Zuul Scheduler talks to Gerrit
939* Nodepool Launcher, Zuul Scheduler, Zuul Web talk to Zookeeper
940* Zuul Executor talks to Zuul Scheduler (using Gearman)
941
942Initial provided config
943=======================
944
945* docker-compose has plumbed in basic config ``etc_zuul/zuul.conf``
946 and ``etc_zuul/main.yaml``
947* Gerrit Connection named "gerrit"
948* Zuul user for that connection
949* Git connection named "zuul-ci.org" for ``zuul-jobs`` standard library
950
951Initial tenant
952==============
953
954* Zuul is (always) multi-tenant
955* Example config contains a tenant called ``example-tenant``
956* Three projects in the ``example-tenant`` tenant:
957 ``zuul-config``, ``test1``, ``test2``
958* Three projects are also in gerrit ready to use
959
960zuul.conf
961=========
962
963::
964
965 [gearman]
966 server=scheduler
967
968 [gearman_server]
969 start=true
970
971 [zookeeper]
972 hosts=zk
973
974 [scheduler]
975 tenant_config=/etc/zuul/main.yaml
976
977 [web]
978 listen_address=0.0.0.0
979
980 [executor]
981 private_key_file=/var/ssh/nodepool
982 default_username=root
983
984zuul.conf part 2
985================
986
987::
988
989 [connection "gerrit"]
990 name=gerrit
991 driver=gerrit
992 server=gerrit
993 sshkey=/var/ssh/zuul
994 user=zuul
995 password=secret
996 baseurl=http://gerrit:8080
997 auth_type=basic
998
999 [connection "zuul-ci.org"]
1000 name=zuul-ci
1001 driver=git
1002 baseurl=https://git.zuul-ci.org/
1003
1004main.yaml
1005=========
1006
1007::
1008
1009 - tenant:
1010 name: example-tenant
1011 source:
1012 gerrit:
1013 config-projects:
1014 - zuul-config
1015 untrusted-projects:
1016 - test1
1017 - test2
1018 zuul-ci.org:
1019 untrusted-projects:
1020 - zuul-jobs:
1021 include:
1022 - job
1023
1024Gerrit Account
1025==============
1026
1027* Need a user account to interact with Gerrit
1028* Gerrit is configured in dev mode - no passwords required
1029* Visit http://localhost:8080
1030* Click "Become"
1031* Click "New Account"
1032* Click "Register"
1033* Enter Full Name
1034* Click "Save Changes"
1035* Enter username in Username field (match your local laptop user)
1036* Copy ``~/.ssh/id_rsa.pub`` contents into SSH Key field
1037* Click Continue
1038
1039Config Repo
1040===========
1041
1042* ``zuul-config`` is a trusted ``config-repo``
1043* Security and functionality of system depend on this repo
1044* Limit its contents to minimum required
1045
1046Config Files vs. Directories
1047============================
1048
1049* Zuul reads config from:
1050 ``.zuul.yaml``, ``zuul.yaml``, ``zuul.d`` or ``.zuul.d``
1051* For projects with substantial zuul config, like ``zuul-config``
1052 ``zuul.d`` directory is likely best.
1053* The directories are read run-parts style.
1054* Recommended practice is splitting by type of object
1055
1056Setting up Gating
1057=================
1058
1059* We want to have changes to ``zuul-config`` be gated
1060* We need to define pipelines: ``check`` and ``gate``
1061* Need to attach ``zuul-config`` to them
1062* Start with builtin ``noop`` job (always return success)
1063* Use regex to attach all projects to ``check`` and ``gate``
1064
1065Pipeline Definitions
1066====================
1067
1068* Zuul has no built-in workflow definitions, let's add ``check`` and ``gate``
1069
1070check pipeline
1071==============
1072
1073::
1074
1075 - pipeline:
1076 name: check
1077 description: |
1078 Newly uploaded patchsets enter this pipeline to receive an
1079 initial +/-1 Verified vote.
1080 manager: independent
1081 require:
1082 gerrit:
1083 open: True
1084 current-patchset: True
1085 trigger:
1086 gerrit:
1087 - event: patchset-created
1088 - event: change-restored
1089 success:
1090 gerrit:
1091 Verified: 1
1092 failure:
1093 gerrit:
1094 Verified: -1
1095
1096gate pipeline
1097=============
1098
1099::
1100 - pipeline:
1101 name: gate
1102 description: |
1103 Changes that have been approved are enqueued in order in this
1104 pipeline, and if they pass tests, will be merged.
1105 manager: dependent
1106 post-review: True
1107 require:
1108 gerrit:
1109 open: True
1110 current-patchset: True
1111 approval:
1112 - Workflow: 1
1113 trigger:
1114 gerrit:
1115 - event: comment-added
1116 approval:
1117 - Workflow: 1
1118 start:
1119 gerrit:
1120 Verified: 0
1121 success:
1122 gerrit:
1123 Verified: 2
1124 submit: true
1125 failure:
1126 gerrit:
1127 Verified: -2
1128
1129Add the pipeline definitions
1130============================
1131
1132.. code-block:: bash
1133
1134 git clone http://localhost:8080/zuul-config
1135 cd zuul-config
1136 mkdir zuul.d
1137 cp ../examples/zuul-config/zuul.d/pipelines.yaml .
1138
1139Shared Project Pipeline Definition
1140==================================
1141
1142In ``examples/zuul-config/zuul.d/projects.yaml``
1143
1144.. code-block:: yaml
1145
1146 - project:
1147 name: ^.*$
1148 check:
1149 jobs: []
1150 gate:
1151 jobs: []
1152
1153 - project:
1154 name: zuul-config
1155 check:
1156 jobs:
1157 - noop
1158 gate:
1159 jobs:
1160 - noop
1161
1162Attach the projects to the pipelines
1163====================================
1164
1165.. code-block:: bash
1166
1167 cp ../examples/zuul-config/zuul.d/projects.yaml .
1168
1169Commit the changes and push up for review
1170=========================================
1171
1172.. code-block:: bash
1173
1174 git add zuul.d
1175 git commit
1176 git review
1177
1178Force merging bootstrap config
1179==============================
1180
1181* Zuul is running with no config, so it won't do anything
1182* For this change (and this change only) we will bypass gating
1183
1184Reviewing normally
1185==================
1186
1187* visit http://localhost:8080/#/c/zuul-config/+/1001/
1188* click reply
1189* vote +2 Code Review +1 Approved
1190
1191Verified +2 is Missing
1192======================
1193
1194Verified +2 is what we have zuul configured to do.
1195
1196::
1197 success:
1198 gerrit:
1199 Verified: 2
1200 submit: true
1201
1202
1203Bypassing Gating
1204================
1205
1206* visit http://localhost:8080/
1207* click 'switch account'
1208* click 'admin'
1209* visit http://localhost:8080/#/c/zuul-config/+/1001/
1210* click reply
1211* vote +2 Verified (normal users do not see this)
1212* click submit (normal users do not see this)
1213* click 'switch account'
1214* click your username
1215
1216Base Job
1217========
1218
1219* Every Zuul installation must define a ``base`` job
1220* Push git repos to build node
1221* Publish logs/artifacts
1222* Any local specific setup
1223* Goes in config repo - because it impacts EVERY job
1224
1225Add Base Job to zuul-config
1226===========================
1227
1228::
1229
1230 cp ../examples/zuul-config/zuul.d/jobs.yaml .
1231 git add jobs.yaml
1232 git commit
1233 git review
1234
1235Then go to http://localhost:8080/#/c/zuul-config/+/1002/ and approve it
1236
1237Zuul should merge the patch
1238===========================
1239
1240zuul-config is configured to use the ``noop`` job
1241
1242Zuul tests syntax automatically
1243===============================
1244
1245* Edit jobs.yaml
1246* Change ``parent: null`` to ``parent: broken``
1247* git commit ; git review
1248* Check out the review in gerrit ... there should be errors!
1249
1250Presentty
1251=========
1252.. hidetitle::
1253.. transition:: pan
1254.. figlet:: Presentty
1255
1256* Console presentations written in reStructuredText
1257* Cross-fade, pan, tilt, cut transitions
1258* https://pypi.python.org/pypi/presentty