diff options
Diffstat (limited to 'src/zuulv3')
-rw-r--r-- | src/zuulv3/overview.rst | 200 |
1 files changed, 136 insertions, 64 deletions
diff --git a/src/zuulv3/overview.rst b/src/zuulv3/overview.rst index a14e671..f1b86dc 100644 --- a/src/zuulv3/overview.rst +++ b/src/zuulv3/overview.rst | |||
@@ -70,7 +70,7 @@ OpenDev | |||
70 | 70 | ||
71 | -- Alex Gaynor | 71 | -- Alex Gaynor |
72 | 72 | ||
73 | "OpenStack Infra are like the SpaceX of CI" | 73 | "like the SpaceX of CI" |
74 | 74 | ||
75 | -- Emily Dunham | 75 | -- Emily Dunham |
76 | 76 | ||
@@ -83,6 +83,7 @@ Zuul | |||
83 | What Zuul Does | 83 | What Zuul Does |
84 | ============== | 84 | ============== |
85 | 85 | ||
86 | * "Speculative Future State" | ||
86 | * gated changes | 87 | * gated changes |
87 | * one or more git repositories | 88 | * one or more git repositories |
88 | * integrated deliverable | 89 | * integrated deliverable |
@@ -93,6 +94,7 @@ Underlying Philosophy | |||
93 | 94 | ||
94 | * All changes flow through code review | 95 | * All changes flow through code review |
95 | * Changes only land if they pass all tests | 96 | * Changes only land if they pass all tests |
97 | * End-to-end integration testing is essential | ||
96 | * Computers are cheaper than humans | 98 | * Computers are cheaper than humans |
97 | 99 | ||
98 | Ramifications of Philosophy | 100 | Ramifications of Philosophy |
@@ -146,8 +148,8 @@ Gating | |||
146 | ====== | 148 | ====== |
147 | 149 | ||
148 | * Changes are pushed to code review | 150 | * Changes are pushed to code review |
149 | * CI system is triggered by code review approval | 151 | * Gating system is triggered by code review approval |
150 | * CI system merges code IFF tests pass | 152 | * Gating system merges code IFF tests pass |
151 | * Proactive - testing code before it lands | 153 | * Proactive - testing code before it lands |
152 | * Future state resulting from merge of code is tested | 154 | * Future state resulting from merge of code is tested |
153 | * Reviewers can fire-and-forget safely | 155 | * Reviewers can fire-and-forget safely |
@@ -400,35 +402,6 @@ Zuul Simulation | |||
400 | .. ansi:: images/zsim-22.ans | 402 | .. ansi:: images/zsim-22.ans |
401 | 403 | ||
402 | 404 | ||
403 | Cross-Project Dependencies | ||
404 | ========================== | ||
405 | |||
406 | Testing or gating dependencies manually specified by developers | ||
407 | |||
408 | .. container:: progressive | ||
409 | |||
410 | * nodepool https://review.openstack.org/612168 | ||
411 | |||
412 | Make functional src jobs actually install from source | ||
413 | * openstacksdk https://review.openstack.org/612186 | ||
414 | |||
415 | Don't start task managers passed in to Connection | ||
416 | |||
417 | Depends-On: https://review.openstack.org/612168 | ||
418 | * openstacksdk https://review.openstack.org/604521 | ||
419 | |||
420 | Add support for per-service rate limits | ||
421 | |||
422 | (git parent is 612186) | ||
423 | * nodepool https://review.openstack.org/612169 | ||
424 | |||
425 | Consume rate limiting task manager from openstacksdk | ||
426 | |||
427 | Depends-On: https://review.openstack.org/604521 | ||
428 | |||
429 | (nodepool-functional-py35-src should pass, but | ||
430 | nodepool-functional-py35 should not fail until openstacksdk release) | ||
431 | |||
432 | Lock Step Changes | 405 | Lock Step Changes |
433 | ================= | 406 | ================= |
434 | 407 | ||
@@ -452,7 +425,7 @@ Live Configuration Changes | |||
452 | config-repos: | 425 | config-repos: |
453 | - opendev/project-config | 426 | - opendev/project-config |
454 | project-repos: | 427 | project-repos: |
455 | - opendev/zuul-jobs | 428 | - zuul/zuul-jobs |
456 | - zuul/zuul | 429 | - zuul/zuul |
457 | - zuul/nodepool | 430 | - zuul/nodepool |
458 | - ansible/ansible | 431 | - ansible/ansible |
@@ -494,16 +467,89 @@ When .zuul.yaml Changes | |||
494 | 467 | ||
495 | ("This change depends on a change to the job definition") | 468 | ("This change depends on a change to the job definition") |
496 | 469 | ||
470 | Explicit Cross-Project Dependencies | ||
471 | =================================== | ||
472 | |||
473 | * Developers can mark changes as being dependent | ||
474 | * Depends-On: footer - in commit or PR | ||
475 | * Zuul uses depends-on when constructing virtual serial queue | ||
476 | * Will not merge changes in gate before depends-on changes | ||
477 | * Works cross-repo AND cross-source | ||
478 | |||
479 | Depends-On Example | ||
480 | ================== | ||
481 | |||
482 | * Service 'nova' talks to service 'ironic' | ||
483 | * Currently using 'python-ironicclient' | ||
484 | * Want to replace python-ironicclient with openstacksdk: | ||
485 | * https://review.openstack.org/643664 | ||
486 | * Need some plumbing in nova first: | ||
487 | * https://review.openstack.org/642899 | ||
488 | * That change "Depends-On" a change to openstacksdk | ||
489 | |||
490 | Depends-On Example - openstacksdk | ||
491 | ================================= | ||
492 | |||
493 | * In openstacksdk, need a new method to extract config differently | ||
494 | * https://review.openstack.org/643601 | ||
495 | * The nova plumbing change adds this: | ||
496 | |||
497 | :: | ||
498 | |||
499 | Depends-On: https://review.openstack.org/643601 | ||
500 | |||
501 | Depends-On Example - keystoneauth | ||
502 | ================================= | ||
503 | |||
504 | * openstacksdk uses 'keystoneauth' library to make REST calls | ||
505 | * Config extraction change wants a new helper method in keystoneauth | ||
506 | * https://review.openstack.org/644251 | ||
507 | * openstacksdk change adds: | ||
508 | |||
509 | :: | ||
510 | |||
511 | Depends-On: https://review.openstack.org/644251 | ||
512 | |||
513 | Depends-On Example - In the Gate | ||
514 | ================================ | ||
515 | |||
516 | * When Zuul prepares git repos for the Ironic nova change: | ||
517 | * Tip of nova, plus nova plumbing change, plus nova ironic change | ||
518 | * Tip of openstacksdk, plus config method change | ||
519 | * Tip of keystoneauth, plus helper method change | ||
520 | * Developers iterate on the nova service change | ||
521 | * BEFORE finalizing and releasing keystoneauth and openstacksdk changes | ||
522 | |||
523 | Zuul Architecture | ||
524 | ================= | ||
525 | |||
526 | We used to call "microservices" "distributed" | ||
527 | |||
528 | * Zuul is comprised of several services (mostly python3) | ||
529 | * zuul-scheduler | ||
530 | * zuul-executor | ||
531 | * zuul-merger | ||
532 | * zuul-web | ||
533 | * zuul-dashboard (javascript/react) | ||
534 | * zuul-fingergw | ||
535 | * zuul-proxy (c++) | ||
536 | * RDBMS | ||
537 | * Gearman | ||
538 | * Zookeeper | ||
539 | |||
497 | Zuul Architecture | 540 | Zuul Architecture |
498 | ================= | 541 | ================= |
499 | 542 | ||
500 | .. ansi:: images/architecture.ans | 543 | .. ansi:: images/architecture.ans |
501 | 544 | ||
545 | Where Does Job Content Run? | ||
546 | =========================== | ||
502 | 547 | ||
503 | Nodepool | 548 | Nodepool |
504 | ======== | 549 | ======== |
505 | 550 | ||
506 | * A separate program that works very closely with *Zuul* | 551 | * A separate program that works very closely with *Zuul* |
552 | * *Zuul* requires *Nodepool* but *Nodepool* can be used independently | ||
507 | * Creates and destroys zero or more node resources | 553 | * Creates and destroys zero or more node resources |
508 | * Resources can include VMs, Containers, COE contexts or Bare Metals | 554 | * Resources can include VMs, Containers, COE contexts or Bare Metals |
509 | * Static driver for allocating pre-existing nodes to jobs | 555 | * Static driver for allocating pre-existing nodes to jobs |
@@ -517,17 +563,12 @@ Nodepool Launcher | |||
517 | * OpenStack | 563 | * OpenStack |
518 | * Static | 564 | * Static |
519 | * Kubernetes | 565 | * Kubernetes |
520 | |||
521 | In review: | ||
522 | |||
523 | * OpenShift | ||
524 | * AWS | 566 | * AWS |
525 | 567 | ||
526 | In work / coming soon: | 568 | In work / coming soon: |
527 | 569 | ||
528 | * Azure | 570 | * Azure |
529 | * GCE | 571 | * GCE |
530 | * Mac Stadium | ||
531 | 572 | ||
532 | What about test/job content? | 573 | What about test/job content? |
533 | ============================ | 574 | ============================ |
@@ -541,16 +582,15 @@ What Zuul Does | |||
541 | 582 | ||
542 | * Listens for code events | 583 | * Listens for code events |
543 | * Prepares appropriate job config and git repo states | 584 | * Prepares appropriate job config and git repo states |
544 | * Allocates nodes for test jobs | 585 | * Requests nodes for test jobs from *Nodepool* |
545 | * Pushes git repo states to nodes | 586 | * Runs user-defined Ansible playbooks with nodes in an inventory |
546 | * Runs user-defined Ansible playbooks | ||
547 | * Collects/reports results | 587 | * Collects/reports results |
548 | * Potentially merges change | 588 | * Potentially merges change |
549 | 589 | ||
550 | Jobs | 590 | Jobs |
551 | ==== | 591 | ==== |
552 | 592 | ||
553 | * Jobs run on nodes from nodepool (static or dynamic) | 593 | * Jobs define test node needs |
554 | * Metadata defined in Zuul's configuration | 594 | * Metadata defined in Zuul's configuration |
555 | * Execution content in Ansible | 595 | * Execution content in Ansible |
556 | * Jobs may be defined centrally or in the repo being tested | 596 | * Jobs may be defined centrally or in the repo being tested |
@@ -570,7 +610,7 @@ Job | |||
570 | nodeset: | 610 | nodeset: |
571 | nodes: | 611 | nodes: |
572 | - name: primary | 612 | - name: primary |
573 | label: centos-7 | 613 | label: ubuntu-bionic |
574 | pre-run: playbooks/base/pre.yaml | 614 | pre-run: playbooks/base/pre.yaml |
575 | post-run: | 615 | post-run: |
576 | - playbooks/base/post-ssh.yaml | 616 | - playbooks/base/post-ssh.yaml |
@@ -669,6 +709,21 @@ Multi-node Job | |||
669 | nodeset: ceph-cluster | 709 | nodeset: ceph-cluster |
670 | run: playbooks/install-ceph.yaml | 710 | run: playbooks/install-ceph.yaml |
671 | 711 | ||
712 | * Creates ansible inventory: | ||
713 | |||
714 | .. code:: yaml | ||
715 | |||
716 | controller ansible_host=1.2.3.4 | ||
717 | compute1 ansible_host=1.2.3.5 | ||
718 | compute2 ansible_host=1.2.3.6 | ||
719 | |||
720 | [ceph-osd] | ||
721 | controller | ||
722 | |||
723 | [ceph-monitor] | ||
724 | controller | ||
725 | compute1 | ||
726 | compute2 | ||
672 | 727 | ||
673 | Multi-node Ceph Job Content | 728 | Multi-node Ceph Job Content |
674 | =========================== | 729 | =========================== |
@@ -696,7 +751,7 @@ Project With Central and Local Config | |||
696 | 751 | ||
697 | .. code:: yaml | 752 | .. code:: yaml |
698 | 753 | ||
699 | # In git.openstack.org/openstack-infra/project-config: | 754 | # In opendev.org/openstack-infra/project-config: |
700 | - project: | 755 | - project: |
701 | name: openstack/nova | 756 | name: openstack/nova |
702 | templates: | 757 | templates: |
@@ -704,11 +759,18 @@ Project With Central and Local Config | |||
704 | 759 | ||
705 | .. code:: yaml | 760 | .. code:: yaml |
706 | 761 | ||
707 | # In git.openstack.org/openstack/nova/.zuul.yaml: | 762 | # In opendev.org/openstack/nova/.zuul.yaml: |
708 | - project: | 763 | - project: |
709 | check: | 764 | check: |
710 | - nova-placement-functional-devstack | 765 | - nova-placement-functional-devstack |
711 | 766 | ||
767 | zuul-jobs standard library | ||
768 | ========================== | ||
769 | |||
770 | * https://opendev.org/openstack-infra/zuul-jobs | ||
771 | * Repo containing general purpose job definitions | ||
772 | * Add the git repo directly to a local Zuul config | ||
773 | |||
712 | Project with Job Dependencies | 774 | Project with Job Dependencies |
713 | ============================= | 775 | ============================= |
714 | 776 | ||
@@ -746,7 +808,7 @@ Secret Example (note, no admins had to enable this) | |||
746 | 808 | ||
747 | .. code:: yaml | 809 | .. code:: yaml |
748 | 810 | ||
749 | # In git.openstack.org/openstack/loci/.zuul.yaml: | 811 | # In opendev.org/openstack/loci/.zuul.yaml: |
750 | - secret: | 812 | - secret: |
751 | name: loci_docker_login | 813 | name: loci_docker_login |
752 | data: | 814 | data: |
@@ -768,7 +830,7 @@ Secret Example | |||
768 | 830 | ||
769 | .. code:: yaml | 831 | .. code:: yaml |
770 | 832 | ||
771 | # In git.openstack.org/openstack/loci/.zuul.yaml: | 833 | # In opendev.org/openstack/loci/.zuul.yaml: |
772 | - job: | 834 | - job: |
773 | name: publish-loci-cinder | 835 | name: publish-loci-cinder |
774 | parent: loci-cinder | 836 | parent: loci-cinder |
@@ -776,7 +838,7 @@ Secret Example | |||
776 | secrets: | 838 | secrets: |
777 | - loci_docker_login | 839 | - loci_docker_login |
778 | 840 | ||
779 | # In git.openstack.org/openstack/loci/playbooks/push.yaml: | 841 | # In opendev.org/openstack/loci/playbooks/push.yaml: |
780 | - hosts: all | 842 | - hosts: all |
781 | tasks: | 843 | tasks: |
782 | - include_vars: vars.yaml | 844 | - include_vars: vars.yaml |
@@ -788,14 +850,14 @@ Secret Example | |||
788 | - command: docker push openstackloci/{{ project }}:{{ branch }}-{{ item.name }} | 850 | - command: docker push openstackloci/{{ project }}:{{ branch }}-{{ item.name }} |
789 | with_items: "{{ distros }}" | 851 | with_items: "{{ distros }}" |
790 | 852 | ||
791 | OpenDev - Largest Known Zuul | 853 | Speculative Conatiner Images |
792 | ==================================== | 854 | ============================ |
793 | 855 | ||
794 | * 2KJPH (2,000 jobs per hour) | 856 | * Gating applied to continuously deployed container images |
795 | * Build Nodes from 16 Regions of 5 Public and 3 Private OpenStack Clouds | 857 | * Build and test images that depend on other images |
796 | * Rackspace, Internap, OVH, Vexxhost, CityCloud | 858 | * Build and test deployments comprising multiple images |
797 | * Linaro (ARM), Limestone, Packethost | 859 | * Without publishing to final location |
798 | * 10,000 changes merged per month | 860 | * Publish the actual image that was built in the gate |
799 | 861 | ||
800 | Zuul is not New | 862 | Zuul is not New |
801 | =============== | 863 | =============== |
@@ -804,13 +866,22 @@ Zuul is not New | |||
804 | * Zuul is now a top-level effort of OpenStack Foundation | 866 | * Zuul is now a top-level effort of OpenStack Foundation |
805 | * Zuul v3 first release where not-OpenStack is first-class use case | 867 | * Zuul v3 first release where not-OpenStack is first-class use case |
806 | 868 | ||
869 | OpenDev - Largest Known Zuul | ||
870 | ============================ | ||
871 | |||
872 | * 2KJPH (2,000 jobs per hour) | ||
873 | * Build Nodes from 16 Regions of 5 Public and 3 Private OpenStack Clouds | ||
874 | * Rackspace, Internap, OVH, Vexxhost, CityCloud | ||
875 | * Linaro (ARM), Limestone, Packethost | ||
876 | * 10,000 changes merged per month | ||
877 | |||
807 | Not just for OpenStack | 878 | Not just for OpenStack |
808 | ====================== | 879 | ====================== |
809 | 880 | ||
810 | * BMW (control plane in OpenShift) | 881 | * BMW (control plane in OpenShift) |
811 | * GoDaddy (control plane in Kubernetes) | 882 | * GoDaddy (control plane in private Kubernetes) |
883 | * GoodMoney (control plane in EKS, adding GKE) | ||
812 | * Le Bon Coin | 884 | * Le Bon Coin |
813 | * GoodMoney | ||
814 | * Easystack | 885 | * Easystack |
815 | * TungstenFabric | 886 | * TungstenFabric |
816 | * OpenLab | 887 | * OpenLab |
@@ -826,8 +897,10 @@ Code Review Systems | |||
826 | In work / coming soon: | 897 | In work / coming soon: |
827 | 898 | ||
828 | * Pagure | 899 | * Pagure |
900 | * Gitea | ||
829 | 901 | ||
830 | Commonly Requested: | 902 | Commonly Requested: |
903 | |||
831 | * GitLab | 904 | * GitLab |
832 | * Bitbucket | 905 | * Bitbucket |
833 | 906 | ||
@@ -846,9 +919,9 @@ Ways to Install Zuul | |||
846 | ==================== | 919 | ==================== |
847 | 920 | ||
848 | * Containers: https://hub.docker.com/_/zuul/ | 921 | * Containers: https://hub.docker.com/_/zuul/ |
849 | * Windmill: http://git.openstack.org/cgit/openstack/windmill | 922 | * Windmill: http://opendev.org/openstack/windmill |
850 | * Software Factory: https://softwarefactory-project.io/ | 923 | * Software Factory: https://softwarefactory-project.io/ |
851 | * Puppet: http://git.openstack.org/cgit/openstack-infra/puppet-zuul | 924 | * Puppet: http://opendev.org/openstack-infra/puppet-zuul |
852 | 925 | ||
853 | Zuul Containers | 926 | Zuul Containers |
854 | =============== | 927 | =============== |
@@ -867,10 +940,9 @@ zuul/zuul-executor | |||
867 | Release Management | 940 | Release Management |
868 | ================== | 941 | ================== |
869 | 942 | ||
870 | * Zuul is a CI system | 943 | * Zuul is run Continuously Delivered and Deployed upstream |
871 | * C stands for "Continuous" | 944 | * Some users deploy Zuul with Zuul |
872 | * It is run Continuously Delivered and Deployed upstream | 945 | * Releases are tagged from code run for OpenDev |
873 | * Releases are tagged from code run upstream | ||
874 | * There is no intent to have a 'stable' release | 946 | * There is no intent to have a 'stable' release |
875 | * 'stable' is a synonym for "old and buggy" | 947 | * 'stable' is a synonym for "old and buggy" |
876 | 948 | ||
@@ -890,10 +962,10 @@ Important Links | |||
890 | =============== | 962 | =============== |
891 | 963 | ||
892 | * https://zuul-ci.org/ | 964 | * https://zuul-ci.org/ |
893 | * https://git.zuul-ci.org/cgit/zuul | ||
894 | * https://zuul-ci.org/docs/zuul | 965 | * https://zuul-ci.org/docs/zuul |
895 | * https://zuul-ci.org/docs/zuul-jobs/ | 966 | * https://zuul-ci.org/docs/zuul-jobs/ |
896 | * freenode:#zuul | 967 | * freenode:#zuul |
968 | * https://opendev.org/zuul (https://git.zuul-ci.org/cgit/zuul) | ||
897 | 969 | ||
898 | Questions | 970 | Questions |
899 | ========= | 971 | ========= |