CI-integrated continuous deployment (Q2, cluster setup)

(this issue has been split from FOLIO-1550 to indicate the cluster-setup work completed in Q2 2019)

Problem statement

Current FOLIO CI infrastructure and reference environments have several limitations that make it hard to scale them the point where a fully continuos and isolated deployment of development builds (e.g from PRs and feature branches, for both UI and backend modules) is possible. Those limitations include:

• the reference envs (e.g snapshot) are recreated in full nightly, in a top-down approach (starting from a platform and resolving its dependencies which are deployed one by one).
• the reference envs are primarily single-tenant (the so-called 'diku' demo tenant)
• the reference envs are constructed from artefacts based on shared branches (master) only and all those artefacts are shared at runtime by the single demo tenant

This is makes it impossible to implement certain development process capabilities (like the PR previews, see FOLIO-1993) without a list of fairly severe limitations (see description of the PR preview PoC for a comprehensive list of those limitations at https://dev.folio.org/guides/branch-preview/)

Proposed approach

We would like to revamp the deployment and orchestration infrastructure, create a new, clustered and multi-tenant, reference environment for development and integration purposes, and update the CI processes (Jenkins, Ansible, etc) to allow for a more continuos and isolated deployment of development artefacts.

Orchestration

Based on the prior work from from Jason Root (TAMU) and Mark Stacy (Core: platform/Colorado, see FOLIO-1408) who have experimented with various orchestration tools and we have concluded that Kubernetes (K8s) has become the defacto standard for container orchestration and it is the orchestration tool of choice across major cloud vendors. It seems very likely that many organisations will use K8s for production-ready FOLIO deployments. K8s also brings many benefits for development deployments: a rich ecosystem of tools that ease provisioning of dependencies (e.g Helm) and widely accepted practices and processes for deployment of multiple development builds.

The Core: platform has undertaken a focused effort to ease K8s integration across FOLIO platform. This effort is tracked on FOLIO-1931.

Clustered reference environment

This work is being done outside of the Platform team, see FOLIO-2053

CI process

We would like to extend the CI process in a way which allows to:

• deploy a backend container immediately after a successful automatic build (Jenkins/CI) on the clustered reference environment. This would include both snapshot and release builds and could be scaled up to allow for a deployment of feature branch builds (at specific points in the PR lifecycle). The deployed container should be registered with Okapi running on the clustered reference environment in order to allow for it being used as a dependency in appropriate tenant configurations.

• create an independent tenant, on the clustered reference environment, for each platform- and ui- module build (of all kinds, including release, snapshot, and feature branch) to allow for running the particular build in isolation (including sample data set isolation). In the case of ui- modules, which are not self-contained units, the build process must be able to embed the module in an appropriate platform to form a complete Stripes bundle. Such bundle should be exposed to users after a succesful build. Okapi remains responsible for providing backend service dependencies in the created tenant.

See FOLIO-2054, FOLIO-2055 and FOLIO-2056

Further read

PR previews: FOLIO-1933
Kubernetes integration: FOLIO-1931
FOLIO RM and CI concepts: https://docs.google.com/document/d/1au2hG4gPekyZ_HxAU7s6sc4NOvRaVovUcBnMLa1iR7E/edit?usp=sharing