Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Spell check

...

...

...

...


Excerpt

Description of the eHealth development and deployment cycle, including description of the use of eHealth environments and flow. 

Table of Contents

Overview of build and deployment 

Applications must be deployed using one of the predefined eHealth Helm chart charts and deployed in eHealth Service Mesh

Telemedicine Solutions provider providers must use the eHealth Infrastructure Deployment Pipeline for deploying to the eHealth environments.

  1. Telemedicine Solutions provider shall use the official eHealth docker images and helm chart for application definition and development

  2. After local development, the Telemedicine Solutions Providers shall upload the docker images to the Docker and Helm chart repository (Harbor)

  3. Telemedicine Solutions Providers

...

  1. use GitLab to define the deployment file or "desired state file" based on Helmsman

  2. A configured Jenkins deployment pipeline monitors the Helmans file in GitLab and

...

  1. pushes updates to Kubernetes 

    1. The Jenkins deployment pipeline is configured by the eHealth Infrastructure provider.

  2. Kubernetes control loop 

...

  1. watches the state of your cluster, then

...

  1. makes or

...

  1. requests changes where needed

The following figure illustrates the deployment pipeline.

Gliffy
imageAttachmentIdatt2233565209
baseUrlhttps://ehealth-dk.atlassian.net/wiki
macroIdc72851e5-d1dc-4a7b-bf1d-89830f727d8c
namebuild-and-deployment-pipeline
diagramAttachmentIdatt2233008167
containerId147783683
timestamp1647609331105

Development and deployment happens happen in four phases.

Phase #1 Local development

All Telemedicine Solution Providers shall start the development of their components in their own local environment.

This could be a complete clone of the official test environments running on the eHealth - platform. But this is not a strict requirement.

Phase #2 Publish build to the eHealth

...

platform

When the Solution Providers provider supplier believes that the application/service/microservice is ready for testing it is published to the eHealth - platform.

This means that the docker image is signed and pushed to the central docker image registry used and hosted by the eHealth - platform.

Phase #3 Deploy to test environment

...

All applications running on an environment is specified as code in the desired state specification for the given environment.

For each application, this includes:

  • The docker image to run

    • Docker repository

    • Image tag

  • The helm chart

    • Helm chart repository

    • Helm chart version

  • Configuration

    • Ports

    • Replicas

    • Memory usage

    • Environment variables

    • Database and queue secrets

    • DNS bindings

    • etc.

When the desired state specification is updated the applications and configuration is automatically rolled on to onto the environment.

Phase #4 Test and Promotion

Notice, that every release must follow a strict release plan where eHealth environments are visited in the required order, that which is deployed and tested on EXTTEST, before being promoted to PREPRODUCTION, and finally deployed to PRODUCTION.

...

Tests should be carried out on the Test environmentTest environment.

When the application has passed QA, it can be promoted to the next environment. This happens by a promotion of a specific desired state specification in Jenkins-test, by users with the right privileges.

...

Going onwards to preprod happens by making a pull - request to the "prod" branch from the "master" branch.Image Removed


...


When the pull - request is approved all updates in the desired state specification for preprod is are rolled on the preprod environment.

...

If serious bugs or security errors is are found in production code, or containers a hotfix can be handled in the following way.Image Removed


...



Rollback

Rollback is handled somewhat semi-automatic, if the problem is visible through the pod probe services' liveness and readiness. In other rollback situation situations manual intervention is necessary.

In the first situation, where either liveness and or readiness is are responding negativenegatively, the new pod instance will never be part of the serving cluster. The cluster will automatically keep sending traffic to the old instances of the service and skip the new pod. What should happen next has to be decided by the one who started the deployment. It could be a configuration fix or a rollback to the old software version.

In the second situation, where both liveness and readiness respond positivepositively, the new pod will receive trafiktraffic. The one who started the deployment will look at the cluster readout and decide on a rollback. That person has to commit to a new cluster configuration. This could be a reverse commit to the desired state file.
Note: if we are using Canary deployment, the negative impact of a software problem can be held at a minimum. That is if the negative impact can be found in prometheusPrometheus.

Diagram of the flow

The flow can allows be viewed as follows.

  1. Application Vendor

...

  1. performs Local Development 

  2. Application Vendor pust application and docker images to the FUT platform

  3. The eHealth infrastructure

...

  1. deploys to different environments, based on the Helmsman files provided by the application vendor.

  2. Application Vendor (or Customer) can test the Telemedicine solution on EXTTEST 

  3. After

...

  1. successful tests, the build can then be promoted to the next eHealth environment (PREPROCUTION). 

  2. After

...

  1. successful tests, the build can then be promoted to the eHealth PRODUCTION environment. 

Info

In the figure is is shown as SRE Team (Software Reliable Engineering) must approve deployment to external test, pre-prod and production. This is not the case for external applications, furthermore, external applications do not have access to Internal Test Environment (INTTEST), but has a special development environment called DEVENVCGI, for their internal test.



...