...
If serious bugs or security errors is found in production code, or containers a hotfix can be handled in the following way.
4.2. Rollback
Rollback is handled somewhat semi-automatic, if the problem is visible through the pod probe services liveness and readiness. In other rollback situation manual intervention is necessary.
...
In the second situation, where both liveness and readiness respond positive, the new pod will receive trafik. The one who started the deployment will look at cluster readout and decide a rollback. That person has to commit a new cluster configuration. This could be a reverse commit to the desired state file.
Note: if we are using Canary deployment, the negative impact of a software problem can be held at a minimum. That is if the negative impact can be found in prometheus.
Diagram of the flow
