Call Tracing
The eHealth infrastructure enables the tracing of all actions across various services and components. This is implemented using Istio and OpenTracing with Jaeger. The Call Tracing applies to both Telemedicine Solutions (3rd party) and eHealth Services (Infrastructure Provider)
The figure below shows the trace flow from initial traffic to an Istio-enabled pod, through the Jaeger system
Header Propagation
For all containers deployed on the infrastructure, an Istio-sidecar proxy is deployed in the same pod. This is configured using the eHealth Helm Charts (see Helm Charts).
The Istio proxy automatically sends spans (the time spent in a single service) to Jaeger.
However, to correlate the spans to an entire trace, we need to propagate the appropriate HTTP headers.
The headers that must be propagated from the incoming request to all outgoing requests are the following:
x-request-idx-b3-traceidx-b3-spanidx-b3-parentspanidx-b3-sampledx-b3-flagsx-ot-span-contextb3 (for forward compatibility)
All headers may not be present for all requests, but if they are, then they must be propagated.
Also, for audit/minlog purposes, all services must forward the “x-audit-restriction-levels“ header. See more on Logging model | MinLog2 logging
Calling external dependencies
If an application deployed on the eHealth infrastructure makes a call to an external dependency, this call should also have the audit and tracing headers on.
This allows the infrastructure to:
Pinpoint potentially slow external dependencies
Calculate how much time is spent waiting for external dependencies for each end-user request
Create audit trails for all requests
More Info
Java client library and great explanation of the different B3 headers: https://github.com/openzipkin/b3-propagation
Istio documentation of distributed tracing using header propagation: https://istio.io/docs/tasks/telemetry/distributed-tracing/overview/
Alternative Java library: https://github.com/jaegertracing/jaeger-client-java
Various alternatives related to server and client instrumentation: https://github.com/opentracing-contrib
E.g. use https://github.com/opentracing-contrib/java-spring-web if using Spring Boot + servlets + Spring RestTemplate clients (other clients also available, e.g. Apache HttpClient: https://github.com/opentracing-contrib/java-apache-httpclient)
Example
Tracing requests to external services
The platform is not able to create automatic traces on encrypted requests to external services.
This is because traces are based on the "x-b3-* headers, and these cannot be read by the proxy when the request is encrypted. Encrypted requests include all requests that use a TLS, SSL or HTTPS protocol.
Because of this, all applications making encrypted requests to external services, where they want to record call time or have trace data, need to create the Jaeger traces manually for the external request.
Jaeger traces should be sent to the Jaeger collector at: http://jaeger-operator-jaeger-collector.jaeger.svc.cluster.local:14268/api/traces