Page Comparison

The figure below show the trace flow from initial traffic to a Istio enabled pod, through the jaeger system all the way to the jaeger web interface and the FUT background service "SLA-metrics" (a part of the SLA calculating system).   

The “SLA-metrics” services extracts response times from Jaeger/Elasticsearch and imports those to Splunk.

Header Propagation

For all containers deployed on the infrastructure, an istioIstio-sidecar proxy is deployed in the same pod. The istio This is configured using the eHealth Helm Charts (see Helm Charts).

The Istio proxy are able to automatically send spans (the time spend in a single service) to Jaeger.

...

The headers that must be propagated from the incoming request to all outgoing requests are the following:

• x-request-id

• x-b3-traceid

• x-b3-spanid

• x-b3-parentspanid

• x-b3-sampled

• x-b3-flags

• x-ot-span-context

• b3 (for forward compatibility)

All headers may not be present for all requests, but if they are, then they must be propagated.

...

If an application deployed on the ehealtheHealth-platform infrastructure makes a call to an external dependency, this call should also have the tracing headers on.

This allows the infrastructure to:

...

• Pinpoint potentially slow external dependencies

...

• Calculate how much time is spend waiting for external dependencies for each end-user request

...

• Create audit

...

• trails for all requests

More Info

Java client library and great explanation of the different B3 headers: https://github.com/openzipkin/b3-propagation

Istio documentation of distributed tracing using headers propagation:   https://istio.io/docs/tasks/telemetry/distributed-tracing/overview/.

Alternative Java library: https://github.com/jaegertracing/jaeger-client-java

Various alternatives related to server and client instrumentation: https://github.com/opentracing-contrib
E.g. use https://github.com/opentracing-contrib/java-spring-web if using Spring Boot + servlets + Spring RestTemplate clients (other clients also available, e.g. Apache HttpClient: https://github.com/opentracing-contrib/java-apache-httpclient)

Example

...

Gliffy
imageAttachmentId att148144293 baseUrl https://ehealth-dk.atlassian.net/wiki macroId 8ddd1f39-ae73-48c7-b07e-b3dd81c1e074 name Istio tracing diagramAttachmentId att148078610 containerId 128843780 timestamp 1554720804082

...

Tracing on requests to external services

The platform is not able to create automatic traces on on encrypted requests to external services.

This is because of the fact that traces is based on the "x-b3-* headers, and these cannot be read by the proxy when the request is encrypted. Encrypted requests include all requests that uses a TLS, SSL or HTTPS protocol.

Because of this all applications making encrypted requests to external services, where they want to record call time or have trace data, needs to create the Jaeger traces manually for the external request. 

Jaeger traces should be sent to the Jaeger collector at: http://jaeger-operator-jaeger-collector.jaeger.svc.cluster.local:14268/api/traces