Configuring opentelemetry, Jaeger, Prometheus, Grafana – 102 processing data

This is one of my blog posts on using the above packages with Docker.

See Configuring opentelemetry, Jaeger, Prometheus, Grafana

• 101 the basics This gives the minimum configuration, and defines some of the terms
• 102 doing processing, this post
• 103 http endpoints
• problem determination

The above packages take tracking data or metrics and can display them in dashboards. I found it a struggle to understand how they were configured, as the documentation assumes you are an expert, and I could not find any “starting from zero” documentation.

This follows on from the 101 the basics.

Doing processing – counting records

With the configuration file

receivers:
  otlp_json_file:
     include:
      - /fooin.file
     start_at: beginning 
exporters:
  file/a:
      path: /fooout.file   
  file/b:
      path: /b.file       

connectors:
  count:

service:
  pipelines:
    traces:
      receivers: [otlp_json_file]
      exporters: [count]
    metrics:
      receivers: [count]
      exporters: [file/b]  

This reads the input file, and copies the data to “count”. Count is a pipeline stage which reads the input, counts the records and outputs a summary of the data to the receiver count, which maps to the file b.file.

This runs in Docker, so there is b.file maps to a real file.

 -v "$(pwd)/foob.json":"/b.file" \

The file needs to exist ( use touch foob.json) and the userid docker needs write to it. I used chmod 777 foob.josn

Another example using span_metrics

This counts the number of span records from opentelemetry collector. It is defined here.

The documentation says

Exporter Pipeline Type	Receiver Pipeline Type
traces	metrics

This says it receives data with a type of metrics, and outputs data with a type of trace.

• traces: … exporters: [span_metrics]
• metrics: receivers: [span_metrics]

receivers:
  otlp_json_file:
     include:
      - /fooin.file
     start_at: beginning 
  
exporters:
  file/a:
      path: /fooout.file   

  file/b:
      path: /b.file       

  debug:
    verbosity: detailed # Options: basic, normal, detailed
    sampling_initial: 2
    sampling_thereafter: 10    


connectors:
  count:
  span_metrics:

processors: 
  batch:

service:
  pipelines:

    traces:
      receivers: [otlp_json_file]
      exporters: [span_metrics]
    metrics:
      receivers: [span_metrics]
      exporters: [file/b,debug]