I had a query
sum (rate(traces_span_metrics_duration_milliseconds_bucket[1m]) ) by(le) / sum(rate(traces_span_metrics_duration_milliseconds_count[1m]) )
and it returned no data. Why?
I discuss how I debugged this problem in Debugging Prometheus queries
The Prometheus documentation has a sections on queries and data.
Prometheus calculations
Data types
There are different sorts of data in Prometheus
- A scalar. This is a single value. It can be a number or a string.
- An array or vector. This could be an empty array [], an array with one element [44], or an array with multiple elements [1,2,3]
- A time series. This has a time stamp and some data.
Basic calculations
I’ll use division as an example:
- Scalar/scalar gives a scalar
- Array/scalar gives an array, with result[i] = array[i]/scalar
- Array/Array when the arrays are the same size. Result[i] = LeftArray[i]/RightArray[i]
- Otherwise no result
Functions
- The sum function takes a vector and returns a vector; for example, sum[1,2,3] = [6]
- The scalar function acts on a vector and creates a scalar; for example, scalar[6] = 6
- The vector function acts on a scalar and returns a vector of size 1; for example, vector(6) gives [6].
Why did the query not return any data
The query
sum (rate(traces_span_metrics_duration_milliseconds_bucket[1m]) ) by(le) / sum(rate(traces_span_metrics_duration_milliseconds_count[1m]) )
- The sum() by() produces a vector, with each element of the vector corresponding to each unique value of le (8 elements in my case)
- The sum() produces a vector with only one element.
From the basic calculations above an array of size 8/array of size 1 does not produce a result, because the arrays are of different sizes.
Putting scalar() around the second query means the calculation is array/scalar, which does work
sum (rate(traces_span_metrics_duration_milliseconds_bucket[1m]) ) by(le) / scalar(sum(rate(traces_span_metrics_duration_milliseconds_count[1m]) ))
