Prometheus alert not peding/firing at second time -
i have simple prometheus alert configured
alert myservicedown if my_custom_metric == 0 15s
scenario follows:
- in correct runtime my_custom_metric 1, , myservicedown alert green @ alert tab.
- then set my_custom_metric 0 , propetheus retrieves 0, can see @ graph -> console tab.
- myservicedown becomes pending (yellow) , after 15 seconds becomes firing (red).
- then "fix bug", my_custom_metric 1 again, , myservicedown alert green.
here comes strange part. break app second time my_custom_metric 0 again. shown @ graph -> console. myservicedown alert never pending or firing. when restart prometheus works good, once.
what doing wrong?
there bug in version 0.17rc1
https://github.com/prometheus/prometheus/issues/1372
it fixed in 0.17rc2
problem solved.
Comments
Post a Comment