-
Notifications
You must be signed in to change notification settings - Fork 487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pyroscope.scrape: change error log level to not swallow errors #5840
Conversation
@@ -228,7 +228,7 @@ func (t *scrapeLoop) scrape() { | |||
} | |||
} | |||
if err := t.fetchProfile(scrapeCtx, profileType, buf); err != nil { | |||
level.Debug(t.logger).Log("msg", "fetch profile failed", "target", t.Labels().String(), "err", err) | |||
level.Error(t.logger).Log("msg", "fetch profile failed", "target", t.Labels().String(), "err", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It makes sense to report this, but if the log line is expected to be printed a lot maybe this should be a metric? That way we can have an alert as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lets try like this in dev and then revert and make it metric when we have them too much.
Also do you know if scraping metric produces error? It feels they should be aligned and have similar experience.
* Allow x-faro-session-id header for faro receiver (#5835) (cherry picked from commit cc7cb37) * misc: follow up on #5835 (#5837) * Add missing CHANGELOG entry * Mirror fix to static mode (cherry picked from commit 7da5726) * Agent Management: Introduce support for template variables (#5788) * Agent Management: Introduce support for template variables - This change allows managing template variables for remote configuration on a per-agent basis. - Both base configurations and snippets can be interpreted as templates and evaluated at load time with the provided template variables. - Templates must follow go's `text/template` syntax. - This greatly increases the flexibility and reusability of snippets. - Template evaluation has been tested in different scenarios and seems really robust. If the variables defined in the template cannot be resolved (even nested ones), and empty string is rendered instead. - Note: templates are only evaluated when the `template_variables` field within the `agent_metadata` remote config field is non-empty. - Note: this feature only applies to static mode. * Improve naming * Check error for template execution * Add tests - Tests different scenarios, including: - Referencing non existing nested objects - Conditionals - Ranges - Character escaping * Update CHANGELOG * Always evaluate templates - This is required because certain agents might start before their labels are synced. If some of the snippets assigned to them contain template variables, loading the config will fail. * Add test for template inside a template - Templates inside templates must be escaped using backticks to avoid them being evaluated by the snippet template execution * Move feature to the next release in CHANGELOG * Document templating functionality * Fix doc (cherry picked from commit d388f94) * pyroscope.scrape: change error log level to not swallow errors (#5840) (cherry picked from commit 2242e4a) * Update windows defaults to use upstream defaults except for enabled collectors. (#5832) Fix #5831 and use the defaults from windows. (cherry picked from commit 15d3d9f) * Bump otelgrpc to fix CVE 2023 47108 (#5806) * Fix CVE-2023-47108 by updating `otelgrpc` from v0.45.0 to v0.46.0. * Stop using the deprecated trace.NewNoopTracerProvider. Using noop.NewTracerProvider from "go.opentelemetry.io/otel/trace/noop" instead. * Reorder changelog Comply with the ordering in: https://github.com/grafana/agent/blob/main/docs/developer/contributing.md#updating-the-changelog --------- Signed-off-by: hainenber <[email protected]> Co-authored-by: Paulin Todev <[email protected]> (cherry picked from commit a2348a0) * fix(otelcol/fanoutconsumer): nil check during fanout consumer creation (#5854) Signed-off-by: hainenber <[email protected]> (cherry picked from commit 84344fb) * prometheus.operator.* - Fix issue with missing targets when one monitor's name is a prefix of another (#5862) Co-authored-by: Paul Bormans <[email protected]> (cherry picked from commit f232fb4) * max_cache_size was being set to 0 (#5869) * max_cache_size was being set to 0, due to issue where it doesnt exist in static but the default wasnt carrying over to river syntax. In truth we should never write it. * Clean up from PR (cherry picked from commit 356c50c) * Add Deploy Mode to usage stats. (#5880) Co-authored-by: Clayton Cornell <[email protected]> (cherry picked from commit 7bf82ea) * Fix promtail converter: docker_sd_configs (#5881) * Fix promtail converter: docker_sd * changelog and lint * typo (cherry picked from commit c5dc968) * prepare for 0.38.1 release (#5891) (cherry picked from commit fc2fd5b) * fix misplaced CHANGELOG entry for #5869 --------- Co-authored-by: Cedric Ziel <[email protected]> Co-authored-by: Jorge Creixell <[email protected]> Co-authored-by: Tolya Korniltsev <[email protected]> Co-authored-by: mattdurham <[email protected]> Co-authored-by: Đỗ Trọng Hải <[email protected]> Co-authored-by: Craig Peterson <[email protected]> Co-authored-by: Piotr <[email protected]>
PR Description
Which issue(s) this PR fixes
Notes to the Reviewer
PR Checklist