Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve alerts #18080

Merged
merged 16 commits into from
Jul 17, 2024
Merged

Improve alerts #18080

merged 16 commits into from
Jul 17, 2024

Conversation

stelfrag
Copy link
Collaborator

@stelfrag stelfrag commented Jul 8, 2024 •
edited
Loading

Summary

Improve the agent to cloud alert communication. The goal is to reduce messages transmitted during agent start or reconnect.

Agents calculate and send a version with each alert. When agent -- cloud link is established the start alerts streaming command carries the version of the alerts the cloud is aware.

A version match on the agent side ensures that we don't transmit transitions that the cloud already knows

New tables

  • alert_queue
    Holds alerts transitions to be processed. Also used to de-duplicate alert transitions
  • aclk_queue
    Holds pending alerts transitions to be send to the cloud
  • alert_version
    Holds last alert version after being submitted to the cloud
  • aclk_alert_xxxx tables have been removed

netdatacli aclk-state adjusted to reflect changes
Removed fields Pending Min Seq ID, Pending Max Seq ID and Last Submitted Seq ID
Added

  • Checkpoints : number of checkpoint requests received from the cloud
  • Alert count : number of alert transitions sent to the cloud
  • Alert snapshot count : number of snapshots sent to the cloud due to checkpoint failures
Test Plan
  • Start agent with new PR. Check alerts

@stelfrag stelfrag force-pushed the improve_alerts branch 3 times, most recently from f7a3144 to b5cc0c4 Compare July 12, 2024 14:22
@stelfrag stelfrag marked this pull request as ready for review July 16, 2024 08:25
@ilyam8 ilyam8 mentioned this pull request Jul 16, 2024
Closed
@stelfrag
Remove extra bind (use UNIXEPOCH())
c5e1728 
Cleanup table aliases
Add DEFINEs for alert delays
Add missing newline
thiagoftsm
thiagoftsm previously approved these changes Jul 16, 2024
View reviewed changes
Copy link
Contributor

@thiagoftsm thiagoftsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After changes, the PR worked as expected on my environment. I had alerts from both parent and child.

@stelfrag stelfrag requested a review from thiagoftsm July 17, 2024 10:15
thiagoftsm
thiagoftsm approved these changes Jul 17, 2024
View reviewed changes
Copy link
Contributor

@thiagoftsm thiagoftsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the branch after last commits, and I am having my alerts on cloud as expected, LGTM!

@stelfrag stelfrag merged commit 3dfe351 into netdata:master Jul 17, 2024
140 checks passed
@stelfrag stelfrag deleted the improve_alerts branch July 19, 2024 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thiagoftsm thiagoftsm thiagoftsm approved these changes

@vkalintiris vkalintiris Awaiting requested review from vkalintiris vkalintiris is a code owner

Assignees
No one assigned
Labels
area/aclk area/daemon area/database area/health area/tests area/web
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@stelfrag @thiagoftsm