Page MenuHomePhabricator
Create Task
Maniphest T374128

haproxykafka features
Open, In Progress, Needs TriagePublic
Actions

Description

Collect here all minor features and work needed by HAProxyKafka

Required for production deployment:

  • Secure connect to kafka cluster (TLS)
  • Debian packaging/CI/Puppetization (T377614, T377613)
  • Tuning Kakfa message batching
  • Trim all unnecessary fields and transformations from produced json (@gmodena can help with this)
  • Better grafana dashboard
  • Alerts

Other features (some still need to be evaluated):

  • Logging level from CLI/Configuration
  • Retry logic on some errors
  • UDP socket support (in addition to UDS)
  • Systemd socket activation
  • Configuration auto-reloading on file change
  • Tests for received kafka.Error errors
  • metrics tests

Fixes

  • Messages sent to DLQ have empty Error field

Details

SubjectRepoBranchLines /-
haproxykafka: profile and hiera filesoperations/puppetproduction 120 -5
haproxykafka: haproxykafka moduleoperations/puppetproduction 398 -1
haproxykafka: start working on haproxykafka moduleoperations/puppetproduction 240 -2
haproxykafka: adding profile and hiera filesoperations/puppetproduction 593 -0
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 In ProgressFabfurT351117 Move analytics log from Varnish to HAProxy
 In ProgressFabfurT370668 New software: haproxykafka
 In ProgressFabfurT374128 haproxykafka features

Event Timeline

Fabfur created this task.Sep 5 2024, 3:58 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 5 2024, 3:58 PM
Fabfur updated the task description. (Show Details)Sep 10 2024, 7:47 PM
Fabfur updated the task description. (Show Details)Sep 13 2024, 10:29 AM
CodeReviewBot added a project: Patch-For-Review.Sep 13 2024, 10:30 AM

fabfur opened https://gitlab.wikimedia.org/repos/sre/haproxykafka/-/merge_requests/37

Some defaults for Kafka message buffering

CodeReviewBot added a comment.Sep 16 2024, 2:58 PM

fabfur merged https://gitlab.wikimedia.org/repos/sre/haproxykafka/-/merge_requests/37

Some defaults for Kafka message buffering

Maintenance_bot removed a project: Patch-For-Review.Sep 16 2024, 3:30 PM
Fabfur updated the task description. (Show Details)Sep 20 2024, 3:07 PM
Fabfur added a subscriber: gmodena.
Fabfur updated the task description. (Show Details)Oct 11 2024, 11:29 AM
gerritbot added a comment.Thu, Oct 17, 9:07 PM

Change #1081264 had a related patch set uploaded (by Fabfur; author: Fabfur):

[operations/puppet@production] haproxykafka: start working on haproxykafka puppetization

https://gerrit.wikimedia.org/r/1081264

gerritbot added a project: Patch-For-Review.Thu, Oct 17, 9:07 PM
Fabfur updated the task description. (Show Details)Fri, Oct 18, 6:35 PM
gerritbot added a comment.Fri, Oct 25, 3:29 PM

Change #1083191 had a related patch set uploaded (by Fabfur; author: Fabfur):

[operations/puppet@production] haproxykafka: adding profile and hiera files

https://gerrit.wikimedia.org/r/1083191

gerritbot added a comment.Fri, Oct 25, 4:07 PM

Change #1083203 had a related patch set uploaded (by Fabfur; author: Fabfur):

[operations/puppet@production] haproxykafka: haproxykafka module

https://gerrit.wikimedia.org/r/1083203

gerritbot added a comment.Fri, Oct 25, 4:07 PM

Change #1083204 had a related patch set uploaded (by Fabfur; author: Fabfur):

[operations/puppet@production] haproxykafka: profile and hiera files

https://gerrit.wikimedia.org/r/1083204

gerritbot added a comment.Fri, Oct 25, 4:07 PM

Change #1083191 abandoned by Fabfur:

[operations/puppet@production] haproxykafka: adding profile and hiera files

Reason:

Messed up with commit split

https://gerrit.wikimedia.org/r/1083191

gerritbot added a comment.Fri, Oct 25, 4:08 PM

Change #1081264 abandoned by Fabfur:

[operations/puppet@production] haproxykafka: start working on haproxykafka module

Reason:

Messed up with commit split

https://gerrit.wikimedia.org/r/1081264

Fabfur changed the task status from Open to In Progress.Wed, Oct 30, 9:45 AM
gerritbot added a comment.Wed, Oct 30, 2:02 PM

Change #1083203 merged by Fabfur:

[operations/puppet@production] haproxykafka: haproxykafka module

https://gerrit.wikimedia.org/r/1083203

gerritbot added a comment.Wed, Oct 30, 2:03 PM

Change #1083204 merged by Fabfur:

[operations/puppet@production] haproxykafka: profile and hiera files

https://gerrit.wikimedia.org/r/1083204

Maintenance_bot removed a project: Patch-For-Review.Wed, Oct 30, 2:31 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits