Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump Kafka to 3.7.1, remove several Kafka CR warnings, and fix exporter dashboard #170

Merged
merged 2 commits into from
Jul 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion debezium/connector/040-Connect.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ metadata:
annotations:
strimzi.io/use-connector-resources: "true"
spec:
version: 3.7.0
version: 3.7.1
replicas: 1
bootstrapServers: anubis-kafka-bootstrap.strimzi-kafka.svc:9093
metricsConfig:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -813,7 +813,7 @@ spec:
"targets": [
{
"datasource": "${DS_THANOS}",
"expr": "sum(delta(kafka_consumergroup_current_offset{consumergroup=~\"$consumergroup\",topic=~\"$topic\", namespace=~\"$kubernetes_namespace\"}[5m])/60) by (consumergroup, topic)",
"expr": "sum(delta(kafka_consumergroup_current_offset{consumergroup=~\"$consumergroup\",topic=~\"$topic\", namespace=~\"$kubernetes_namespace\"}[5m])/300) by (consumergroup, topic)",
"format": "time_series",
"intervalFactor": 1,
"legendFormat": "{{consumergroup}} (topic: {{topic}})",
Expand Down
49 changes: 1 addition & 48 deletions strimzi/kraft-mode/kafka/00-kafka-route.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,8 @@ metadata:
strimzi.io/kraft: enabled
spec:
kafka:
version: 3.7.0
version: 3.7.1
# ignored by nodepools, needed just for schema validation
replicas: 5
listeners:
- name: plain
port: 9092
Expand All @@ -94,13 +93,7 @@ spec:
log.retention.hours: -1 # i.e., not time limit is applied
log.retention.bytes: 128000000 # 128 MB
log.segment.bytes: 64000000 # 64 MB
log.message.format.version: "3.6"
inter.broker.protocol.version: "3.6"
min.insync.replicas: 2
# ignored by nodepools, needed just for schema validation
storage:
type: persistent-claim
size: 100Gi
authorization:
type: simple
superUsers:
Expand Down Expand Up @@ -155,46 +148,6 @@ spec:
matchLabels:
# label on Kafka pods
strimzi.io/name: horus-kafka
zookeeper:
replicas: 3
storage:
type: persistent-claim
size: 10Gi
class: standard-csi
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
key: horus-zookeeper-jmx-exporter-configuration.yaml
name: horus-zookeeper-jmx-exporter-configuration
template:
podDisruptionBudget:
maxUnavailable: 0
pod:
tolerations:
- key: "nodetype"
operator: "Equal"
value: "kafka"
effect: "NoSchedule"
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: nodetype
operator: In
values:
- kafka
topologySpreadConstraints:
# Deploy Zookeeper pods with spread with difference max 2 pod
- maxSkew: 4
# All nodes with same key are considered as one group, so we should pick a key, which has different values on the nodes
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
# label on Zookeper pods
strimzi.io/kind: Kafka

entityOperator:
template:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
labels:
mode: kraft
spec:
version: 3.7.0
version: 3.7.1
replicas: 3
resources:
requests:
Expand Down
12 changes: 3 additions & 9 deletions strimzi/zookeeper-mode/kafka/00-kafka-route.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,7 @@ metadata:
strimzi.io/node-pools: enabled
spec:
kafka:
version: 3.7.0
# ignored by nodepools, needed just for schema validation
replicas: 7
version: 3.7.1
listeners:
- name: plain
port: 9092
Expand All @@ -69,13 +67,9 @@ spec:
log.retention.hours: -1 # i.e., not time limit is applied
log.retention.bytes: 128000000 # 128 MB
log.segment.bytes: 64000000
log.message.format.version: "3.6"
inter.broker.protocol.version: "3.6"
log.message.format.version: "3.7"
inter.broker.protocol.version: "3.7"
min.insync.replicas: 2
# ignored by nodepools, needed just for schema validation
storage:
type: persistent-claim
size: 300Gi
authorization:
type: simple
superUsers:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
labels:
mode: zookeeper
spec:
version: 3.7.0
version: 3.7.1
replicas: 3
resources:
requests:
Expand Down
Loading