Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[stable/cluster-overprovisioner] add chart #13

Merged
merged 19 commits into from
Aug 20, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions ci/helm-conftest-policies/ignored_charts.rego
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package main

ignored_charts_deprecations := {
"cluster-overprovisioner"
}
7 changes: 7 additions & 0 deletions ci/helm-conftest-policies/v1.16-deprecations.rego
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
package main

deny[msg] {
not ignored_charts_deprecations
input.apiVersion == "v1"
input.kind == "List"
obj := input.items[_]
msg := _deny with input as obj
}

deny[msg] {
not ignored_charts_deprecations
input.apiVersion != "v1"
input.kind != "List"
msg := _deny
Expand All @@ -17,13 +19,15 @@ deny[msg] {

# All resources under apps/v1beta1 and apps/v1beta2 - use apps/v1 instead
_deny = msg {
not ignored_charts_deprecations
apis := ["apps/v1beta1", "apps/v1beta2"]
input.apiVersion == apis[_]
msg := sprintf("%s/%s: API %s has been deprecated, use apps/v1 instead.", [input.kind, input.metadata.name, input.apiVersion])
}

# daemonsets, deployments, replicasets resources under extensions/v1beta1 - use apps/v1 instead
_deny = msg {
not ignored_charts_deprecations
resources := ["DaemonSet", "Deployment", "ReplicaSet"]
input.apiVersion == "extensions/v1beta1"
input.kind == resources[_]
Expand All @@ -32,20 +36,23 @@ _deny = msg {

# networkpolicies resources under extensions/v1beta1 - use networking.k8s.io/v1 instead
_deny = msg {
not ignored_charts_deprecations
input.apiVersion == "extensions/v1beta1"
input.kind == "NetworkPolicy"
msg := sprintf("%s/%s: API extensions/v1beta1 for NetworkPolicy has been deprecated, use networking.k8s.io/v1 instead.", [input.kind, input.metadata.name])
}

# podsecuritypolicies resources under extensions/v1beta1 - use policy/v1beta1 instead
_deny = msg {
not ignored_charts_deprecations
input.apiVersion == "extensions/v1beta1"
input.kind == "PodSecurityPolicy"
msg := sprintf("%s/%s: API extensions/v1beta1 for PodSecurityPolicy has been deprecated, use policy/v1beta1 instead.", [input.kind, input.metadata.name])
}

# PriorityClass resources will no longer be served from scheduling.k8s.io/v1beta1 and scheduling.k8s.io/v1alpha1 in v1.17.
_deny = msg {
not ignored_charts_deprecations
apis := ["scheduling.k8s.io/v1beta1", "scheduling.k8s.io/v1alpha1"]
input.apiVersion == apis[_]
input.kind == "PriorityClass"
Expand Down
3 changes: 3 additions & 0 deletions ci/helm-conftest-policies/v1.17-deprecations.rego
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
package main

deny[msg] {
not ignored_charts_deprecations
input.apiVersion == "v1"
input.kind == "List"
obj := input.items[_]
msg := _deny with input as obj
}

deny[msg] {
not ignored_charts_deprecations
input.apiVersion != "v1"
input.kind != "List"
msg := _deny
Expand All @@ -17,6 +19,7 @@ deny[msg] {

# PriorityClass resources will no longer be served from scheduling.k8s.io/v1beta1 and scheduling.k8s.io/v1alpha1 in v1.17.
_deny = msg {
not ignored_charts_deprecations
apis := ["scheduling.k8s.io/v1beta1", "scheduling.k8s.io/v1alpha1"]
input.apiVersion == apis[_]
input.kind == "PriorityClass"
Expand Down
21 changes: 21 additions & 0 deletions stable/cluster-overprovisioner/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*~
# Various IDEs
.project
.idea/
*.tmproj
27 changes: 27 additions & 0 deletions stable/cluster-overprovisioner/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
apiVersion: v1
appVersion: "1.0"
description: |
This chart provide a buffer for cluster autoscaling to allow overprovisioning of cluster nodes. This is desired when you have work loads that need to scale up quickly without waiting for the new cluster nodes to be created and join the cluster.

It works but creating a deployment that creates pods of a lower than default `PriorityClass`. These pods request resources from the cluster but don't actually consume any resources. These pods are then evicted allowing other normal pods are created while also triggering a scale-up by the [cluster-autoscaler](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler).

This approach is the [current recommended method to achieve overprovisioning](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler).
name: cluster-overprovisioner
home: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is not correct

Copy link
Member

@max-rocket-internet max-rocket-internet Aug 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But maybe it's the most appropriate as there is "home" for this tool?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is wrong?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK maybe we just leave this as it is.

version: 0.4.0
maintainers:
- name: max-rocket-internet
email: [email protected]
- name: mmingorance-dh
email: [email protected]
engine: gotpl
icon: https://github.com/kubernetes/kubernetes/raw/master/logo/logo.png
keywords:
- cluster
- autoscaling
- overprovision
- cluster-autoscaler
sources:
- https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler
- https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler
- https://github.com/kubernetes/kubernetes/tree/master/build/pause
6 changes: 6 additions & 0 deletions stable/cluster-overprovisioner/OWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
approvers:
- max-rocket-internet
- mmingorance-dh
reviewers:
- max-rocket-internet
- mmingorance-dh
84 changes: 84 additions & 0 deletions stable/cluster-overprovisioner/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# cluster-overprovisioner

![Version: 0.4.0](https://img.shields.io/badge/Version-0.4.0-informational?style=flat-square) ![AppVersion: 1.0](https://img.shields.io/badge/AppVersion-1.0-informational?style=flat-square)

This chart provide a buffer for cluster autoscaling to allow overprovisioning of cluster nodes. This is desired when you have work loads that need to scale up quickly without waiting for the new cluster nodes to be created and join the cluster.

It works but creating a deployment that creates pods of a lower than default `PriorityClass`. These pods request resources from the cluster but don't actually consume any resources. These pods are then evicted allowing other normal pods are created while also triggering a scale-up by the [cluster-autoscaler](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler).

This approach is the [current recommended method to achieve overprovisioning](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler).

**Homepage:** <https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler>

## How to install this chart

Add Delivery Hero public chart repo:

```console
helm repo add deliveryhero https://charts.deliveryhero.io/public
```

A simple install with default values:

```console
helm install deliveryhero/cluster-overprovisioner
```

To install the chart with the release name `my-release`:

```console
helm install my-release deliveryhero/cluster-overprovisioner
```

To install with some set values:

```console
helm install my-release deliveryhero/cluster-overprovisioner --set values_key1=value1 --set values_key2=value2
```

To install with custom values file:

```console
helm install my-release deliveryhero/cluster-overprovisioner -f values.yaml
```

## Source Code

* <https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler>
* <https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler>
* <https://github.com/kubernetes/kubernetes/tree/master/build/pause>

## Values

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| deployments | list | {} | Define optional additional deployments - A default deployment is included by default |
| deployments[0].affinity | object | `{}` | Default Deployment - Map of node/pod affinities |
| deployments[0].annotations | object | `{}` | Default Deployment - Annotations to add to the deployment |
| deployments[0].labels | object | `{}` | Default Deployment - Optional labels tolerations |
| deployments[0].name | string | `"default"` | Default Deployment - Name for additional deployments (will be added as label cluster-over-provisioner-name, so you can match it with affinity rules) |
| deployments[0].nodeSelector | object | `{}` | Default Deployment - Node labels for pod assignment |
| deployments[0].replicaCount | int | `3` | Default Deployment - Number of replicas |
| deployments[0].resources.limits.cpu | string | `"1000m"` | Default Deployment - CPU limit for the overprovision pods |
| deployments[0].resources.limits.memory | string | `"1000Mi"` | Default Deployment - Memory limit for the overprovision pods |
| deployments[0].resources.requests.cpu | string | `"1000m"` | Default Deployment - CPU requested for the overprovision pods |
| deployments[0].resources.requests.memory | string | `"1000Mi"` | Default Deployment - Memory requested for the overprovision pods |
| deployments[0].tolerations | list | `[]` | Default Deployment - Optional deployment tolerations |
| fullnameOverride | string | `""` | Override the fullname of the chart |
| image.pullPolicy | string | `"IfNotPresent"` | Container pull policy |
| image.repository | string | `"k8s.gcr.io/pause"` | Image repository |
| image.tag | float | `3.1` | Image tag |
| nameOverride | string | `""` | Override the name of the chart |
| podSecurityContext | object | `{}` | Pod security context object |
| priorityClassDefault.enabled | bool | `true` | If true, enable default priorityClass |
| priorityClassDefault.name | string | `"default"` | Name of the default priorityClass |
| priorityClassDefault.value | int | `0` | Priority value of the default priorityClass |
| priorityClassOverprovision.name | string | `"overprovisioning"` | Name of the overprovision priorityClass |
| priorityClassOverprovision.value | int | `-1` | Priority value of the overprovision priorityClass |

## Maintainers

| Name | Email | Url |
| ---- | ------ | --- |
| max-rocket-internet | [email protected] | |
| mmingorance-dh | [email protected] | |
18 changes: 18 additions & 0 deletions stable/cluster-overprovisioner/ci/ct-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
deployments:
- name: ci-one
annotations: {}
replicaCount: 1
nodeSelector: {}
resources: {}
tolerations: []
affinity: {}
labels: {}

- name: ci-two
annotations: {}
replicaCount: 0
nodeSelector: {}
resources: {}
tolerations: []
affinity: {}
labels: {}
3 changes: 3 additions & 0 deletions stable/cluster-overprovisioner/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
To verify that the cluster-overprovisioner pods have started, run:

kubectl --namespace={{ .Release.Namespace }} get pods -l "app.kubernetes.io/name={{ template "cluster-overprovisioner.name" . }},app.kubernetes.io/instance={{ .Release.Name }}"
66 changes: 66 additions & 0 deletions stable/cluster-overprovisioner/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "cluster-overprovisioner.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "cluster-overprovisioner.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "cluster-overprovisioner.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Return the appropriate apiVersion for PriorityClass.
*/}}
{{- define "PriorityClass.apiVersion" -}}
{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
{{- print "scheduling.k8s.io/v1" -}}
{{- else if semverCompare ">=1.11-0, <1.14-0" .Capabilities.KubeVersion.GitVersion -}}
{{- print "scheduling.k8s.io/v1beta1" -}}
{{- else if semverCompare "<1.11-0" .Capabilities.KubeVersion.GitVersion -}}
{{- print "scheduling.k8s.io/v1alpha1" -}}
{{- end -}}
{{- end -}}

{{/*
Common labels
*/}}
{{- define "cluster-overprovisioner.labels" -}}
app.kubernetes.io/name: {{ include "cluster-overprovisioner.name" . }}
helm.sh/chart: {{ include "cluster-overprovisioner.chart" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}

{{/*
Common labels
*/}}
{{- define "cluster-overprovisioner.matchLabels" -}}
app.kubernetes.io/name: {{ include "cluster-overprovisioner.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}
73 changes: 73 additions & 0 deletions stable/cluster-overprovisioner/templates/deployments.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
{{- $relName := .Release.Name -}}
{{- $relService := .Release.Service -}}
{{- $fullname := include "cluster-overprovisioner.fullname" . -}}
{{- $name := include "cluster-overprovisioner.name" . -}}
{{- $chart := include "cluster-overprovisioner.chart" . -}}
{{- $commonLabels := include "cluster-overprovisioner.labels" . -}}
{{- $matchLabels := include "cluster-overprovisioner.matchLabels" . -}}
{{- $chartName := .Chart.Name }}
{{- $podSecurityContext := .Values.podSecurityContext }}
{{- $priorityClassName := .Values.priorityClassOverprovision.name }}
{{- $repository := .Values.image.repository }}
{{- $imageTag := .Values.image.tag }}
{{- $pullPolicy := .Values.image.pullPolicy }}
{{- $imagePullSecrets := .Values.image.pullSecrets }}

{{ range .Values.deployments }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: "{{ $fullname }}-{{ .name }}"
labels:
{{- $commonLabels | nindent 4 }}
cluster-overprovisioner-deployment: {{ .name }}
spec:
strategy:
type: Recreate
replicas: {{ .replicaCount }}
selector:
matchLabels:
{{- $matchLabels | nindent 6 }}
cluster-overprovisioner-deployment: {{ .name }}
template:
metadata:
{{- with .annotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- $commonLabels | nindent 8 }}
cluster-overprovisioner-deployment: {{ .name }}
{{- with .labels }}
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
priorityClassName: {{ $priorityClassName }}
securityContext:
{{- toYaml $podSecurityContext | nindent 8 }}
containers:
- name: {{ $chartName }}
image: "{{ $repository }}:{{ $imageTag }}"
imagePullPolicy: {{ $pullPolicy }}
resources:
{{- toYaml .resources | nindent 12 }}
{{- if $imagePullSecrets }}
imagePullSecrets:
{{- range $imagePullSecrets }}
- name: {{ . }}
{{- end }}
{{- end }}
{{- with .nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
---
{{- end }}
Loading