Skip to content

Commit

Permalink
Feature/Support of Oracle Cloud Infrastructure(OCI) for KubeIP. (#161)
Browse files Browse the repository at this point in the history
* added support for OCI

* fix lint issues
  • Loading branch information
prateek010 authored Jan 14, 2025
1 parent d7f419c commit a78270d
Show file tree
Hide file tree
Showing 21 changed files with 2,845 additions and 45 deletions.
93 changes: 90 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Welcome to KubeIP v2, a complete overhaul of the popular [DoiT](https://www.doit
KubeIP [v1-main](https://github.com/doitintl/kubeip/tree/v1-main) open-source project, originally developed
by [Aviv Laufer](https://github.com/avivl).

KubeIP v2 expands its support beyond Google Cloud (as in v1) to include AWS, and it's designed to be extendable to other cloud providers
KubeIP v2 expands its support beyond Google Cloud (as in v1) to include AWS and Oracle Cloud Infrastructure(OCI), and it's designed to be extendable to other cloud providers
that allow assigning static public IP to VMs. We've also transitioned from a Kubernetes controller to a standard DaemonSet, enhancing
reliability and ease of use.

Expand Down Expand Up @@ -252,6 +252,93 @@ To use this feature, add the `filter` flag (or set `FILTER` environment variable
value: "labels.env=dev;labels.app=streamer"
```

### Oracle Cloud Infrastructure (OCI)

Make sure that KubeIP DaemonSet is deployed on nodes that have a public IP (node running in public subnet). Set the [compartment OCID](https://docs.oracle.com/en-us/iaas/Content/GSG/Tasks/contactingsupport_topic-Locating_Oracle_Cloud_Infrastructure_IDs.htm#Finding_the_OCID_of_a_Compartment) in the `project` flag (or
set `FILTER` environment variable) to the KubeIP DaemonSet:

```yaml
- name: PROJECT
value: "ocid1.compartment.oc1..test"
```

KubeIP will also need certain permissions to communicate with the OCI APIs. Follow these steps to set up the necessary permissions and generate the required API key and place it in the KubeIP DaemonSet:

1. Create a [user and group](https://docs.oracle.com/en/cloud/paas/integration-cloud/oracle-integration-gov/create-iam-group.html) in the OCI console and add the following [policy](https://docs.oracle.com/en-us/iaas/Content/Identity/Tasks/managingpolicies.htm) to the group:

```
Allow group <group_ocid> to manage public-ips in compartment id <compartment_ocid>
Allow group <group_ocid> to manage private-ips in compartment id <compartment_ocid>
Allow group <group_ocid> to manage vcns in compartment id <compartment_ocid>
```

2. Generate an [API Key](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm#two) for the user and download the private key. Config file will look like this:

```
[DEFAULT]
user=ocid1.user.oc1..test
fingerprint=
key_file=/root/.oci/oci_api_key.pem
tenancy=ocid1.tenancy.oc1..test
region=us-ashburn-1
```
3. Add the following [secret](https://kubernetes.io/docs/concepts/configuration/secret/) to the KubeIP DaemonSet:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: kubeip-oci-secret
namespace: kube-system
type: Opaque
data:
config: <base64_encoded_oci_config>
oci_api_key.pem: <base64_encoded_oci_api_key>
```

4. Create a volume and mount in the KubeIP DaemonSet to mount the secret:

```yaml
volumes:
- name: oci-config
secret:
secretName: kubeip-oci-secret
```
```yaml
volumeMounts:
- name: oci-config
mountPath: /root/.oci
```
5. Add the following environment variables to the KubeIP DaemonSet:
```yaml
- name: OCI_CONFIG_FILE
value: /root/.oci/config
```
KubeIP supports filtering of reserved Public IPs using tags. To use this feature, add the `filter` flag (or
set `FILTER` environment variable) to the KubeIP DaemonSet:

```yaml
- name: FILTER
value: "freeformTags.env=dev"
```

KubeIP OCI filter supports the following filter syntax:

- `freeformTags.<key>=<value>`

To specify multiple filters, separate them with a semicolon (`;`). For example:

```yaml
- name: FILTER
value: "freeformTags.env=dev;freeformTags.app=streamer"
```

In the case of multiple filters, they are joined with an `AND`, and the request returns only results that match all the specified filters.

## How to contribute to KubeIP?

KubeIP is an open-source project, and we welcome your contributions!
Expand Down Expand Up @@ -287,8 +374,8 @@ OPTIONS:
--kubeconfig value path to Kubernetes configuration file (not needed if running in node) [$KUBECONFIG]
--node-name value Kubernetes node name (not needed if running in node) [$NODE_NAME]
--order-by value order by for the IP addresses [$ORDER_BY]
--project value name of the GCP project or the AWS account ID (not needed if running in node) [$PROJECT]
--region value name of the GCP region or the AWS region (not needed if running in node) [$REGION]
--project value name of the GCP project or the AWS account ID (not needed if running in node) or OCI compartment OCID (required for OCI) [$PROJECT]
--region value name of the GCP region or the AWS region or the OCI region (not needed if running in node) [$REGION]
--release-on-exit release the static public IP address on exit (default: true) [$RELEASE_ON_EXIT]
--taint-key value specify a taint key to remove from the node once the static public IP address is assigned [$TAINT_KEY]
--retry-attempts value number of attempts to assign the static public IP address (default: 10) [$RETRY_ATTEMPTS]
Expand Down
15 changes: 15 additions & 0 deletions chart/templates/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ spec:
imagePullPolicy: Always
resources:
{{- toYaml .Values.daemonSet.resources | nindent 12 }}
{{- if eq .Values.cloudProvider "oci" }}
volumeMounts:
- name: oci-config
mountPath: /root/.oci
{{- end }}
env:
- name: NODE_NAME
valueFrom:
Expand All @@ -53,10 +58,20 @@ spec:
value: {{ .Values.daemonSet.env.LOG_LEVEL | quote }}
- name: LOG_JSON
value: {{ .Values.daemonSet.env.LOG_JSON | quote }}
{{- if eq .Values.cloudProvider "oci" }}
- name: OCI_CONFIG_FILE
value: /root/.oci/config
{{- end }}
securityContext:
privileged: false
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
{{- if eq .Values.cloudProvider "oci" }}
volumes:
- name: oci-config
secret:
secretName: oci-config
{{- end }}
11 changes: 11 additions & 0 deletions chart/templates/secrets.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{{- if .Values.secrets.create }}
apiVersion: v1
kind: Secret
metadata:
name: kubeip-oci-secret
namespace: {{ .Values.namespaceOverride }}
type: Opaque
data:
oci_config: {{ .Values.secrets.oci_config }}
oci_oci_api_key.pem: {{ .Values.secrets.oci_oci_api_key }}
{{- end }}
8 changes: 7 additions & 1 deletion chart/values.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# The cloud provider where your Kubernetes cluster is running.
# This value determines the appropriate annotations for the Service Account.
# Currently acceptable values are 'gcp' or 'aws'.
# Currently acceptable values are 'gcp' or 'aws' or 'oci'.
cloudProvider: gcp

# The namespace where the kubeip-agent will be deployed.
Expand All @@ -27,6 +27,12 @@ rbac:
create: true
allowNodesPatchPermission: false

# Secret configuration for oci users.
secrets:
create: true
oci_config: "" # base64 encoded oci config file
oci_oci_api_key: "" # base64 encoded oci api key file

# DaemonSet configuration.
daemonSet:
terminationGracePeriodSeconds: 30
Expand Down
4 changes: 2 additions & 2 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -316,13 +316,13 @@ func main() {
},
&cli.StringFlag{
Name: "project",
Usage: "name of the GCP project or the AWS account ID (not needed if running in node)",
Usage: "name of the GCP project or the AWS account ID (not needed if running in node) or OCI compartment OCID (required for OCI)",
EnvVars: []string{"PROJECT"},
Category: "Configuration",
},
&cli.StringFlag{
Name: "region",
Usage: "name of the GCP region or the AWS region (not needed if running in node)",
Usage: "name of the GCP region or the AWS region or the OCI region (not needed if running in node)",
EnvVars: []string{"REGION"},
Category: "Configuration",
},
Expand Down
5 changes: 4 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ require (
github.com/aws/aws-sdk-go-v2 v1.26.0
github.com/aws/aws-sdk-go-v2/config v1.27.9
github.com/aws/aws-sdk-go-v2/service/ec2 v1.152.0
github.com/oracle/oci-go-sdk/v65 v65.80.0
github.com/pkg/errors v0.9.1
github.com/sirupsen/logrus v1.9.3
github.com/stretchr/testify v1.9.0
Expand All @@ -15,6 +16,7 @@ require (
k8s.io/api v0.29.3
k8s.io/apimachinery v0.29.3
k8s.io/client-go v0.29.3
k8s.io/utils v0.0.0-20240310230437-4693a0247e57
sigs.k8s.io/controller-runtime v0.17.2
)

Expand All @@ -41,6 +43,7 @@ require (
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/gofrs/flock v0.8.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
Expand All @@ -60,6 +63,7 @@ require (
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sony/gobreaker v0.5.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/xrash/smetrics v0.0.0-20240312152122-5f08fbb34913 // indirect
Expand All @@ -85,7 +89,6 @@ require (
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/klog/v2 v2.120.1 // indirect
k8s.io/kube-openapi v0.0.0-20240322212309-b815d8309940 // indirect
k8s.io/utils v0.0.0-20240310230437-4693a0247e57 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
Expand Down
8 changes: 8 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+Gr
github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
Expand Down Expand Up @@ -138,6 +140,8 @@ github.com/onsi/ginkgo/v2 v2.14.0 h1:vSmGj2Z5YPb9JwCWT6z6ihcUvDhuXLc3sJiqd3jMKAY
github.com/onsi/ginkgo/v2 v2.14.0/go.mod h1:JkUdW7JkN0V6rFvsHcJ478egV3XH9NxpD27Hal/PhZw=
github.com/onsi/gomega v1.30.0 h1:hvMK7xYz4D3HapigLTeGdId/NcfQx1VHMJc60ew99+8=
github.com/onsi/gomega v1.30.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ=
github.com/oracle/oci-go-sdk/v65 v65.80.0 h1:Rr7QLMozd2DfDBKo6AB3DzLYQxAwuOG118+K5AAD5E8=
github.com/oracle/oci-go-sdk/v65 v65.80.0/go.mod h1:IBEV9l1qBzUpo7zgGaRUhbB05BVfcDGYRFBCPlTcPp0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
Expand All @@ -149,6 +153,8 @@ github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg=
github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
Expand All @@ -161,6 +167,7 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/urfave/cli/v2 v2.27.1 h1:8xSQ6szndafKVRmfyeUMxkNUJQMjL1F2zmsZ+qHpfho=
Expand Down Expand Up @@ -226,6 +233,7 @@ golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
Expand Down
2 changes: 2 additions & 0 deletions internal/address/assigner.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ func NewAssigner(ctx context.Context, logger *logrus.Entry, provider types.Cloud
return &azureAssigner{}, nil
} else if provider == types.CloudProviderGCP {
return NewGCPAssigner(ctx, logger, cfg.Project, cfg.Region, cfg.IPv6)
} else if provider == types.CloudProviderOCI {
return NewOCIAssigner(ctx, logger, cfg)
}
return nil, ErrUnknownCloudProvider
}
2 changes: 1 addition & 1 deletion internal/address/aws.go
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ func (a *awsAssigner) tryAssignAddress(ctx context.Context, address *types.Addre

func (a *awsAssigner) getNetworkInterfaceID(instance *types.Instance) (string, error) {
// get network interface ID
if instance.NetworkInterfaces == nil || len(instance.NetworkInterfaces) == 0 {
if len(instance.NetworkInterfaces) == 0 {
return "", errors.Errorf("no network interfaces found for instance %s", *instance.InstanceId)
}
// get primary network interface ID with public IP address (DeviceIndex == 0)
Expand Down
Loading

0 comments on commit a78270d

Please sign in to comment.