feat: Allow applications in any namespace #612

iandelahorne · 2023-08-20T13:48:05Z

Add --namespaced flag, that when true will only monitor the configured namespace for applications. If it is false, then monitor all namespaces for applications.

By default this is set to true to preserve the current behavior.

fixes #601

iandelahorne · 2023-08-20T13:50:47Z

👋 Looking for feedback on this, specifically around the default behavior of this flag. Ideally it should default to be global (i e false), but I didn't want to break compatibility without discussing it.

I suspect I will have to add documentation updates as well as RBAC and helm chart updates, happy to do so if this is something the maintainers think is a good idea.

codecov-commenter · 2023-09-03T22:49:25Z

Codecov Report

Merging #612 (de9d4e4) into master (82d59ef) will increase coverage by 0.13%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #612      +/-   ##
==========================================
+ Coverage   65.35%   65.48%   +0.13%     
==========================================
  Files          22       22              
  Lines        2084     2092       +8     
==========================================
+ Hits         1362     1370       +8     
  Misses        588      588              
  Partials      134      134

Files	Coverage Δ
pkg/argocd/argocd.go	`66.66% <100.00%> (+0.27%)`	⬆️
pkg/argocd/update.go	`67.14% <100.00%> (+0.59%)`	⬆️
pkg/kube/kubernetes.go	`85.50% <100.00%> (+0.21%)`	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

jannfis · 2023-09-28T23:48:54Z

@iandelahorne Thanks a lot for your contribution, and apologies that I again haven't found the time to react timely on it.

I will try to review your PR the coming days.

jannfis

First quick review, have a couple of comments.

Generally, I believe the pattern should follow a similar pattern than what Argo CD does, i.e. instead of having a bool to turn it on, give it a list of namespaces that it's allowed to dig into. Then, ignore those applications that are popping up and that are not in that namespace.

jannfis · 2023-09-28T23:50:14Z

pkg/argocd/argocd.go

@@ -176,17 +176,16 @@ func FilterApplicationsForUpdate(apps []v1alpha1.Application, patterns []string,
 	var appsForUpdate = make(map[string]ApplicationImages)

 	for _, app := range apps {
-		logCtx := log.WithContext().AddField("application", app.GetName())
+		appNSName := fmt.Sprintf("%s/%s", app.GetNamespace(), app.GetName())


You can use app.QualifiedName() to get the representation of <namespace>/<name> of an app instead of rolling your own here :)

Oh nice, i'll go ahead and use those.

jannfis · 2023-09-28T23:51:07Z

pkg/argocd/argocd.go

 		// Check whether application has our annotation set
 		annotations := app.GetAnnotations()
 		if _, ok := annotations[common.ImageUpdaterAnnotation]; !ok {
-			logCtx.Tracef("skipping app '%s' of type '%s' because required annotation is missing", app.GetName(), app.Status.SourceType)


Why did you remove this trace statement?

I honestly thought I had added it and was removing it before commiting, good catch.

jannfis · 2023-09-28T23:52:11Z

pkg/argocd/argocd.go

 			continue
 		}

 		// Check for valid application type
 		if !IsValidApplicationType(&app) {
-			logCtx.Warnf("skipping app '%s' of type '%s' because it's not of supported source type", app.GetName(), app.Status.SourceType)


Why did you remove this warning statement?

jannfis · 2023-09-28T23:52:42Z

pkg/argocd/argocd.go

@@ -198,16 +197,16 @@ func FilterApplicationsForUpdate(apps []v1alpha1.Application, patterns []string,

 		// Check if application carries requested label
 		if !matchAppLabels(app.GetName(), app.GetLabels(), appLabel) {
-			logCtx.Debugf("Skipping app '%s' because it does not carry requested label", app.GetName())
+			logCtx.Debugf("Skipping app '%s' because it does not carry requested label", appNSName)


As mentioned earlier, you can use app.QualifiedName() here

jannfis · 2023-09-28T23:53:18Z

pkg/argocd/argocd.go

 			continue
 		}

-		logCtx.Tracef("processing app '%s' of type '%v'", app.GetName(), app.Status.SourceType)
+		logCtx.Tracef("processing app '%s' of type '%v'", appNSName, app.Status.SourceType)


As mentioned earlier, you can use app.QualifiedName() here

jannfis · 2023-09-28T23:53:22Z

pkg/argocd/argocd.go

 		imageList := parseImageList(annotations)
 		appImages := ApplicationImages{}
 		appImages.Application = app
 		appImages.Images = *imageList
-		appsForUpdate[app.GetName()] = appImages
+		appsForUpdate[appNSName] = appImages


As mentioned earlier, you can use app.QualifiedName() here

iandelahorne · 2023-09-29T02:21:40Z

Thanks for the feedback. It does sound cleaner to specify the list of namespaces or wildcard, and if none is specifed default to the current namespace. I'll take a stab at implementing that soon if that's the route you think we should go. In the meantime I addressed the most immediate PR feedback.

villisco · 2023-10-02T18:40:13Z

This cluster role would be needed as well:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/name: argocd-image-updater-cluster-apps
    app.kubernetes.io/instance: argocd
    app.kubernetes.io/part-of: argocd-image-updater
    app.kubernetes.io/component: controller
  annotations:
    argocd.argoproj.io/tracking-id: argocd:rbac.authorization.k8s.io/Role:argocd/argocd-image-updater-cluster-apps
  name: argocd-image-updater-cluster-apps
rules:
  - apiGroups:
      - ''
    verbs:
      - get
      - list
      - watch
    resources:
      - secrets
      - configmaps
  - apiGroups:
      - argoproj.io
    verbs:
      - get
      - list
      - update
      - patch
    resources:
      - applications
  - apiGroups:
      - ''
    verbs:
      - create
    resources:
      - events
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/name: argocd-image-updater-cluster-apps
    app.kubernetes.io/instance: argocd
    app.kubernetes.io/part-of: argocd-image-updater
    app.kubernetes.io/component: controller
  name: argocd-image-updater-cluster-apps
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: argocd-image-updater-cluster-apps
subjects:
  - kind: ServiceAccount
    name: argocd-image-updater
    namespace: argocd

villisco · 2023-10-02T18:41:43Z

Hope it gets merged soon. much needed feature! :)

villisco · 2023-10-14T07:03:00Z

Any news on this?

Add `--namespaced` flag, that when true will only monitor the configured namespace for applications. If it is false, then monitor all namespaces for applications. By default this is set to `true` to preserve the current behavior. fixes argoproj-labs#611 Signed-off-by: Ian Delahorne <[email protected]> Signed-off-by: Ian Delahorne <[email protected]>

Use `app.QualifiedName()` instead of creating a namespace/name string. In addition, I had erroneously removed trace statements, which are now back. Signed-off-by: Ian Delahorne <[email protected]>

iandelahorne · 2023-10-31T23:46:11Z

@villisco Sorry for the delay, I've been busy with my day job. I'm starting to take a look at refactoring this to take a list of namespaces.

villisco · 2023-11-04T09:19:27Z

iandelahorne

Been using this PR image and works great so far! Thank you @iandelahorne Great work! :)

torfjor · 2023-11-14T14:02:30Z

This PR does not address the git write back method for applications in any namespace. Application overrides are still written to the file .argocd-source-appName.yaml. Moving that file to .argocd-source-appNamespace_appName.yaml seems to make things work.

ElanHasson · 2023-12-21T04:57:59Z

Any updates on this one?

iandelahorne · 2023-12-22T22:45:41Z

@torfjor thanks for pointing that out, i'll be sure to address it @ElanHasson I'll try to address this over the holiday break - like I've mentioned above my day job got in the way.

…

On Wed, Dec 20, 2023 at 8:58 PM Elan Hasson ***@***.***> wrote: Any updates on this one? — Reply to this email directly, view it on GitHub <#612 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABNFBWIA3RJ2LHPAOUG4TTYKO6WDAVCNFSM6AAAAAA3XJQKYSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVGQ4DCNRVGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- // ian

ElanHasson · 2023-12-23T16:31:34Z

@iandelahorne I understand. I am happy to help in any way I can, just let me know!

I also know @renchap wants this feature as well.

mcanevet · 2024-01-23T10:35:34Z

@iandelahorne any news on this?

I get this error in the logs when using this branch with git write back:

2024-01-23 11:42:32 | panic: runtime error: invalid memory address or nil pointer dereference
-- | --
2024-01-23 11:42:32 | [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xc4e82d]
2024-01-23 11:42:32 |  
2024-01-23 11:42:32 | goroutine 1954 [running]:
2024-01-23 11:42:32 | github.com/argoproj/argo-cd/v2/util/git.HTTPSCreds.Environ({{0xc00178fba8, 0x6}, {0xc001649c00, 0x1a}, 0x0, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}, ...})
2024-01-23 11:42:32 | /go/pkg/mod/github.com/argoproj/argo-cd/[email protected]/util/git/creds.go:218 +0xb6d
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/ext/git.(*nativeGitClient).runCredentialedCmd(0xc00099f9d0, {0x24afa3d?, 0xc001271f20?}, {0xc0017df2c0?, 0x6?, 0x28341b8?})
2024-01-23 11:42:32 | /src/argocd-image-updater/ext/git/client.go:595 +0x8b
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/ext/git.(*nativeGitClient).Fetch(0xc00099f9d0, {0x0?, 0x0?})
2024-01-23 11:42:32 | /src/argocd-image-updater/ext/git/client.go:332 +0x236
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/pkg/argocd.commitChangesGit(0xc000eacc00, 0xc001808420, {0xc000f597d0, 0x1, 0x1}, 0x25e8610)
2024-01-23 11:42:32 | /src/argocd-image-updater/pkg/argocd/git.go:159 +0x365
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/pkg/argocd.commitChanges(0xc000be4400?, 0xc0007b9100?, {0xc000f597d0?, 0xc001289c30?, 0xc00146b680?})
2024-01-23 11:42:32 | /src/argocd-image-updater/pkg/argocd/update.go:574 +0x5b
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/pkg/argocd.commitChangesLocked(0xc001276500?, 0xc001808420?, 0x34?, {0xc000f597d0?, 0x1?, 0x1?})
2024-01-23 11:42:32 | /src/argocd-image-updater/pkg/argocd/update.go:554 +0x110
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/pkg/argocd.UpdateApplication(0xc0017dfb88, 0x19?)
2024-01-23 11:42:32 | /src/argocd-image-updater/pkg/argocd/update.go:337 +0x1fcd
2024-01-23 11:42:32 | main.runImageUpdater.func1({_, _}, {{{{0xc0011f2df0, 0xb}, {0xc000969e60, 0x14}}, {{0xc0009c2ea0, 0x26}, {0x0, 0x0}, ...}, ...}, ...})
2024-01-23 11:42:32 | /src/argocd-image-updater/cmd/run.go:322 +0x287
2024-01-23 11:42:32 | created by main.runImageUpdater
2024-01-23 11:42:32 | /src/argocd-image-updater/cmd/run.go:308 +0xa1e
2024-01-23 11:42:32 | panic: runtime error: invalid memory address or nil pointer dereference
2024-01-23 11:42:32 | [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xc4e82d]
2024-01-23 11:42:32 |  
2024-01-23 11:42:32 | goroutine 2056 [running]:
2024-01-23 11:42:32 | github.com/argoproj/argo-cd/v2/util/git.HTTPSCreds.Environ({{0xc0011ec070, 0x6}, {0xc000f342a0, 0x1a}, 0x0, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}, ...})
2024-01-23 11:42:32 | /go/pkg/mod/github.com/argoproj/argo-cd/[email protected]/util/git/creds.go:218 +0xb6d
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/ext/git.(*nativeGitClient).runCredentialedCmd(0xc0001b45b0, {0x24afa3d?, 0xc001701260?}, {0xc000ec92c0?, 0x6?, 0x28341b8?})
2024-01-23 11:42:32 | /src/argocd-image-updater/ext/git/client.go:595 +0x8b
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/ext/git.(*nativeGitClient).Fetch(0xc0001b45b0, {0x0?, 0x0?})
2024-01-23 11:42:32 | /src/argocd-image-updater/ext/git/client.go:332 +0x236
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/pkg/argocd.commitChangesGit(0xc001973800, 0xc001808580, {0xc0003f1038, 0x1, 0x1}, 0x25e8610)
2024-01-23 11:42:32 | /src/argocd-image-updater/pkg/argocd/git.go:159 +0x365
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/pkg/argocd.commitChanges(0xc000be4400?, 0xc000a3ca20?, {0xc0003f1038?, 0xc0009b5250?, 0xc0009d7680?})
2024-01-23 11:42:32 | /src/argocd-image-updater/pkg/argocd/update.go:574 +0x5b
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/pkg/argocd.commitChangesLocked(0xc001276a40?, 0xc001808580?, 0x34?, {0xc0003f1038?, 0x1?, 0x1?})
2024-01-23 11:42:32 | /src/argocd-image-updater/pkg/argocd/update.go:554 +0x110
2024-01-23 11:42:32 | github.com/argoproj-labs/argocd-image-updater/pkg/argocd.UpdateApplication(0xc000ec9b88, 0x19?)
2024-01-23 11:42:32 | /src/argocd-image-updater/pkg/argocd/update.go:337 +0x1fcd
2024-01-23 11:42:32 | main.runImageUpdater.func1({_, _}, {{{{0xc000a52950, 0xb}, {0xc000a33458, 0x14}}, {{0xc00133e660, 0x20}, {0x0, 0x0}, ...}, ...}, ...})
2024-01-23 11:42:32 | /src/argocd-image-updater/cmd/run.go:322 +0x287
2024-01-23 11:42:32 | created by main.runImageUpdater
2024-01-23 11:42:32 | /src/argocd-image-updater/cmd/run.go:308 +0xa1e

When I cherry-picked 674da709ce72e8b2c8bedd28ea59a58df0c0f127 on top of v0.12.2. It's better, but somehow it can't find the ConfigMap containing the credentials:

Could not update application spec: could not get creds for repo 'https://gitlab.com/<myorg>/<my-application-repo>.git': configmap \"argocd-cm\" not found" application=<my-application>

iandelahorne · 2024-01-25T17:30:19Z

@mcanevet @ElanHasson Apologies for the delays and for abandoning this. I just started a new position and ArgoCD is no longer a part of my daily life. If someone else wants to take this up and move it forward, I still believe this is a valuable feature. I just can't commit currently to updating this at any point in the near future sadly.

rossigee · 2024-01-26T00:45:03Z

Thanks @iandelahorne - I updated your PR and re-submitted it here. This PR can be closed now.

iandelahorne · 2024-01-31T23:06:24Z

Thanks @rossigee for taking this over!

iandelahorne force-pushed the namespace-support branch from 1294f52 to 6ce47d1 Compare August 20, 2023 13:48

iandelahorne force-pushed the namespace-support branch 3 times, most recently from da55471 to b378e4b Compare September 3, 2023 22:45

jannfis reviewed Sep 29, 2023

View reviewed changes

iandelahorne and others added 2 commits October 31, 2023 16:31

Use app.QualifiedName, add back trace statements

de9d4e4

Use `app.QualifiedName()` instead of creating a namespace/name string. In addition, I had erroneously removed trace statements, which are now back. Signed-off-by: Ian Delahorne <[email protected]>

iandelahorne force-pushed the namespace-support branch from 949b420 to de9d4e4 Compare October 31, 2023 23:32

ElanHasson mentioned this pull request Dec 21, 2023

Documentation: ArgoCD apps in any namespace incompatibility #541

Merged

rossigee mentioned this pull request Jan 26, 2024

Namespace support #672

Closed

iandelahorne closed this Jan 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Allow applications in any namespace #612

feat: Allow applications in any namespace #612

iandelahorne commented Aug 20, 2023 •

edited

Loading

iandelahorne commented Aug 20, 2023

codecov-commenter commented Sep 3, 2023 •

edited

Loading

jannfis commented Sep 28, 2023

jannfis left a comment

jannfis Sep 28, 2023

iandelahorne Sep 29, 2023

jannfis Sep 28, 2023

iandelahorne Sep 29, 2023

jannfis Sep 28, 2023

jannfis Sep 28, 2023

jannfis Sep 28, 2023

jannfis Sep 28, 2023

iandelahorne commented Sep 29, 2023 •

edited

Loading

villisco commented Oct 2, 2023 •

edited

Loading

villisco commented Oct 2, 2023 •

edited

Loading

villisco commented Oct 14, 2023

iandelahorne commented Oct 31, 2023

villisco commented Nov 4, 2023

torfjor commented Nov 14, 2023 •

edited

Loading

ElanHasson commented Dec 21, 2023

iandelahorne commented Dec 22, 2023 via email

ElanHasson commented Dec 23, 2023 •

edited

Loading

mcanevet commented Jan 23, 2024 •

edited

Loading

iandelahorne commented Jan 25, 2024

rossigee commented Jan 26, 2024

iandelahorne commented Jan 31, 2024

feat: Allow applications in any namespace #612

feat: Allow applications in any namespace #612

Conversation

iandelahorne commented Aug 20, 2023 • edited Loading

iandelahorne commented Aug 20, 2023

codecov-commenter commented Sep 3, 2023 • edited Loading

Codecov Report

jannfis commented Sep 28, 2023

jannfis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iandelahorne commented Sep 29, 2023 • edited Loading

villisco commented Oct 2, 2023 • edited Loading

villisco commented Oct 2, 2023 • edited Loading

villisco commented Oct 14, 2023

iandelahorne commented Oct 31, 2023

villisco commented Nov 4, 2023

torfjor commented Nov 14, 2023 • edited Loading

ElanHasson commented Dec 21, 2023

iandelahorne commented Dec 22, 2023 via email

ElanHasson commented Dec 23, 2023 • edited Loading

mcanevet commented Jan 23, 2024 • edited Loading

iandelahorne commented Jan 25, 2024

rossigee commented Jan 26, 2024

iandelahorne commented Jan 31, 2024

iandelahorne commented Aug 20, 2023 •

edited

Loading

codecov-commenter commented Sep 3, 2023 •

edited

Loading

iandelahorne commented Sep 29, 2023 •

edited

Loading

villisco commented Oct 2, 2023 •

edited

Loading

villisco commented Oct 2, 2023 •

edited

Loading

torfjor commented Nov 14, 2023 •

edited

Loading

ElanHasson commented Dec 23, 2023 •

edited

Loading

mcanevet commented Jan 23, 2024 •

edited

Loading