Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SURE-3400 related] Outbound type is restored to LoadBalancer if UserDefinedRouting is selected #750

Closed
3 tasks
valaparthvi opened this issue Dec 11, 2024 · 6 comments
Assignees
Labels
JIRA Must shout kind/bug Something isn't working
Milestone

Comments

@valaparthvi
Copy link
Contributor

valaparthvi commented Dec 11, 2024

If an AKS cluster is created with an outbound type: userdefinedrouting, it is restored to LoadBalancer after the cluster is provisioned successfully. This change appears to be triggered from Azure console, because the change appears in UpstreamSpec first and then in AKSConfig.

The same was reproduced when I provisioned AKS via Terraform, ref: #709.

Rancher version: v2.10-c4944b3ca0b39c474121df9e7ef31f7aff4e4f7a-head.
AKS Operator version: rancher-aks-operator:105.1.0+up1.10.1-rc.2

PR's:

@kkaempf kkaempf added the kind/bug Something isn't working label Dec 12, 2024
@kkaempf kkaempf added this to the v2.10.2 milestone Dec 12, 2024
@kkaempf kkaempf added the JIRA Must shout label Dec 12, 2024
@kkaempf kkaempf changed the title Outbound type is restored to LoadBalancer if UserDefinedRouting is selected [SURE-3400 related] Outbound type is restored to LoadBalancer if UserDefinedRouting is selected Dec 12, 2024
@mjura mjura self-assigned this Dec 13, 2024
@mjura mjura moved this from Backlog to In Progress (8 max) in CAPI & Hosted Kubernetes providers (EKS/AKS/GKE) Dec 13, 2024
@mjura mjura moved this from In Progress (8 max) to PR to be reviewed in CAPI & Hosted Kubernetes providers (EKS/AKS/GKE) Dec 24, 2024
@mjura
Copy link
Contributor

mjura commented Dec 24, 2024

@mantis-toboggan-md Rancher UI needs to be also fixed

Image

  1. Box name where users are selecting LoadBalancer or User Defined Routing needs to be called Outbound Type, currently is DNS Prefix
  2. When User Defined Routing is selected then Virtual Network has to be provided, it can't be empty
  3. When User Defined Routing is selected then Network Plugin Azure CNI has to be selected

@kkaempf
Copy link

kkaempf commented Jan 9, 2025

PRs got all merged, moving to "To Test"

@valaparthvi valaparthvi self-assigned this Jan 15, 2025
@valaparthvi
Copy link
Contributor Author

Tested this on v2.10-d352f3ad3f7dec680c91e8bf7d5c87574f2ece2a-head, and it appears to have been fixed.

  • Returned error when Azure CNI was not selected
  • Returned error when subnet was not selected.
  • Successfully provisioned a cluster to use UserDefinedRouting, thanks to @cpinjani for verifying this case 🙌

@valaparthvi
Copy link
Contributor Author

I was too quick in testing 2.9. It is not fixed on 2.9.

  • No error was thrown on not using Azure CNI
  • UserDefinedRouting was reverted back to LoadBalancer.

@valaparthvi valaparthvi reopened this Jan 20, 2025
@kkaempf kkaempf modified the milestones: v2.10.2, v2.9.6 Jan 20, 2025
@valaparthvi
Copy link
Contributor Author

@cpinjani tried testing this on 2.9 and the issue seems to have been fixed, it seems I was not using stgregistry which is why it did not work. Closing the issue.

@cpinjani
Copy link
Contributor

Tested on v2.9-dde222feabec581e9f15256620a27a47a2e630a2-head, aks-operator:v1.9.6-rc.10
Cluster spec for provisioned cluster:

aksConfig:
    authBaseUrl: null
    authorizedIpRanges: null
    azureCredentialSecret: cattle-global-data:cc-2r9s2
    baseUrl: null
    clusterName: cpinjani-udr
    dnsPrefix: cpinjani-udr
    dnsServiceIp: 10.1.0.10
    dockerBridgeCidr: null
    httpApplicationRouting: null
    imported: false
    kubernetesVersion: 1.30.4
    linuxAdminUsername: azureuser
    loadBalancerSku: standard
    logAnalyticsWorkspaceGroup: null
    logAnalyticsWorkspaceName: null
    managedIdentity: null
    monitoring: null
    networkPlugin: azure
    networkPolicy: azure
    nodePools:
      - availabilityZones:
          - '1'
          - '2'
          - '3'
        count: 1
        maxPods: 110
        maxSurge: '1'
        mode: System
        name: agentpool
        orchestratorVersion: 1.30.4
        osDiskSizeGB: 128
        osDiskType: Managed
        osType: Linux
        vmSize: Standard_DS2_v2
        vnetSubnetID: >-
          /subscriptions/<REDACTED>/resourceGroups/<REDACTED>/providers/Microsoft.Network/virtualNetworks/<REDACTED>/subnets/aks-subnet
    outboundType: userDefinedRouting
    podCidr: null
    privateCluster: false
    privateDnsZone: null
    resourceGroup: <REDACTED>
    resourceLocation: eastus
    serviceCidr: 10.1.0.0/16
    subnet: null
    tags:
      Team: container-es
    userAssignedIdentity: null
    virtualNetwork: null
    virtualNetworkResourceGroup: null

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
JIRA Must shout kind/bug Something isn't working
Development

No branches or pull requests

4 participants