Project Selection: Replace text input with a project selection dropdown

[zacdezgeo]

The current project selection workflow in the plugin requires users to manually input a Google Cloud project name as a text string. This input is then sent to Earth Engine (EE) without any validation, which can lead to errors if the project name is misspelled or unregistered for Earth Engine access.

To improve the user experience, the plugin should allow users to select a Google Cloud project from a dropdown list, similar to how the Earth Engine Code Editor handles project selection. The dropdown should fetch and display only active projects associated with the user’s account.

Related links:

• Authenticate and set project during initialize #156
• Link Earth Engine to Cloud Projects #150

[alukach]

I spent some time working through this but have been unable to retrieve a list of projects for a given set of credentials.

After authenticating, I've been trying the following:

from googleapiclient import discovery
from ee.data import get_persistent_credentials

credentials = get_persistent_credentials()
service = discovery.build("cloudresourcemanager", "v1", credentials=credentials)
request = service.projects().list()
while request is not None:
    response = request.execute()
    for project in response.get("projects", []):
        print(f'{project["projectId"]=}')
    request = service.projects().list_next(
        previous_request=request, previous_response=response
    )

However, this yields the following error:

googleapiclient.errors.HttpError: 
<HttpError 403 when requesting https://cloudresourcemanager.googleapis.com/v1/projects?alt=json 
returned "Cloud Resource Manager API has not been used in project 517222506229 before 
or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=517222506229 
then retry. If you enabled this API recently, wait a few minutes for the action to 
propagate to our systems and retry.". 
Details: "[
    {
        '@type': 'type.googleapis.com/google.rpc.Help', 
        'links': [
            {
                'description': 'Google developers console API activation', 
                'url': 'https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=517222506229'
            }
        ]
    }, 
    {
        '@type': 'type.googleapis.com/google.rpc.ErrorInfo', 
        'reason': 'SERVICE_DISABLED', 
        'domain': 'googleapis.com', 
        'metadata': {
            'service': 'cloudresourcemanager.googleapis.com', 
            'consumer': 'projects/517222506229'
        }
    }
]">

The project 517222506229 mentioned comes from the OAuth credentials hardcoded into the earthengine API.

Unless an administrator of the OAuth client for the earthengine API is willing/capable of updating the client configuration to enable the cloud resource manager API, I don't think we can make such a query.

I will note that I don't think scopes are a factor regarding this issue, as the default scopes used includes https://www.googleapis.com/auth/cloud-platform, which is one of the required for the resource manager's project.list call.

[jdbcode]

@naschmitz for awareness. RE project listing and selection.

[naschmitz]

This is something the Earth Engine Python team is thinking about as well.

517222506229 is the ee-api-python-authenticator project. We shouldn't be using that client project for Cloud Resource Manager requests.

I'm unable to reproduce the issue you're seeing, but I have a couple ideas for fixes:

• We still support the notebook auth mode. Authenticate with ee.Authenticate(auth_mode="notebook") and a project selector appears in the notebook authenticator. The application default credentials will store project information, so no need to specify project in ee.Initialize. I believe this is what the QGIS plugin was doing before we switched the default auth mode but I'm not sure.
• You could try using the Cloud Resource Manager client library: https://cloud.google.com/python/docs/reference/cloudresourcemanager/0.30.5. It'll be easier and more robust than using the cloudresourcemanager Discovery API.
• You could also try creating credentials using the google.auth module instead of using the ones created by Earth Engine: import google.auth and credentials, project = google.auth.default().

I'm happy to dig into this problem more later in the week. Let me know whether the suggestions worked.

[alukach]

@naschmitz Thanks for the input. Having spent some time with your suggestions:

I'm unable to reproduce the issue you're seeing

I'm not entirely sure what you're saying here. Are you suggesting that you are able to successfully query projects with the ee-api-python-authenticator client?

• We still support the notebook auth mode. Authenticate with ee.Authenticate(auth_mode="notebook") and a project selector appears in the notebook authenticator. The application default credentials will store project information, so no need to specify project in ee.Initialize. I believe this is what the QGIS plugin was doing before we switched the default auth mode but I'm not sure.

I experimented with this. I was happy to see that the goal of Project Selection was handled within the browser during the auth flow. However, two disadvantages seemed to stand out to me:

1. The messaging during the notebook auth process is centered around authenticating notebooks (to no surprise). This could be a bit confusing to end users.
2. It appears that the output of this auth flow is to provide an auth token (edit: auth code) that is to be copied/pasted into the notebook. This makes sense, as notebooks are unable to support redirects in the same way as traditional web servers (at least, as far as I know). This means that to use this flow within our QGIS application, we would need to present an input form to the user to allow them to provide their auth token to QGIS. This is not difficult in terms of implementation, however we need agreement that this is an acceptable UX for users of this QGIS plugin.

Screenshots

Project Selection

Auth Token Output

• You could try using the Cloud Resource Manager client library: https://cloud.google.com/python/docs/reference/cloudresourcemanager/0.30.5. It'll be easier and more robust than using the cloudresourcemanager Discovery API.
• You could also try creating credentials using the google.auth module instead of using the ones created by Earth Engine: import google.auth and credentials, project = google.auth.default().

Regarding both of these suggestions, I don't believe that using either alternative SDK resolves the core issue of permissions. Take the following script for example:

#!/usr/bin/env python3
# /// script
# dependencies = [
#     "google-auth",
#     "google-auth-oauthlib",
#     "google-cloud-resource-manager",
#     "earthengine-api",
# ]
# ///

import google_auth_oauthlib.flow
from google.cloud import resourcemanager_v3
import ee


client_config = {
    "installed": {
        "client_id": ee.oauth.CLIENT_ID,
        "client_secret": ee.oauth.CLIENT_SECRET,
        "auth_uri": ee.oauth.AUTH_URI,
        "token_uri": ee.oauth.TOKEN_URI,
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "redirect_uris": ["http://localhost:8080"],
    }
}

flow = google_auth_oauthlib.flow.InstalledAppFlow.from_client_config(
    client_config, ee.oauth.SCOPES
)

flow.run_local_server()
session = flow.authorized_session()

client = resourcemanager_v3.ProjectsClient(credentials=session.credentials)
for project in client.list_projects():
    print(project)

Output

▶ uv run script.py --python /Applications/QGIS-LTR.app/Contents/MacOS/bin/python3        
Reading inline script metadata from `script.py`
Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=517222506229-vsmmajv00ul0bs7p89v5m89qs8eb9359.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2F&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fearthengine+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.full_control&state=BJUU2n5fmph7hA0g5WrGBLj57DObGx&access_type=offline
Traceback (most recent call last):
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/script.py", line 35, in <module>
    for project in client.list_projects():
                   ~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/extlibs/google/cloud/resourcemanager_v3/services/projects/client.py", line 904, in list_projects
    response = rpc(
        request,
    ...<2 lines>...
        metadata=metadata,
    )
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/extlibs/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/extlibs/google/api_core/retry/retry_unary.py", line 293, in retry_wrapped_func
    return retry_target(
        target,
    ...<3 lines>...
        on_error=on_error,
    )
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/extlibs/google/api_core/retry/retry_unary.py", line 153, in retry_target
    _retry_error_helper(
    ~~~~~~~~~~~~~~~~~~~^
        exc,
        ^^^^
    ...<6 lines>...
        timeout,
        ^^^^^^^^
    )
    ^
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/extlibs/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
    raise final_exc from source_exc
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/extlibs/google/api_core/retry/retry_unary.py", line 144, in retry_target
    result = target()
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/extlibs/google/api_core/timeout.py", line 120, in func_with_timeout
    return func(*args, **kwargs)
  File "/Users/alukach/github/gee-community/qgis-earthengine-plugin/extlibs/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.PermissionDenied: 403 Cloud Resource Manager API has not been used in project 517222506229 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=517222506229 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry. [reason: "SERVICE_DISABLED"
domain: "googleapis.com"
metadata {
  key: "service"
  value: "cloudresourcemanager.googleapis.com"
}
metadata {
  key: "serviceTitle"
  value: "Cloud Resource Manager API"
}
metadata {
  key: "containerInfo"
  value: "517222506229"
}
metadata {
  key: "consumer"
  value: "projects/517222506229"
}
metadata {
  key: "activationUrl"
  value: "https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=517222506229"
}
, locale: "en-US"
message: "Cloud Resource Manager API has not been used in project 517222506229 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=517222506229 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry."
, links {
  description: "Google developers console API activation"
  url: "https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=517222506229"
}
]

---

It seems like our solutions are to either:

1. Create a new OAuth2 client with the required permissions. I am unaware of what the downstream effects of that could be.
2. Update the existing OAuth2 client to have permission to list projects. I am unaware of what the downstream effects of that could be.
3. Adapt the QGIS Plugin auth flow to follow a copy/paste auth code strategy as described above. With this strategy, project selection takes place within the browser.
4. Keep the plugin as it is today where a project ID must be manually provided by text input.