Listing all projects under a Google Cloud organization with Python

When working on the Google Cloud Platform, we often need to list all the google cloud projects under the organization. This can be easily done using GCP resource manager and Python.

In this blog post, I will walk through the steps in detail.

What is Google Cloud Resource Manager?

Google Cloud Resource Manager is a service provided by Google Cloud Platform that helps you manage and organise your cloud resources in a hierarchical structure. It acts as a central hub for resource management and provides features to create, manage, and monitor projects, folders, and organisations within your Google Cloud environment.

What is so good about Google Cloud Resource Manager?

Resource Manager enables you to create and manage projects(serve as logical containers for your cloud resources) and structure them under folders in a hierarchical manner under an organization.

It provides fine-grained access control through IAM policies and supports organization-level policies enforcing compliance, security, and governance rules across projects and folders.

Resource Manager provides visibility into your resource hierarchy, and relationships between projects, folders, and organizations. It let you track resource usage, view audit logs, and monitor billing information for individual projects.

---

Resource Manager offers APIs and SDKs that allow you to programmatically manage your cloud resources.

Google Cloud Resource Manager with Python

We will use Python to walk through the organization tree and list out all the projects distributed across different folders.

Note: I will be running the Python script from the google cloud shell by impersonating a service account.

---

Let’s create a service account

#set the project in cloud shell
gcloud config set project someproject

gcloud iam service-accounts create rm-svc \
    --description="DESCRIPTION" \
    --display-name="resource-manager-list"

Assigning organization-level permission to service account

gcloud organizations add-iam-policy-binding \ 
ORGANIZATION_ID --member="serviceAccount:rm-svc@someproject.iam.gserviceaccount.com" \
 --role="roles/resourcemanager.folderViewer"

Creating the key for the service account

gcloud iam service-accounts keys create rm-svc.json --iam-account  \
rm-svc@someproject.iam.gserviceaccount.com

The key will look something like this with actual values instead of placeholders

Let’s impersonate the service account so you can run the Python code to scan through the GCP organization.

gcloud auth activate-service-account —key-file=rm-svc.json

Note: There are many better ways to run the code than creating keys, such as using short-lived credentials, but I want to keep it simple for demo purposes.

Now we are ready to write some Python code. Let’s first install the Python client for the resource manager

#assuming you already have python and pip installed
pip install google-cloud-resource-manager 

Create a file main.py and paste the below code

from google.cloud import resourcemanager_v3

def get_folders(
    parent_id = "organizations/ORGANIZATION_ID",
    folders = None):

# This function will return a list of folder_id for all the folders and 
# subfolders respectively

    if folders is None:
        folders = []

# Creating folder client 
    client = resourcemanager_v3.FoldersClient()
    request = resourcemanager_v3.ListFoldersRequest(
        parent=parent_id,
    )

    page_result = client.list_folders(request=request)
    for pages in page_result:
        folders.append(pages.name)
        get_folders(parent_id=pages.name, folders=folders)
    return folders


def search_projects(folder_id):
# This function will take folder_id input and returns
# the list of project_id under a given folder_id

    client = resourcemanager_v3.ProjectsClient()

    query = f"parent:{folder_id}"
    request = resourcemanager_v3.SearchProjectsRequest(query=query)
    page_result = client.search_projects(request=request)
    search_result = []
    for pages in page_result:
        search_result.append(pages)
    return search_result


def list_projects():
# will returns the list of all active projects(project_id)

    active_project = []
    for folders in get_folders(parent_id=organizations/ORGANIZATION_ID, folders=None):
        for projects in search_projects(folders):
            if str(projects.state) == "State.ACTIVE":
                active_project.append(projects.project_id)

    return active_project


if __name__ == "__main__":
print(list_projects())

Finally, run the script

python3 main.py

We can use Python with resource manager API to do so much more than this.