Google Cloud credentials control access to your resources hosted on Google Cloud. To help keep your data secure and protected from attackers, you must handle your credentials with utmost care.
We recommend that you protect all of your Google Cloud credentials from unintended access. These credentials include but are not limited to the following:
Service credentials:
- Service account private keys (JSON and p12 files)
- API keys
- OAuth2 client ID secrets
User credentials that are created and managed on developer workstations or other computers:
Browser cookies
Google Cloud CLI credentials are stored in the
user's home directory. You can list them in Google Cloud CLI using the gcloud
auth list command.
Application Default Credentials are stored on the developer's workstation.
Browser cookies are browser-specific, but are typically stored on the developer's
workstation.
If you suspect that any of your credentials have been compromised, you must take immediate action to limit the impact of the compromise on your Google Cloud account.
Monitor for credential compromise
To monitor for potential compromise, consider the following:
Monitor for suspicious account activity such as privilege escalation and multiple account creations. Monitor for these activities using Cloud Audit Logs, Policy Intelligence and Security Command Center. Use the following Security Command Center services and capabilities:
- Event Threat Detection to identify threats that are based on administrator activities, Groups changes, and Identity and Access Management (IAM) permission changes.
- Sensitive Actions Service to track actions in your organization, folders, and projects that could be damaging to your business if they are taken by a malicious actor.
- Cloud Infrastructure Entitlement Management (CIEM) (Preview) to manage access for identities and generate findings for misconfigurations.
Monitor user logins in Google Workspace and Cloud Identity. To better track issues, consider exporting the logs to Cloud Logging.
Monitor for secrets in your code repositories, using tools such as Anomaly Detection or secret scanning.
Monitor for anomalies in service account key usage using Cloud Monitoring or CIEM.
Ensure that your security operations center (SOC) is notified promptly and has the playbooks, tools, and access that are required to respond quickly to a suspected credential compromise. Use the Security Command Center Enterprise tier to enable SIEM and SOAR capabilities such as playbooks, response workflows, and automated actions. You can also integrate Security Command Center with your existing SIEM or import logs into Google Security Operations for further analysis.
Protect your Google Cloud resources from a compromised credential
Complete the steps in the following sections as soon as you can to help protect your resources if you suspect a credential is compromised.
Revoke and reissue credentials
If you suspect a credential is compromised, revoke and re-issue it. Proceed carefully to ensure you don't suffer a service outage as a result of revoking credentials.
In general, to reissue credentials, you generate a new credential, push it to all services and users that need it, and then revoke the old credential.
The following sections provide specific instructions for each type of credential.
Replace a service account key
In the Google Cloud console, go to the Service accounts page.
Locate the affected service account.
Create a new key for the service account.
Push the new key to all the locations in which the old key was in use.
Delete the old key.
For more information, see Create service accounts.
Regenerate API keys
In the Google Cloud console, go to the Credentials page.
Create a new API key using the Create credentials button. Configure the new key to be the same as the compromised API key. The restrictions on the API key must match; otherwise you might suffer an outage.
Push the API key to all locations in which the old key was in use.
Delete the old key.
For more information, see Authenticate by using API keys.
Reset an OAuth2 client ID secret
Changing a client ID secret will cause a temporary outage while the secret is rotated.
In the Google Cloud console, go to the Credentials page.
Select the compromised OAuth2 client ID and edit it.
Click Reset Secret.
Push the new secret to your application.
For more information, see Setting up OAuth 2.0 and Using OAuth 2.0 to access Google APIs.
Remove Google Cloud CLI credentials as an administrator
As a Google Workspace administrator, remove access to Google Cloud CLI from the user's list of connected apps. For more information, see View and remove access to third-party applications.
When the user accesses Google Cloud CLI again, it will automatically ask them to re-authorize the application.
Remove Google Cloud CLI credentials as a user
Open the list of apps with access to your Google Account.
Remove Google Cloud CLI from the list of connected apps.
When you access Google Cloud CLI again, it will automatically ask you to re-authorize the application.
Revoke Application Default Credentials as an administrator
If you suspect that an Application Default Credential is compromised, you can revoke it. This procedure can cause a temporary outage until the credentials file is recreated.
As a Google Workspace administrator, remove access to the Google Auth Library from the user's list of connected apps. For more information, see View and remove access to third-party applications.
Revoke Application Default Credentials as a user
If you suspect that an Application Default Credential that you created is compromised, you can revoke it. This procedure can cause a temporary outage until the credentials file is recreated. This procedure can only be completed by the owner of the compromised credential.
Install and initialize the Google Cloud CLI, if you haven't already.
Authorize gcloud CLI with your user identity, not with a service account:
gcloud auth loginFor more information, see Authorize the gcloud CLI.
Revoke the credentials:
gcloud auth application-default revokeOptionally, delete the
application_default_credentials.jsonfile. The location depends on your operating system:- Linux, macOS:
$HOME/.config/gcloud/ - Windows:
%APPDATA%\gcloud\
- Linux, macOS:
Recreate the credentials file:
gcloud auth application-default login
Invalidate browser cookies as an administrator
If you suspect browser cookies are compromised, Google Workspace administrators can sign a user out of their account.
In addition, immediately force a password change.
These actions invalidate all existing cookies, and the user is asked to sign in again.
Invalidate browser cookies as a user
If you suspect browser cookies are compromised, sign out of your Google Account and change your password immediately.
These actions invalidate all your existing cookies. The next time you access Google Cloud, you must sign in again.
Look for unauthorized access and resources
After you revoke compromised credentials and restored your service, review all access to your Google Cloud resources. You can use Logging or Security Command Center.
In Logging, complete the following:
Examine your audit logs in the Google Cloud console.
Search all potentially affected resources, and make sure that all account activity (especially related to the compromised credentials) are as expected.
In Security Command Center, complete the following:
In the Google Cloud console, go to the Security Command Center Findings page.
If necessary, select your Google Cloud project or organization.
In the Quick filters section, click an appropriate filter to display the finding that you need in the Findings query results table. For example, if you select Event Threat Detection or Container Threat Detection in the Source display name subsection, only findings from the selected service appear in the results.
The table is populated with findings for the source you selected.
To view details of a specific finding, click the finding name under
Category. The finding details pane expands to display a summary of the finding's details.To display all findings that were caused by the same user's actions:
- On the finding details pane, copy the email address next to Principal email.
- Close the pane.
In Query editor, enter the following query:
access.principal_email="USER_EMAIL"Replace USER_EMAIL with the email address that you previously copied.
Security Command Center displays all findings that are associated with actions taken by the user that you specified.
Delete all unauthorized resources
Make sure that there are no unexpected resources, such as VMs, App Engine apps, service accounts, Cloud Storage buckets, and so forth, that the compromised credential could access.
After you are satisfied that you have identified all unauthorized resources, you can choose to delete these resources immediately. This is especially important for Compute Engine resources, because attackers can use compromised accounts to exfiltrate data or otherwise compromise your production systems.
Alternatively, you can try to isolate unauthorized resources to allow your own forensics teams to perform additional analysis.
Contact Cloud Customer Care
For help with finding the Google Cloud logs and tools that you require for your investigation and mitigation steps, contact Customer Care and open a support case.
Best practices to avoid compromised credentials
This section describes best practices that you can implement to help you avoid compromised credentials.
Separate credentials from code
Manage and store your credentials separately from your source code. It is extremely common to accidentally push both credentials and source code to a source management site like GitHub, which makes your credentials vulnerable to attack.
If you are using GitHub or other public repository, you can implement tools such as Anomaly Detection or secret scanning, which warns you about exposed secrets in your GitHub repositories. To stop keys from being committed to your GitHub repositories, consider using tools such as git-secrets.
Use secret management solutions such as Secret Manager and Hashicorp Vault to store your secrets, rotate them regularly, and apply least privilege.
Implement service account best practices
To help protect service accounts, review the best practices for working with service accounts.
Limit session lengths
To force periodic re-authentication, limit the time that sessions remain active for Google and Google Cloud accounts. For more information, see the following:
Use VPC Service Controls to limit access
To limit the impact of compromised credentials, create service perimeters using VPC Service Controls. When you configure VPC Service Controls, resources inside the perimeter can only communicate with other resources inside the perimeter.