AWS vs GCP vs Azure: Service Mapping for ML Engineers

Most ML engineers know one cloud well and the other two loosely. Interviewers know this, so cross-cloud questions usually test whether you understand the underlying concept well enough to translate it, not whether you memorized three product catalogs.

How to use this guide in an interview:

1. Name the concept first. "We need a workload identity for the training job so it can read the bucket without a static key."
2. Then name the service on the cloud you know. "On AWS that is the SageMaker AI execution role."
3. Then translate, and say where the analogy breaks. "On GCP it would be a service account attached to the Vertex AI custom job; the difference is that GCP grants the role on the bucket or project rather than through a policy document attached to the role."

This guide focuses on the infrastructure around ML: identity, networking, storage, compute, data, GenAI and cost. For the ML platforms themselves (Amazon SageMaker AI, Vertex AI, Azure Machine Learning) feature by feature, see the Cloud ML Platforms Comparison; the ML platform table below is intentionally brief.

Product names in this space change often (several services in these tables were renamed in 2024 and 2025). Names are believed current as of 2026, and the ones most likely to drift are marked "check current name". In an interview, getting the concept right matters more than getting the newest brand name.

Table of Contents

1. Service mapping tables
   • Identity and access
   • Networking
   • Storage
   • Compute
   • Data and streaming
   • ML platform
   • Generative AI
   • Observability and cost
2. Key behavioural differences
3. Cloud-agnostic ML architecture
4. Migration and multi-cloud scenarios
5. Terraform example: one versioned bucket on three clouds
6. Interview Q&A
7. Common Pitfalls
8. Related Topics

Service mapping tables

Read each row as "same job, different implementation". Where the Notes column says there is no clean equivalent, say that in an interview instead of forcing a match: it signals real experience.

Identity and access

Networking

Storage

Compute

Data and streaming

ML platform

This table is deliberately short. The Cloud ML Platforms Comparison covers the platforms feature by feature, and the provider guides (SageMaker, Vertex AI, Azure ML) cover each one in depth.

Generative AI

Observability and cost

Key behavioural differences

Mapping tables hide the differences that actually cause incidents and wrong interview answers. These are the ones worth explaining out loud.

Global VPC on GCP vs regional networks on AWS and Azure

• GCP: a VPC network is a global resource. Subnets are regional, but VMs in us-central1 and europe-west4 on the same VPC reach each other over internal IPs with no peering. Firewall rules and routes are defined once for the whole network.
• AWS: a VPC lives in one Region, and each subnet sits in one Availability Zone. Multi-region means multiple VPCs connected with peering or Transit Gateway, and route tables to match.
• Azure: a VNet lives in one region, and subnets span the region's availability zones. Multi-region means multiple VNets with peering or Virtual WAN.

Why it matters for ML: on GCP a training job in one region reading from a store in another region "just works" on the network, which makes it easy to pay for cross-region traffic without noticing. On AWS and Azure the network topology forces you to make the cross-region decision explicitly.

Projects vs accounts vs subscriptions and resource groups

A common mapping: one AWS account per environment maps to one GCP project per environment and to one Azure subscription per environment, with resource groups per application inside it. Azure ML adds a twist: a workspace has dependent resources (storage account, key vault, container registry, Application Insights) that usually sit in the same resource group, so deleting the group deletes all of them.

IAM models

• AWS: policies are JSON documents of Effect, Action, Resource and Condition. They attach to principals (identity-based) or to resources (resource-based, such as S3 bucket policies and KMS key policies). Evaluation starts from an implicit deny; an explicit Deny anywhere (identity policy, resource policy, permission boundary, SCP, RCP) wins. Cross-account access usually needs both sides: the caller's identity policy and the target's resource or trust policy.
• GCP: allow policies bind principals to roles on a resource (organization, folder, project or individual resource) and are inherited downward. You cannot remove an inherited allow lower in the tree; you either grant it lower in the first place or add an IAM deny policy, which is evaluated before allow policies. IAM Conditions add attribute-based restrictions.
• Azure: a role assignment binds a principal to a role definition at a scope (management group, subscription, resource group or resource) and is inherited downward. Azure Policy is a separate system that governs what resources may look like, not who can act. Two traps: control-plane roles (such as Owner or Contributor) do not automatically grant data-plane access (for example reading blobs needs a data role like Storage Blob Data Reader), and Microsoft Entra directory roles are a separate permission system from Azure RBAC roles. Deny assignments exist but are created by platform features (such as deployment stacks or managed applications), not directly by you.

Interview shorthand: AWS is "policy documents with explicit deny", GCP is "role bindings inherited down a resource tree, plus deny policies", Azure is "role assignments at scopes, plus Azure Policy for configuration".

BigQuery vs Redshift vs Synapse and Fabric

• BigQuery is serverless: there is no cluster to size. Storage and compute are separate, and you pay either for data scanned (on-demand) or for reserved compute capacity (slots via editions). For ML this means ad hoc feature exploration needs no infrastructure, but unpartitioned SELECT * queries over large tables become a cost problem fast. BigQuery ML and remote models let you train or call models from SQL.
• Redshift started as a provisioned cluster you size, now with managed storage and a Serverless option that scales compute automatically. Tuning concepts (distribution keys, sort keys, workload management) still matter more than in BigQuery.
• Azure is in transition: Synapse dedicated SQL pools are provisioned, Synapse serverless SQL pools query the lake on demand, and Microsoft Fabric bundles warehouse, lakehouse, Spark and pipelines on a shared capacity model with OneLake storage. Many Azure ML teams instead use Azure Databricks as the warehouse and feature engineering layer.

When an interviewer asks "what is the Azure BigQuery?", the honest answer is "it depends on whether the company is on Fabric, Synapse or Databricks", followed by the concept: a columnar warehouse with separated storage and compute.

Egress and data movement charges

All three clouds charge for data leaving the cloud to the internet, and for traffic between regions. Traffic between zones in the same region is also billed on some clouds and not others, and the rules change, so check the current pricing pages rather than quoting from memory. Ingress is generally free. Consequences for ML:

• Training where the data lives is usually cheaper than moving the data to the GPUs.
• Cross-cloud architectures pay egress on every training read and every replication run, so design for one bulk copy plus incremental syncs, not repeated reads across clouds.
• Container image pulls, dataset downloads from public hubs and log shipping to a third-party observability vendor are easy to forget.
• All three providers have announced waived egress for customers moving data out to leave the cloud (on request, with conditions). That helps one-time migrations, not ongoing multi-cloud traffic.

Cloud-agnostic ML architecture

"Cloud-agnostic" should not mean "use nothing managed". It means keeping the parts that are expensive to rewrite portable, and accepting managed services where the switching cost is small or the value is large.

What to keep portable

Where managed services usually win

• Warehouses (BigQuery, Redshift, Fabric or Synapse): running your own is rarely worth it; keep data in open formats so the engine is swappable.
• Foundation model APIs (Bedrock, Gemini on Vertex AI, Foundry): frontier closed models are only available as managed APIs. Hide them behind an internal gateway or client library so the provider is a configuration choice.
• Identity, KMS, secrets and audit: always use the native service. Abstracting these usually weakens security.
• Spot and accelerator scheduling: managed training services handle capacity, retries and checkpoint plumbing that is tedious to rebuild.
• Small teams: a single managed ML platform beats a portable platform nobody has time to operate.

Portable training and serving stack

                         +----------------------------------------+
   git push ------------>|  CI (OIDC to each cloud, no keys)      |
                         |  build images, terraform/tofu plan     |
                         +-------------------+--------------------+
                                             |
   ==================== PORTABLE CORE (same code everywhere) ====================
                                             v
   +-------------------+   +-------------------------+   +-----------------------+
   | Data              |   | Training                |   | Serving               |
   | Iceberg / Delta   |-->| container image         |-->| container image       |
   | tables on object  |   | (PyTorch, JAX, sklearn) |   | (vLLM, Triton, FastAPI|
   | storage; feature  |   | run by Kubeflow / Argo  |   |  or KServe)           |
   | logic in SQL/dbt  |   | or a managed job        |   | stable HTTP/gRPC API  |
   +-------------------+   +-----------+-------------+   +-----------+-----------+
                                       |                             |
                                       v                             v
                           +-----------------------+     +-----------------------+
                           | MLflow tracking and   |     | OpenTelemetry SDK     |
                           | model registry        |     | and Collector         |
                           +-----------------------+     +-----------------------+
   ==============================================================================
                                             |
                 thin cloud-specific adapters (Terraform modules, config)
                                             |
        +--------------------------+---------+---------+--------------------------+
        v                          v                   v                          v
   AWS adapter               GCP adapter          Azure adapter          Managed extras
   S3, EKS, ECR,             GCS, GKE,            Blob/ADLS, AKS, ACR,   (opt-in, behind
   Pod Identity or IRSA,     Artifact Registry,   Entra Workload ID,     interfaces)
   KMS, Secrets Manager,     WIF for GKE, KMS,    Key Vault,             Bedrock, Gemini,
   CloudWatch                Secret Manager,      Azure Monitor          Foundry, BigQuery,
                             Cloud Monitoring                            managed endpoints

The adapters should be thin and boring: storage URIs, identity bindings, registry endpoints and exporters. If business logic leaks into an adapter, portability is gone.

Migration and multi-cloud scenarios

Migrating a SageMaker AI pipeline to Vertex AI

Assume a SageMaker Pipeline that runs a processing job, a training job, an evaluation step, registers a model with an approval status, and deploys to a real-time endpoint.

1. Inventory the contract, not the code. List inputs (S3 prefixes, Feature Store groups), outputs (model artifacts, metrics), the IAM execution role's permissions, triggers (EventBridge rules, schedules) and consumers of the endpoint.
2. Move data first. Bulk-copy training data from S3 to GCS with Storage Transfer Service, then decide whether the source of truth moves or stays (if it stays on AWS, you now have a cross-cloud data path; see the next scenario). If features live in SageMaker Feature Store, rebuild them as BigQuery tables registered with Vertex AI Feature Store.
3. Adapt the training container. SageMaker containers read channels from /opt/ml/input/data/<channel> (or SM_CHANNEL_* variables) and write the model to /opt/ml/model (SM_MODEL_DIR). Vertex AI custom training passes an output location in AIP_MODEL_DIR (a gs:// path) and exposes Cloud Storage buckets through Cloud Storage FUSE under /gcs/. Make the script take paths as arguments so both work.
4. Adapt the serving container. SageMaker expects GET /ping and POST /invocations on port 8080. Vertex AI custom containers read the port and routes from AIP_HTTP_PORT, AIP_HEALTH_ROUTE and AIP_PREDICT_ROUTE, and requests use an {"instances": [...]} envelope with {"predictions": [...]} responses. A small routing shim keeps one image working on both.
5. Rewrite the pipeline definition. SageMaker Pipelines steps become KFP v2 components and a pipeline compiled for Vertex AI Pipelines. This is a rewrite, not a translation; keep step logic in the container so the DSL layer is thin.
6. Replace governance. SageMaker Model Registry approval status has no direct equivalent: use Vertex AI Model Registry versions with aliases (for example champion) and labels, with the approval gate in CI or a pipeline step. Replace EventBridge triggers with Cloud Scheduler, Pub/Sub or Eventarc.
7. Translate identity. The SageMaker AI execution role becomes a dedicated service account on the custom job and endpoint, with roles granted on the specific bucket, dataset and Artifact Registry repository. Replace any CI access keys with Workload Identity Federation.
8. Run in parallel, then cut over. Shadow traffic or dual-run batch scoring, compare predictions and latency, then shift traffic and keep the SageMaker endpoint available for rollback until confidence is high.

Training on one cloud, serving on another

Typical reason: TPUs or reserved GPUs on GCP for training, while the product and its users run on AWS.

• Data gravity: keep training data next to the training compute. Replicate only what training needs (curated feature tables, not raw logs), on a schedule, in an open format.
• Egress: the big transfer should be the training dataset copy, and that should be incremental. Model artifacts are small by comparison; ship them once per release.
• Artifacts: export the model in a framework-neutral or serving-ready format (for example SavedModel, ONNX or safetensors weights) and register it in a shared MLflow registry or copy it to the serving cloud's object storage. Build the serving image in CI and push it to the serving cloud's registry.
• Identity federation between clouds: do not copy access keys across clouds. A GCP workload can obtain AWS credentials with AssumeRoleWithWebIdentity using a Google-issued OIDC token for its service account. In the other direction, GCP Workload Identity Federation accepts AWS identities directly, and Microsoft Entra workload identity federation accepts external OIDC issuers. For workloads without a cloud identity, AWS IAM Roles Anywhere uses X.509 certificates.
• Serving parity: feature computation at serving time on AWS must match what training used on GCP. Keep feature logic in shared code and run a skew check that compares online features against the offline training snapshot.
• Networking: for private paths use VPN or a cross-cloud interconnect product instead of the public internet if data is sensitive; check current offerings, as cross-cloud interconnect products are relatively new.

Choosing a cloud for a new ML team

Structure the answer around constraints, in this order:

1. Where the data and the company already are. Existing identity, security review, data warehouse and procurement agreements usually decide it. Fighting the company's main cloud costs more than any feature gap.
2. Accelerator access. If the roadmap needs large-scale training, ask which cloud can actually provide GPU or TPU capacity and quota in your regions, and whether the team can use Trainium or TPU software stacks.
3. Model access for GenAI. If a specific model family is required (OpenAI models, Gemini, a particular partner model), check which cloud offers it in the required regions with the needed data-residency terms.
4. Team skills. A team fluent in one cloud ships faster there.
5. Compliance and residency. Regions, certifications and sovereign cloud offerings.
6. Total cost of ownership, not list prices: engineering time, egress, idle resources and negotiated commitments.

Then keep the portable core so the decision is not irreversible.

Terraform example: one versioned bucket on three clouds

The same intent (a private object store with versioning) expressed for each provider. Provider configuration is kept minimal: in practice you would split this into per-cloud modules, use remote state and supply credentials through OIDC federation in CI. Bucket and account names must be globally unique (the Azure storage account name must be 3 to 24 lowercase letters and digits).

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
    google = {
      source  = "hashicorp/google"
      version = "~> 7.0"
    }
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.20"
    }
  }
}

# Minimal provider configuration. Credentials come from the environment
# (for example OIDC federation in CI), never from this file.
provider "aws" {
  region = "us-east-1"
}

provider "google" {
  project = "my-ml-project"
  region  = "us-central1"
}

provider "azurerm" {
  features {}
  # subscription_id is read from ARM_SUBSCRIPTION_ID if not set here.
}

# ---------- AWS: S3 bucket with versioning ----------
resource "aws_s3_bucket" "ml_artifacts" {
  bucket = "example-ml-artifacts-aws"
}

resource "aws_s3_bucket_versioning" "ml_artifacts" {
  bucket = aws_s3_bucket.ml_artifacts.id

  versioning_configuration {
    status = "Enabled"
  }
}

# ---------- GCP: Cloud Storage bucket with versioning ----------
resource "google_storage_bucket" "ml_artifacts" {
  name                        = "example-ml-artifacts-gcp"
  location                    = "US-CENTRAL1"
  uniform_bucket_level_access = true
  public_access_prevention    = "enforced"

  versioning {
    enabled = true
  }
}

# ---------- Azure: storage account + container with blob versioning ----------
resource "azurerm_resource_group" "ml" {
  name     = "rg-ml-artifacts"
  location = "eastus"
}

resource "azurerm_storage_account" "ml_artifacts" {
  name                            = "examplemlartifacts"
  resource_group_name             = azurerm_resource_group.ml.name
  location                        = azurerm_resource_group.ml.location
  account_tier                    = "Standard"
  account_replication_type        = "ZRS"
  min_tls_version                 = "TLS1_2"
  allow_nested_items_to_be_public = false

  blob_properties {
    versioning_enabled = true
  }
}

resource "azurerm_storage_container" "ml_artifacts" {
  name                  = "ml-artifacts"
  storage_account_id    = azurerm_storage_account.ml_artifacts.id
  container_access_type = "private"
}

Points worth saying in an interview:

• Versioning lives in different places. AWS uses a separate aws_s3_bucket_versioning resource (since AWS provider v4 split bucket settings out), GCP uses a block on the bucket, and Azure sets it on the storage account, so it applies to every container in that account.
• Public access defaults. New S3 buckets block public access by default. On GCP, public_access_prevention and uniform bucket-level access make IAM the only access path. On Azure, allow_nested_items_to_be_public = false stops containers from being made public.
• Azure's extra layer. The storage account is where redundancy, network rules, TLS and versioning live; the container is closer to an S3 or GCS bucket. Older azurerm examples pass storage_account_name to the container; recent 4.x releases use storage_account_id.
• Versioning costs storage. Pair it with lifecycle rules that expire noncurrent versions, or checkpoints will accumulate indefinitely.

Interview Q&A

What is the GCP equivalent of an AWS IAM role for a workload?

The closest equivalent is a service account attached to the workload: the VM, the Vertex AI custom job, the Cloud Run service, or a Kubernetes service account mapped through Workload Identity Federation for GKE. Like an AWS role used by EC2 or SageMaker AI, it gives the workload short-lived credentials from the metadata server, so there are no static keys. The difference is where permissions live. On AWS you attach policy documents to the role. On GCP you grant roles to the service account on the resources it needs (a bucket, a dataset, a project), and those grants are inherited down the hierarchy. A GCP service account is also a resource, so you control who can impersonate it, which is the rough counterpart of an AWS role trust policy. On Azure the equivalent is a managed identity plus Azure RBAC role assignments at the right scope. A common mistake is downloading service account keys on GCP "because it is easier"; organization policy can and usually should block key creation.

How do PrivateLink, Private Service Connect and Private Endpoint compare?

All three put a private IP address for a managed service (or a partner's service) inside your own network, so traffic never traverses the public internet and you can disable public access to the service. AWS PrivateLink creates interface endpoints per service per VPC, and S3 and DynamoDB also have cheaper gateway endpoints that work through route tables. GCP Private Service Connect creates an endpoint for Google APIs or for a published service; Private Google Access is a separate, simpler option that lets VMs without external IPs reach Google APIs. Azure Private Endpoint attaches a NIC for a specific resource (for example one storage account or one Azure ML workspace), which is more granular than per-service. In every case DNS is what usually breaks: the public hostname must resolve to the private IP from inside the network, and Azure in particular relies on privatelink.* private DNS zones. For ML, remember that a private workspace or endpoint also needs private paths to its dependencies (storage, registry, key vault), or jobs fail in confusing ways.

How do AWS accounts, GCP projects and Azure subscriptions map to each other?

For environment separation the usual mapping is one AWS account, one GCP project, or one Azure subscription per team per environment. They are not equally strong boundaries, though. An AWS account is a hard boundary: separate IAM, separate quotas, and cross-account access must be explicitly allowed on both sides. A GCP project is cheaper to create and is the unit for APIs, quotas and billing linkage, but IAM granted on a parent folder or the organization flows into it. An Azure subscription is the billing, quota and policy boundary, while resource groups inside it are lightweight lifecycle groupings with their own role assignments. So "one ML workspace per resource group in a per-environment subscription" on Azure is roughly analogous to "one project per environment" on GCP. The tradeoff is blast radius against overhead: more boundaries mean better isolation but more networking, IAM and quota requests to manage.

How would you let a GitHub Actions workflow deploy to all three clouds without long-lived keys?

Use OIDC federation on each cloud. GitHub issues a signed OIDC token per workflow run that includes claims such as repository, branch and environment. On AWS you register GitHub as an IAM OIDC identity provider and create a role whose trust policy restricts the sub claim; the workflow calls AssumeRoleWithWebIdentity. On GCP you create a workload identity pool and provider with an attribute condition on the repository, then either grant roles directly to the federated principal or let it impersonate a service account. On Azure you add a federated credential to an app registration or user-assigned managed identity matching the repository and environment. The benefits are no secrets to rotate or leak and a clear audit trail per run. The main risk is overly broad trust conditions, such as trusting every repository in an organization or every branch, which would let a pull request from a fork or a feature branch deploy to production.

When is multi-cloud worth it for ML, and when is it a trap?

It is worth it when there is a concrete reason that a single cloud cannot satisfy: accelerator capacity or a specific chip (for example TPUs) available only elsewhere, a required foundation model only offered on another cloud, customers or regulators requiring deployment in their cloud, or an acquisition that brought a second cloud with it. It is a trap when the motivation is vague "avoid lock-in" or "negotiate leverage" for a team that has not yet mastered one cloud. The costs are real: duplicate identity and network setups, egress on every cross-cloud data path, two sets of on-call knowledge, lowest-common-denominator tooling and slower delivery. A middle ground that usually works is a portable core (containers, Terraform, MLflow, open table formats) on one primary cloud, with a second cloud used for one well-bounded job such as training or a specific model API. I would also separate "multi-cloud capable" (could move in months) from "active multi-cloud" (runs on two clouds today), because the second is much more expensive.

How would you design a feature pipeline that could move between clouds?

Separate feature definitions from feature infrastructure. Write the transformations in SQL (for example dbt models) or Python in a framework that runs on several engines (Spark, Beam or DuckDB for small data), and keep them in version control with tests. Store offline features as Iceberg or Delta tables on object storage, so BigQuery, Redshift, Spark on Databricks or Fabric can all read them. For online serving, put a thin interface in front of the key-value store (DynamoDB, Bigtable, Cosmos DB or Redis) so the serving code calls get_features(entity_id) and not a cloud SDK; an open-source feature store such as Feast can provide that layer. Orchestrate with Airflow or another portable scheduler, and keep cloud-specific connection details in configuration. The tradeoff is that you give up some managed conveniences, such as native BigQuery-backed feature serving in Vertex AI Feature Store, so do this only if moving clouds is a realistic requirement.

Why might the same training job cost very differently across clouds?

Several factors differ even when the GPU model is identical. Instance shapes bundle different amounts of CPU, memory, local disk and network bandwidth with the same GPU, and some shapes only come in large multi-GPU sizes. Spot availability and interruption rates differ by cloud, region and time, and frequent interruptions without good checkpointing waste paid compute. Storage throughput matters: if the data loader cannot keep GPUs busy, you pay for idle accelerators, and the fast-storage options (Lustre-based services, local NVMe caching) differ. Data location adds egress if the data lives on another cloud or region. Managed ML services price their instances separately from raw VMs, and discount instruments (savings plans, committed use discounts, reservations) have different scopes and terms. Finally, switching hardware (for example to TPUs or Trainium) can change cost a lot in either direction, but only after engineering work to port and tune the model. I would compare cost per completed training run, including failed and restarted attempts, not hourly instance price.

What is the Azure equivalent of BigQuery, and why is the answer not a single service?

The concept is a managed columnar warehouse with separated storage and compute. On Azure that can be a Microsoft Fabric Warehouse or Lakehouse, Azure Synapse Analytics (dedicated SQL pools are provisioned, serverless SQL pools query the lake on demand), or Databricks SQL on Azure Databricks. Microsoft is steering new analytics work toward Fabric, but many enterprises still run Synapse or standardize on Databricks. BigQuery differs from most of these in being serverless by default: there is no cluster to size, and cost scales with data scanned or reserved slots. That changes ML habits, because exploration is easy but careless queries are expensive, so partitioning, clustering and query cost controls matter. In an interview I would ask which data platform the company uses before mapping, and then talk about where features are computed and how the ML platform reads them.

How does Pub/Sub differ from Kinesis Data Streams and Event Hubs for ML event pipelines?

Kinesis Data Streams and Event Hubs are partitioned logs: producers write to shards or partitions, ordering is per partition, and consumers track their own position and can replay within the retention window. Pub/Sub is a topic-and-subscription service: each subscription gets every message, delivery is tracked per message with acknowledgements, ordering is opt-in through ordering keys, and replay uses seek to a timestamp or snapshot when retention is configured. For ML this changes how you build things like feature backfills and exactly-once processing. Log semantics make "reprocess the last day from offset X" natural, while Pub/Sub is simpler to scale and operate with no shard management. If portability matters, managed Kafka exists on all three clouds (Amazon MSK, Managed Service for Apache Kafka on GCP, and the Kafka endpoint on Event Hubs), which keeps the client code and semantics consistent.

If you know Amazon Bedrock, how would you explain Vertex AI and Microsoft Foundry to a team?

All three are managed APIs for foundation models with similar building blocks: a model catalog, pay-per-token and provisioned throughput, fine-tuning, managed RAG, agent services, guardrails and evaluation. On Google Cloud, Gemini is the first-party model family accessed through Vertex AI, and Model Garden adds partner and open models that can be called as APIs or deployed to your own endpoints. On Azure, Microsoft Foundry (formerly Azure AI Foundry) is the umbrella for model access, including Azure OpenAI models, plus agents, evaluation and content safety. The practical differences are which models are available in which regions, data residency terms, quota models and how identity and networking integrate (IAM roles, service accounts, Entra ID and private endpoints). Agent and RAG product names change frequently, so I would map them by function. For portability, put an internal client or gateway in front of all providers and keep prompts, evaluation sets and guardrail policies in your own repository.

How do you prevent data exfiltration from an ML environment on each cloud?

Combine identity, network and organization controls, because any one layer alone has gaps. On GCP, VPC Service Controls is the most direct tool: it draws a perimeter around projects so that even a valid credential cannot copy data from BigQuery or Cloud Storage to a project outside the perimeter. On AWS you build a data perimeter from SCPs and RCPs that restrict access to trusted identities and resources, VPC endpoint policies that limit which buckets can be reached from inside the network, and bucket policies that require access through your endpoints. On Azure you disable public network access on storage and workspaces, use private endpoints, restrict outbound traffic (Azure ML managed virtual networks support an outbound mode that allows only approved destinations), and enforce it with Azure Policy. On all three, notebooks with open internet egress are the weak point, so restrict outbound traffic and package mirrors as well. The tradeoff is developer friction: package installs and model downloads need approved mirrors or allow-lists.

How would you serve a model trained on TPUs to users whose application runs on AWS?

First, make sure the model artifact is not tied to the TPU: JAX or PyTorch/XLA training can export weights in a standard format, and the serving stack on AWS will run them on GPUs or CPUs (or on Inferentia after compiling with the Neuron SDK). I would build the serving image in CI and push it to Amazon ECR, then deploy to a SageMaker AI endpoint or to EKS with an open model server. The artifact moves once per release from GCS to S3, using a pipeline step whose identity is federated (no long-lived keys). Online features must be computed on AWS with the same logic used for training on GCP, so I would keep feature code shared and run a skew check. Monitoring data flows back to GCP for retraining in batches, not per request, to limit egress. The tradeoff is operating two clouds for one model; it is worth it if TPU access substantially lowers training cost or time.

What breaks first when you lift a SageMaker AI training script onto Vertex AI or Azure ML?

Usually the container contract, not the model code. SageMaker AI injects channel paths and the model output directory through /opt/ml/... paths and SM_* environment variables, and uploads /opt/ml/model to S3 at the end. Vertex AI custom training passes AIP_MODEL_DIR and mounts buckets through Cloud Storage FUSE under /gcs/, while Azure ML passes inputs and outputs as command arguments bound to datastore URIs. Next come credentials and SDK calls (boto3 calls to S3 or Secrets Manager inside the script), distributed training environment setup, and checkpoint locations for spot recovery. The fix is to make the script take every path and secret as arguments or environment variables, read and write through fsspec-style URIs or mounted paths, and keep cloud SDK calls out of training code. Then a thin launcher per cloud maps the platform's contract onto the script's arguments.

How do spot capacity semantics differ, and how do you make training robust to them?

All three clouds sell interruptible capacity at a discount with a short warning before reclaim: EC2 Spot gives a two-minute notice, while GCP Spot VMs and Azure Spot VMs give a much shorter notice (around 30 seconds). GCP Spot VMs have no maximum runtime, unlike the older preemptible VMs, and Azure lets you choose eviction behavior and a maximum price. Managed ML services wrap this differently: SageMaker AI Managed Spot Training syncs a checkpoint directory to S3 and resumes, Vertex AI custom training can run on Spot VMs, and Azure ML compute clusters offer a low-priority tier. Regardless of cloud, the job must checkpoint to durable storage frequently enough that losing the time since the last checkpoint is acceptable, and resume idempotently, including optimizer state and data loader position. For multi-node training, one interrupted node stops the whole job, so elastic training or smaller jobs reduce wasted compute. I would use spot for fault-tolerant training and hyperparameter search, and on-demand or reserved capacity for latency-sensitive serving.

How do you keep cost attribution consistent across three clouds?

Define one tagging schema (for example team, env, project, cost_center, model) and enforce it with each cloud's policy engine: SCPs or tag policies on AWS, organization policies and required labels in Terraform on GCP, and Azure Policy on Azure. Remember the naming differences: AWS tags must be activated as cost allocation tags, GCP uses labels for billing and separate tags for policy, and Azure tags are not inherited by default. Export detailed billing data from each cloud (Cost and Usage Report data exports, BigQuery billing export, Cost Management exports) into one place, ideally normalised to the FOCUS format, and build dashboards on that. ML-specific gaps include shared GPU clusters, where you need Kubernetes-level allocation by namespace or label, and managed services that create untagged child resources. Enforce tags in CI through Terraform modules, because tags added by hand drift quickly.

Common Pitfalls

Related Topics