Cracking ML Interviews

Open in the interactive study hub

Python for AI Engineering

The foundation under every LLM app and agent. Before RAG, tools, or multi-agent orchestration, you need fluent, idiomatic, production Python. This guide covers the slice of Python that AI engineers use daily, and that interviewers expect you to write without thinking.

Why it's listed first in the roadmap: agent code is concurrent, schema-driven, API-heavy, and failure-prone. Weak Python here shows up immediately as flaky, slow, unobservable agents.

Table of Contents

  1. Core Python You Actually Use
  2. Type Hints
  3. async / await & Concurrency
  4. Pydantic Models
  5. Environment & Dependency Management
  6. Working with APIs
  7. Error Handling & Retries
  8. Logging & Observability
  9. Project Hygiene
  10. Interview Questions

1. Core Python You Actually Use

Feature Why it matters in AI code
Comprehensions / generators Stream large datasets/embeddings without loading everything into memory
Context managers (with) Manage clients, files, DB sessions, spans cleanly
Decorators Wrap functions with retries, caching, tracing, timing
dataclasses / Enum Lightweight typed records and fixed choice sets
functools (lru_cache, partial) Cache deterministic calls; pre-bind config
pathlib Safe, cross-platform path handling (and traversal checks) 
from functools import lru_cache

@lru_cache(maxsize=1024)
def embed_cached(text: str) -> tuple[float, ...]:
    """Cache deterministic embeddings so repeated text isn't re-embedded."""
    return tuple(embed(text))

2. Type Hints

Type hints make agent code maintainable and power schema generation (Pydantic, tool definitions) and editor/static checks.

from typing import Literal, Optional, Any

def call_model(
    prompt: str,
    model: str = "claude-sonnet-5",
    temperature: Optional[float] = None,  # None = omit; current Claude models reject non-default values
    tools: Optional[list[dict[str, Any]]] = None,
    role: Literal["system", "user", "assistant"] = "user",
) -> str:
    ...

3. async / await & Concurrency

Agents make many network calls. Doing them sequentially is the #1 latency sink. asyncio lets independent calls run concurrently.

import asyncio

async def fetch_tool(call):
    return await run_tool_async(call)

async def run_parallel_tools(tool_calls):
    # Execute independent tool calls concurrently, gather all results
    return await asyncio.gather(*(fetch_tool(c) for c in tool_calls))

sem = asyncio.Semaphore(10)  # respect provider rate limits
async def guarded_call(req):
    async with sem:
        return await client_async.call(req)

4. Pydantic Models

Pydantic is the backbone of structured outputs, tool schemas, and config. It validates and coerces data at the boundary so the rest of your code trusts its types.

from pydantic import BaseModel, Field

class SearchArgs(BaseModel):
    query: str = Field(description="Search query")
    top_k: int = Field(default=5, ge=1, le=50)

class Settings(BaseModel):
    model: str = "claude-opus-5-5"
    temperature: float | None = None  # leave unset: Claude Opus 4.7+ and Sonnet 5 reject non-default sampling params
    max_tokens: int = 4096

→ Deeper: Pydantic for AI Systems · Structured Outputs

5. Environment & Dependency Management

Tool Use
venv Built-in isolated environments
uv Fast installer + resolver + lockfile (modern default)
poetry / pip-tools Dependency management + locking
.env + python-dotenv / pydantic-settings Load secrets/config from env 
# config.py: never hardcode keys
from pydantic_settings import BaseSettings, SettingsConfigDict

class AppConfig(BaseSettings):
    # v2 style; the inner `class Config:` form is deprecated
    model_config = SettingsConfigDict(env_file=".env")

    anthropic_api_key: str
    vector_db_url: str = "http://localhost:6333"

config = AppConfig()  # reads from environment / .env

6. Working with APIs

import httpx

async def post_json(url: str, payload: dict, timeout: float = 30.0) -> dict:
    async with httpx.AsyncClient(timeout=timeout) as client:
        resp = await client.post(url, json=payload)
        resp.raise_for_status()
        return resp.json()

→ Production HTTP/API patterns: FastAPI for AI Backends

7. Error Handling & Retries

Networks fail, providers rate-limit, and models occasionally return junk. Handle it.

import time, random

def with_retry(fn, *, retries=5, base=1.0, max_delay=30.0):
    for attempt in range(retries):
        try:
            return fn()
        except RateLimitError:          # 429: retryable
            pass
        except ServerError:             # 5xx: retryable
            pass
        except BadRequestError:         # 4xx: do NOT retry
            raise
        delay = min(base * 2 ** attempt + random.uniform(0, 1), max_delay)
        time.sleep(delay)
    raise RuntimeError("exhausted retries")

8. Logging & Observability

Agents are non-deterministic: logs are how you debug them.

import logging, json, uuid

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("agent")

def log_event(event: str, **fields):
    """Structured log line with a correlation id for tracing a run."""
    log.info(json.dumps({"event": event, **fields}))

run_id = str(uuid.uuid4())
log_event("tool_call", run_id=run_id, tool="search", args={"query": "..."})

→ Deeper: Observability & Tracing · LLMOps

9. Project Hygiene

10. Interview Questions

  1. When do you reach for async in an AI app? → I/O-bound concurrency: parallel model/tool/DB calls. Use asyncio.gather, cap with a semaphore for rate limits; don't use it for CPU-bound work.
  2. Why Pydantic in an LLM pipeline? → Validate/coerce model output and tool args at the boundary, auto-generate JSON schemas, and load typed config.
  3. How do you handle a 429 vs a 400? → Retry 429 (and 5xx) with exponential backoff + jitter; never retry 400: it's a client error, fix the request.
  4. How do you make a non-deterministic agent debuggable? → Structured logs with a per-run correlation id, log every model/tool call with tokens/latency, and end-to-end tracing.
  5. How do you keep secrets out of code? → .env (git-ignored) + a settings loader locally, secrets manager / CI secrets in production; never hardcode.
  6. Sync vs async SDK client: does it matter? → Yes: calling a blocking sync client inside an async handler stalls the event loop; use the async client or offload to an executor.

View or edit this guide on GitHub