diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml new file mode 100644 index 0000000..1467bec --- /dev/null +++ b/.github/workflows/check.yml @@ -0,0 +1,20 @@ +name: Check briefs + +on: + pull_request: + workflow_dispatch: + +permissions: + contents: read + +jobs: + check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-python@v6 + with: + python-version: '3.12' + - run: python scripts/lint_briefs.py + - run: python -m pip install -r requirements-docs.txt + - run: zensical build --clean --strict diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 0225c99..2e5b2c0 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -25,6 +25,7 @@ jobs: - uses: actions/setup-python@v6 with: python-version: '3.12' + - run: python scripts/lint_briefs.py - run: python -m pip install -r requirements-docs.txt - run: zensical build --clean --strict - uses: actions/configure-pages@v6 diff --git a/AGENTS.md b/AGENTS.md index 3cb6d82..b8e29f4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,9 +6,11 @@ This repo is a set of concise topic briefs for technical interview prep. The rea - `docs/topics/` — one brief per topic (e.g. `docs/topics/golang.md`). - `docs/index.md` — the reading site's homepage. +- `scripts/lint_briefs.py` — checks the format rules below; CI runs it with the strict site build on every pull request. - `.agents/skills/quiz/` — the `quiz` skill, which quizzes the user on brief concepts. `.claude/skills/quiz` is a symlink to it so Claude Code and Codex share one copy. - File names are lowercase, single word or kebab-case. - `README.md` and `docs/index.md` list every topic with a link and a one-line summary, and `zensical.toml` lists it in the site nav. Update all three when adding or renaming a file. +- Topics are grouped the same way in all three: **Fundamentals**, **Languages**, **Web**, **Infrastructure**, **Domains** — groups ordered by breadth, topics alphabetical within a group. ## Target format @@ -41,12 +43,12 @@ What experienced Go engineers forget before an interview, grouped by subtopic. - **Title**: `# ` (e.g. `# Go`, `# Distributed Systems`). No "Cheatsheet" suffix. - **Intro**: one line: "What experienced engineers forget before an interview, grouped by subtopic." Adjust wording for non-language topics (e.g. "Key system design building blocks and trade-offs, grouped by subtopic."). -- **`##` = subtopic** (Concurrency, Memory management, …). Named, not numbered. Ordered by how often the area comes up in interviews. -- **`###` = concept** — a noun phrase (`### Scheduler (GMP)`, `### Isolation anomalies`), never a question, no numbers. Each concept should be self-contained: the quiz picks `###` headings at random. +- **`##` = subtopic** (Concurrency, Memory management, …). Named, not numbered. Ordered by how often the area comes up in interviews. At least 2 concepts each; fold a lone concept into a related subtopic. +- **`###` = concept** — a noun phrase (`### Scheduler (GMP)`, `### Isolation anomalies`), never a question, no numbers. Each concept should be self-contained (no "as above"): the quiz picks `###` headings at random. - **Body of a concept**: 2–6 bullets, or one short paragraph, or one compact table. One fact per bullet, one line where possible. - **Tables** for comparisons and matrices (channel axioms, isolation levels vs anomalies, deployment strategies, promise combinators). - **Code** only when shorter than prose; ≤ 6 lines. -- **Bold** the term or number worth memorizing — at most 2–3 per concept. +- **Bold** the term or number worth memorizing — at most 3 per concept, not counting tables. - **Versions**: note when behavior changed ("since Go 1.22", "Python 3.14+", "Pectra, May 2025"). - **Cross-topic**: never repeat a concept owned by another file; link to it instead: `see [security.md](security.md)`. Link to files, not anchors (the strict build validates anchors; only use one if you've checked the slug). - Suggested size: 5–10 sections, 3–8 concepts each. Shorter is better if nothing is lost. diff --git a/README.md b/README.md index eb6e769..13fed7b 100644 --- a/README.md +++ b/README.md @@ -8,24 +8,38 @@ Each topic in `docs/topics/` is a brief grouped by subtopic, covering what an ex ## Topics -- [Go](docs/topics/golang.md) — concurrency, memory management, slices/maps/strings, types and interfaces, errors, standard library gotchas, modules, testing and profiling -- [Algorithms](docs/topics/algorithms.md) — complexity, array and string patterns, sorting, data structures, graphs, dynamic programming, greedy methods, NP-hardness -- [Distributed Systems](docs/topics/distributed.md) — theory, consistency models, replication, partitioning, consensus, clocks and IDs, transactions, conflict resolution, failure handling, Kafka and stream processing -- [Ethereum](docs/topics/ethereum.md) — accounts and transactions, gas, EVM, security, upgradeability, signatures, PoS consensus, scaling, DeFi and MEV -- [System Design](docs/topics/system.md) — interview framework, traffic and edge, real-time delivery, caching, data storage, async processing, rate limiting, reliability, architecture, classic designs +### Fundamentals + +- [Algorithms](docs/topics/algorithms.md) — complexity, arrays and sequences, strings, sorting, data structures, graphs, dynamic programming, greedy and backtracking, NP-hardness - [Databases](docs/topics/database.md) — indexes, isolation, MVCC and locking, storage engines, query tuning, SQL gotchas, Redis and NoSQL, schema design, scaling -- [Networking](docs/topics/networking.md) — TCP, HTTP/1.1 vs 2 vs 3, QUIC, DNS, IP addressing, NAT, MTU, BGP and anycast -- [Operating Systems](docs/topics/os.md) — processes and signals, scheduling and cgroup throttling, virtual memory and OOM, file descriptors, epoll and io_uring, fsync, synchronization, perf tools -- [AI Engineering](docs/topics/ai.md) — LLM fundamentals, inference and serving, RAG, fine-tuning, agents, tools and MCP, evaluation, cost, security -- [DevOps](docs/topics/devops.md) — deployment strategies, containers, Kubernetes workloads/networking/operations, Terraform, GitOps -- [AWS](docs/topics/aws.md) — IAM and KMS, VPC networking, compute, storage and databases, events, operations -- [Security](docs/topics/security.md) — web vulnerabilities, auth and sessions, tokens and OAuth, access control, cryptography, TLS, supply chain, threat modeling +- [Distributed Systems](docs/topics/distributed.md) — theory, consistency models, replication, partitioning, consensus, clocks and IDs, transactions, conflict resolution, failure handling, Kafka and stream processing +- [Networking](docs/topics/networking.md) — TCP, keepalive and congestion control, HTTP/1.1 vs 2 vs 3, QUIC, DNS, IP addressing, NAT, MTU, BGP and anycast +- [Operating Systems](docs/topics/os.md) — processes and signals, scheduling and cgroup throttling, virtual memory, OOM and cgroup limits, NUMA, file descriptors, epoll and io_uring, fsync, synchronization, perf tools +- [System Design](docs/topics/system.md) — interview framework, load balancing, caching and consistency, storage choices, queues, rate limiting, reliability, architecture, probabilistic and geospatial structures, classic designs + +### Languages + +- [Go](docs/topics/golang.md) — concurrency, memory management, slices/maps/strings, types and interfaces, errors, standard library gotchas, modules and GODEBUG, testing, profiling and PGO +- [JavaScript](docs/topics/javascript.md) — event loop, scope and closures, `this` and prototypes, numbers, async, modules, memory, metaprogramming, Node.js runtime, DOM events, recent additions +- [Python](docs/topics/python.md) — runtime and the GIL, memory management, data structures, concurrency, gotchas, object model, functions, recent features, typing, tooling - [Rust](docs/topics/rust.md) — ownership and borrowing, lifetimes, traits, closures and iterators, smart pointers, error handling, concurrency, async, unsafe, macros and Cargo -- [Python](docs/topics/python.md) — runtime and the GIL, memory management, data structures, concurrency, gotchas, object model, typing, tooling -- [TypeScript](docs/topics/typescript.md) — type system semantics, narrowing, type-level programming, compilation, strictness, declarations, typing patterns -- [JavaScript](docs/topics/javascript.md) — event loop, scope and closures, `this` and prototypes, numbers, async, modules, memory, DOM events, recent additions -- [Backend](docs/topics/backend.md) — API design, idempotency and concurrency, service calls and webhooks, async work, data access, lifecycle, security, testing -- [Frontend](docs/topics/frontend.md) — rendering strategies, browser pipeline, performance, caching, React internals and concurrency, state management, accessibility +- [TypeScript](docs/topics/typescript.md) — type system semantics, narrowing, type-level programming, compilation and TypeScript 6/7, strictness, declarations, typing patterns + +### Web + +- [Backend](docs/topics/backend.md) — API design, idempotency and concurrency, service calls and webhooks, async work, data access, lifecycle and health, security, testing +- [Frontend](docs/topics/frontend.md) — rendering strategies, browser pipeline, CSS, performance, caching, React rendering and hooks, state management, accessibility +- [Security](docs/topics/security.md) — server-side and browser-side attacks, auth and sessions, tokens and OAuth, authorization, cryptography, TLS, supply chain, threat modeling + +### Infrastructure + +- [AWS](docs/topics/aws.md) — IAM and KMS, VPC networking, compute, storage and databases, events, operations +- [DevOps](docs/topics/devops.md) — deployment strategies, containers, Kubernetes workloads/storage/networking/operations, Terraform, CI/CD and GitOps + +### Domains + +- [AI Engineering](docs/topics/ai.md) — LLM fundamentals, attention and MoE, reasoning models, inference and serving, RAG, fine-tuning, agents, tools and MCP, evaluation, cost, security +- [Ethereum](docs/topics/ethereum.md) — accounts and transactions, gas, EVM, Solidity, calls and ABI, security, upgradeability, signatures and standards, PoS consensus, scaling, DeFi and MEV ## Quiz diff --git a/docs/index.md b/docs/index.md index 855fbbb..6c5e637 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,24 +4,38 @@ Topic briefs on what engineers forget before technical interviews, grouped by su ## Topics -- [Go](topics/golang.md) — concurrency, memory management, slices/maps/strings, types and interfaces, errors, standard library gotchas, modules, testing and profiling -- [Algorithms](topics/algorithms.md) — complexity, array and string patterns, sorting, data structures, graphs, dynamic programming, greedy methods, NP-hardness -- [Distributed Systems](topics/distributed.md) — theory, consistency models, replication, partitioning, consensus, clocks and IDs, transactions, conflict resolution, failure handling, Kafka and stream processing -- [Ethereum](topics/ethereum.md) — accounts and transactions, gas, EVM, security, upgradeability, signatures, PoS consensus, scaling, DeFi and MEV -- [System Design](topics/system.md) — interview framework, traffic and edge, real-time delivery, caching, data storage, async processing, rate limiting, reliability, architecture, classic designs +### Fundamentals + +- [Algorithms](topics/algorithms.md) — complexity, arrays and sequences, strings, sorting, data structures, graphs, dynamic programming, greedy and backtracking, NP-hardness - [Databases](topics/database.md) — indexes, isolation, MVCC and locking, storage engines, query tuning, SQL gotchas, Redis and NoSQL, schema design, scaling -- [Networking](topics/networking.md) — TCP, HTTP/1.1 vs 2 vs 3, QUIC, DNS, IP addressing, NAT, MTU, BGP and anycast -- [Operating Systems](topics/os.md) — processes and signals, scheduling and cgroup throttling, virtual memory and OOM, file descriptors, epoll and io_uring, fsync, synchronization, perf tools -- [AI Engineering](topics/ai.md) — LLM fundamentals, inference and serving, RAG, fine-tuning, agents, tools and MCP, evaluation, cost, security -- [DevOps](topics/devops.md) — deployment strategies, containers, Kubernetes workloads/networking/operations, Terraform, GitOps -- [AWS](topics/aws.md) — IAM and KMS, VPC networking, compute, storage and databases, events, operations -- [Security](topics/security.md) — web vulnerabilities, auth and sessions, tokens and OAuth, access control, cryptography, TLS, supply chain, threat modeling +- [Distributed Systems](topics/distributed.md) — theory, consistency models, replication, partitioning, consensus, clocks and IDs, transactions, conflict resolution, failure handling, Kafka and stream processing +- [Networking](topics/networking.md) — TCP, keepalive and congestion control, HTTP/1.1 vs 2 vs 3, QUIC, DNS, IP addressing, NAT, MTU, BGP and anycast +- [Operating Systems](topics/os.md) — processes and signals, scheduling and cgroup throttling, virtual memory, OOM and cgroup limits, NUMA, file descriptors, epoll and io_uring, fsync, synchronization, perf tools +- [System Design](topics/system.md) — interview framework, load balancing, caching and consistency, storage choices, queues, rate limiting, reliability, architecture, probabilistic and geospatial structures, classic designs + +### Languages + +- [Go](topics/golang.md) — concurrency, memory management, slices/maps/strings, types and interfaces, errors, standard library gotchas, modules and GODEBUG, testing, profiling and PGO +- [JavaScript](topics/javascript.md) — event loop, scope and closures, `this` and prototypes, numbers, async, modules, memory, metaprogramming, Node.js runtime, DOM events, recent additions +- [Python](topics/python.md) — runtime and the GIL, memory management, data structures, concurrency, gotchas, object model, functions, recent features, typing, tooling - [Rust](topics/rust.md) — ownership and borrowing, lifetimes, traits, closures and iterators, smart pointers, error handling, concurrency, async, unsafe, macros and Cargo -- [Python](topics/python.md) — runtime and the GIL, memory management, data structures, concurrency, gotchas, object model, typing, tooling -- [TypeScript](topics/typescript.md) — type system semantics, narrowing, type-level programming, compilation, strictness, declarations, typing patterns -- [JavaScript](topics/javascript.md) — event loop, scope and closures, `this` and prototypes, numbers, async, modules, memory, DOM events, recent additions -- [Backend](topics/backend.md) — API design, idempotency and concurrency, service calls and webhooks, async work, data access, lifecycle, security, testing -- [Frontend](topics/frontend.md) — rendering strategies, browser pipeline, performance, caching, React internals and concurrency, state management, accessibility +- [TypeScript](topics/typescript.md) — type system semantics, narrowing, type-level programming, compilation and TypeScript 6/7, strictness, declarations, typing patterns + +### Web + +- [Backend](topics/backend.md) — API design, idempotency and concurrency, service calls and webhooks, async work, data access, lifecycle and health, security, testing +- [Frontend](topics/frontend.md) — rendering strategies, browser pipeline, CSS, performance, caching, React rendering and hooks, state management, accessibility +- [Security](topics/security.md) — server-side and browser-side attacks, auth and sessions, tokens and OAuth, authorization, cryptography, TLS, supply chain, threat modeling + +### Infrastructure + +- [AWS](topics/aws.md) — IAM and KMS, VPC networking, compute, storage and databases, events, operations +- [DevOps](topics/devops.md) — deployment strategies, containers, Kubernetes workloads/storage/networking/operations, Terraform, CI/CD and GitOps + +### Domains + +- [AI Engineering](topics/ai.md) — LLM fundamentals, attention and MoE, reasoning models, inference and serving, RAG, fine-tuning, agents, tools and MCP, evaluation, cost, security +- [Ethereum](topics/ethereum.md) — accounts and transactions, gas, EVM, Solidity, calls and ABI, security, upgradeability, signatures and standards, PoS consensus, scaling, DeFi and MEV ## Quiz diff --git a/docs/topics/ai.md b/docs/topics/ai.md index 7298773..b8fd725 100644 --- a/docs/topics/ai.md +++ b/docs/topics/ai.md @@ -9,9 +9,26 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F - Models read **tokens** (subword pieces from BPE-style tokenizers): roughly **4 characters or ¾ of a word** of English per token; code and non-English text use more. - The context window holds input **and** output tokens; price and latency scale with both, and output tokens cost several times more than input. +### Attention cost + +- Self-attention compares every token with every other: prefill compute grows **O(n²)** with sequence length; each decoded token attends to all cached ones. +- **GQA/MQA** share key/value heads across query heads, shrinking the KV cache several-fold. +- **FlashAttention** computes exact attention in tiles held in on-chip memory — same result, far less memory traffic. + +### Mixture of experts + +- An **MoE** layer routes each token to a few of many expert feed-forward networks, so **active parameters** per token are a fraction of the total. +- Compute follows active parameters, but GPUs must hold **all** weights; uneven routing complicates serving. + +### Reasoning models + +- Reasoning models spend **test-time compute**: they generate a chain of thought before answering, trading latency and cost for accuracy on math, code, and planning. +- **Thinking tokens are billed as output** and count against the context window; APIs expose a thinking budget or effort level. +- Prompt them with goals and constraints — "think step by step" is already built in. + ### Sampling parameters -- **Temperature** scales logits before softmax: low → focused and repeatable, high → diverse. **Top-p** (nucleus) samples from the smallest set covering probability `p`; **top-k** from the `k` likeliest tokens. +- **Temperature** scales logits before softmax: low → focused and repeatable, high → diverse. **Top-p** (nucleus) samples from the smallest set covering probability `p`; top-k from the `k` likeliest tokens. - Temperature 0 is near-greedy but **not guaranteed deterministic** — batching and floating-point order change results. ### Long-context behavior @@ -21,7 +38,8 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F ### Structured output -- **Constrained decoding** masks tokens so output always matches a JSON schema or grammar — it guarantees **shape, not correctness**; still validate values in code. +- **Constrained decoding** masks tokens so output always matches a JSON schema or grammar. +- It guarantees **shape, not correctness** — still validate values in code. ### Tool calling @@ -32,8 +50,8 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F ### Prefill vs decode -- **Prefill** processes the whole prompt in parallel — compute-bound, drives **time to first token (TTFT)**. -- **Decode** emits one token at a time — memory-bandwidth-bound, drives **time per output token (TPOT)**; total latency ≈ TTFT + TPOT × output tokens. +- Prefill processes the whole prompt in parallel — compute-bound, drives **time to first token (TTFT)**. +- Decode emits one token at a time — memory-bandwidth-bound, drives **time per output token (TPOT)**; total latency ≈ TTFT + TPOT × output tokens. ### KV cache @@ -47,7 +65,8 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F ### Speculative decoding -- A small **draft model** proposes several tokens; the large model verifies them in one pass and keeps the accepted prefix — faster, with the **same output distribution**. +- A small **draft model** proposes several tokens; the large model verifies them in one pass and keeps the accepted prefix. +- Faster decode with the **same output distribution**; the gain depends on how often drafts are accepted. ### Quantization @@ -83,8 +102,8 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F ### Hybrid search and reranking -- **BM25** (lexical) catches exact names and IDs; **dense vectors** catch paraphrases; merge with **reciprocal rank fusion**. -- A **cross-encoder reranker** rescores a small candidate set — but can't recover a passage that was never retrieved, so tune **recall@k** first. +- **BM25** (lexical) catches exact names and IDs; dense vectors catch paraphrases; merge with **reciprocal rank fusion**. +- A cross-encoder reranker rescores a small candidate set — but can't recover a passage that was never retrieved, so tune **recall@k** first. ### Permission-aware retrieval @@ -133,7 +152,8 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F ### Context management -- Long agent runs overflow the window: **compact** old turns into summaries, keep tool outputs short, and store bulky state in files or memory the agent can re-read. +- Long agent runs overflow the window: **compact** old turns into summaries and keep tool outputs short. +- Store bulky state in files or memory the agent can re-read; **subagents** keep exploratory work out of the main context. ## Tools and integration @@ -144,12 +164,13 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F ### Model Context Protocol -- **MCP** standardizes how apps discover and call **tools**, read **resources**, and fetch **prompts** — JSON-RPC 2.0 over **stdio** (local) or **Streamable HTTP** (remote, replaced HTTP+SSE in 2025). +- **MCP** standardizes how apps discover and call tools, read resources, and fetch prompts — JSON-RPC 2.0 over stdio (local) or **Streamable HTTP** (remote, replaced HTTP+SSE in 2025). - Remote servers authorize with **OAuth 2.1**; MCP doesn't replace per-action authorization or business validation. ### Browser automation as a fallback -- Use it only without an API — layouts change and clicks have ambiguous effects; isolate its credentials and require approval before irreversible submits. +- Use it only without an API — layouts change and clicks have ambiguous effects. +- Isolate its credentials and require **approval** before irreversible submits. ## Evaluation @@ -170,7 +191,8 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F ### Tracing -- One trace per run across model calls, retrieval, tool calls, approvals, and writes: latency, tokens, cost, errors, and whether a side effect already happened. +- **One trace per run** across model calls, retrieval, tool calls, approvals, and writes. +- Record latency, tokens, cost, errors, and whether a side effect already happened — so a retry doesn't repeat it. ## Cost and latency @@ -180,6 +202,11 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F - **Semantic caching** reuses answers to similar queries; its key must include user permissions, source version, and freshness, or it leaks. - Measure cost **per completed task**, not per call. +### Batch and streaming + +- **Batch APIs** run requests asynchronously (results within ~24 h) at about **half price** — for evals, backfills, and offline labeling. +- **Streaming** doesn't cut total time but shows the first tokens at TTFT, which is what users perceive. + ## Security and oversight ### Prompt injection @@ -189,7 +216,8 @@ What experienced AI engineers forget before an interview, grouped by subtopic. F ### Lethal trifecta -- **Private data access + untrusted content + an exfiltration channel** in one agent enables data theft; removing any one of the three closes most of the risk. +- **Private data access + untrusted content + an exfiltration channel** in one agent enables data theft. +- Removing any one of the three closes most of the risk — e.g. no outbound network from an agent that reads private data. ### Human approval gates diff --git a/docs/topics/algorithms.md b/docs/topics/algorithms.md index 0b3b927..8207d93 100644 --- a/docs/topics/algorithms.md +++ b/docs/topics/algorithms.md @@ -6,67 +6,71 @@ What experienced engineers forget before an algorithms interview, grouped by sub ### Time and space bounds -- State the **input size**, then the worst-case time and **auxiliary space**; an output of size `k` already costs Ω(k) time to produce. -- Nested loops are not automatically O(n²): if two pointers each advance at most `n` times, total work is O(n). -- A recursive algorithm's stack counts toward auxiliary space; balanced divide-and-conquer recursion uses O(log n) stack frames, a linear chain uses O(n). +- State the **input size**, then worst-case time and **auxiliary space**; producing an output of size `k` already costs Ω(k). +- Nested loops aren't automatically O(n²): if two pointers each advance at most `n` times, total work is O(n). +- Recursion stack counts as space: balanced divide-and-conquer uses O(log n) frames, a linear chain O(n). ### Amortized analysis -- A dynamic array's occasional **O(n) resize** does not make every append O(n): geometric growth yields **O(1) amortized** append over a sequence of operations. -- Amortized is a guarantee over that sequence; hash-table lookup is usually **expected O(1)** under hashing assumptions, but collisions can make one operation O(n). +- A dynamic array's occasional **O(n) resize** still gives **O(1) amortized** append, because capacity grows geometrically. +- Amortized is a guarantee over a sequence of operations; hash-table lookup is **expected** O(1), and collisions can make one operation O(n). ### Recurrence patterns -- **Binary search** halves one subproblem: `T(n) = T(n/2) + O(1) = O(log n)`. -- **Merge sort** solves two halves then merges in linear time: `T(n) = 2T(n/2) + O(n) = O(n log n)`. -- A branching recursion without memoization can revisit the same states exponentially often; count distinct **states × work per state** after memoizing. +- **Binary search**: `T(n) = T(n/2) + O(1) = O(log n)`. +- **Merge sort**: `T(n) = 2T(n/2) + O(n) = O(n log n)`. +- Recursion without memoization can revisit states exponentially often; after memoizing, cost = **states × work per state**. -## Arrays and search +## Arrays and sequences ### Binary search boundaries -- Search for a **boundary** in a monotonic predicate; maintain a known false side and a known true side until adjacent. -- **Lower bound** finds the first element `>= x`; **upper bound** finds the first `> x`. Their difference counts occurrences in a sorted array. -- Binary search on an answer needs a monotonic feasibility test; complexity is `O(log R × cost(test))` over an integer range of size `R`. +- Search for the **boundary** of a monotonic predicate: keep a known-false and a known-true index until they're adjacent. +- **Lower bound** = first element `>= x`; upper bound = first `> x`; their difference counts occurrences. +- Binary search on the answer needs a monotonic feasibility test: O(log R × cost(test)) over a range of size `R`. ### Two pointers and sliding windows -- On a sorted array, move the left or right pointer according to whether a pair's sum is too small or too large; each pointer moves at most `n` times. -- A variable sliding window works when expanding and shrinking preserve a useful **monotonic condition**; the usual sum-based window fails with negative values. +- On a sorted array, move the left or right pointer depending on whether the pair sum is too small or too large — each moves at most `n` times. +- A variable window needs a **monotonic condition** when expanding and shrinking; sum-based windows break with negative values. - A fixed-size window adds the new element and removes the old one in O(1) per step. ### Prefix sums and difference arrays -- With `P[0]=0` and `P[i+1]=P[i]+a[i]`, half-open range sum `[l,r)` is **`P[r]-P[l]`**: O(n) setup, O(1) per query. -- For many range increments, mark `D[l]+=v`, `D[r]-=v`, then take a prefix sum once; each update is O(1). +- With `P[0]=0`, `P[i+1]=P[i]+a[i]`, the sum of `[l,r)` is **`P[r]-P[l]`**: O(n) setup, O(1) per query. +- For many range increments, mark `D[l]+=v`, `D[r]-=v`, then take one prefix sum. ### Monotonic stacks -- For a **next greater/smaller element**, pop indices that the new value makes obsolete; the remaining top is the nearest qualifying candidate. -- Each index is pushed and popped at most once, so the whole scan is **O(n)**; choose strict versus non-strict comparison deliberately for duplicates. +- **Next greater/smaller element**: pop indices the new value makes obsolete; the remaining top is the answer. +- Each index is pushed and popped once, so the scan is **O(n)**; pick strict vs non-strict comparison for duplicates. ### Monotonic deque -- **Sliding window maximum**: keep indices in a deque with decreasing values; pop smaller values from the back, drop the front once it leaves the window — **O(n)** total. +- **Sliding window maximum**: keep indices with decreasing values; pop smaller values from the back, drop the front once it leaves the window. +- Every index enters and leaves once — **O(n)** total instead of O(nk). ### Intervals - **Merge overlapping intervals**: sort by start, extend the last merged interval while the next start ≤ its end — O(n log n). -- **Minimum meeting rooms**: sweep sorted start and end points (or a min-heap of end times); the max overlap is the answer. +- **Minimum meeting rooms**: sweep sorted start and end points (or a min-heap of end times); the maximum overlap is the answer. ### Cycle detection in sequences -- **Floyd's tortoise and hare**: a slow and a fast pointer meet inside a cycle; restart one from the head and advance both by one to find the cycle's entry — O(n) time, **O(1) space**. +- **Floyd's tortoise and hare**: a slow and a fast pointer meet inside a cycle — O(n) time, **O(1) space**. +- Restart one pointer from the head and advance both by one; they meet at the cycle's entry. + +## Strings ### KMP prefix function -- **KMP** preprocesses a pattern's longest proper prefix that is also a suffix, then reuses matches after a mismatch instead of restarting the text scan. -- Pattern search takes **O(n+m)** time for text length `n` and pattern length `m`, with O(m) preprocessing space. +- **KMP** precomputes, for each pattern prefix, the longest proper prefix that is also a suffix, and reuses it after a mismatch instead of rescanning the text. +- Search takes **O(n+m)** time for text length `n` and pattern length `m`, with O(m) extra space. ### Rolling hash (Rabin-Karp) -- A **polynomial rolling hash** updates a window's hash in O(1) as it slides, so pattern search is O(n + m) expected. -- Hash matches must be verified (or use two moduli) because collisions happen; also useful for finding duplicate substrings. +- A **polynomial rolling hash** updates a window's hash in O(1) as it slides, so search is O(n + m) expected. +- Verify matches (or use two moduli) because collisions happen; also finds duplicate substrings via binary search on length. ## Sorting and selection @@ -78,66 +82,68 @@ What experienced engineers forget before an algorithms interview, grouped by sub | Quicksort | O(n log n) expected, O(n²) worst | O(log n) expected stack | no | randomized pivot; in-place partitioning | | Heapsort | O(n log n) worst | O(1) | no | bounded extra space | -These are properties of the standard array implementations; stability and space can differ in variants. +Standard array implementations; variants differ in stability and space. ### Sorting lower bound and counting sort -- Any **comparison sort** needs Ω(n log n) comparisons in the worst case for arbitrary distinct keys: it must distinguish `n!` possible orders. -- **Counting sort** uses O(n + k) time for integer keys in a range of size `k`; counts need O(k) space, while stable placement also needs O(n) output space. +- Any **comparison sort** needs Ω(n log n) comparisons in the worst case — it must distinguish `n!` orders. +- **Counting sort** runs in O(n + k) for integer keys in a range of size `k`, with O(k) counts plus O(n) output for stable placement. ### Quickselect and top-k -- **Quickselect** partitions around a randomized pivot and recurses into one side: O(n) expected, O(n²) worst with poor pivots. -- A size-`k` **min-heap** tracks the largest `k` of `n` items in O(n log k) time and O(k) space; sorting all items costs O(n log n). +- **Quickselect** partitions around a random pivot and recurses into one side: O(n) expected, O(n²) worst. +- A size-`k` **min-heap** keeps the largest `k` of `n` items in O(n log k) time and O(k) space. ## Data structures ### Heap invariants -- A binary heap is a **complete tree** stored in an array; the root is the minimum or maximum, but siblings are not sorted. -- Peek is O(1); insert and remove-root are O(log n); **bottom-up heap construction is O(n)**. +- A binary heap is a **complete tree** in an array; the root is the min or max, siblings are unordered. +- Peek O(1); insert and remove-root O(log n); **bottom-up construction is O(n)**. ### Hash tables and collisions -- Hash maps trade ordering for expected O(1) lookup/update; **collisions** require chaining or probing, and resizable tables rehash at a chosen load threshold. -- Equal keys must have equal hashes; mutable keys break lookup if fields used by the hash change after insertion. +- Expected O(1) lookup and update, no ordering; **collisions** are resolved by chaining or probing, and the table rehashes past a load threshold. +- Equal keys must hash equally; mutating a key's hashed fields after insertion makes it unfindable. ### Disjoint-set union -- **Union-find** tracks connected components with `find` (representative) and `union` (merge); it does not support efficient deletion or path queries. -- **Path compression + union by size/rank** gives amortized O(α(n)) per operation, effectively constant at practical sizes. +- **Union-find** tracks components with `find` (representative) and `union` (merge); no efficient deletion or path queries. +- **Path compression + union by size/rank** gives amortized O(α(n)) — effectively constant. ### Tries and prefix lookup -- A **trie** follows one edge per symbol: lookup is O(length of key), independent of the number of stored keys under a bounded alphabet. -- Prefix search is natural, but storing a node/edge per prefix can cost much more space than a hash map; compressed tries merge single-child paths. +- A **trie** follows one edge per symbol: lookup is O(key length), independent of the number of keys. +- Prefix search is natural, but a node per prefix costs much more space than a hash map; compressed tries merge single-child paths. ### LRU cache -- **Hash map + doubly linked list**: the map finds a node in O(1); the list keeps recency, moving a node to the front on access and evicting from the tail — **O(1) get and put**. -- In Python, `OrderedDict.move_to_end` + `popitem(last=False)`; in Java, `LinkedHashMap` with access order. +- **Hash map + doubly linked list**: the map finds a node in O(1), the list keeps recency — move to front on access, evict from the tail. +- Python: `OrderedDict.move_to_end` + `popitem(last=False)`; Java: `LinkedHashMap` in access order. ### Fenwick and segment trees -- A **Fenwick tree** (binary indexed tree) gives prefix sums with point updates, both **O(log n)**, in one array. -- A **segment tree** handles any associative range query (min, max, sum) with updates in O(log n); **lazy propagation** adds range updates. +- A **Fenwick tree** (binary indexed tree) gives prefix sums with point updates, both O(log n), in one array. +- A **segment tree** answers any associative range query (min, max, sum) with O(log n) updates; **lazy propagation** adds range updates. ## Trees and graphs ### Tree traversal and BST ordering -- **Inorder** traversal of a binary search tree yields sorted keys; preorder suits serialization, postorder suits bottom-up aggregation. -- An unbalanced BST can degrade to **O(n)** height; AVL/red-black rotations keep search, insert, and delete O(log n). +- **Inorder** traversal of a BST yields sorted keys; preorder suits serialization, postorder bottom-up aggregation. +- An unbalanced BST degrades to **O(n)** height; AVL/red-black rotations keep operations O(log n). ### Graph representation and traversal -- An **adjacency list** uses O(V+E) space and supports BFS/DFS in O(V+E); a matrix uses O(V²) space but tests an edge in O(1). -- **BFS** finds minimum-edge paths in an unweighted graph; **DFS** exposes cycles, connected components, and finishing order. Mark a node when enqueuing it to avoid duplicate BFS work. +- An **adjacency list** takes O(V+E) space and supports BFS/DFS in O(V+E); a matrix takes O(V²) but tests an edge in O(1). +- **BFS** gives shortest paths by edge count; DFS exposes cycles, components, and finishing order. +- Mark a node visited when **enqueuing** it, not when dequeuing, to avoid duplicate BFS work. ### Topological ordering -- A **topological order exists only for a DAG**; edges point from earlier to later vertices. -- Kahn's algorithm repeatedly removes zero-indegree vertices; fewer than `V` removals means a cycle. DFS can instead reverse finishing order, while detecting back edges. Both take O(V+E). +- A topological order exists **only for a DAG**. +- **Kahn's algorithm** repeatedly removes zero-indegree vertices; fewer than `V` removals means a cycle. +- DFS alternative: reverse finishing order, detecting back edges. Both O(V+E). ### Shortest-path choices @@ -148,54 +154,57 @@ These are properties of the standard array implementations; stability and space | negative allowed | Bellman-Ford | O(VE) | detects reachable negative cycles | | DAG, any weights | topological relaxation | O(V+E) | needs acyclic graph | - ### Minimum spanning tree -- An **MST** minimizes total edge weight while connecting all vertices of an undirected, connected graph; it is not a shortest-path tree. -- **Kruskal** sorts edges and accepts those joining different union-find components; **Prim** grows one tree using a minimum-edge priority queue. +- An **MST** connects all vertices of an undirected graph with minimum total weight — not a shortest-path tree. +- **Kruskal** sorts edges and keeps those joining different union-find components; **Prim** grows one tree from a min-edge priority queue. -## Dynamic programming and search +## Dynamic programming ### DP state and order -- Define a **state** containing everything future decisions need, a recurrence, base cases, and an evaluation order where dependencies are ready. -- **Memoization** explores reachable states top-down; tabulation fills them bottom-up. Complexity is number of states × transitions per state. +- Define a **state** holding everything future decisions need, a recurrence, base cases, and an evaluation order where dependencies are ready. +- Top-down **memoization** visits only reachable states; bottom-up tabulation fills them all. Cost = states × transitions. ### Knapsack and one-dimensional DP -- For **0/1 knapsack**, process capacities downward for each item so it cannot be reused in the same iteration; upward iteration models **unbounded** reuse. -- O(nC) time is **pseudopolynomial**: `C` is a numeric capacity, whose encoded input length is only O(log C). +- **0/1 knapsack**: iterate capacities **downward** per item so it isn't reused; upward iteration models unbounded reuse. +- O(nC) is **pseudopolynomial**: `C` is a number whose input length is only O(log C). ### Longest increasing subsequence -- The O(n log n) method stores the **smallest possible tail** for each subsequence length and binary-searches where each new value belongs. -- Use first tail `>= x` for **strictly increasing** subsequences; the tails array alone gives the length, not necessarily an actual subsequence. +- The O(n log n) method keeps the **smallest tail** for each length and binary-searches where each value goes. +- For strictly increasing, replace the first tail `>= x`; the tails array gives the length, not the subsequence itself. ### Two-sequence DP: LCS and edit distance -- **LCS**: `dp[i][j] = dp[i-1][j-1] + 1` if characters match, else `max(dp[i-1][j], dp[i][j-1])`. -- **Edit distance**: `1 + min(insert, delete, replace)` on a mismatch, diagonal on a match; both O(nm) time, reducible to **O(min(n, m)) space** with two rows. +- **LCS**: `dp[i][j] = dp[i-1][j-1] + 1` on a match, else `max(dp[i-1][j], dp[i][j-1])`. +- **Edit distance**: diagonal on a match, else `1 + min(insert, delete, replace)`. +- Both O(nm) time, reducible to **O(min(n, m)) space** with two rows. ### Bitmask DP -- Represent a subset as bits and index DP by it: travelling salesman is `dp[mask][last]` in **O(2ⁿ · n²)** — feasible up to n ≈ 20. - -### Bit manipulation +- Index DP by a subset encoded as bits: travelling salesman is `dp[mask][last]` in **O(2ⁿ · n²)** — feasible up to n ≈ 20. +- Iterate submasks of `mask` with `s = (s - 1) & mask`; over all masks that totals O(3ⁿ). -- `x & (x − 1)` clears the lowest set bit (power-of-two test: result is 0); `x & −x` isolates it. -- XOR of all elements cancels pairs — finds the single non-duplicated number in O(1) space. +## Greedy, backtracking, and hardness ### Greedy proof pattern -- A greedy choice needs an **exchange argument** or a maintained invariant; choosing the locally best-looking option is not enough. -- For maximum number of non-overlapping intervals, sort by **earliest finish time** and take the next compatible interval; sorting by start time or shortest duration can fail. +- A greedy choice needs an **exchange argument** or a maintained invariant — locally best isn't enough. +- Maximum non-overlapping intervals: sort by **earliest finish time**; sorting by start or by length fails. ### Backtracking and pruning -- Backtracking explores a **decision tree**, undoing each choice after recursion; track state that lets you reject invalid partial solutions early. -- Exponential worst-case search may still be necessary to enumerate exponentially many outputs; prune only when a branch cannot yield a required answer or improve the best score. +- Backtracking walks a **decision tree**, undoing each choice after recursing; reject invalid partial solutions early. +- Prune only branches that can't produce an answer or beat the best score; enumerating all outputs stays exponential. + +### Bit manipulation + +- `x & (x − 1)` clears the lowest set bit (zero result = power of two); **`x & −x`** isolates it. +- XOR of all elements cancels pairs — finds the single unpaired number in O(1) space. ### Recognizing NP-hard problems - Traveling salesman, subset sum, knapsack, graph coloring, SAT, Hamiltonian path, and set cover are **NP-hard** in general. -- Signals in an interview: exact answers need exponential search (backtracking, bitmask DP) or pseudopolynomial DP on small numbers; otherwise use a greedy or approximation algorithm and say so. +- Exact answers then need exponential search or pseudopolynomial DP on small numbers; otherwise use a greedy or **approximation** algorithm and say so. diff --git a/docs/topics/aws.md b/docs/topics/aws.md index ab26634..e001d7e 100644 --- a/docs/topics/aws.md +++ b/docs/topics/aws.md @@ -6,10 +6,9 @@ What experienced AWS developers and DevOps engineers forget before an interview, ### Policy evaluation -- Requests start with an **implicit deny**; any applicable **explicit deny** wins over every allow. -- Identity and resource policies **grant**; permissions boundaries, session policies, and SCPs/RCPs only **cap** what can be granted. +- Requests start with an implicit deny; any applicable **explicit deny** wins over every allow. +- Identity and resource policies grant; permissions boundaries, session policies, and SCPs/RCPs only **cap** what can be granted. - A role needs both a **trust policy** allowing the principal to assume it and permissions for the resulting session to perform actions. -- An **SCP** caps permissions in member accounts; it does not grant permissions by itself. ### Workload credentials @@ -43,31 +42,31 @@ What experienced AWS developers and DevOps engineers forget before an interview, ### Route 53 routing -- Policies: simple, **weighted** (canary), **latency**, **failover** (with health checks), geolocation, geoproximity, multivalue. +- Policies: simple, **weighted** (canary), latency, **failover** (with health checks), geolocation, geoproximity, multivalue. - **Alias records** work at the zone apex (`example.com` → ALB/CloudFront), where a CNAME isn't allowed, and are free to query. ## Compute ### Workload choice -| Service | Use when | Operational boundary | +| Service | Use when | You still manage | |---|---|---| -| EC2 | OS, runtime, or host control matters | manage instances and scaling | -| ECS on Fargate | containers without worker-node management | manage tasks, services, and capacity settings | -| EKS | Kubernetes APIs and ecosystem are required | manage workloads; worker capacity can be managed with EKS Auto Mode or Fargate | -| Lambda | event-driven, bounded work | manage function concurrency, timeouts, and event handling | +| EC2 | OS, runtime, or host control matters | instances, patching, scaling | +| ECS on Fargate | containers without managing nodes | task definitions, services | +| EKS | Kubernetes APIs and ecosystem are required | workloads; nodes unless Auto Mode or Fargate | +| Lambda | event-driven, bounded work | concurrency, timeouts, event handling | ### Lambda limits and concurrency -- Maximum timeout **15 minutes**; synchronous payloads **6 MB** each way (streamed responses allow more). -- **Lambda Managed Instances** allow up to **90 minutes** for async and event-source invocations (Sept 2026); synchronous calls stay at 15. -- **Reserved concurrency** guarantees capacity and caps a function; **provisioned concurrency** pre-initializes environments to reduce startup latency and incurs separate charges. +- Maximum timeout **15 minutes**; synchronous payloads 6 MB each way (streamed responses allow more). +- Lambda Managed Instances allow up to **90 minutes** for async and event-source invocations (Sept 2026); synchronous calls stay at 15. +- **Reserved concurrency** guarantees capacity and caps a function; provisioned concurrency pre-initializes environments to reduce startup latency and incurs separate charges. ### Lambda cold starts and scaling - A cold start downloads code, starts the runtime, and runs init code; the **INIT phase is billed since August 2025**. -- Reduce it with **provisioned concurrency** or **SnapStart** (Java, Python, .NET), which restores a snapshot of an initialized environment — make sure randomness and connections are recreated after restore. -- Default **1,000** concurrent executions per account per Region; each function scales by up to **1,000 more every 10 seconds**. +- Reduce it with provisioned concurrency or **SnapStart** (Java, Python, .NET), which restores a snapshot of an initialized environment — make sure randomness and connections are recreated after restore. +- Default 1,000 concurrent executions per account per Region; each function scales by up to **1,000 more every 10 seconds**. ### API Gateway vs ALB @@ -76,8 +75,8 @@ What experienced AWS developers and DevOps engineers forget before an interview, ### ECS service mechanics -- An ECS **task definition** specifies containers and resources; a **service** maintains the desired number of tasks and replaces failed ones. -- An Application Load Balancer routes at **L7** by HTTP attributes; a Network Load Balancer routes at **L4** for TCP/UDP-style traffic. +- A **task definition** specifies containers and resources; a **service** keeps the desired number of tasks running and replaces failed ones. +- Services register tasks with a load balancer target group and roll out new task definitions with rolling or blue/green deployments. ## Storage and databases @@ -99,20 +98,20 @@ What experienced AWS developers and DevOps engineers forget before an interview, ### Managed database behavior -- An RDS **Multi-AZ DB instance** uses a synchronous standby for failover; that standby does **not** serve reads. Use a read replica or Multi-AZ DB cluster for read scaling. +- An RDS **Multi-AZ DB instance** uses a synchronous standby for failover; that standby does not serve reads. Use a read replica or Multi-AZ DB cluster for read scaling. - DynamoDB reads are **eventually consistent by default**; strong reads are available on tables and local secondary indexes, **not GSIs**. -- DynamoDB global tables support both **multi-Region eventual** and **multi-Region strong** consistency modes (strong mode since June 2025); specify the mode before stating cross-Region read behavior. +- DynamoDB global tables support multi-Region eventual and multi-Region strong consistency (strong since June 2025). ### Aurora storage - The storage volume keeps **6 copies across 3 AZs**; a write needs **4 of 6**, a read **3 of 6** — it survives losing an AZ plus one more copy for reads. -- Up to **15 replicas** share the same storage volume, so replica lag is usually well under 100 ms and failover doesn't copy data. +- Up to 15 replicas share the same storage volume, so replica lag is usually well under 100 ms and failover doesn't copy data. ### DynamoDB partitions and indexes - Each partition serves about **3,000 RCU and 1,000 WCU** and holds ~10 GB; a hot partition key throttles even if table capacity is fine. -- **GSIs** have their own capacity and are eventually consistent; a throttled GSI **throttles writes to the base table**. LSIs must be defined at table creation. -- Items are at most **400 KB**; transactions cover up to **100 items**; TTL deletes are background work that can lag by days. +- GSIs have their own capacity and are eventually consistent; a throttled GSI **throttles writes to the base table**. LSIs must be defined at table creation. +- Items are at most **400 KB**; transactions cover up to 100 items; TTL deletes are background work that can lag by days. ## Events and workflows @@ -128,15 +127,15 @@ What experienced AWS developers and DevOps engineers forget before an interview, ### Routing and orchestration -- **SNS** pushes to subscribers; **EventBridge** matches event patterns and routes to targets, with optional archive/replay; **SQS** retains work for consumers to poll. Choose based on routing and consumer behavior, not service names. +- SNS pushes to subscribers; EventBridge matches event patterns and routes to targets, with optional archive/replay; SQS retains work for consumers to poll. - Step Functions **Standard** workflows use exactly-once execution unless a task has explicit retries; **asynchronous Express** is at-least-once, while **synchronous Express** is at-most-once. Retried external side effects still need idempotency. ## Operations ### Observability and audit -- **CloudWatch** holds metrics, logs, and alarms; **CloudTrail** records account API activity. CloudTrail **data events** such as S3 object access are not logged by default when creating a trail. -- **VPC Flow Logs** show accepted/rejected IP traffic metadata, not packet payloads; use them when route and security-rule debugging needs evidence. +- CloudWatch holds metrics, logs, and alarms; **CloudTrail** records account API activity. CloudTrail **data events** such as S3 object access are not logged by default when creating a trail. +- **VPC Flow Logs** record accepted/rejected IP traffic metadata, not packet payloads — for debugging routes and security rules. ### Infrastructure changes and secrets @@ -145,6 +144,6 @@ What experienced AWS developers and DevOps engineers forget before an interview, ### Capacity and cost guardrails -- Service **quotas** can stop scaling before the application reaches its own limits; check the relevant account and Region quota before a load test or launch. -- **AWS Budgets** alerts follow billing-data updates, at least daily; use CloudWatch service metrics for operational alarms. **Cost Explorer** is for spend analysis. +- Service **quotas** are per account and Region and often stop scaling before the application does — many are soft and raised by request. +- AWS Budgets alerts follow billing-data updates, at least daily; use CloudWatch service metrics for operational alarms. Cost Explorer is for spend analysis. - EC2 **Spot** capacity can be reclaimed; stop/terminate interruption notices give about **2 minutes** on a best-effort basis (hibernation starts immediately). Checkpoint work and replace capacity proactively. diff --git a/docs/topics/backend.md b/docs/topics/backend.md index eb8a706..8c0e9cf 100644 --- a/docs/topics/backend.md +++ b/docs/topics/backend.md @@ -11,9 +11,12 @@ What experienced backend engineers forget before an interview, grouped by subtop ### Status codes people confuse -- **401** not authenticated vs **403** authenticated but not allowed (or **404** to hide that the resource exists). -- **409** conflicts with current state vs **422** well-formed but fails validation. -- **202** accepted for async processing; **204** success with no body; **412** precondition (`If-Match`) failed; **429** rate limited. +| Codes | Meaning | +|---|---| +| **401** vs **403** | not authenticated vs authenticated but not allowed (or 404 to hide that the resource exists) | +| **409** vs **422** | conflicts with current state vs well-formed but fails validation | +| 202 / 204 | accepted for async processing / success with no body | +| 412 / 429 | precondition (`If-Match`) failed / rate limited | ### Method semantics @@ -24,6 +27,11 @@ What experienced backend engineers forget before an interview, grouped by subtop | POST | no | no | | PATCH | no | not guaranteed — depends on the patch format | +### PATCH formats + +- **JSON Merge Patch** (RFC 7396): send a partial object; `null` deletes a field — so it can't set a value to null or edit one array element. +- **JSON Patch** (RFC 6902): a list of operations (`add`, `remove`, `replace`, `test`) on JSON Pointer paths; `test` makes the patch conditional. + ### REST vs gRPC vs GraphQL | | REST | gRPC | GraphQL | @@ -59,6 +67,7 @@ What experienced backend engineers forget before an interview, grouped by subtop ### Optimistic concurrency with ETags - `GET` returns an **`ETag`**; the client sends `PUT` with **`If-Match: `**, and the server answers **412** if the resource changed — no lost updates between clients. +- `If-Match` needs a strong ETag (not `W/`); `If-None-Match: *` makes a `PUT` create-only. ### Transactional outbox @@ -67,7 +76,8 @@ What experienced backend engineers forget before an interview, grouped by subtop ### Inbox deduplication -- Consumers record processed message IDs (in the same transaction as their effect) and skip repeats — the receiving side of the outbox. +- Consumers record processed message IDs **in the same transaction** as their effect and skip repeats — the receiving side of the outbox. +- Expire entries only after the broker's maximum redelivery window. ## Service calls @@ -81,7 +91,7 @@ What experienced backend engineers forget before an interview, grouped by subtop - **Sign** each payload (HMAC over timestamp + body) and have receivers verify it with a **constant-time** compare; reject stale timestamps to stop replays. - Deliver **at least once** with backoff, so receivers dedupe by event ID; receivers should ack fast (2xx) and process asynchronously. -- Fetching a user-supplied webhook URL is an **SSRF** risk — see [security.md](security.md). +- Fetching a user-supplied webhook URL is an SSRF risk — see [security.md](security.md). ## Async work @@ -98,10 +108,12 @@ What experienced backend engineers forget before an interview, grouped by subtop ### Background jobs - At-least-once delivery means jobs must be **idempotent**; persistently failing jobs go to a **dead-letter queue** with alerting. +- Give every job a timeout; long jobs checkpoint progress so a retry resumes instead of restarting. ### Scheduled jobs -- Guard against **overlapping runs** (a run longer than its interval) and against every replica firing the same cron — use a lock (DB row, advisory lock, lease) or a single scheduler. +- Guard against **overlapping runs**, where one run lasts longer than the interval. +- With several replicas, each fires the same cron — use a lock (DB row, advisory lock, lease) or a single scheduler. ## Data access @@ -117,7 +129,8 @@ What experienced backend engineers forget before an interview, grouped by subtop ### Transaction boundaries -- Never make a network call inside an open DB transaction — it holds locks and a connection for the whole round trip. +- Never make a network call inside an open DB transaction — it holds **locks and a connection** for the whole round trip. +- For "commit, then notify another service", use the outbox instead of calling inside the transaction. ### File uploads @@ -131,6 +144,11 @@ What experienced backend engineers forget before an interview, grouped by subtop - On **`SIGTERM`**: fail readiness → stop accepting new connections → finish in-flight requests within a deadline → close pools and flush telemetry → exit. - Workers stop pulling jobs and finish or checkpoint the current one. Kubernetes timing: see [devops.md](devops.md). +### Health endpoints + +- Separate endpoints: **liveness** checks only that the process can make progress; **readiness** checks what serving needs (warm-up done, required dependencies) and fails while draining. +- Keep them cheap and on an internal port; probe configuration and pitfalls: see [devops.md](devops.md). + ### Configuration and secrets - Config that varies by environment comes from the **environment**, so one immutable artifact promotes from staging to production. @@ -141,18 +159,26 @@ What experienced backend engineers forget before an interview, grouped by subtop ### Mass assignment - Binding the request body straight onto a model lets clients set `isAdmin: true` — **allowlist** bindable fields or use separate DTOs. +- Serialize responses through explicit DTOs too, so a new sensitive column doesn't leak. ### Input validation -- Validate type, format, and range at the boundary before business logic (**400**); business-rule failures come later (**409/422**). +- Validate type, format, and range at the boundary before business logic (**400**); business-rule failures come later (409/422). +- Cap sizes — body, array lengths, string lengths, nesting depth — to prevent resource exhaustion. ### Secrets in logs - Request logging middleware can capture tokens, auth headers, and PII — **redact centrally** in the logging layer, not per call site. +- Tokens in URL query strings end up in proxy and CDN logs — send them in headers. ## Testing -### Integration and contract tests +### Integration tests with real dependencies - **Testcontainers** runs real dependencies (Postgres, Kafka) in tests, catching SQL and serialization bugs that mocks hide. -- **Consumer-driven contract tests** (Pact) check that a provider still satisfies what each consumer relies on, without deploying both. +- A mock only checks your assumptions about a dependency; test the real one at least at the adapter boundary. + +### Consumer-driven contract tests + +- **Consumer-driven contract tests** (Pact): each consumer records the requests and fields it relies on, and the provider's CI verifies them. +- Catches breaking API changes without deploying both services together. diff --git a/docs/topics/database.md b/docs/topics/database.md index 487f562..13e9238 100644 --- a/docs/topics/database.md +++ b/docs/topics/database.md @@ -56,8 +56,8 @@ What experienced database engineers forget before an interview, grouped by subto ### PostgreSQL isolation -- **Default: Read Committed**; Repeatable Read is **snapshot isolation** — no phantoms, and a concurrent update of the same row **aborts** (no lost update), but write skew is possible. -- **Serializable** is **SSI**: it prevents write skew by aborting with a serialization error, so the app must **retry**. +- **Default: Read Committed**; Repeatable Read is **snapshot isolation** — no phantoms, and a concurrent update of the same row aborts (no lost update), but write skew is possible. +- Serializable is **SSI**: it prevents write skew by aborting with a serialization error, so the app must retry. ### MySQL InnoDB isolation @@ -68,18 +68,19 @@ What experienced database engineers forget before an interview, grouped by subto ### PostgreSQL MVCC -- Readers read a **snapshot** and never block writers, or vice versa; writers still lock rows against each other. -- Old row versions stay in the table, tagged **`xmin`/`xmax`**; **VACUUM** reclaims them, and a long-running transaction blocks it, causing **bloat**. +- Readers read a snapshot and never block writers, or vice versa; writers still lock rows against each other. +- Old row versions stay in the table, tagged **`xmin`/`xmax`**; **VACUUM** reclaims them, and a long-running transaction blocks it, causing bloat. - **XID wraparound** (32-bit transaction IDs) forces aggressive anti-wraparound vacuuming. -- **HOT updates** skip index maintenance when no indexed column changed and the page has room. +- HOT updates skip index maintenance when no indexed column changed and the page has room. ### InnoDB MVCC - Updates happen in place; older versions are rebuilt from the **undo log**, and a long transaction makes undo history grow. -### Row locks and queue tables +### Row locks, queue tables, advisory locks - `SELECT ... FOR UPDATE` locks the rows it returns; **`SKIP LOCKED`** lets workers claim different rows of a queue table without blocking. +- PostgreSQL **advisory locks** are app-defined locks on a number, not a row — for leader election or job dedup without a table to lock. ### Optimistic locking @@ -96,10 +97,6 @@ What experienced database engineers forget before an interview, grouped by subto - Most `ALTER TABLE` forms take an **`ACCESS EXCLUSIVE`** lock. If it waits behind a long query, **every later query queues behind it** — a brief outage from a "fast" migration. - Set **`lock_timeout`** (a few seconds) on migrations and retry; rollout patterns in [devops.md](devops.md). -### Advisory locks - -- PostgreSQL **advisory locks** are app-defined locks on a number, not a row — for leader election or job dedup without a table to lock. - ## Storage engines and durability ### B-tree vs LSM-tree @@ -120,7 +117,7 @@ What experienced database engineers forget before an interview, grouped by subto ### Backups and PITR -- **Logical** backups (`pg_dump`): portable, slow to restore at scale. **Physical** (`pg_basebackup`, snapshots): fast restore, same major version. +- Logical backups (`pg_dump`): portable, slow to restore at scale. Physical (`pg_basebackup`, snapshots): fast restore, same major version. - **PITR** = physical base backup + WAL replayed to a chosen moment. Replicas are **not backups** — they replicate mistakes too. ## Query execution and tuning @@ -156,7 +153,7 @@ What experienced database engineers forget before an interview, grouped by subto ### Window functions - `fn() OVER (PARTITION BY ... ORDER BY ...)` computes per row without collapsing rows. -- **`ROW_NUMBER`** 1,2,3; **`RANK`** 1,1,3 (gaps); **`DENSE_RANK`** 1,1,2. +- `ROW_NUMBER` 1,2,3; **`RANK`** 1,1,3 (gaps); **`DENSE_RANK`** 1,1,2. - The default frame with `ORDER BY` is **`RANGE ... CURRENT ROW`**, which includes all peer rows with equal sort keys — use `ROWS` for a true running total. ### CTE materialization @@ -174,17 +171,17 @@ What experienced database engineers forget before an interview, grouped by subto - Commands execute on **one thread**, so each command (and Lua script or `MULTI` block) is atomic; a slow command (`KEYS *`, big `SMEMBERS`) blocks everyone. - Structures: strings, hashes, lists, sets, **sorted sets** (skip list + hash — leaderboards, rate limiters), streams, HyperLogLog. -- **Redis Cluster** splits keys into **16,384 hash slots**; multi-key operations need the same slot, forced with **hash tags** (`{user42}:cart`). +- Redis Cluster splits keys into **16,384 hash slots**; multi-key operations need the same slot, forced with hash tags (`{user42}:cart`). ### Redis persistence and licensing -- **RDB**: periodic fork + snapshot (copy-on-write), loses writes since the last one. **AOF**: logs every write; `appendfsync everysec` (default) loses up to **~1 s**. +- RDB: periodic fork + snapshot (copy-on-write), loses writes since the last one. AOF: logs every write; `appendfsync everysec` (default) loses up to **~1 s**. - Replication is **asynchronous** — failover can drop acknowledged writes. -- The 2024 license change led to the **Valkey** fork (Linux Foundation); **Redis 8** (2025) added AGPLv3 as an option. +- The 2024 license change led to the **Valkey** fork (Linux Foundation); Redis 8 (2025) added AGPLv3 as an option. ### Wide-column key design (Cassandra, DynamoDB) -- The **partition key** picks the node; the **sort/clustering key** orders rows inside the partition — a query must supply the partition key. +- The **partition key** picks the node; the sort/clustering key orders rows inside the partition — a query must supply the partition key. - Model **tables per query** (denormalized), not per entity; unbounded partitions (all events of a popular user) become hot and huge — add a time bucket to the key. - Deletes write **tombstones**, which slow reads until compaction purges them. @@ -198,12 +195,12 @@ What experienced database engineers forget before an interview, grouped by subto ### Normalization -- **1NF** atomic values → **2NF** no partial-key dependency → **3NF/BCNF** no dependency on non-key columns: each fact stored once. +- 1NF atomic values → 2NF no partial-key dependency → **3NF/BCNF** no dependency on non-key columns: each fact stored once. - **Denormalize** measured hot paths only (copied columns, counters, materialized views), accepting harder writes. ### Surrogate keys -- **Auto-increment bigint**: compact and ordered, but reveals counts and needs one sequence. +- Auto-increment bigint: compact and ordered, but reveals counts and needs one sequence. - **UUIDv4** scatters inserts across the B-tree; **UUIDv7** is time-ordered and native via **`uuidv7()` since PostgreSQL 18** — ID schemes in [distributed.md](distributed.md). ### Constraints and soft deletes @@ -221,13 +218,13 @@ What experienced database engineers forget before an interview, grouped by subto ### Table partitioning -- **Range** (most common, by date), **list**, or **hash**; **partition pruning** skips irrelevant partitions. +- **Range** (most common, by date), list, or hash; **partition pruning** skips irrelevant partitions. - Dropping an old partition is instant, unlike a bloating bulk `DELETE`; PostgreSQL unique constraints must include the partition key. ### Connection pooling - Each PostgreSQL connection is a **process** (a few MB), so thousands of direct connections hurt — pool them. -- PgBouncer **session mode** keeps `SET` and advisory locks; **transaction mode** scales further but loses session state; prepared statements work in it **since PgBouncer 1.21**. +- PgBouncer session mode keeps `SET` and advisory locks; **transaction mode** scales further but loses session state; prepared statements work in it **since PgBouncer 1.21**. ### OLTP vs OLAP diff --git a/docs/topics/devops.md b/docs/topics/devops.md index 67b2cf9..f089604 100644 --- a/docs/topics/devops.md +++ b/docs/topics/devops.md @@ -22,6 +22,7 @@ What experienced DevOps engineers forget before an interview, grouped by subtopi ### Rollbacks - Roll back by shifting traffic or redeploying the previous **immutable artifact**; it never undoes data changes already made. +- So the previous version must run against the new schema — keep migrations backward compatible. ### Zero-downtime schema changes @@ -34,8 +35,8 @@ What experienced DevOps engineers forget before an interview, grouped by subtopi ### How containers work - A container is a Linux process with **namespaces** (isolated view: PID, network, mount, UTS, IPC, user) and **cgroups** (resource limits: CPU, memory, I/O). -- The filesystem is image layers stacked with **OverlayFS** plus a thin writable layer; all containers share the **host kernel**, unlike VMs. -- The OCI runtime (**runc**) sets this up; containerd/CRI-O manage images and lifecycles above it. +- The filesystem is image layers stacked with OverlayFS plus a thin writable layer; all containers share the **host kernel**, unlike VMs. +- The OCI runtime (runc) sets this up; containerd/CRI-O manage images and lifecycles above it. ### Dockerfile practices @@ -56,7 +57,7 @@ What experienced DevOps engineers forget before an interview, grouped by subtopi ### Control plane -- The **API server** is the only component talking to **etcd**; the **scheduler** assigns Pods to nodes; **controllers** reconcile actual state toward desired state. +- The **API server** is the only component talking to **etcd**; the scheduler assigns Pods to nodes; **controllers** reconcile actual state toward desired state. - Everything is declarative and level-triggered: `kubectl apply` succeeding means the object was stored, not that the app is healthy. ### Node components @@ -69,7 +70,7 @@ What experienced DevOps engineers forget before an interview, grouped by subtopi ### Workload types - **Deployment**: interchangeable stateless Pods via ReplicaSets. **StatefulSet**: stable names (`db-0`), ordered rollout, a volume per Pod. -- **DaemonSet**: one Pod per node (log agents, CNI). **Job**/**CronJob**: run to completion, once or on a schedule. +- **DaemonSet**: one Pod per node (log agents, CNI). Job/CronJob: run to completion, once or on a schedule. - Never run a bare Pod for anything that should come back after failure. ### Rolling update tuning @@ -84,19 +85,26 @@ What experienced DevOps engineers forget before an interview, grouped by subtopi ### Scheduling constraints -- **Taints** on nodes repel Pods unless they have a matching **toleration** (dedicated GPU nodes). -- **Node affinity** attracts Pods to labeled nodes; **pod anti-affinity** or **topology spread constraints** spread replicas across nodes and zones. +- **Taints** on nodes repel Pods unless they have a matching toleration (dedicated GPU nodes). +- **Node affinity** attracts Pods to labeled nodes; pod anti-affinity or **topology spread constraints** spread replicas across nodes and zones. ### PodDisruptionBudgets - A **PDB** (`minAvailable` or `maxUnavailable`) limits **voluntary** disruptions — node drains, upgrades, autoscaler scale-down — not crashes. - A PDB that allows zero disruptions blocks node drains forever. +### Persistent volumes + +- A **PVC** requests storage; a **StorageClass** provisions a matching PV dynamically. +- Access modes: `ReadWriteOnce` (one node — most block storage) vs `ReadWriteMany` (shared file systems such as EFS or NFS). +- **`WaitForFirstConsumer`** binding delays provisioning until the Pod is scheduled, so a zonal disk lands in the Pod's zone. +- Dynamic volumes default to reclaim policy `Delete` (the disk goes with the PVC); `Retain` keeps it. + ## Kubernetes networking ### Services -- A **Service** gives a changing set of Pods (chosen by label selector) a stable virtual IP and DNS name: **ClusterIP** (internal), **NodePort**, **LoadBalancer** (cloud LB). +- A **Service** gives a changing set of Pods (chosen by label selector) a stable virtual IP and DNS name: **ClusterIP** (internal), NodePort, LoadBalancer (cloud LB). - No traffic? Check the selector and readiness first: `kubectl get endpointslices` (the Endpoints API is deprecated **since 1.33**). - General service-discovery patterns: see [system.md](system.md). @@ -114,23 +122,23 @@ What experienced DevOps engineers forget before an interview, grouped by subtopi ### Probes -- **Readiness** gates Service traffic; **liveness** restarts a stuck container; **startup** holds off both during slow boots. +- **Readiness** gates Service traffic; **liveness** restarts a stuck container; startup holds off both during slow boots. - A liveness probe that checks dependencies or times out under load causes **restart cascades** — keep it a cheap local check. ### Requests, limits, QoS -- **Requests** drive scheduling; **limits** cap usage — over the CPU limit the container is **throttled**, over the memory limit it's **OOMKilled**. -- Requests vs limits set the **QoS class** (Guaranteed / Burstable / BestEffort), which decides eviction order under node pressure. -- **In-place Pod resize** (GA in **1.35**) changes CPU/memory requests without restarting the Pod. +- Requests drive scheduling; limits cap usage — over the CPU limit the container is **throttled**, over the memory limit it's **OOMKilled**. +- Requests vs limits set the QoS class (Guaranteed / Burstable / BestEffort), which decides eviction order under node pressure. +- In-place Pod resize (GA in **1.35**) changes CPU/memory requests without restarting the Pod. ### Autoscaling -- **HPA** scales replicas from metrics; **VPA** adjusts requests; **Cluster Autoscaler**/**Karpenter** add nodes when Pods can't be scheduled. +- **HPA** scales replicas from metrics; **VPA** adjusts requests; Cluster Autoscaler/**Karpenter** add nodes when Pods can't be scheduled. - More replicas don't help when the bottleneck is a saturated downstream database. ### Graceful Pod shutdown -- On deletion the Pod is removed from endpoints **and** gets `SIGTERM` at the same time, so it may still receive traffic briefly — a short **`preStop`** sleep covers that race. +- On deletion the Pod is removed from endpoints and gets `SIGTERM` at the same time, so it may still receive traffic briefly — a short **`preStop`** sleep covers that race. - After **`terminationGracePeriodSeconds`** (default **30 s**) the kubelet sends `SIGKILL`; app-side steps are in [backend.md](backend.md). ### ConfigMaps and Secrets @@ -143,6 +151,12 @@ What experienced DevOps engineers forget before an interview, grouped by subtopi - **`Pending`**: unschedulable (resources, quota, taints, unbound volume). **`ImagePullBackOff`**: wrong image or registry credentials. **`CrashLoopBackOff`**: the process or its liveness probe keeps failing. - `kubectl describe pod` shows events; `kubectl logs --previous` shows the crashed container's last output. +### ServiceAccounts and workload identity + +- Each Pod runs as a **ServiceAccount**; RBAC Roles bound to it limit what it may do through the Kubernetes API. +- Its tokens are projected, short-lived, and audience-bound; set `automountServiceAccountToken: false` when the app doesn't call the API. +- **Workload identity** maps a ServiceAccount to a cloud role (EKS IRSA or Pod Identity, GKE Workload Identity) — no static cloud keys in Secrets. + ### Packaging and extension - **Helm** templates charts with values and tracks releases; **Kustomize** patches plain YAML with overlays, no templating. @@ -162,15 +176,23 @@ What experienced DevOps engineers forget before an interview, grouped by subtopi ### Refactoring resources -- **`moved`** blocks record a renamed address so the plan doesn't destroy and recreate; **`import`** blocks (**1.5**) adopt existing resources; **`removed`** (**1.7**) forgets a resource without destroying it. +- **`moved`** blocks record a renamed address, so the plan doesn't destroy and recreate it. +- **`import`** blocks (1.5) adopt existing resources into state. +- **`removed`** blocks (1.7) forget a resource without destroying it. ### Modules, workspaces, licensing -- **Modules** package resources behind inputs and outputs. -- **Workspaces** give one configuration several states but are **not an access boundary** — isolate environments with separate backends or root modules. +- Modules package resources behind inputs and outputs. +- Workspaces give one configuration several states but are **not an access boundary** — isolate environments with separate backends or root modules. - Terraform moved to the **BSL license in 2023**; **OpenTofu** is the open-source fork with the same workflow. -## GitOps +## CI/CD and GitOps + +### CI pipeline practices + +- **Trunk-based development**: short-lived branches merged at least daily, unfinished work behind flags. +- **Build once, promote** the same image digest through environments — never rebuild per environment. +- Cache dependencies keyed on the lock-file hash, run fast checks first, and pin third-party CI actions by commit SHA. ### Pull-based reconciliation diff --git a/docs/topics/distributed.md b/docs/topics/distributed.md index 234803a..221a753 100644 --- a/docs/topics/distributed.md +++ b/docs/topics/distributed.md @@ -6,9 +6,9 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### CAP and PACELC -- During a **network partition**, choose **Consistency** (reject or block — etcd, ZooKeeper) or **Availability** (serve possibly stale data — Cassandra, DynamoDB default). -- CAP's "C" is **linearizability**, not ACID's "C" (application invariants). -- **PACELC**: if Partitioned, A vs C; **Else**, **Latency vs Consistency** — the everyday cost when nothing is broken. +- During a **network partition**, choose Consistency (reject or block — etcd, ZooKeeper) or Availability (serve possibly stale data — Cassandra, DynamoDB default). +- CAP's "C" is linearizability, not ACID's "C" (application invariants). +- **PACELC**: if Partitioned, A vs C; Else, **Latency vs Consistency** — the everyday cost when nothing is broken. ### FLP and Two Generals @@ -17,23 +17,21 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### Byzantine faults -- Nodes that lie need **`3f + 1`** nodes to tolerate `f` faults (vs **`2f + 1`** for crash faults); classic algorithm **PBFT**. Most internal systems assume crash faults only. +- Tolerating `f` nodes that lie needs **`3f + 1`** nodes, vs `2f + 1` for crash faults; the classic algorithm is **PBFT**. +- Most internal systems assume crash faults only; blockchains must assume Byzantine ones. ## Consistency models ### Consistency hierarchy -- From strongest: **strict serializable** > **linearizable** > **sequential** > **causal** > **eventual**. +- From strongest: strict serializable > linearizable > sequential > causal > eventual. - **Linearizability**: single-object recency — once a write completes, every later read (in real time) sees it. - **Serializability**: multi-object transaction isolation — equivalent to *some* serial order, not necessarily real-time order. - -### Strict serializability - -- Serializability + real-time order: transactions behave like linearizable operations (Spanner, FoundationDB). CockroachDB is serializable but linearizable only **per key**. +- **Strict serializability** adds real-time order (Spanner, FoundationDB); CockroachDB is serializable but linearizable only per key. ### Session guarantees -- **Read-your-writes**, **monotonic reads**, **monotonic writes**, **writes-follow-reads** — per-client guarantees layered on eventual consistency. +- **Read-your-writes**, **monotonic reads**, **monotonic writes**, writes-follow-reads — per-client guarantees layered on eventual consistency. - Typical implementations: sticky routing to one replica, or reading from a replica that has caught up to the client's last-seen version. ## Replication @@ -41,7 +39,7 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### Single-leader replication - All writes go to one leader; **sync** followers are durable but add latency, **async** followers are fast but lose recent writes on failover. -- **Semi-sync** waits for at least one follower. +- Semi-sync waits for at least one follower. - **Replication lag** breaks read-your-writes: a user can't see their own update right after writing. ### Multi-leader and leaderless @@ -64,11 +62,12 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### Hash vs range partitioning - **Hash** spreads load evenly but loses range scans; **range** keeps keys sorted but sequential keys (timestamps) create hot spots. +- Compound keys combine both: hash the first component to pick a partition, keep rows sorted by the rest (Cassandra partition + clustering key). ### Consistent and rendezvous hashing - **Consistent hashing**: nodes and keys on a ring; a key belongs to the next node clockwise, so adding a node moves ~**`1/N`** of keys. **Virtual nodes** smooth the load. -- **Rendezvous hashing**: owner = node with the highest `hash(node, key)` — no ring, same minimal movement. +- Rendezvous hashing: owner = node with the highest `hash(node, key)` — no ring, same minimal movement. ### Hot keys and secondary indexes @@ -84,8 +83,8 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### Raft -- **Leader election**: randomized timeouts avoid split votes; one vote per node per **term**. -- **Log replication**: an entry commits once a **majority** stores it. +- **Leader election**: randomized timeouts avoid split votes; one vote per node per term. +- Log replication: an entry commits once a **majority** stores it. - **Election restriction**: only a candidate with an up-to-date log can win, so committed entries survive. ### Raft reads and membership @@ -95,29 +94,27 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### Paxos -- Two phases — **prepare/promise**, then **accept/accepted** — across proposers and acceptors; same guarantees as Raft, harder to implement (Multi-Paxos adds a stable leader). +- Two phases across proposers and acceptors: **prepare/promise**, then **accept/accepted**. +- Same safety as Raft but harder to implement; **Multi-Paxos** adds a stable leader to skip the first phase. ### Leases and fencing tokens -- A **lease** is a lock with a TTL; if the holder dies, it expires and someone else takes over. +- A lease is a lock with a TTL; if the holder dies, it expires and someone else takes over. - A holder paused past expiry (long GC) still thinks it's the leader — **split brain**. **Fencing tokens** (increasing per lease, checked by the storage) reject its stale writes. - **Redlock** depends on timing assumptions; use a consensus store (etcd, ZooKeeper) when correctness matters. ## Time, ordering, and IDs -### Physical clocks +### Physical clocks and TrueTime -- Clocks drift and NTP can step them backward, so wall-clock timestamps can't order events across nodes — last-write-wins silently loses data. +- Clocks drift and NTP can step them backward, so wall-clock timestamps can't order events across nodes — **last-write-wins** silently loses data. +- Spanner's **TrueTime** returns an uncertainty interval; a commit **waits out** the uncertainty (~ms) before becoming visible, giving external consistency. ### Logical clocks - **Lamport clock**: increment per event; on receive, `max(local, received) + 1`. A total order consistent with causality, but can't detect concurrency. -- **Vector clock**: one counter per node; comparing two vectors tells **happened-before** from **concurrent**. -- **Hybrid logical clock (HLC)**: physical time + logical counter — causal and close to wall time (CockroachDB). - -### TrueTime - -- Spanner's **TrueTime** returns an uncertainty interval; a commit **waits out** the uncertainty (~ms) before becoming visible, giving external consistency. +- **Vector clock**: one counter per node; comparing two vectors tells **happened-before** from concurrent. +- Hybrid logical clock (HLC): physical time + logical counter — causal and close to wall time (CockroachDB). ### Distributed IDs @@ -132,13 +129,13 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### Two-phase commit -- Coordinator asks participants to **prepare** (vote and hold locks), then **commit** or **abort**. +- Coordinator asks participants to prepare (vote and hold locks), then commit or abort. - Atomic but **blocking**: if the coordinator dies after prepare, participants hold locks until it returns. **3PC** avoids that only without partitions, so it's rarely used. ### Sagas - A chain of local transactions, each with a **compensating action**; eventual consistency, **no isolation** (others see intermediate states). -- **Orchestration** (a central coordinator) vs **choreography** (services react to each other's events). +- **Orchestration** (a central coordinator) vs choreography (services react to each other's events). ### Effectively-once processing @@ -155,7 +152,7 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### CRDTs - Merges are **commutative, associative, idempotent**, so replicas converge without coordination. -- **G-Counter** (per-node counts, summed), **PN-Counter** (two G-Counters), **OR-Set** (add wins over concurrent remove), **LWW-Register**. +- **G-Counter** (per-node counts, summed), PN-Counter (two G-Counters), **OR-Set** (add wins over concurrent remove), LWW-Register. ### OT vs CRDT for collaborative editing @@ -166,13 +163,14 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. ### Timeouts and retries -- Put a **timeout** on every remote call. +- Put a timeout on every remote call. - Retry only **idempotent** operations, with **exponential backoff + jitter** so retries don't synchronize. - A **retry budget** (e.g. retries ≤ 10% of requests) stops retries from multiplying load during an outage. ### Circuit breakers - **Closed** (normal) → **open** after an error threshold (fail fast) → **half-open** (let a probe through, then close or reopen). +- Keep one breaker per dependency and pair it with a fallback (cached or default response). ### Bulkheads, backpressure, load shedding @@ -215,13 +213,13 @@ Key distributed-systems building blocks and trade-offs, grouped by subtopic. - Each partition has a leader and followers; the **ISR** (in-sync replicas) are followers caught up within `replica.lag.time.max.ms`. - **`acks=all`** + **`min.insync.replicas=2`** (with replication factor 3) acknowledges a write only once 2 replicas have it — survives one broker loss without losing acknowledged data. -- **`unclean.leader.election.enable=false`** (default) refuses to elect an out-of-sync replica, choosing unavailability over data loss. +- `unclean.leader.election.enable=false` (default) refuses to elect an out-of-sync replica, choosing unavailability over data loss. ### Kafka rebalancing, compaction, KRaft -- A **rebalance** reassigns partitions when consumers join or leave; the **KIP-848** consumer protocol (GA in **Kafka 4.0**) makes it incremental and broker-driven instead of stop-the-world. +- A rebalance reassigns partitions when consumers join or leave; the **KIP-848** consumer protocol (GA in Kafka 4.0) makes it incremental and broker-driven instead of stop-the-world. - **Log compaction** keeps only the latest record per key (a delete = `null` tombstone) — for changelogs and state restoration. -- **Kafka 4.0 (March 2025) removed ZooKeeper**: metadata lives in a Raft quorum of controllers (**KRaft**). +- **Kafka 4.0 (March 2025) removed ZooKeeper**: metadata lives in a Raft quorum of controllers (KRaft). ### Event sourcing and CQRS diff --git a/docs/topics/ethereum.md b/docs/topics/ethereum.md index b92ff93..abb66c3 100644 --- a/docs/topics/ethereum.md +++ b/docs/topics/ethereum.md @@ -7,82 +7,148 @@ What experienced Ethereum engineers forget before an interview, grouped by subto ### Account model - **EOA** (controlled by a secp256k1 key) vs **contract account** (controlled by its code); only EOAs originate transactions. -- **Nonce** counts sent transactions per EOA — prevents replay and orders transactions; a stuck transaction blocks all later ones until replaced by the same nonce with a higher fee. -- Address derivation: EOA = last 20 bytes of `keccak256(pubkey)`; `CREATE` = last 20 bytes of `keccak256(rlp([sender, nonce]))`; `CREATE2` = deterministic from a salt and init code, so a not-yet-deployed address is predictable. +- The **nonce** counts an EOA's sent transactions — it prevents replay and orders them. +- A stuck transaction blocks every later nonce until replaced by one with the same nonce and a higher fee. -### Transaction types and lifecycle +### Address derivation -- Types: legacy, **2930** (access lists), **1559** (base fee + priority fee), **4844** (blob), **7702** (set-code, EOA delegates to contract code, from Pectra, May 2025). -- Lifecycle: signed → propagated to the **mempool** → a **builder** assembles a block (MEV-Boost / PBS separates block-building from proposing) → the proposer publishes it → validators **attest** → finalized after **2 epochs (~12.8 min)**. -- Before finality a block can be **reorged**; exchanges and bridges wait for finality or a number of confirmations. +- EOA: last 20 bytes of `keccak256(pubkey)`. +- **`CREATE`**: last 20 bytes of `keccak256(rlp([sender, nonce]))`. +- **`CREATE2`**: derived from the deployer, a salt, and the init code hash, so an address is predictable before deployment (counterfactual wallets). + +### Transaction types + +| Type | EIP | Adds | +|---|---|---| +| 0 | legacy | single `gasPrice` | +| 1 | 2930 | access lists (pre-warmed addresses and slots) | +| 2 | **1559** | base fee + priority fee | +| 3 | **4844** | blobs for rollup data | +| 4 | **7702** | set-code: an EOA delegates to contract code (Pectra, May 2025) | + +### Transaction lifecycle + +- Signed → propagated to the **mempool** → a builder assembles a block (**PBS** via MEV-Boost separates building from proposing) → the proposer publishes it → validators attest. +- **Finalized after 2 epochs (~12.8 min)**; before that a block can be reorged, so exchanges and bridges wait for finality or several confirmations. ## Gas and fees ### EIP-1559 fee market -- Fee = `gasUsed × (baseFee + priorityFee)`. **Base fee** is set by the protocol, moves **±12.5%** per block toward a 50%-full target, and is **burned**. **Priority fee** (tip) goes to the proposer. -- Users set `maxFeePerGas` / `maxPriorityFeePerGas`; unused headroom is refunded. -- Only the **base fee** is burned (the tip is not), so ETH supply shrinks whenever burn exceeds new issuance. +- Fee = `gasUsed × (baseFee + priorityFee)`; users set `maxFeePerGas` and `maxPriorityFeePerGas`, and unused headroom is refunded. +- The **base fee** moves up to **±12.5%** per block toward a 50%-full target and is **burned**; the tip goes to the proposer. +- ETH supply shrinks whenever the burn exceeds new issuance. + +### Calldata costs -### Gas costs and refunds +- A transaction costs **21,000** gas base; calldata costs 16 gas per non-zero byte, 4 per zero byte. +- **EIP-7623** (Pectra) adds a floor of 40/10 gas per non-zero/zero byte for data-heavy transactions, pushing bulk data to blobs. -- Base transaction cost **21,000** gas; calldata costs **16 gas/non-zero byte, 4 gas/zero byte**. -- **EIP-7623** (Pectra, May 2025) adds a calldata **floor price** (40/10 gas per non-zero/zero byte) for data-heavy transactions, pushing bulk data to blobs. -- **EIP-7825** (Fusaka, Dec 2025) caps a single transaction at **2²⁴ ≈ 16.7M gas**, independent of the block gas limit. -- **Cold/warm access** (EIP-2929): first touch of a storage slot in a transaction (cold `SLOAD`) costs **2,100**; first touch of an address (cold `CALL`/`BALANCE`/`EXT*`) costs **2,600**; any warm (already-touched) access costs **100**. -- Clearing a storage slot to zero gives a partial gas refund, capped at **1/5** of the transaction's total gas (EIP-3529, down from the pre-London 1/2). -- The per-block gas limit is not fixed by protocol — it moves by validator vote, roughly ±1/1024 per block — so treat any specific number as a snapshot, not a constant. +### Storage access costs + +- EIP-2929: the first (**cold**) `SLOAD` of a slot costs **2,100**; the first touch of an address (`CALL`, `BALANCE`, `EXT*`) costs 2,600; any warm access costs 100. +- Clearing a slot to zero earns a refund, capped at **1/5** of the transaction's gas since London (EIP-3529, was 1/2). + +### Gas limits + +- **EIP-7825** (Fusaka, December 2025) caps one transaction at **2²⁴ ≈ 16.7M gas**, whatever the block limit. +- The block gas limit isn't fixed by protocol: validators move it by up to ~1/1024 per block, so any number is a snapshot. ## EVM and storage ### Execution model -- Stack-based, **256-bit words**, stack depth **1024** (only the top 16 items are directly reachable — the source of Solidity's "stack too deep"). -- Memory expansion cost is **quadratic** in size; storage (`SSTORE`/`SLOAD`) is the most expensive resource, priced per 32-byte slot. -- Storage layout: variables under 32 bytes pack into shared slots; a `mapping`'s value slot is `keccak256(key . baseSlot)`; dynamic arrays store length at the slot and elements at `keccak256(slot)`. +- Stack-based, **256-bit words**, stack depth 1024 — only the top **16** items are reachable, the source of "stack too deep". +- Memory expansion cost is **quadratic** in size; storage is the most expensive resource, priced per 32-byte slot. + +### Storage layout + +- Variables under 32 bytes **pack** into shared slots in declaration order. +- A `mapping` value lives at **`keccak256(key . slot)`**; a dynamic array stores its length at the slot and elements from `keccak256(slot)`. + +### Transient storage and SELFDESTRUCT -### Transient storage and logs +- **Transient storage** (`TSTORE`/`TLOAD`, EIP-1153, Cancun) costs 100 gas and is cleared after each transaction — cheap reentrancy locks. +- **`SELFDESTRUCT`** only sends ETH since Cancun (**EIP-6780**); code and storage are deleted only when called in the same transaction as creation. -- **Transient storage** (`TSTORE`/`TLOAD`, EIP-1153, Cancun) is cleared after each transaction — cheap reentrancy locks without permanent storage cost. -- **Logs** (events) live in the receipt, not storage: up to **3 indexed** parameters become searchable topics (plus the event signature hash as topic 0); contracts cannot read their own logs. -- `CREATE`/`CREATE2` addresses as above; **`SELFDESTRUCT`** no longer deletes code/storage or refunds gas except when called in the same transaction as creation (EIP-6780, neutered post-Cancun). +### Events and logs + +- Logs live in the **receipt**, not storage — far cheaper, but contracts can't read them. +- Up to **3 indexed** parameters become searchable topics, plus the event signature hash as topic 0. ### World state and light clients -- World state (address → account) is a **Merkle Patricia Trie**; each contract's storage is its own trie. A block header commits to three roots: **state root**, **transactions root**, **receipts root**. -- A single root hash lets a **light client** verify a balance, transaction, or receipt with a compact **Merkle proof**, without downloading the full state — the mechanism behind "trust-minimized" wallets and cross-chain bridges that verify L1 state. +- World state (address → account) is a **Merkle Patricia Trie**; each contract's storage is its own trie. +- A block header commits to the state, transactions, and receipts roots, so a **light client** verifies a balance or receipt with a Merkle proof instead of the full state. + +## Solidity + +### Data locations + +- **`storage`** is persistent state; `memory` is per call; **`calldata`** is read-only input — the cheapest location for external array and struct parameters. +- Storage → memory assignment **copies**; memory → memory copies only the reference; a local `storage` variable is a pointer into state. + +### Checked arithmetic + +- Overflow and underflow **revert since 0.8** (`Panic(0x11)`); `unchecked { }` opts out for proven-safe math. +- Division **truncates**, so multiply before dividing to keep precision. + +### Gas optimization patterns + +- **Pack** small variables that are read and written together into one slot. +- **`constant`/`immutable`** values are embedded in bytecode — no `SLOAD`. +- Cache storage reads in local variables inside loops; use `calldata` parameters. +- **Custom errors** (0.8.4) are cheaper than revert strings; 0.8.22 made simple loop-counter increments unchecked automatically. ## Calls and ABI -### Dispatch and low-level calls +### Function selectors and fallback + +- Calldata starts with a 4-byte **selector** (first 4 bytes of `keccak256("fn(types)")`) followed by ABI-encoded arguments; selectors can collide, a risk in proxy dispatchers. +- No matching selector → **`fallback()`**; plain ETH with empty calldata → **`receive()`** (or a payable fallback); a contract with neither rejects plain ETH. + +### call, delegatecall, staticcall + +- `call` runs in the callee's context; **`delegatecall`** runs the callee's code on the **caller's** storage, `msg.sender`, and `msg.value`; `staticcall` forbids state changes. +- The **63/64 rule**: a call forwards at most 63/64 of remaining gas, keeping 1/64 for the caller. -- A call's `data` starts with a 4-byte **selector** (first 4 bytes of `keccak256("fn(types)")`) followed by ABI-encoded arguments; different functions can collide on a selector by chance, a real risk in proxy dispatchers. -- No matching selector → **`fallback()`** runs; plain ETH transfer with empty calldata → **`receive()`** (or payable `fallback` if no `receive`); a contract with neither rejects plain ETH. -- **`call`** runs in the callee's context; **`delegatecall`** runs the callee's code in the **caller's** storage/`msg.sender`/`msg.value`; **`staticcall`** forbids state changes. -- The **63/64 rule**: a call forwards at most 63/64 of remaining gas, leaving 1/64 with the caller — relevant for reentrancy-guard gas-griefing analysis. -- Always check the `bool` return of a low-level `call`; `abi.encodePacked` of several dynamic arguments can collide (`("a","bc")` vs `("ab","c")`) — use `abi.encode` when hashing for signatures. +### ABI encoding pitfalls + +- Low-level `call` returns a **`bool`** instead of reverting — always check it. +- **`abi.encodePacked`** of several dynamic values can collide (`("a","bc")` vs `("ab","c")`) — hash `abi.encode` output for signatures. ## Smart contract security -### Reentrancy, access control, oracles +### Reentrancy + +- An external call re-enters before state updates: **single-function**, cross-function, or **read-only** (a view function returns pre-update state to another protocol). +- Fix with **Checks-Effects-Interactions** plus a reentrancy guard. + +### Access control + +- Missing modifiers and **unprotected initializers** let anyone call admin functions. +- Authenticate with `msg.sender`, never **`tx.origin`** — a malicious contract the owner calls can pass a `tx.origin` check. + +### Oracle manipulation + +- A DEX **spot price** can be moved within one transaction using a flash loan, then read by a victim protocol. +- Use **TWAPs** or a push oracle such as Chainlink, and check staleness. -- **Reentrancy** (single-function, cross-function, and **read-only** reentrancy via view functions reading pre-update state) — fix with **Checks-Effects-Interactions** plus a guard. -- **Access control** — missing `onlyOwner`, unprotected initializers, using `tx.origin` instead of `msg.sender` for auth. -- **Oracle/price manipulation** — spot DEX prices moved within one transaction via flash loan; use TWAPs or a push oracle like Chainlink instead. -- **Signature replay/malleability** — missing nonce/chainId/deadline in signed messages; `s` can be flipped (`n - s`) unless the verifier enforces low-`s`. +### Signature replay and malleability -### Unchecked calls, proxies, token quirks +- Signed messages without a **nonce, chainId, and deadline** can be replayed on another chain or later. +- ECDSA `s` can be flipped (`n − s`) into a second valid signature — enforce **low-`s`** and never use a signature as a unique ID. -- **Unchecked calls** — ignoring a low-level `call`'s return, or `transfer`/`transferFrom` on non-standard ERC-20s (use SafeERC20). -- **Delegatecall storage collisions**, **uninitialized proxies**, **fee-on-transfer/rebasing tokens** breaking balance assumptions, **unbounded-loop DoS**. -- Integer over/underflow is checked by default **since Solidity 0.8** — still possible inside `unchecked` blocks. +### Token integration quirks + +- Non-standard ERC-20s return nothing or `false` from `transfer` — use **SafeERC20**. +- **Fee-on-transfer** and rebasing tokens break "amount sent = amount received" assumptions — measure balance deltas. +- Unbounded loops over user-growable arrays become a gas **denial of service**. ## Upgradeability ### Proxy patterns -- All upgrade patterns route calls through a proxy that holds **storage** and `delegatecall`s to an **implementation**. - | Pattern | Upgrade logic lives in | Note | |---|---|---| | **Transparent** | proxy (admin-only path) | admin can't call the implementation | @@ -90,57 +156,92 @@ What experienced Ethereum engineers forget before an interview, grouped by subto | **Beacon** | shared beacon contract | upgrades many proxies at once | | **Diamond** (EIP-2535) | proxy routing to many facets | per-selector implementations | -- Implementation address stored at the pseudo-random **EIP-1967** slot to avoid collision with the implementation's own variables; new versions must only **append** storage, never reorder — or use **ERC-7201 namespaced storage**. -- Constructors don't run through a proxy — use **initializer** functions, and call `_disableInitializers()` in the implementation's own constructor so it can't be initialized directly. +- Every pattern keeps **storage in the proxy** and `delegatecall`s the implementation. + +### Proxy storage and initializers + +- The implementation address sits at the pseudo-random **EIP-1967** slot, away from the implementation's own variables. +- New versions only append storage variables, never reorder — or use **ERC-7201** namespaced storage. +- Constructors don't run through a proxy: use **initializers**, and call `_disableInitializers()` in the implementation's constructor. ## Signatures and standards ### Signing -- ECDSA over **secp256k1**; signature is `(v, r, s)`. **`ecrecover`** returns `address(0)` on an invalid signature — always check. -- **EIP-191** (`personal_sign`) prefixes messages so a signed message can never double as a valid transaction. -- **EIP-712** signs typed structured data with a domain separator (name, version, chainId, verifying contract) — human-readable in wallets, not replayable across contracts/chains. -- **ERC-1271** lets smart-contract wallets "sign" via `isValidSignature`; **EIP-2612 `permit`** and **Permit2** allow ERC-20 approvals via an off-chain signature instead of a transaction. +- ECDSA over **secp256k1**, signature `(v, r, s)`; **`ecrecover`** returns `address(0)` for an invalid signature — always check. +- **EIP-191** (`personal_sign`) prefixes messages so a signed message can never double as a transaction. + +### EIP-712 and permits -### Token and account standards +- **EIP-712** signs typed data with a domain separator (name, version, chainId, contract) — readable in wallets, not replayable across contracts or chains. +- **ERC-1271** lets contract wallets "sign" via `isValidSignature`. +- **EIP-2612 `permit`** and Permit2 grant ERC-20 approvals by signature instead of a transaction. -- **ERC-20** fungible, **ERC-721** NFTs (unique `tokenId`, `safeTransferFrom` checks `onERC721Received`), **ERC-1155** multi-token with batch transfers, **ERC-4626** tokenized yield vaults. -- **ERC-4337** account abstraction: users sign **UserOperations**, **bundlers** submit them to a singleton **EntryPoint**, **paymasters** can sponsor gas — all without protocol changes. -- **EIP-7702** complements it: an EOA can delegate to smart-wallet code while keeping its address, avoiding fund migration. +### Token standards + +| Standard | Kind | Note | +|---|---|---| +| ERC-20 | fungible | `approve` + `transferFrom`; infinite approvals are a risk | +| ERC-721 | NFT, unique `tokenId` | `safeTransferFrom` calls `onERC721Received` (a reentrancy point) | +| ERC-1155 | multi-token | batch transfers | +| ERC-4626 | tokenized vault | share/asset math; first-depositor inflation attack | + +### Account abstraction + +- **ERC-4337**: users sign UserOperations, bundlers submit them to a singleton EntryPoint, and **paymasters** can sponsor gas — no protocol change. +- **EIP-7702** lets an existing EOA delegate to smart-wallet code while keeping its address. ## Consensus (PoS) -### Gasper +### Slots, epochs, finality -- Time is divided into **12 s slots** and **32-slot (~6.4 min) epochs**; each slot has a randomly chosen proposer and a committee that **attests**. -- **Gasper** = **LMD-GHOST** (fork choice: heaviest attestation weight) + **Casper FFG** (finality: 2/3-stake-voted checkpoints become justified, then finalized ~2 epochs later). -- **Slashing** burns stake for provably malicious behavior (double proposing, surrounding/contradictory attestations); smaller **inactivity leak** penalties hit offline validators, worse during non-finality. -- **Maximum effective balance raised to 2048 ETH** (EIP-7251, Pectra) lets large operators consolidate validators instead of running many at the old 32 ETH cap. +- **12 s slots**, 32-slot (~6.4 min) epochs; each slot has a random proposer and a committee that attests. +- **Gasper** = LMD-GHOST (fork choice by attestation weight) + **Casper FFG** (checkpoints with 2/3 of stake become justified, then finalized). + +### Slashing and penalties + +- **Slashing** burns stake for provable misbehavior: double proposals or contradictory (surround or double) attestations. +- The **inactivity leak** drains offline validators while the chain can't finalize. +- EIP-7251 (Pectra) raised the **maximum effective balance to 2048 ETH**, so large operators consolidate validators. ## Scaling ### Rollups -- **Optimistic** (Arbitrum, Optimism, Base) assumes validity, allows **fraud proofs** during a challenge window (~7 days) — slow withdrawals to L1. -- **ZK** (zkSync, Starknet, Scroll, Linea) posts a validity proof per batch — fast finality on L1 once verified, at the cost of expensive proof generation. -- Both post data to L1 for **data availability**; most still rely on a centralized **sequencer** for ordering, with forced-inclusion mechanisms as a censorship-resistance backstop. +| Type | Examples | Proof | L1 withdrawal | +|---|---|---|---| +| **Optimistic** | Arbitrum, OP Mainnet, Base | fraud proofs in a challenge window | ~7 days | +| **ZK** | zkSync, Starknet, Scroll, Linea | validity proof per batch | once the proof verifies | + +### Sequencers and data availability + +- Rollups post transaction data to L1 for **data availability**, so anyone can rebuild state and challenge or exit. +- Most use a **centralized sequencer** for ordering, with forced inclusion through L1 as the censorship backstop. ### Blobs and PeerDAS -- **EIP-4844** blobs are cheap, temporary data separate from calldata, pruned after a retention window measured in days, not permanently stored. -- **Fusaka**'s **PeerDAS** lets nodes verify blob availability by sampling instead of downloading every blob, and ships alongside **Blob Parameter Only (BPO)** forks that step up blob target/max capacity without further protocol changes. +- **EIP-4844** blobs are cheap, temporary data (pruned after ~18 days), separate from calldata. +- **PeerDAS** (Fusaka) lets nodes check blob availability by **sampling**; Blob Parameter Only (BPO) forks then raise blob capacity without other changes. ## DeFi and MEV ### AMMs -- Uniswap v2: constant product **`x·y = k`**; larger trades move price more (**price impact**), hence **slippage** limits. -- **Impermanent loss**: as price moves, the pool sells the rising asset, so LPs can end up worse off than holding, offset by trading fees. -- **v3** adds **concentrated liquidity** in chosen price ranges (positions as NFTs, more capital-efficient); **v4** uses a singleton contract plus **hooks** for custom pool logic. +- Uniswap v2: constant product **`x·y = k`**; larger trades move the price more, hence slippage limits. +- **Impermanent loss**: the pool sells the rising asset, so LPs can end up worse than holding, offset by fees. +- v3 adds **concentrated liquidity** in chosen price ranges; v4 uses a singleton contract with hooks. + +### Lending and flash loans + +- Lending (Aave, Compound) is over-collateralized; a **health factor** below 1 allows liquidation at a discount. +- **Flash loans** are uncollateralized but must be repaid in the same transaction or everything reverts — turning manipulable prices into large exploits. + +### MEV + +- **Sandwich attacks** (front-run and back-run a victim's swap), arbitrage back-running, and liquidations. +- Mitigations: private order flow (Flashbots Protect), tight **slippage limits**, and PBS so small validators earn MEV too. -### Lending and MEV +### Bridge exploits -- Lending (Aave, Compound): over-collateralized borrowing; **health factor** below 1 triggers **liquidation** at a discount. -- **Flash loans** — uncollateralized, must be repaid in the same transaction or the whole transaction reverts; legitimate for arbitrage/liquidation/collateral swaps, but turn manipulable spot prices into large exploits. -- **MEV** types: sandwich attacks, back-running/arbitrage, liquidations. Mitigations: private order flow (e.g. Flashbots Protect), slippage limits, PBS to keep small validators competitive. -- Bridges are prime exploit targets since they hold large locked pools: **Ronin** (compromised validator keys), **Wormhole** (signature verification bug), **Nomad** (bad initialization let anyone forge messages). +- Bridges hold large locked pools, making them prime targets. +- **Ronin** (5 of 9 validator keys compromised), **Wormhole** (signature verification bypass), **Nomad** (bad initialization let anyone forge messages). diff --git a/docs/topics/frontend.md b/docs/topics/frontend.md index 38d9bc7..5a89226 100644 --- a/docs/topics/frontend.md +++ b/docs/topics/frontend.md @@ -24,11 +24,16 @@ What experienced frontend engineers forget before an interview, grouped by subto - **Islands architecture**: the page is static HTML; only interactive islands ship JS and hydrate independently. - **React Server Components** render only on the server and send serialized output, not code — less JS, but props crossing into client components must be serializable. +### Client-side routing + +- The router intercepts navigation and updates the URL with the **History API** (`pushState`), no full reload. +- A refresh on `/profile` hits the **server**, which must fall back to `index.html` for unknown paths or it 404s. + ## Browser pipeline ### Critical rendering path -- HTML → **DOM**, CSS → **CSSOM** → **render tree** (visible nodes) → **layout** → **paint** → **composite**. +- HTML → DOM, CSS → CSSOM → **render tree** (visible nodes) → layout → paint → composite. - CSS is **render-blocking**; a classic `