From a354cc8d5c2d37b704946afb73c3d37d3db2f657 Mon Sep 17 00:00:00 2001 From: Adam Djellouli Date: Thu, 12 Nov 2026 15:18:16 +0100 Subject: [PATCH 1/2] Add interview pattern guides to graph, DP, backtracking, sets, searching and sorting notes Each note was checked against the common LeetCode problems in its area. The missing patterns are now explained in short sections, every major technique starts with a "First step" box, and each note ends with a "Reading the Problem Statement" table that maps problem wording to code decisions. Existing content is kept; one misleading sentence about cycle detection in the topological sort section was reworded. Co-Authored-By: Claude Opus 5.5 --- notes/backtracking.md | 425 +++++++++++++++++++++++++++++++++ notes/dynamic_programming.md | 357 +++++++++++++++++++++++++++ notes/graphs.md | 365 +++++++++++++++++++++++++++- notes/math_set_relationship.md | 348 +++++++++++++++++++++++++++ notes/searching.md | 366 ++++++++++++++++++++++++++++ notes/sorting.md | 311 ++++++++++++++++++++++++ 6 files changed, 2170 insertions(+), 2 deletions(-) diff --git a/notes/backtracking.md b/notes/backtracking.md index 8e3976c..1d54827 100644 --- a/notes/backtracking.md +++ b/notes/backtracking.md @@ -297,8 +297,148 @@ One practical “do/don’t” with this style: - Do make `place/unplace` symmetrical (every change you make must be undone). - Don’t mutate shared state without undoing it, or you’ll get “ghost effects” where one branch pollutes another. +### Choose, Explore, Unchoose + +> **First step:** Before writing code, name the one decision made at each level of the recursion, what the partial solution holds, and the test that says it is complete. Everything else in the function follows from those three answers. + +The generic framework above has five callbacks, but in interview problems you rarely write them as separate functions. The shape collapses into three lines inside a loop. You choose a candidate by changing shared state, explore by recursing, and unchoose by reverting exactly the change you made. The shared state is usually a `path` list plus whatever bookkeeping makes validity checks cheap, such as a set of used columns or a running sum. + +The recorded answer must be a copy. `path` is a single list that every branch mutates, so `result.append(path)` stores a reference that later pops will empty, and the result ends up full of identical empty lists. Write `result.append(path[:])` or `path.copy()`. When the partial answer is a string built with `s + ch`, a new string is created at every step, so no copy is needed, at the price of $O(L)$ work per step. + +The Python-ish framework above already has this shape, with `partial.append` as the choose and `partial.pop` as the unchoose. Total time is roughly the number of nodes in the recursion tree times the work done per node, plus the cost of copying each answer. The most common mistake is an unchoose that does not mirror the choose, for example adding to a set before the recursive call and forgetting to remove it on one of the return paths. + +### Estimating Cost From the Output Size + +When a problem asks you to return every solution, the size of the answer is a lower bound on the running time. If there are $S$ answers of length $L$, copying them alone costs $O(S \cdot L)$, and no pruning can beat that. So the first estimate is to count the answers. The three counts that appear constantly are + +$$ +\text{subsets: } 2^n \qquad \text{permutations: } n! \qquad \text{k-combinations: } \binom{n}{k} = \frac{n!}{k!(n-k)!} +$$ + +This gives the familiar bounds. Subsets (LeetCode 78) is $O(n \cdot 2^n)$, Permutations (LeetCode 46) is $O(n \cdot n!)$, and Combinations (LeetCode 77) is $O(k \cdot \binom{n}{k})$. It also explains the input limits. Since $2^{20}$ is about a million and $10!$ is about 3.6 million, subset problems allow $n$ up to about 20 while permutation problems stop around 8 to 11. + +For problems that return a single answer, a yes or no, or a count (Sudoku, N-Queens II, Partition to K Equal Sum Subsets), the output is tiny and the cost is the size of the pruned tree, which is hard to state exactly. There you quote a loose bound such as $O(k^n)$ for assigning $n$ items to $k$ buckets and rely on pruning in practice. The counting rules themselves are derived in [Math Set Relationships](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/notes/math_set_relationship.md). + +The common mistake is treating a counting question as an enumeration question. If only the number of answers is required, dynamic programming or a formula is often exponentially faster than listing them. + +### Start Index: Subsets and Combinations + +> **First step:** Each level picks the next element to add, and it may only pick from positions after the last one used. The partial solution is the chosen list, and for subsets every node of the tree is already an answer. + +Order does not matter in a subset or a combination, so {1, 2} and {2, 1} are the same answer. The standard way to generate each set exactly once is to always pick elements in increasing index order. The recursion carries a `start` index and the loop only considers positions from `start` onward. After picking index `i`, the child call receives `i + 1`, so nothing at or before `i` can be picked again. + +For Subsets (LeetCode 78) you record the path at every node, including the empty path at the root. For Combinations (LeetCode 77) and Combination Sum III (LeetCode 216) you record only when the path reaches length $k$ (and, for 216, when the sum equals the target). Combinations also has a cheap prune: if fewer elements remain than slots still to fill, the loop can stop early. + +```python +def combine(n, k): + result, path = [], [] + def backtrack(start): + if len(path) == k: + result.append(path[:]) + return + need = k - len(path) + for i in range(start, n - need + 2): # leave room for the remaining picks + path.append(i) + backtrack(i + 1) + path.pop() + backtrack(1) + return result +``` + +Letter Combinations of a Phone Number (LeetCode 17) and Letter Case Permutation (LeetCode 784) are the same shape with a different decision per level. The level is a digit or a character position, and the choices are its letters or its two cases. There is no start index because each position is decided exactly once, in order. Binary Watch (LeetCode 401) can be solved by choosing which LEDs are on with a start index, but a direct scan of all 720 times that counts set bits is simpler. + +Subsets runs in $O(n \cdot 2^n)$ and Combinations in $O(k \cdot \binom{n}{k})$. The most common mistake is passing `start + 1` instead of `i + 1` to the child, which lets later picks reach back to earlier elements and produces duplicates. + +### Used Array: Permutations + +> **First step:** Each level fills the next position of the arrangement, and the choice is any element not already placed. The path is complete when its length equals $n$. + +In a permutation order matters, so after choosing `nums[2]` the next position may still take `nums[0]`. A start index cannot express that. Instead keep a boolean array `used` (or a bitmask) and loop over every index, skipping those already in the path. + +```python +def permute(nums): + result, path = [], [] + used = [False] * len(nums) + def backtrack(): + if len(path) == len(nums): + result.append(path[:]) + return + for i in range(len(nums)): + if used[i]: + continue + used[i] = True + path.append(nums[i]) + backtrack() + path.pop() + used[i] = False + backtrack() + return result +``` + +Beautiful Arrangement (LeetCode 526) is this template with one extra validity check: value $v$ may go at position $p$ only if $v$ divides $p$ or $p$ divides $v$. Because the check runs before recursing, most branches die after a few levels, and since only the count is required, no path list is needed at all. + +Permutations run in $O(n \cdot n!)$ time with $O(n)$ extra space besides the output. The most common mistake is testing `nums[i] in path` instead of keeping a used array, which costs $O(n)$ per test and breaks as soon as the input contains equal values. + +### Reuse Allowed: Combination Sum + +> **First step:** Each level picks the next candidate to add, from the current index onward. The path is complete when the remaining target is exactly zero and invalid once it would go negative. + +Combination Sum (LeetCode 39) allows each number to be used any number of times, but [2, 2, 3] and [2, 3, 2] still count as the same answer. Keep the start index so the order is fixed, and pass `i` instead of `i + 1` to the child so the same element can be picked again. Passing `i + 1` gives each element at most once, as in Combination Sum II (LeetCode 40). Looping from 0 at every level would count ordered sequences instead, which is what Combination Sum IV (LeetCode 377) asks for, and since it only wants a count it is solved with dynamic programming. + +Sorting the candidates enables an early exit. Once `candidates[i]` exceeds the remaining target, every later candidate does too, so the loop can `break` rather than `continue`. + +```python +def combination_sum(candidates, target): + candidates.sort() + result, path = [], [] + def backtrack(start, remaining): + if remaining == 0: + result.append(path[:]) + return + for i in range(start, len(candidates)): + if candidates[i] > remaining: + break # sorted, nothing later fits + path.append(candidates[i]) + backtrack(i, remaining - candidates[i]) # i, not i + 1: reuse allowed + path.pop() + backtrack(0, target) + return result +``` + +The depth is at most the target divided by the smallest candidate, so the tree is exponential in that depth and the output size usually dominates. The most common mistake is keeping the `break` on unsorted input, where it silently drops valid answers. + +### Skipping Duplicates After Sorting + +> **First step:** Sort the input so equal values sit next to each other. At any one level of the tree each distinct value may be tried only once, while deeper levels may still use the next copy of it. + +Subsets II (LeetCode 90), Combination Sum II (LeetCode 40), and Permutations II (LeetCode 47) take input that may contain duplicates and ask for unique answers. Generating everything and deduplicating with a set of tuples works, but it wastes time exploring identical subtrees. The better fix is never to start two identical subtrees from the same parent. + +For the start-index family the rule is `if i > start and nums[i] == nums[i - 1]: continue`. The condition is `i > start` and not `i > 0` because the first candidate at a level is always allowed. If it equals the element just before it, that earlier element was the parent's choice, and taking another copy one level deeper is legitimate, since [1, 2, 2] must be produced. What is forbidden is two siblings with the same value, because they would grow identical subtrees. + +```python +def subsets_with_dup(nums): + nums.sort() + result, path = [], [] + def backtrack(start): + result.append(path[:]) + for i in range(start, len(nums)): + if i > start and nums[i] == nums[i - 1]: + continue # this value was already tried at this level + path.append(nums[i]) + backtrack(i + 1) + path.pop() + backtrack(0) + return result +``` + +Permutations II has no start index, so the rule becomes `if i > 0 and nums[i] == nums[i - 1] and not used[i - 1]: continue`. It forces equal values to be placed in their original left-to-right order, so each distinct arrangement appears once. Non-decreasing Subsequences (LeetCode 491) cannot be sorted, because sorting changes which subsequences exist. There the rule is a fresh `seen` set inside each call, skipping any value already tried at that level, and the same trick works whenever sorting is not allowed. + +The worst case is unchanged, but the work now tracks the number of distinct answers instead of every duplicate copy. The most common mistake is writing `i > 0` instead of `i > start`, which removes valid answers such as [1, 2, 2]. + ### N-Queens Problem +> **First step:** One decision per level is the column of the queen in the current row, so rows never conflict by construction. The partial solution is the list of columns chosen so far, complete when every row has a queen, and a column is invalid if it shares a column or diagonal with an earlier queen. + The N-Queens problem is a classic puzzle in which the goal is to place $N$ queens on an $N \times N$ chessboard such that no two queens threaten each other. In chess, a queen can move any number of squares along a row, column, or diagonal. Therefore, no two queens can share the same row, column, or diagonal. Objective: @@ -521,8 +661,68 @@ This is a good moment to connect the “why should I care?” dot: many real pro - Using heuristics to choose the order of columns to try first. - Converting the recursive solution to an iterative one using explicit stacks to handle larger values of $N$ without stack overflow. +#### Constant-Time Conflict Checks + +The implementation above rescans all earlier rows for every candidate. The usual solution for N-Queens (LeetCode 51) and N-Queens II (LeetCode 52) keeps three sets instead. Every square on the same descending diagonal has the same value of $r - c$, and every square on the same ascending diagonal has the same value of $r + c$. A placement is safe when its column, its $r - c$, and its $r + c$ are all unused. Arrays of size $2n - 1$ work too if $r - c$ is shifted by $n - 1$. + +```python +def total_n_queens(n): + cols, diag, anti = set(), set(), set() + def place(r): + if r == n: + return 1 + count = 0 + for c in range(n): + if c in cols or (r - c) in diag or (r + c) in anti: + continue + cols.add(c); diag.add(r - c); anti.add(r + c) + count += place(r + 1) + cols.remove(c); diag.remove(r - c); anti.remove(r + c) + return count + return place(0) +``` + +Each node now does $O(n)$ work instead of $O(n^2)$, and the number of nodes is still bounded by roughly $n!$. The most common mistake is using `abs(r - c)` as the diagonal key, which merges two different diagonals and rejects valid boards. + +### Sudoku Solver + +> **First step:** One decision is the digit for the next empty cell. The partial solution is the grid itself, complete when no empty cell remains, and a digit is invalid if its row, column, or 3 by 3 box already contains it. + +Sudoku Solver (LeetCode 37) needs one valid fill, not all of them. Collect the empty cells once, and keep a set of used digits for each row, each column, and each box, where the box index is `(r // 3) * 3 + c // 3`. The recursion walks the list of empty cells and returns `True` the moment it reaches the end. That `True` must travel all the way up without undoing anything, otherwise the solved grid is erased on the way out. + +```python +def solve_sudoku(board): + rows, cols, boxes = [set() for _ in range(9)], [set() for _ in range(9)], [set() for _ in range(9)] + empty = [] + for r in range(9): + for c in range(9): + d = board[r][c] + if d == '.': + empty.append((r, c)) + else: + rows[r].add(d); cols[c].add(d); boxes[r // 3 * 3 + c // 3].add(d) + def fill(k): + if k == len(empty): + return True + r, c = empty[k] + b = r // 3 * 3 + c // 3 + for d in "123456789": + if d in rows[r] or d in cols[c] or d in boxes[b]: + continue + board[r][c] = d; rows[r].add(d); cols[c].add(d); boxes[b].add(d) + if fill(k + 1): + return True + board[r][c] = '.'; rows[r].remove(d); cols[c].remove(d); boxes[b].remove(d) + return False + fill(0) +``` + +A large speedup comes from filling the empty cell with the fewest legal digits first instead of going in reading order. The worst case is $O(9^m)$ for $m$ empty cells, though real puzzles prune far below that. The most common mistake is continuing the loop after a solution was found, so later undo steps wipe the answer from the board. + ### Example: Maze Solver +> **First step:** One decision is the direction of the next move. The partial solution is the path of cells from the start, complete when it reaches the goal, and a move is invalid if it leaves the grid, hits a wall, or steps on a cell already on the path. + Given a maze represented as a 2D grid, find a path from the starting point to the goal using backtracking. The maze consists of open paths and walls, and movement is allowed in four directions: up, down, left, and right (no diagonal moves). The goal is to determine a sequence of moves that leads from the start to the goal without crossing any walls. Maze solving makes backtracking feel instantly real. You’re making choices (directions), you hit walls or loops (constraints), and you undo moves when you get stuck. Even if you never care about mazes, you do care about the pattern: this is the same structure as exploring possible decisions in a game AI, navigating states in a program, or searching combinations until you find one that works. @@ -766,6 +966,204 @@ Choose the search method according to the required result: The reusable pattern is to represent partial choices, check constraints, explore a candidate, and restore state. Understanding these responsibilities makes it easier to adapt the method to other constraint problems. +### Grid Search With In-Place Marking + +> **First step:** One decision is the neighbour to step to next. The partial solution is the current path, marked directly in the grid, and a step is invalid if it leaves the grid, hits a blocked cell, revisits a cell on the current path, or does not match what the problem expects next. + +Word Search (LeetCode 79) asks whether a word can be traced through horizontally or vertically adjacent cells without reusing a cell. Start a search from every cell. The recursive call carries the index of the letter being matched, temporarily overwrites the current cell with a marker such as `'#'`, tries the four neighbours, and writes the letter back before returning. + +The restore is essential here, unlike in the any-path maze. Whether a cell can continue a match depends on which letter you are on when you reach it, so a cell that failed as the fourth letter may succeed as the sixth letter along another route. A permanent visited set would wrongly rule it out. + +```python +def exist(board, word): + R, C = len(board), len(board[0]) + def dfs(r, c, k): + if k == len(word): + return True + if not (0 <= r < R and 0 <= c < C) or board[r][c] != word[k]: + return False + board[r][c] = '#' # mark as on the current path + found = (dfs(r + 1, c, k + 1) or dfs(r - 1, c, k + 1) or + dfs(r, c + 1, k + 1) or dfs(r, c - 1, k + 1)) + board[r][c] = word[k] # restore before returning + return found + return any(dfs(r, c, 0) for r in range(R) for c in range(C)) +``` + +Unique Paths III (LeetCode 980) uses the same mark and restore, but it counts paths from the start square to the end square that walk over every non-obstacle square exactly once. Count the walkable squares first. A path is complete only when it stands on the end square and the number of squares visited equals that count. Reaching the end early is a dead end, not an answer. The grid has at most 20 cells, which is the hint that exponential search is expected. + +Word Search costs $O(R \cdot C \cdot 3^{L})$ for a word of length $L$, since after the first step at most three directions are unvisited. The most common mistake is checking `k == len(word)` after the bounds check, which reports failure when the word ends on a border cell and the next call steps off the grid. + +### Word Search II With a Trie + +Word Search II (LeetCode 212) gives one board and many words. Running the single-word search for each word repeats the same board walks again and again. Instead put all words in a trie and walk the board and the trie together. A branch dies the moment the current prefix is not a child in the trie, which is far stronger pruning than any per-word check. + +Two details keep it fast. Store the complete word at its final trie node and remove it once found, so it is reported once. After returning from a child, delete trie nodes that have become empty, so later starting cells do not walk into exhausted branches. + +```python +def find_words(board, words): + root = {} + for w in words: + node = root + for ch in w: + node = node.setdefault(ch, {}) + node['$'] = w + R, C, found = len(board), len(board[0]), [] + def dfs(r, c, parent): + ch = board[r][c] + node = parent.get(ch) + if node is None: + return # no word continues this prefix + if '$' in node: + found.append(node.pop('$')) # report each word once + board[r][c] = '#' + for nr, nc in ((r + 1, c), (r - 1, c), (r, c + 1), (r, c - 1)): + if 0 <= nr < R and 0 <= nc < C: + dfs(nr, nc, node) + board[r][c] = ch + if not node: + del parent[ch] # prune an exhausted branch + for r in range(R): + for c in range(C): + dfs(r, c, root) + return found +``` + +The worst case is $O(R \cdot C \cdot 3^{L})$ for the longest word length $L$, plus the size of the trie, but the trie pruning makes typical runs much faster. The most common mistake is leaving found words in the trie, which reports the same word several times. + +### Partitioning a String + +> **First step:** One decision is where the next piece ends. The partial solution is the list of pieces so far, complete when the cut position reaches the end of the string, and a cut is invalid if the new piece breaks the rule (not a palindrome, not a valid IP octet, not one less than the previous number). + +Palindrome Partitioning (LeetCode 131) asks for every way to cut a string into palindromes. The recursion carries the start of the next piece and tries every end position. Checking each piece with slicing costs $O(n)$, so it pays to precompute a table `pal[i][j]` that says whether `s[i..j]` is a palindrome. The table fills from short substrings to long ones in $O(n^2)$. + +```python +def partition(s): + n = len(s) + pal = [[False] * n for _ in range(n)] + for i in range(n - 1, -1, -1): + for j in range(i, n): + pal[i][j] = s[i] == s[j] and (j - i < 2 or pal[i + 1][j - 1]) + result, path = [], [] + def backtrack(start): + if start == n: + result.append(path[:]) + return + for end in range(start, n): + if pal[start][end]: + path.append(s[start:end + 1]) + backtrack(end + 1) + path.pop() + backtrack(0) + return result +``` + +Restore IP Addresses (LeetCode 93) is the same cut-position recursion with exactly four pieces. Each piece has one to three digits, a value of at most 255, and no leading zero unless the piece is exactly "0". It also prunes on length: the characters left must be at least the number of pieces left and at most three times that number. Splitting a String Into Descending Consecutive Values (LeetCode 1849) tries every first piece except the whole string, after which each next piece must equal the previous value minus one. Since the next value is forced, the tree is very narrow. Compare values as integers, because pieces may have leading zeros. + +A string of length $n$ has $2^{n-1}$ ways to cut it, so Palindrome Partitioning is $O(n \cdot 2^n)$ in the worst case, for example a string of identical letters. The most common mistake in Restore IP Addresses is forgetting the leading-zero rule, which accepts pieces like "01". + +### Generating Balanced Parentheses + +> **First step:** Each level appends either an opening or a closing bracket. The partial solution is the string so far with its open and close counts, complete at length $2n$, and a bracket is invalid if it would make open exceed $n$ or close exceed open. + +Generate Parentheses (LeetCode 22) shows how much a validity check during the search saves. Generating all $2^{2n}$ strings and filtering at the end is wasteful. Allowing `(` only while fewer than $n$ have been used, and `)` only while it closes something, means every leaf is already a valid answer. + +```python +def generate_parenthesis(n): + result = [] + def backtrack(s, open_, close): + if len(s) == 2 * n: + result.append(s) + return + if open_ < n: + backtrack(s + "(", open_ + 1, close) + if close < open_: + backtrack(s + ")", open_, close + 1) + backtrack("", 0, 0) + return result +``` + +Remove Invalid Parentheses (LeetCode 301) runs the logic in reverse. First scan once to find how many `(` and how many `)` must be removed at minimum. Then backtrack over positions, deciding to keep or delete each bracket while those two budgets last, and accept a string when both budgets are zero and it is balanced. To avoid duplicate results, when several identical brackets sit in a row, only try deleting the first of the run at each level. A breadth-first search that deletes one character per level is a common alternative. + +The number of answers to Generate Parentheses is the Catalan number $\frac{1}{n+1}\binom{2n}{n}$, which grows roughly like $4^n / n^{3/2}$. The most common mistake is checking balance only at the end, which explores every one of the $2^{2n}$ strings. + +### Building Expressions With Operators + +> **First step:** One decision is the next operand (a run of digits) and the operator placed before it. The partial state is the expression text, its value so far, and the last term added, complete when all digits are used, and an operand is invalid if it has a leading zero. + +Expression Add Operators (LeetCode 282) inserts `+`, `-`, or `*` between digits to hit a target. Addition and subtraction are easy to evaluate on the fly. Multiplication binds tighter, so when you append `* cur` the last term has already been added to the running value and must be taken back out. If the value is `value` and the last term is `last`, the new value is `value - last + last * cur` and the new last term is `last * cur`. For subtraction the last term is stored as `-cur`, so a following multiplication is handled the same way. + +```python +def add_operators(num, target): + result = [] + def backtrack(i, expr, value, last): + if i == len(num): + if value == target: + result.append(expr) + return + for j in range(i, len(num)): + if j > i and num[i] == '0': + break # no leading zeros in an operand + s = num[i:j + 1] + cur = int(s) + if i == 0: + backtrack(j + 1, s, cur, cur) + else: + backtrack(j + 1, expr + "+" + s, value + cur, cur) + backtrack(j + 1, expr + "-" + s, value - cur, -cur) + backtrack(j + 1, expr + "*" + s, value - last + last * cur, last * cur) + backtrack(0, "", 0, 0) + return result +``` + +Between each pair of adjacent digits there are four options (join, plus, minus, times), so the search is $O(n \cdot 4^n)$ including the string building. The most common mistake is evaluating strictly left to right, which turns `2+3*2` into 10 instead of 8. + +### Filling K Buckets + +> **First step:** One decision is which bucket receives the next item, with items taken largest first. The partial solution is the list of bucket sums, complete when every item is placed, and a placement is invalid if it pushes a bucket over the target. + +Partition to K Equal Sum Subsets (LeetCode 698) asks whether an array can be split into $k$ groups with equal sums. The target per bucket is the total divided by $k$, and if the total is not divisible or some item exceeds the target the answer is immediately false. Sorting the items in decreasing order makes failures happen near the root, where they cut the most. + +The decisive prune is symmetry. Buckets are interchangeable, so placing the current item into two buckets that hold the same sum leads to identical subtrees. Skip a bucket whose sum was already tried for this item. This covers the important special case where an item fails in an empty bucket, after which every other empty bucket would fail too. + +```python +def can_partition_k(nums, k): + total = sum(nums) + if total % k: + return False + target = total // k + nums.sort(reverse=True) + if nums[0] > target: + return False + buckets = [0] * k + def place(i): + if i == len(nums): + return True # no bucket exceeds target, so all equal it + tried = set() + for b in range(k): + if buckets[b] + nums[i] > target or buckets[b] in tried: + continue + tried.add(buckets[b]) # equal sums give identical subtrees + buckets[b] += nums[i] + if place(i + 1): + return True + buckets[b] -= nums[i] + return False + return place(0) +``` + +Matchsticks to Square (LeetCode 473) is this function with $k = 4$. Find Minimum Time to Finish All Jobs (LeetCode 1723) and Fair Distribution of Cookies (LeetCode 2305) use the same item-to-bucket recursion but minimize the largest bucket, so they keep a global best and abandon any branch whose largest bucket already reaches it. Campus Bikes II (LeetCode 1066) assigns each worker a distinct bike, which is a used-array search that works for its tiny limits, though dynamic programming over a bitmask of used bikes is the safer answer. + +The worst case is $O(k^n)$, and a bitmask dynamic programming alternative runs in $O(n \cdot 2^n)$. The most common mistake is omitting the symmetry prune, which explores each partition up to $k!$ times and times out. + +### Other Common Variations + +Maximum Length of a Concatenated String with Unique Characters (LeetCode 1239) is an include-or-exclude search over the strings. Drop any string that repeats a letter internally, turn each remaining string into a 26-bit mask, and include a string only when its mask shares no bit with the current mask. With at most 16 strings, the $2^n$ subsets are cheap. + +Path Sum II (LeetCode 113) is backtracking on a tree. Append the node to the path, subtract its value from the remaining sum, recurse into both children, and pop on the way out. Record a copy of the path only at a leaf where the remaining sum is zero. Values may be negative, so a running sum that overshoots the target is not a reason to stop early. + +Gray Code (LeetCode 89) can be built by backtracking that flips one bit at a time and skips codes already used. The direct formula `i ^ (i >> 1)` produces a valid sequence in $O(2^n)$ with no search, so it is usually the better answer. + ## List of Problems ### Permutations @@ -830,3 +1228,30 @@ Develop an algorithm to determine the optimal move for a player in a game of tic - [C++ Solution](https://github.com/djeada/Algorithms-And-Data-Structures/tree/master/src/backtracking/cpp/minimax) - [Python Solution](https://github.com/djeada/Algorithms-And-Data-Structures/tree/master/src/backtracking/python/minimax) + +## Reading the Problem Statement + +The wording and the constraints of a backtracking problem usually tell you which template to use before you design anything. This table maps common phrases to the decision they imply. + +| If the problem says | What it means for your code | +|---|---| +| "Return all" subsets, combinations, or arrangements | Backtracking that records a copy at each answer, with the output size setting the running time. | +| "Return the number of" ways | Look for dynamic programming or a formula first, and backtrack only when n is tiny. | +| "Return any valid solution" or "return true if possible" | Return True up the stack as soon as one complete solution is found and skip the undo on that path. | +| "May contain duplicates" and "unique" answers | Sort, then skip `nums[i] == nums[i - 1]` when `i > start`, or when the earlier copy is unused for permutations. | +| "Each number may only be used once" | Pass `i + 1` to the child call. | +| "The same number may be chosen an unlimited number of times" | Pass `i` to the child call. | +| "Order does not matter" or "combinations" | Use a start index so each set is built in one fixed order. | +| "All possible orderings" or "permutations" | Loop over every index with a used array or bitmask. | +| n up to about 8 to 11 | An $n!$ search over arrangements fits. | +| n up to about 15 to 20 | A $2^n$ search over subsets, or dynamic programming over a bitmask, fits. | +| n around 30 to 40 with a subset question | Plain subset search is too slow, so think of meet in the middle or dynamic programming. | +| "All candidates are positive" | After sorting, `break` once a candidate exceeds the remaining target. | +| "Subsequence" or the input order must be kept | Do not sort, and deduplicate with a set per recursion level. | +| A grid path that "walks over every non-obstacle square exactly once" | Mark cells in place, count visited cells, restore on return, and accept only at the end square with all cells used. | +| "The same cell may not be used more than once" in a word on a grid | Use path-local marks restored on return, never a permanent visited set. | +| A list of many words to find on one board | Build a trie and walk it during the search. | +| "Partition into k subsets with equal sums" | Check divisibility, sort descending, fill buckets, and skip buckets with equal sums. | +| "Split the string" or "insert dots" or "insert operators" | Recurse on the cut position and validate each piece, including leading zeros. | +| "Minimize the maximum" load over an assignment | Keep a global best and prune branches that reach it, or binary search the answer with a feasibility check. | +| "Well-formed" or "valid" parentheses | Track open and close counts and never let close exceed open. | diff --git a/notes/dynamic_programming.md b/notes/dynamic_programming.md index be90102..ab657a4 100644 --- a/notes/dynamic_programming.md +++ b/notes/dynamic_programming.md @@ -64,6 +64,8 @@ You can think of these as two different writing styles for the same story. Memoi ### 1. Memoization (Top-Down Approach) +> **First step:** Write the function signature with exactly the parameters that define a subproblem, and say in one sentence what it returns. Write the base cases before the recursive case, and make sure every argument that changes between calls is part of the cache key. + Memoization is a technique used to optimize recursive problem-solving by storing the results of solved subproblems in a data structure, such as a hash table or an array. When the algorithm encounters a subproblem, it first checks if the result is already stored. If it is, the algorithm simply retrieves the cached result instead of recomputing it. This approach reduces redundant calculations, making the solution process faster and more efficient. A “do” for memoization: keep the recursion because it matches the problem’s natural definition, but make every subproblem cheap after the first time. A “don’t”: let your memo structure accidentally depend on hidden state (like mutable defaults or changing globals) in a way that makes results inconsistent. @@ -104,6 +106,8 @@ One practical note when learning: top-down DP is often easier to write correctly ### 2. Tabulation (Bottom-Up Approach) +> **First step:** Decide the table's dimensions and what one cell means, then find which cells each cell reads from. Choose loop directions so those cells are always filled first, and note which cell holds the final answer. + Tabulation is a technique that solves a problem by working from the smallest subproblems up to the larger ones in a step-by-step, iterative way. It stores the solutions to these subproblems in a table, often an array, and uses these stored values to solve bigger subproblems until the final solution is reached. Unlike memoization, which works recursively and checks if a solution is already computed, tabulation systematically builds the solution from the ground up. This method is efficient and avoids the overhead of recursive calls. The “why” behind tabulation is control: you decide the exact order states are computed, and you avoid recursion limits and call overhead. Fill the table in dependency order. Do not fill it in a convenient order that violates dependencies, because then you’ll be reading values that aren’t valid yet. @@ -156,6 +160,8 @@ A recurrence specifies how the answer for a state depends on other states. Defin ### Formulating Recurrence Relations +> **First step:** Look at the last decision made in an optimal solution, such as whether the last characters match or whether the last item is taken. Each possible last decision becomes one term of the recurrence, and the empty or smallest input becomes the base case. + Example: Longest Common Subsequence (LCS) Given two sequences $X = x_1, x_2, ..., x_m$ and $Y = y_1, y_2, ..., y_n$, let $LCS(i,j)$ be the LCS length of their first $i$ and $j$ elements. The recurrence uses one-based character indices and a zero row and column for empty prefixes: @@ -177,6 +183,8 @@ The reason LCS is such a beloved DP example is that the “choices” are easy t ### State Representation +> **First step:** Write the state in words, for example "dp[i][w] is the best value using the first i items with capacity w", and check that two situations with the same state really have the same best future. If they do not, the state is missing information and needs another dimension. + Properly defining the state is crucial for dynamic programming. - State variables are the parameters that uniquely define each subproblem, helping to break down the problem into smaller, manageable components. @@ -211,6 +219,8 @@ A key “do” here is to interpret your state in plain language. If you can’t ### Memory Optimization +> **First step:** Get the full-table version working first, then mark exactly which earlier cells each cell reads. If it reads only the previous row, two rows are enough, and a single array often works when the loop direction guarantees that old values are read before they are overwritten. + In some cases, we can optimize space complexity by noticing dependencies between states. This is where DP graduates from “works” to “works well.” Once the recurrence is correct, you can ask: “Do I truly need all previous states, or only a slice of them?” Many DP solutions only depend on the previous row, previous column, or a small window, so storing everything is optional. @@ -240,6 +250,325 @@ This distinction matters because DP isn’t “better recursion”, it’s “re Example: Merge Sort algorithm divides the list into halves, sorts each half, and then merges the sorted halves. +## Common DP Patterns + +Most interview DP problems are variations on a small number of shapes. Once you recognize the shape, the state, the transition, and the loop order follow almost mechanically. The sections below cover the shapes behind the most common LeetCode DP questions. + +### Linear DP over a Sequence + +> **First step:** Write down in words what `dp[i]` means for the prefix ending at position `i`, then list the few earlier indices it can come from. Decide the values for the first one or two positions and whether the answer is `dp[n]` or the best over all `i`. + +The simplest DP shape walks along an array or string once, and each position depends on a constant number of earlier positions. Climbing Stairs (LeetCode 70) uses $dp[i] = dp[i-1] + dp[i-2]$ because the last move was either one step or two; the version in the problem list below also allows jumps of three and adds $dp[i-3]$. House Robber (LeetCode 198) either skips house $i$ or robs it on top of the best answer two houses back, so $dp[i] = \max(dp[i-1], dp[i-2] + a_i)$. + +Recognize this pattern when the input is processed left to right and each decision only interacts with a short window behind it. House Robber II (LeetCode 213) puts the houses in a circle, which you break by running the linear version twice, once without the first house and once without the last. Decode Ways (LeetCode 91) adds $dp[i-1]$ when the single digit is 1 to 9 and $dp[i-2]$ when the two-digit number is 10 to 26. Word Break (LeetCode 139) sets $dp[i]$ to true when some dictionary word ends at position $i$ and $dp$ is true where that word starts. + +Longest Valid Parentheses (LeetCode 32) stores the length of the longest valid substring ending at $i$. When s[i] is ')' and the character at $j = i - dp[i-1] - 1$ is '(', then $dp[i] = dp[i-1] + 2 + dp[j-1]$, taking $dp[j-1]$ as 0 when $j = 0$, and the answer is the maximum over all $i$. + +```python +def rob_line(houses): + prev2, prev1 = 0, 0 # best up to i-2 and up to i-1 + for amount in houses: + prev2, prev1 = prev1, max(prev1, prev2 + amount) + return prev1 + +def rob_circle(houses): # LeetCode 213 + if len(houses) == 1: + return houses[0] + return max(rob_line(houses[1:]), rob_line(houses[:-1])) +``` + +These run in $O(n)$ time and $O(1)$ extra space when each state looks back a constant number of steps, while Word Break looks back up to the longest word length $L$ and costs about $O(n L^2)$ when each candidate word is sliced and looked up in a hash set. The most common mistake is in Decode Ways, where a '0' cannot be decoded on its own and a pair starting with '0', such as "06", is not a valid two-digit code. + +### Kadane's Algorithm and Running Best + +> **First step:** Define the local state as the best subarray that ends exactly at index `i`, because that is what makes a one-step recurrence possible. Keep a separate global best, since the answer can end anywhere. + +Maximum Subarray (LeetCode 53) asks for the largest sum of a contiguous subarray. Let $cur_i$ be the best sum of a subarray that ends exactly at index $i$. That subarray either extends the best one ending at $i-1$ or starts fresh at $i$, so $cur_i = \max(a_i, cur_{i-1} + a_i)$, and the answer is the largest $cur_i$ seen, not the last one. + +Maximum Product Subarray (LeetCode 152) needs two running values, because multiplying by a negative number turns the smallest product into the largest. Track both the maximum and the minimum product ending at $i$, and swap them when $a_i$ is negative. Recognize this family by the word "contiguous" next to "maximum" or "minimum". + +```python +def max_subarray(nums): # LeetCode 53 + cur = best = nums[0] + for x in nums[1:]: + cur = max(x, cur + x) + best = max(best, cur) + return best + +def max_product(nums): # LeetCode 152 + hi = lo = best = nums[0] + for x in nums[1:]: + if x < 0: + hi, lo = lo, hi + hi = max(x, hi * x) + lo = min(x, lo * x) + best = max(best, hi) + return best +``` + +Both run in $O(n)$ time and $O(1)$ space. The most common mistake is starting `best` at 0, which returns 0 for an array whose elements are all negative. + +### 0/1 and Unbounded Knapsack + +> **First step:** Decide whether each item may be used once or any number of times, because that fixes the direction of the capacity loop. Then decide whether a cell holds a Boolean, a count, or a minimum or maximum, and initialize `dp[0]` and the other cells to match. + +The one-dimensional knapsack in Memory Optimization iterates capacity from high to low, and that direction is what makes it 0/1. When `best[c]` reads `best[c - w]`, that cell still holds the value from before the current item was considered. Iterating capacity from low to high means `best[c - w]` may already include the current item, so the item can be taken again, which is exactly the unbounded knapsack. + +Partition Equal Subset Sum (LeetCode 416) is 0/1 subset sum: an odd total is impossible, otherwise ask whether some subset reaches half the total. Target Sum (LeetCode 494) becomes 0/1 counting after a little algebra. If $P$ is the sum of the numbers given a plus sign and $S$ is the total, then $P - (S - P) = t$, so $P = (S + t)/2$ and you count subsets with that sum. Coin Change (LeetCode 322) and Perfect Squares (LeetCode 279) are unbounded minimization with $dp[c] = \min(dp[c], dp[c - x] + 1)$, where every cell except $dp[0] = 0$ starts at infinity. + +```python +def can_partition(nums): # LeetCode 416, 0/1, Boolean + total = sum(nums) + if total % 2: + return False + target = total // 2 + reach = [True] + [False] * target + for x in nums: + for c in range(target, x - 1, -1): # downward: each number once + reach[c] = reach[c] or reach[c - x] + return reach[target] + +def coin_change(coins, amount): # LeetCode 322, unbounded, minimum + INF = float("inf") + dp = [0] + [INF] * amount + for coin in coins: + for c in range(coin, amount + 1): # upward: coin can be reused + dp[c] = min(dp[c], dp[c - coin] + 1) + return dp[amount] if dp[amount] != INF else -1 +``` + +Both run in $O(n T)$ time and $O(T)$ space, where $n$ is the number of items and $T$ the target sum or amount. The most common mistake in Target Sum is forgetting to return 0 when $S + t$ is odd or $|t| > S$, since the subset target is then not a valid non-negative integer. + +### Counting Combinations Versus Permutations + +> **First step:** Decide whether [1, 2] and [2, 1] are the same way or two different ways. If they are the same, put the item loop outside; if they are different, put the target loop outside. + +Coin Change II (LeetCode 518) and Combination Sum IV (LeetCode 377) look identical: count the ways to reach a target using numbers that can be reused. The only difference is whether order matters, and that is controlled entirely by which loop is on the outside. + +With coins in the outer loop, each way is built by first deciding how many of the first coin to use, then how many of the second, and so on, so each multiset of coins is counted exactly once. With the amount in the outer loop, $dp[t]$ sums over every possible last element, so 1 + 2 and 2 + 1 are counted as different sequences. Climbing Stairs is the permutation version with steps of 1 and 2. + +```python +def count_combinations(coins, amount): # LeetCode 518, order ignored + dp = [1] + [0] * amount + for coin in coins: + for t in range(coin, amount + 1): + dp[t] += dp[t - coin] + return dp[amount] + +def count_permutations(nums, target): # LeetCode 377, order matters + dp = [1] + [0] * target + for t in range(1, target + 1): + for x in nums: + if x <= t: + dp[t] += dp[t - x] + return dp[target] +``` + +Both take $O(k T)$ time for $k$ choices and target $T$, with $O(T)$ space. The most common mistake is setting $dp[0] = 0$, but the empty selection is the one way to make 0 and every other count is built from it. + +### Grid DP + +> **First step:** Decide which neighbors can move into a cell, usually the one above and the one to the left, and fill the grid in an order where they are already done, such as row by row. Set the first row, the first column, and any blocked cells explicitly before the main loop. + +In grid problems the state is a cell $(i, j)$ and the value is the number of ways, or the best cost, to reach it from the start. Unique Paths (LeetCode 62) sums the top and left neighbors, and Unique Paths II (LeetCode 63) forces blocked cells to 0. Minimum Path Sum (LeetCode 64) takes the smaller of the two neighbors plus the cell's own cost. Triangle (LeetCode 120) is easiest bottom-up: each entry becomes its value plus the smaller of the two entries below it, and the apex ends up holding the answer. + +Maximal Square (LeetCode 221) uses a different state: $dp[i][j]$ is the side of the largest all-ones square whose bottom-right corner is $(i, j)$. If the cell is 1, $dp[i][j] = 1 + \min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1])$, and the answer is the square of the largest side seen anywhere. + +Each row depends only on the row above and on cells already filled in the current row, so a single array of width $n$ is enough. Before the update, `dp[j]` still holds the value from the row above. + +```python +def unique_paths_with_obstacles(grid): # LeetCode 63 + cols = len(grid[0]) + dp = [0] * cols + dp[0] = 1 + for row in grid: + for j in range(cols): + if row[j] == 1: + dp[j] = 0 + elif j > 0: + dp[j] += dp[j - 1] # from above plus from the left + return dp[-1] +``` + +This takes $O(mn)$ time and $O(n)$ space. The most common mistake is filling the whole first row and first column with 1, which ignores obstacles, since every cell after a blocked cell in the first row or column must be 0. + +### Longest Increasing Subsequence + +> **First step:** Decide whether you need only the length or the subsequence itself, and whether "increasing" is strict. Strict uses `bisect_left` and non-decreasing uses `bisect_right`, and recovering the subsequence needs parent indices. + +Longest Increasing Subsequence (LeetCode 300) has a direct $O(n^2)$ DP. Let $dp[i]$ be the length of the longest increasing subsequence that ends at index $i$; then $dp[i] = 1 + \max dp[j]$ over all $j < i$ with $a_j < a_i$, or 1 if there is no such $j$, and the answer is the largest $dp[i]$. + +The faster patience method keeps an array `tails`, where `tails[k]` is the smallest possible last value of an increasing subsequence of length $k + 1$ seen so far. This array is always sorted, so each new value is placed with binary search. It either extends the longest subsequence or replaces the first tail that is greater than or equal to it, which makes that length easier to extend later. Russian Doll Envelopes (LeetCode 354) reduces to this after sorting by width ascending and, for equal widths, height descending. + +```python +from bisect import bisect_left + +def length_of_lis(nums): # LeetCode 300 + tails = [] + for x in nums: + k = bisect_left(tails, x) # first tail >= x + if k == len(tails): + tails.append(x) + else: + tails[k] = x + return len(tails) +``` + +The patience method runs in $O(n \log n)$ time and $O(n)$ space. The most common mistake is returning `tails` as the answer subsequence, which it generally is not; to rebuild a real one, record for each element the index that sat at position $k - 1$ when it was placed. + +### Two-String DP + +> **First step:** Define `dp[i][j]` on prefixes, the first `i` characters of one string and the first `j` of the other, in an (m+1) by (n+1) table whose row 0 and column 0 stand for the empty prefix. Fill those borders first, because most bugs in this family live there. + +LCS above is the template for a large family. The cell $dp[i][j]$ looks at the last characters of the two prefixes and at three neighbors: the diagonal (both characters used), the cell above (drop a character of the first string), and the cell to the left (drop a character of the second). Problems differ only in what those moves cost and what the borders hold. Delete Operation for Two Strings (LeetCode 583) is just $m + n$ minus twice the LCS length (LeetCode 1143). + +Edit Distance (LeetCode 72) takes the diagonal for free when the characters match, and otherwise 1 plus the minimum of replace (diagonal), delete (above), and insert (left), with borders $dp[i][0] = i$ and $dp[0][j] = j$. Distinct Subsequences (LeetCode 115) counts how many times $t$ appears as a subsequence of $s$: $dp[i][j] = dp[i-1][j]$, plus $dp[i-1][j-1]$ when the characters match, with $dp[i][0] = 1$. Interleaving String (LeetCode 97) stores whether the first $i$ characters of s1 and the first $j$ of s2 can form the first $i + j$ characters of s3. + +Wildcard Matching (LeetCode 44) and Regular Expression Matching (LeetCode 10) put the text on one axis and the pattern on the other. In 44, a cell whose pattern character is '*' is true if $dp[i][j-1]$ (the star matches nothing) or $dp[i-1][j]$ (the star absorbs one more character). In 10, a pair such as 'x*' is handled together: $dp[i][j] = dp[i][j-2]$ for zero copies, or $dp[i-1][j]$ when the text character matches x. Row 0 must be computed too, because patterns like "a*b*" match the empty string. + +```python +def edit_distance(a, b): # LeetCode 72 + m, n = len(a), len(b) + dp = [[0] * (n + 1) for _ in range(m + 1)] + for i in range(m + 1): + dp[i][0] = i # delete all of a[:i] + for j in range(n + 1): + dp[0][j] = j # insert all of b[:j] + for i in range(1, m + 1): + for j in range(1, n + 1): + if a[i - 1] == b[j - 1]: + dp[i][j] = dp[i - 1][j - 1] + else: + dp[i][j] = 1 + min(dp[i - 1][j - 1], # replace + dp[i - 1][j], # delete + dp[i][j - 1]) # insert + return dp[m][n] +``` + +This takes $O(mn)$ time and $O(mn)$ space, which drops to $O(n)$ by keeping two rows or one row plus the saved diagonal value. The most common mistake is mixing table indices with string indices, since $dp[i][j]$ compares a[i - 1] with b[j - 1]. + +### Interval DP and Palindromes + +> **First step:** Define `dp[i][j]` on the range from `i` to `j` and decide what the last action inside that range is, such as matching the two ends or picking a split point. Fill ranges in order of increasing length, starting from ranges of length one or zero as the base case. + +Some problems are about a contiguous range whose answer depends on smaller ranges inside it. The table is indexed by both ends, and every range must be computed after the ranges it contains, which is easiest by looping over the length, or by running $i$ downward and $j$ upward from $i$. + +Longest Palindromic Subsequence (LeetCode 516) compares the two ends: if $s_i = s_j$ then $dp[i][j] = dp[i+1][j-1] + 2$, otherwise $dp[i][j] = \max(dp[i+1][j], dp[i][j-1])$. Minimum Insertion Steps to Make a String Palindrome (LeetCode 1312, listed below) is $n$ minus that length. Longest Palindromic Substring (LeetCode 5) can use a Boolean table where $pal[i][j]$ is true when the ends match and $pal[i+1][j-1]$ is true, but expanding around each of the $2n - 1$ centers gives the same $O(n^2)$ time with $O(1)$ space. + +Burst Balloons (LeetCode 312) is the split-point version. Pad the array with a 1 on each side and let $dp[i][j]$ be the most coins from bursting every balloon strictly between $i$ and $j$. The trick is to choose the balloon $k$ that bursts last in that range, because then its neighbors are exactly $i$ and $j$, and the two sides become independent subproblems. + +```python +def max_coins(nums): # LeetCode 312 + a = [1] + nums + [1] + n = len(a) + dp = [[0] * n for _ in range(n)] + for length in range(2, n): # length = j - i + for i in range(n - length): + j = i + length + for k in range(i + 1, j): # k is the last balloon burst + dp[i][j] = max(dp[i][j], + dp[i][k] + a[i] * a[k] * a[j] + dp[k][j]) + return dp[0][n - 1] +``` + +With $O(n^2)$ ranges and $O(n)$ split points this takes $O(n^3)$ time and $O(n^2)$ space, which is fine for $n$ up to a few hundred. The most common mistake is choosing the balloon that bursts first, which leaves the neighbors of the remaining balloons unknown. + +### State Machine DP for Stock Problems + +> **First step:** List the states you can be in at the end of a day, such as holding a share, not holding, or cooling down, and draw the allowed moves between them with their gain or cost. Each move becomes one term of the transition, and every state is updated from yesterday's values. + +The Best Time to Buy and Sell Stock family looks like several problems but is one pattern. Each day you are in one of a few states, and buying or selling moves you between them. Keep the best profit for each state, update all of them from the previous day, and the answer is the best not-holding state on the last day. + +With unlimited transactions (LeetCode 122) the states are hold and free, with $hold = \max(hold, free - p)$ and $free = \max(free, hold + p)$. A transaction fee (LeetCode 714) subtracts the fee on the selling move. A cooldown (LeetCode 309) adds a third state for the day right after a sale, from which you may not buy. At most $k$ transactions (LeetCode 188, and LeetCode 123 with $k = 2$) gives a hold and a free state for each transaction count. + +```python +def max_profit_cooldown(prices): # LeetCode 309 + hold, sold, rest = float("-inf"), 0, 0 + for p in prices: + hold, sold, rest = (max(hold, rest - p), # keep, or buy after a rest day + hold + p, # sell today + max(rest, sold)) # idle, cooldown ends + return max(sold, rest) + +def max_profit_k(k, prices): # LeetCode 188 + buy = [float("-inf")] * (k + 1) + sell = [0] * (k + 1) + for p in prices: + for t in range(1, k + 1): + buy[t] = max(buy[t], sell[t - 1] - p) + sell[t] = max(sell[t], buy[t] + p) + return sell[k] +``` + +This is $O(n)$ time and $O(1)$ space for a fixed number of states, and $O(nk)$ time with $O(k)$ space for $k$ transactions. The most common mistake is updating one state in place and then reading its new value for another state of the same day, which in the cooldown problem lets you buy the day after selling; assign all states together or use temporaries. + +### DP on Trees + +> **First step:** Decide what each subtree should return to its parent, often a pair such as "best if this node is used" and "best if it is not". Use a post-order traversal so children are finished first, and keep any answer that cannot be passed upward in a separate variable. + +On a tree the subproblems are subtrees, and a post-order DFS already visits them in dependency order, so no explicit table is needed. Each call returns a small tuple of values for its subtree and the parent combines its children's tuples. + +House Robber III (LeetCode 337) returns two numbers per node: the best total if this node is robbed, in which case both children must be skipped, and the best total if it is not, in which case each child takes its better option. Binary Tree Maximum Path Sum (LeetCode 124) returns the best path that starts at the node and goes down, which may use at most one child, while a separate global value records paths that bend through the node and use both children. Diameter of Binary Tree (LeetCode 543) has the same shape. + +```python +def rob_tree(root): # LeetCode 337 + def dfs(node): # returns (robbed, skipped) + if node is None: + return 0, 0 + l_rob, l_skip = dfs(node.left) + r_rob, r_skip = dfs(node.right) + robbed = node.val + l_skip + r_skip + skipped = max(l_rob, l_skip) + max(r_rob, r_skip) + return robbed, skipped + return max(dfs(root)) + +def max_path_sum(root): # LeetCode 124 + best = float("-inf") + def down(node): # best path from node going down + nonlocal best + if node is None: + return 0 + left = max(down(node.left), 0) # drop negative branches + right = max(down(node.right), 0) + best = max(best, node.val + left + right) + return node.val + max(left, right) + down(root) + return best +``` + +Both visit each node once, so they take $O(n)$ time and $O(h)$ stack space for tree height $h$. The most common mistake in LeetCode 124 is returning the bent path through both children to the parent, which stops being a valid path once it continues upward. + +### Bitmask DP + +> **First step:** Check that the number of items is small, about 20 or fewer, and decide what one bit means, usually "this item is already used". Then decide what the state needs besides the mask, such as the last item visited or how full the current group is. + +When the state must remember exactly which items have been used and $n$ is small, encode that set as an integer whose bit $i$ is 1 when item $i$ is used. There are $2^n$ masks, about a million for $n = 20$, and each transition adds one item whose bit is still 0, so a larger mask always comes after a smaller one. + +Partition to K Equal Sum Subsets (LeetCode 698) fills groups one at a time. The total of the used numbers modulo the group size tells you how full the current group is, so a mask only needs a reachable flag. Minimum Number of Work Sessions to Finish the Tasks (LeetCode 1986) stores a pair per mask, the fewest sessions and the smallest load of the last session, compared in that order. Find the Shortest Superstring (LeetCode 943) uses $dp[mask][last]$, the classic travelling salesman state. + +```python +def can_partition_k(nums, k): # LeetCode 698 + total = sum(nums) + if total % k: + return False + side, n = total // k, len(nums) + reachable = [False] * (1 << n) + filled = [0] * (1 << n) # load of the current group + reachable[0] = True + for mask in range(1 << n): + if not reachable[mask]: + continue + for i in range(n): + if mask & (1 << i) or filled[mask] + nums[i] > side: + continue # already used, or group overflows + nxt = mask | (1 << i) + reachable[nxt] = True + filled[nxt] = (filled[mask] + nums[i]) % side + return reachable[(1 << n) - 1] +``` + +This takes $O(2^n n)$ time and $O(2^n)$ space. The most common mistake is operator precedence in C++ and Java, where `mask & (1 << i) == 0` performs the comparison before the AND, so always write `(mask & (1 << i)) == 0`. + ## Common Terms These terms show up constantly around DP because DP is really about breaking a big idea into structured pieces. The more comfortable you are with these building blocks, the faster DP problems start to feel like patterns instead of puzzles. @@ -540,3 +869,31 @@ This problem involves finding the minimum number of insertions needed to transfo - [C++](https://github.com/djeada/Algorithms-And-Data-Structures/tree/master/src/dynamic_programming/cpp/minimum_insertions_for_palindrome) - [Python](https://github.com/djeada/Algorithms-And-Data-Structures/tree/master/src/dynamic_programming/python/minimum_insertions_for_palindrome) + +## Reading the Problem Statement + +The wording and the constraints of a problem usually tell you which DP shape is intended before you write any code. Use this table to translate the statement into the decisions described above. + +| If the problem says | What it means for your code | +| --- | --- | +| "number of ways" or "how many" | Transitions add counts, the base case is one way to do nothing, and counts usually need modulo $10^9 + 7$ after every addition. | +| "minimum cost" or "maximum profit" | Transitions take a min or a max, unreachable states start at positive or negative infinity, and an infinite final value means impossible, often returned as -1. | +| "is it possible" or "can you" | A Boolean table combined with or, often a simpler version of the counting DP. | +| "each item can be used at most once" | 0/1 knapsack, with the item loop outside and the capacity loop running downward. | +| "unlimited supply" or "can be reused" | Unbounded knapsack, with the capacity loop running upward so the current item can feed itself. | +| "combinations" or "order does not matter" | Items in the outer loop and the target in the inner loop, as in Coin Change II. | +| "different orders count as different" | Target in the outer loop and items in the inner loop, as in Combination Sum IV. | +| two strings or two arrays | An (m+1) by (n+1) table indexed by prefix lengths, with row 0 and column 0 for the empty prefix. | +| "subsequence" | Elements may be skipped, so a mismatch falls back to dropping one side and taking the better or summed neighbor. | +| "substring" or "subarray" | Elements must be contiguous, so a mismatch resets the state to 0 and the answer is the best over all end positions. | +| "contiguous" with "maximum sum" or "maximum product" | Kadane, with a best-ending-here value and a separate global best, plus a running minimum for products. | +| "palindrome" | Interval states on the range from i to j filled by increasing length, or expansion around centers for substrings. | +| "adjacent elements cannot both be chosen" | The House Robber shape, take or skip with a look back of two positions. | +| "cooldown", "fee", or "at most k transactions" | A state machine with one variable per state, each updated from the previous day's values. | +| "burst", "merge", or neighbors join after a removal | Interval DP that picks the last operation inside each range. | +| a binary tree with "maximum" or "rob" | Post-order DFS that returns a small tuple of states per subtree. | +| n up to $10^5$ or more | Only $O(n)$ or $O(n \log n)$ total work, so one-dimensional states with constant or logarithmic transitions. | +| n up to a few thousand | $O(n^2)$ is fine, such as the simple LIS or a two-string table. | +| n up to about 500 | $O(n^3)$ fits, which usually signals interval DP with a split point. | +| n up to about 20 | Bitmask DP over the $2^n$ subsets is intended. | +| "return the path" or "list the actual items" | Store the choice made at each state as a parent pointer, or walk the filled table backward from the answer cell. | diff --git a/notes/graphs.md b/notes/graphs.md index 59c42a5..5531d50 100644 --- a/notes/graphs.md +++ b/notes/graphs.md @@ -400,6 +400,8 @@ A useful “do” before you choose BFS vs DFS: decide what “close” means in Breadth-First Search (BFS) is a graph traversal algorithm that explores a graph level by level from a specified start vertex. It first visits all vertices at distance 1 from the start, then all vertices at distance 2, and so on. This makes BFS the natural choice whenever “closest in number of edges” matters. +> **First step:** Build an adjacency list from the edge list, adding both directions if the graph is undirected. Decide whether you need only reachability, the distance in edges (keep a `dist` map), or the actual path (keep a `parent` map), and mark the start as visited before the loop begins. + To efficiently keep track of the traversal, BFS employs two primary data structures: - A queue (often named `queue` or `unexplored`) that stores vertices pending exploration in first-in, first-out (FIFO) order. @@ -521,6 +523,8 @@ A practical “do” for BFS code: use a real queue (like `collections.deque` in Depth-First Search (DFS) is a graph traversal algorithm that explores as far as possible along each branch before backtracking. Starting from a source vertex, it dives down one neighbor, then that neighbor’s neighbor, and so on, only backing up when it runs out of new vertices to visit. +> **First step:** Build the adjacency list and decide what the DFS has to report: reachability, a component label, a cycle, or a post-order. If the graph can be a long chain (around $10^4$ vertices or more), plan for the iterative form or a raised recursion limit before you start. + To track the traversal efficiently, DFS typically uses: - A call stack via recursion or an explicit stack data structure (LIFO). @@ -715,8 +719,205 @@ Implementation tips: - If neighbor order matters (e.g., lexicographic traversal), control push order (push in reverse for stacks) or sort adjacency lists. - For sparse graphs, adjacency lists are preferred over adjacency matrices for time/space efficiency. +### Common BFS and DFS Patterns + +Most interview graph problems are a plain BFS or DFS with one twist. The twist is usually in what counts as a node, where the search starts, or what extra information travels with each node. The patterns below are the twists that come up again and again. + +> **First step:** Before writing the loop, write down in one line what a node is (a vertex, a grid cell, a word, a `(vertex, mask)` pair), how its neighbors are produced, and which nodes go into the queue or stack at the start. + +Grids are the most common disguise. Each cell is a node with 4 or 8 neighbors, and the only extra work is the bounds check. Number of islands, flood fill, and shortest paths on a maze are worked through in the "Grids as Graphs" part of the [matrices notes](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/notes/matrices.md), and every pattern below applies to grids unchanged. + +Multi-source BFS by levels. When the question asks how many minutes something takes to spread, or how far each cell is from the nearest source, put every source in the queue before the loop starts. Then process one level at a time by reading `len(queue)` at the start of each round. In 994 Rotting Oranges every rotten orange is a source and each round is one minute. In 542 01 Matrix every zero cell is a source and the BFS distance is the answer for every cell. + +``` +queue = deque(rotten_cells) # all sources start together +minutes = 0 +while queue and fresh > 0: + for _ in range(len(queue)): # exactly one level per round + r, c = queue.popleft() + for nr, nc in four_neighbors(r, c): + if in_bounds(nr, nc) and grid[nr][nc] == FRESH: + grid[nr][nc] = ROTTEN + fresh -= 1 + queue.append((nr, nc)) + minutes += 1 +return minutes if fresh == 0 else -1 +``` + +Search backwards from the targets. If many starting points share a small set of goals, reverse the question and start from the goals. In 417 Pacific Atlantic Water Flow, searching from every cell is quadratic, so instead run one search from all Pacific border cells and one from all Atlantic border cells, moving only to neighbors of equal or greater height, and return the cells reached by both. In 130 Surrounded Regions, search from every `O` on the border and mark what you reach; every unmarked `O` is surrounded and gets flipped. + +BFS on an implicit graph. Sometimes the graph is never given and you generate neighbors on the fly. In 127 Word Ladder a node is a word and its neighbors are words in the dictionary that differ in one letter; try all 26 letters at each position and look them up in a set. The answer counts words, so the start word has distance 1. In 752 Open the Lock a node is a 4-digit string with 8 neighbors (each wheel up or down). Add the dead ends to `visited` before starting, and return -1 at once if `"0000"` is itself a dead end. + +BFS over states. When the same node can be visited again with different progress, the node alone is not enough. In 847 Shortest Path Visiting All Nodes the state is `(node, mask)`, where the mask records visited nodes. Start from every `(i, 1 << i)` at once and stop when the mask is full. There are $n \cdot 2^n$ states, which is fine for $n \le 12$. The same idea with a different state appears in 1293 Shortest Path in a Grid with Obstacles Elimination, where the state is `(row, col, removals left)`. + +Copying a graph. In 133 Clone Graph keep a map from each original node to its copy. Create the copy and store it in the map before visiting the neighbors, so a cycle finds the existing copy instead of recursing forever. + +``` +def clone(node, copies): + if node in copies: + return copies[node] + copy = Node(node.val) + copies[node] = copy # register before recursing + for nb in node.neighbors: + copy.neighbors.append(clone(nb, copies)) + return copy +``` + +Peeling leaves. In 310 Minimum Height Trees the best roots are the centers of the tree. Put every leaf (degree 1) in a queue, remove one layer of leaves at a time while lowering their neighbors' degrees, and stop when at most two vertices remain. Handle $n = 1$ separately, where the answer is `[0]`. + +Three-color DFS. Give each vertex a state: 0 unvisited, 1 on the current DFS path, 2 finished. Reaching a state-1 vertex means a directed cycle, which is how 207 Course Schedule can be solved with DFS. In 802 Find Eventual Safe States a node is safe exactly when every path from it avoids cycles, so a node is safe when all of its neighbors are safe; nodes left in state 1 are unsafe for good. + +``` +def is_safe(u): # state: 0 new, 1 on path, 2 safe + if state[u] != 0: + return state[u] == 2 + state[u] = 1 + for v in G[u]: + if not is_safe(v): + return False # u stays 1, so it stays unsafe + state[u] = 2 + return True +``` + +0-1 BFS. When every edge costs 0 or 1, a deque replaces Dijkstra's heap. Relax edges as in Dijkstra, push the neighbor to the front for a 0-cost edge and to the back for a 1-cost edge. This runs in $O(V+E)$ and fits problems such as 1368 Minimum Cost to Make at Least One Valid Path in a Grid and 2290 Minimum Obstacle Removal to Reach Corner. + +``` +dist = {s: 0} +dq = deque([s]) +while dq: + u = dq.popleft() + for v, w in G[u]: # w is 0 or 1 + if dist[u] + w < dist.get(v, INF): + dist[v] = dist[u] + w + if w == 0: + dq.appendleft(v) + else: + dq.append(v) +``` + +Validating a tree. An undirected graph on $n$ vertices is a tree exactly when it has $n-1$ edges and is connected. For 261 Graph Valid Tree, check the edge count first, then run one BFS, DFS, or union-find pass and confirm that all $n$ vertices were reached. + +Building the edges yourself. In 269 Alien Dictionary the graph is hidden in the word order. Compare each pair of adjacent words, and the first position where they differ gives one edge from the earlier letter to the later one; ignore the rest of the pair. If there is no differing position and the first word is longer (as in `"abc"` before `"ab"`), the input is invalid. Every letter that appears anywhere must be a vertex, even with no edges, and a topological order shorter than the number of letters means a cycle. + +Weighted relations. In 399 Evaluate Division, the equation $a / b = k$ becomes an edge $a \to b$ with weight $k$ and an edge $b \to a$ with weight $1/k$. A query $x / y$ is the product of the weights along any path from $x$ to $y$, found with BFS or DFS. If either variable never appeared the answer is -1.0, and $x / x$ is 1.0 only if $x$ appeared. + +All of these keep the $O(V+E)$ cost of the underlying search, measured on whatever graph the nodes and edges really are (for example $n \cdot 2^n$ states in 847). The most common mistake in the plain BFS patterns is marking a node visited when it is popped instead of when it is pushed, which lets the same node enter the queue many times and can blow up both time and memory (0-1 BFS is the exception, since it relaxes distances like Dijkstra). + Once you can represent a problem as a graph and you’re fluent in storing it (matrix/list) and traversing it (BFS/DFS), you’ve unlocked the foundation for almost every other graph technique, shortest paths, minimum spanning trees, flows, matchings, connectivity analysis, and more. Graphs are the doorway; traversal is the first step through it. +## Graph Coloring + +Coloring gives each vertex a color so that the two ends of every edge differ. In programming problems it rarely says "color". It says "split the people into two groups so that no two who dislike each other are together" or "give each garden a flower type different from its neighbors". + +> **First step:** Build an undirected adjacency list (a dislike or conflict goes in both directions) and a color array filled with -1 for uncolored. Plan an outer loop over every vertex, because nothing promises the graph is connected. + +Two colors is the bipartite check. A graph can be 2-colored exactly when it has no odd cycle. Start a BFS or DFS at an uncolored vertex with color 0, give each newly discovered neighbor the opposite color, and fail as soon as an edge joins two vertices of the same color. The trap is that the graph may be disconnected, so one search from vertex 0 can miss a whole component with an odd cycle. 785 Is Graph Bipartite gives the adjacency list directly, and 886 Possible Bipartition gives dislike pairs on labels 1 to n. + +``` +def is_bipartite(n, G): + color = [-1] * n + for s in range(n): # start from every uncolored vertex + if color[s] != -1: + continue + color[s] = 0 + queue = deque([s]) + while queue: + u = queue.popleft() + for v in G[u]: + if color[v] == -1: + color[v] = 1 - color[u] + queue.append(v) + elif color[v] == color[u]: + return False # odd cycle + return True +``` + +With more colors, a simple greedy rule often suffices. Visit the vertices in any order and give each the smallest color not used by its already-colored neighbors. A vertex has at most $\Delta$ neighbors, where $\Delta$ is the maximum degree, so among $\Delta + 1$ colors one is always free. That is why 1042 Flower Planting With No Adjacent needs no search at all: every garden has at most 3 paths, so 4 flower types always work and greedy never gets stuck. + +Greedy does not find the minimum number of colors in general, and deciding whether a graph can be colored with 3 colors is NP-complete. So interview coloring problems always come with special structure: exactly two colors, a degree bound like in 1042, or a graph small enough for backtracking. The backtracking version is the "K-Colorable Configurations" exercise in the [backtracking notes](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/notes/backtracking.md). + +Both the bipartite check and greedy coloring run in $O(V+E)$ time with $O(V)$ extra space. The most common mistake is searching only from vertex 0 and reporting "bipartite" while another component still contains an odd cycle. + +## Bridges, Articulation Points, and Strongly Connected Components + +A bridge is an edge whose removal disconnects the graph, and an articulation point is a vertex whose removal does the same. Problems phrase this as "critical connections" or "which link, if cut, splits the network". Removing each edge and running a fresh BFS costs $O(E(V+E))$, which is far too slow for 1192 Critical Connections in a Network with $n$ up to $10^5$. One DFS with low-link values finds all of them. + +During the DFS, `tin[u]` is the time $u$ was discovered. `low[u]` is the smallest `tin` reachable from $u$'s DFS subtree using tree edges downward and at most one back edge. If a child $v$ has `low[v] > tin[u]`, nothing below $v$ can climb back to $u$ or above, so the edge $u - v$ is a bridge. For articulation points the test is `low[v] >= tin[u]` for some child $v$, except that the DFS root is an articulation point only when it has two or more DFS children. + +> **First step:** Build an undirected adjacency list, set up `tin` and `low` arrays plus a timer, and decide how to skip the edge back to the parent: by parent vertex for simple graphs, by edge id when parallel edges may exist. + +``` +def bridges(n, G): + tin = [-1] * n + low = [0] * n + timer = 0 + result = [] + + def dfs(u, parent): + nonlocal timer + tin[u] = low[u] = timer + timer += 1 + for v in G[u]: + if v == parent: + continue # skip the edge we arrived by + if tin[v] == -1: # tree edge + dfs(v, u) + low[u] = min(low[u], low[v]) + if low[v] > tin[u]: + result.append((u, v)) + else: # back edge + low[u] = min(low[u], tin[v]) + + for s in range(n): + if tin[s] == -1: + dfs(s, -1) + return result +``` + +This runs in $O(V+E)$ time and $O(V)$ space, but the recursion can be $V$ deep, so in Python an input with $10^5$ vertices needs an iterative version or a raised recursion limit. The most common mistake is mixing up the two tests: bridges use the strict `low[v] > tin[u]`, articulation points use `low[v] >= tin[u]`. + +In a directed graph the matching idea is the strongly connected component (SCC): a maximal set of vertices where each one can reach every other. Kosaraju's algorithm runs one DFS to record finishing order, reverses every edge, and then runs DFS on the reversed graph in decreasing finishing time; each tree of that second pass is one SCC. Tarjan's algorithm does it in a single DFS with the same `low` values plus a stack, closing a component when `low[u] == tin[u]`, and it only lowers `low[u]` through vertices still on the stack. Both are $O(V+E)$. Collapsing each SCC to a single vertex always gives a DAG, so a problem with cycles can often be reduced to topological sort or DP on that smaller graph. + +## Union-Find (Disjoint Set Union) + +Union-Find keeps elements split into groups and answers two questions quickly: which group is $x$ in (`find`), and merge the groups of $x$ and $y$ (`union`). Each group is a tree of parent pointers, and the root is the group's representative. Recognize it when edges or "these two belong together" facts arrive one at a time, and you only need to know whether two things are connected or how many groups exist, never the actual path. It cannot remove edges. + +Two tricks keep the trees flat. Path compression points nodes closer to the root while `find` walks up, and union by size (or rank) always hangs the smaller tree under the larger root. Kruskal's algorithm below uses this same structure. + +> **First step:** Map every element to an integer id from 0 to n-1, using a dictionary when the elements are strings such as emails. Then create `parent[i] = i`, `size[i] = 1`, and a component counter equal to n. + +``` +parent = list(range(n)) +size = [1] * n +components = n + +def find(x): + while parent[x] != x: + parent[x] = parent[parent[x]] # path halving, a form of compression + x = parent[x] + return x + +def union(a, b): + ra, rb = find(a), find(b) + if ra == rb: + return False # already connected: edge closes a cycle + if size[ra] < size[rb]: + ra, rb = rb, ra + parent[rb] = ra # smaller tree under larger root + size[ra] += size[rb] + components -= 1 # needs nonlocal or global in real code + return True +``` + +The typical uses follow directly from `union` returning `False` and from the counter: + +- Counting components, as in 323 Number of Connected Components in an Undirected Graph and 547 Number of Provinces: union every edge and read `components`. +- Finding the redundant edge in 684 Redundant Connection: process edges in input order, and the first one whose `union` returns `False` closes the only cycle, which makes it the last cycle edge in the input as the problem requires. +- 1319 Number of Operations to Make Network Connected: if there are fewer than $n-1$ cables the answer is -1; otherwise every extra cable can be moved, so the answer is `components - 1`. +- 721 Accounts Merge: give each email an id the first time you see it, union every email of an account with that account's first email, then group emails by `find(id)`, sort each group, and put the owner's name in front. + +With both path compression and union by size, a sequence of operations costs $O(\alpha(n))$ amortized each, where $\alpha$ is the inverse Ackermann function and is at most 4 for any practical $n$; space is $O(n)$. The most common mistake is comparing `parent[a] == parent[b]` instead of `find(a) == find(b)`, or linking `a` under `b` directly instead of linking their roots. + ## Shortest paths A common task when dealing with weighted graphs is to find the shortest route between two vertices, such as from vertex $A$ to vertex $B$. Note that there might not be a unique shortest path, since several paths could have the same length. @@ -725,6 +926,8 @@ A common task when dealing with weighted graphs is to find the shortest route be Dijkstra’s algorithm computes shortest paths from a specified start vertex in a graph with non-negative edge weights. It grows a “settled” region outward from the start, always choosing the unsettled vertex with the smallest known distance and relaxing its outgoing edges to improve neighbors’ distances. +> **First step:** Confirm that every edge weight is non-negative, build a weighted adjacency list of `(neighbor, weight)` pairs, and fill `dist` with infinity except `dist[source] = 0`. Decide up front whether you can stop as soon as the target is popped. + To efficiently keep track of the traversal, Dijkstra’s algorithm employs two primary data structures: - A min-priority queue (often named `pq`, `open`, or `unexplored`) keyed by each vertex’s current best known distance from the start. @@ -868,10 +1071,44 @@ Implementation: Implementation tip: If your PQ has no decrease-key, push duplicates on improvement and, when popping a vertex, skip it if it’s already finalized or if the popped key doesn’t match `dist[u]`. This “lazy” approach is simple and fast in practice. +### Dijkstra Variants: Minimax Paths and Maximum Probability + +Dijkstra is not tied to adding weights. It works whenever extending a path can never make its cost better, because then the first time a node is popped its cost is final. Two common variants use this. The first minimizes the largest edge on the path instead of the sum. The second maximizes a product of probabilities, each between 0 and 1. + +> **First step:** Write down how the path cost changes when you add one edge, for example `max(cost, w)` or `cost * p`, and check that adding an edge can never improve it. Then reuse Dijkstra with that combine step and the right heap direction. + +In 1631 Path With Minimum Effort the effort of a path is the largest height difference between neighboring cells on it. In 778 Swim in Rising Water the time needed is the largest elevation on the path, including the starting cell. Both are minimax problems: the relaxation becomes `max(d, w)` instead of `d + w`. + +``` +def minimax_path(G, s, t, start_cost=0): # 778: start_cost = grid[0][0] + best = {s: start_cost} + heap = [(start_cost, s)] + while heap: + d, u = heappop(heap) + if u == t: + return d + if d > best[u]: + continue # stale entry + for v, w in G[u]: + nd = max(d, w) # bottleneck instead of sum + if nd < best.get(v, INF): + best[v] = nd + heappush(heap, (nd, v)) + return INF +``` + +There is a second way to solve minimax problems. "Can I reach the target using only edges of weight at most $L$?" gets easier as $L$ grows, so binary search on $L$ and answer each question with a BFS or DFS that ignores heavier edges. Adding edges in increasing weight order with union-find until source and target join also works, which is Kruskal's idea. + +For 1514 Path with Maximum Probability, multiplying by a probability $p \le 1$ never increases the product, so Dijkstra works with a max-heap (push the negated probability with Python's `heapq`). Start with probability 1 at the source and 0 everywhere else, and relax with `prob[u] * p > prob[v]`. This is the same as ordinary Dijkstra on weights $-\log p$, which are non-negative. + +Both variants keep Dijkstra's $O((V+E)\log V)$ time and $O(V+E)$ space, and the binary-search version costs $O((V+E)\log W)$ for a weight range $W$. The most common mistake in 1514 is using a min-heap, which pops the worst probability first, and in 1631 it is summing the differences instead of taking their maximum. + ### Bellman–Ford Algorithm Bellman–Ford computes shortest paths from a start vertex in graphs that may have negative edge weights (but no negative cycles reachable from the start). It works by repeatedly relaxing every edge; each full pass can reduce some distances until they stabilize. A final check detects negative cycles: if an edge can still be relaxed after $(V-1)$ passes, a reachable negative cycle exists. +> **First step:** Put all edges in one flat list of `(u, v, w)` triples, adding both directions for undirected edges (an undirected negative edge is already a negative cycle). Initialize `dist` with infinity except the source, and decide whether you need negative-cycle detection or a limit on the number of edges. + To efficiently keep track of the computation, Bellman–Ford employs two primary data structures: - A `dist` map (or array) with the best-known distance to each vertex (initialized to ∞ except the start). @@ -1002,6 +1239,28 @@ Implementation: Implementation tip: For all-pairs on sparse graphs with possible negative edges, use Johnson’s algorithm: run Bellman–Ford once from a super-source to reweight edges (no negatives), then run Dijkstra from each vertex. +### Shortest Paths With at Most k Edges + +The passes of Bellman–Ford have an exact meaning if every pass reads only the distances left by the previous pass. After pass $i$, `dist[v]` is then the cheapest cost of reaching $v$ with at most $i$ edges. The in-place version above can chain several edges inside a single pass, because it reads values written earlier in the same pass. That is harmless when you only want the final shortest paths, but it breaks any limit on the number of edges. + +787 Cheapest Flights Within K Stops is this situation. $k$ stops means $k+1$ flights, so run exactly $k+1$ passes, and in each pass relax from a copy of `dist` taken at the start of the pass. + +> **First step:** Translate the limit into edges (k stops means k + 1 edges), then make sure each pass reads from a frozen copy of `dist` and writes into the live array. + +``` +def cheapest_within_k_stops(n, flights, src, dst, k): + dist = [INF] * n + dist[src] = 0 + for _ in range(k + 1): # k stops = k + 1 edges + prev = dist[:] # freeze the last pass + for u, v, w in flights: + if prev[u] != INF and prev[u] + w < dist[v]: + dist[v] = prev[u] + w + return dist[dst] if dist[dst] != INF else -1 +``` + +This takes $O(k(V+E))$ time, counting the copy in each pass, and $O(V)$ space. The most common mistake is relaxing in place without the copy, which lets one pass use several flights and silently exceeds the stop limit. Plain Dijkstra on the node alone is also wrong here, because a cheap route with too many stops can finalize a node before a pricier route with fewer stops arrives; Dijkstra over `(node, stops used)` states does work. + ### A* (A-Star) Algorithm A* is a best-first search that finds a least-cost path from a start to a goal by minimizing @@ -1015,6 +1274,8 @@ where: - $g(n)$ = cost from start to $n$ (so far), - $h(n)$ = heuristic estimate of the remaining cost from $n$ to the goal. +> **First step:** Check that edge costs are non-negative and pick a heuristic that never overestimates the remaining cost, such as Manhattan distance on a 4-direction grid with unit steps. If no such heuristic is available, plain Dijkstra is the safer choice. + Assume a finite graph, non-negative edge costs, and $h(goal)=0$. With a consistent heuristic, A* is optimal without reopening expanded nodes. With an admissible but inconsistent heuristic, reopen a node whenever a better path reaches it. These guarantees apply when the goal is removed from the queue with a current entry, not when it is first discovered. [Berkeley CS 188 notes](https://inst.eecs.berkeley.edu/~cs188/fa22/assets/notes/cs188-fa22-note02.pdf). Core data structures @@ -1197,6 +1458,39 @@ Implementation: Implementation tip: If your PQ lacks decrease-key, push duplicates with improved keys and ignore stale entries when popped (check if popped `g` matches current `g[u]`). This is simple and fast in practice. +### Floyd–Warshall Algorithm + +Floyd–Warshall computes the shortest distance between every pair of vertices at once. It keeps an $n \times n$ matrix and, for each vertex $k$ in turn, asks whether going through $k$ shortens the path from $i$ to $j$: + +$$ +dist[i][j] = \min(dist[i][j],\ dist[i][k] + dist[k][j]) +$$ + +After the outer loop has handled $k$, `dist[i][j]` is the shortest path whose intermediate vertices all come from the first $k+1$ vertices. That invariant only holds if $k$ is the outermost loop. Negative edges are fine, and a negative cycle shows up as `dist[i][i] < 0` at the end. + +Reach for it when the problem asks about many pairs and $n$ is small. With $n = 400$ the triple loop does $6.4 \times 10^7$ steps, which is fine in compiled languages and about the practical limit in Python. In 1334 Find the City With the Smallest Number of Neighbors at a Threshold Distance, $n \le 100$: run Floyd–Warshall, count for each city how many others are within the threshold, and return the city with the smallest count, breaking ties by the larger index. The boolean version, with `or` and `and` in place of `min` and `+`, answers reachability queries such as 1462 Course Schedule IV. + +> **First step:** Build an $n \times n$ matrix with 0 on the diagonal, the edge weight for each edge (both directions if undirected, keeping the smaller weight for parallel edges), and infinity elsewhere. Check that $n^3$ fits the time limit before going further. + +``` +def floyd_warshall(n, edges): + dist = [[INF] * n for _ in range(n)] + for i in range(n): + dist[i][i] = 0 + for u, v, w in edges: + dist[u][v] = min(dist[u][v], w) + for k in range(n): # intermediate vertex, outermost + for i in range(n): + if dist[i][k] == INF: + continue + for j in range(n): + if dist[i][k] + dist[k][j] < dist[i][j]: + dist[i][j] = dist[i][k] + dist[k][j] + return dist +``` + +Time is $O(V^3)$ and space is $O(V^2)$. The most common mistake is putting `k` in an inner loop, which still runs without errors but returns wrong distances. + ## Minimum Spanning Trees Suppose we have a graph that represents a network of houses. Weights represent the distances between vertices, which each represent a single house. All houses must have water, electricity, and internet, but we want the cost of installation to be as low as possible. We need to identify a subgraph of our graph with the following properties: @@ -1211,6 +1505,8 @@ Such a subgraph is called a minimum spanning tree (MST). It minimizes total edge Prim’s algorithm builds a minimum spanning tree (MST) of a weighted, undirected graph by growing a tree from a start vertex. At each step it adds the cheapest edge that connects a vertex inside the tree to a vertex outside the tree. +> **First step:** Confirm the graph is undirected, build a weighted adjacency list with both directions, and decide what a disconnected input should return. For a complete graph given as points, as in 1584 Min Cost to Connect All Points, skip the adjacency list and use the $O(V^2)$ array version that computes distances on the fly. + To efficiently keep track of the construction, Prim’s algorithm employs two primary data structures: - A min-priority queue (often named `pq`, `open`, or `unexplored`) keyed by a vertex’s best known connection cost to the current tree. @@ -1353,6 +1649,8 @@ For dense graphs ($E \approx V^2$), skip heaps: store `key` in an array and, at Kruskal’s algorithm builds a minimum spanning tree (MST) for a weighted, undirected graph by sorting all edges by weight (lightest first) and repeatedly adding the next lightest edge that does not create a cycle. It grows the MST as a forest of trees that gradually merges until all vertices are connected. +> **First step:** Collect the edges as `(weight, u, v)` triples and sort them, then set up the structure from the Union-Find (Disjoint Set Union) section above; the helpers below are the same idea. Count accepted edges so you can stop at $V-1$ and notice when the graph is disconnected. + To efficiently keep track of the construction, Kruskal’s algorithm employs two primary data structures: - A sorted edge list arranged in ascending order of weights ensures that Kruskal’s algorithm always considers the lightest available edge next; without this ordering, the method cannot guarantee that the resulting spanning tree has minimum total weight. @@ -1402,7 +1700,8 @@ Kruskal(V, E): return mst_edges, total -# Union-Find helpers (path compression + union by rank): +# Union-Find helpers (path compression + union by rank), +# see the Union-Find (Disjoint Set Union) section for details: find(x): if parent[x] != x: parent[x] = find(parent[x]) @@ -1498,6 +1797,8 @@ On huge graphs that stream from disk, you can external-sort edges by weight, the Topological sort orders the vertices of a directed acyclic graph (DAG) so that every directed edge $u \rightarrow v$ goes from left to right in the order (i.e., $u$ appears before $v$). It’s the canonical tool for scheduling tasks with dependencies. +> **First step:** Decide the edge direction from the wording (in 207 Course Schedule, the pair `[a, b]` means take `b` before `a`, so the edge is `b → a`). Build the adjacency list and the indegree array in the same pass, and include every vertex, even ones with no edges. + To efficiently keep track of the process (Kahn’s algorithm), we use: - A queue (or min-heap if you want lexicographically smallest order) holding all vertices with indegree = 0 (no unmet prerequisites). @@ -1508,7 +1809,7 @@ Useful additions in practice: - Maintaining a visited count or tracking the length of the output order lets you detect cycles, since producing fewer than $V$ vertices indicates that some could not be placed due to a cycle; without this check, algorithms like Kahn’s may silently return incomplete results on cyclic task graphs. - Using a min-heap instead of a simple FIFO queue ensures that, among available candidates, the smallest-indexed vertex is always chosen, yielding the lexicographically smallest valid topological order; without this modification, the output order depends on arbitrary queueing, which may vary between runs. -- A DFS-based alternative computes a valid topological order by recording vertices in reverse postorder, also in $O(V+E)$ time, while detecting cycles via a three-color marking or recursion stack; without DFS, cycle detection must be handled separately in Kahn’s algorithm. +- A DFS-based alternative computes a valid topological order by recording vertices in reverse postorder, also in $O(V+E)$ time, while detecting cycles via a three-color marking or recursion stack. Kahn’s algorithm detects cycles differently, through the output count described above. Algorithm Steps (Kahn’s algorithm) @@ -1628,3 +1929,63 @@ Implementation tips: - Use a deque for FIFO behavior; use a min-heap to get the lexicographically smallest topological order. - When the graph is large and sparse, store adjacency as lists and compute indegrees in one pass for $O(V+E)$. - DFS variant (brief): color states `0=unseen,1=visiting,2=done`; on exploring `u`, mark `1`; DFS to neighbors; if you see `1` again, there’s a cycle; on finish, push `u` to a stack. Reverse the stack for the order. + +## Eulerian Paths with Hierholzer’s Algorithm + +An Eulerian path uses every edge exactly once. Problems hide it behind words like "use all the tickets", "each domino once", or "a string containing every code". The existence conditions are in the terminology list at the top: all edges must lie in one connected piece, and an undirected graph needs zero or two odd-degree vertices. In a directed graph every vertex must have equal indegree and outdegree, except that one vertex may have one extra outgoing edge (the start) and one may have one extra incoming edge (the end). + +Hierholzer’s algorithm builds the path in linear time. Walk from the start along unused edges, deleting each edge as you use it. When you reach a vertex with no unused edges left, append it to the result and step back. The result comes out in reverse, so reverse it at the end. When the conditions hold, the first vertex where the walk gets stuck must be the end of the whole path, which is why the post-order append is correct. Any cycle the walk skipped is spliced in automatically when the recursion steps back to a vertex that still has unused edges. + +> **First step:** Check the degree conditions and choose the start: the vertex with one extra outgoing edge if there is one, otherwise any vertex that has edges. Store each adjacency list so used edges can be removed cheaply, for example sorted in reverse so `pop()` returns the smallest neighbor. + +332 Reconstruct Itinerary asks for the lexicographically smallest itinerary from `"JFK"` that uses every ticket. Always taking the smallest airport first is not enough on its own, because it can walk into a dead end too early. The post-order append fixes this, since the dead end is appended first and therefore ends up last in the reversed route. + +``` +def find_itinerary(tickets): + G = defaultdict(list) + for a, b in sorted(tickets, reverse=True): + G[a].append(b) # reverse order: pop() gives smallest + route = [] + + def visit(u): + while G[u]: + visit(G[u].pop()) # each edge is used once + route.append(u) # post-order: append when stuck + + visit("JFK") + return route[::-1] +``` + +In an undirected graph each edge sits in two adjacency lists, so give edges ids and keep a `used` array instead of popping from one side only. The same algorithm solves 753 Cracking the Safe (a de Bruijn sequence) and 2097 Valid Arrangement of Pairs. The repository’s [Python implementation](https://github.com/djeada/Algorithms-And-Data-Structures/tree/master/src/graphs/python/eulerian_path) uses Fleury’s algorithm instead, which avoids bridges at every step and is easier to follow but costs about $O(E \cdot (V+E))$ with simple bridge checks. + +Hierholzer runs in $O(V+E)$ time, plus $O(E \log E)$ when neighbors must be sorted, and $O(V+E)$ space; the recursion can be $E$ deep, so large inputs need an explicit stack. The most common mistake is appending a vertex when it is first visited instead of when it gets stuck, which gives a wrong route whenever the walk hits a dead end early. + +## Reading the Problem Statement + +Graph problems are usually decided by a few words in the statement and by the constraints. This table maps the common phrases to what they mean for your code. + +| If the problem says | What it means for your code | +| --- | --- | +| "undirected", "bidirectional", "two-way", or "friends" | Add every edge in both directions when building the adjacency list. | +| Nothing promises the graph is connected | Loop over all vertices and start a search from each unvisited one; the number of starts is the number of components. | +| "tree", or n nodes with n - 1 edges | Connected and acyclic, so every path is unique; to validate a tree, check for exactly n - 1 edges and one component. | +| "prerequisites", "order", "depends on", "must come before" | A directed acyclic graph; use Kahn's algorithm, and fewer than n vertices in the output means a cycle. | +| "group", "merge", "same set", "redundant edge" | Union-Find; the first union whose two ends are already joined is the redundant edge. | +| "no two adjacent the same", or "split into two groups" | Bipartite 2-coloring with BFS or DFS, started from every uncolored vertex. | +| "minimum steps", "moves", or "minutes" on an unweighted graph | BFS; with several starting points, put all of them in the queue together at distance 0. | +| Every move costs 0 or 1 | 0-1 BFS with a deque, pushing 0-cost moves to the front. | +| Non-negative weights, "cheapest", "fastest" | Dijkstra with a min-heap and a check that skips stale entries. | +| Negative weights, or "at most k stops" | Bellman–Ford; for k stops run k + 1 passes, each reading from a copy of dist. | +| "minimize the maximum", "the largest step on the path" | Minimax Dijkstra with max instead of plus, or binary search on the answer plus BFS. | +| "maximum probability", or a product of factors at most 1 | Dijkstra with a max-heap on the product. | +| All pairs, or many pair queries, with n up to a few hundred | Floyd–Warshall with k as the outermost loop. | +| "minimum cost to connect all" | Minimum spanning tree with Kruskal or Prim. | +| "use every edge (or ticket) exactly once" | Eulerian path; check degrees, then Hierholzer with post-order append and a final reverse. | +| "critical connection", or removing it disconnects the network | Bridges (articulation points for vertices) with low-link values from one DFS. | +| "eventually safe", "every path ends at a terminal" | Three-color DFS, or Kahn's algorithm on the reversed graph. | +| A grid of cells | Each cell is a node with 4 or 8 neighbors; check bounds before indexing. | +| Nodes labeled 1 to n | Shift every label by one or allocate n + 1 slots. | +| Edges given as a list of pairs | Build the adjacency list (and indegree array if directed) before anything else. | +| Nodes are strings such as words, emails, or airports | Map each string to an integer id, or key a dictionary by the string. | +| n up to $10^5$ or more | You need $O(V+E)$ or $O(E \log V)$, and deep recursion in Python needs an iterative version. | +| n up to a few hundred | $O(V^3)$ algorithms such as Floyd–Warshall are acceptable. | diff --git a/notes/math_set_relationship.md b/notes/math_set_relationship.md index f864ff8..14a2a60 100644 --- a/notes/math_set_relationship.md +++ b/notes/math_set_relationship.md @@ -94,6 +94,8 @@ A practical do/don’t: do stream results when possible (generate and consume im If Cartesian products feel like “two-loop land,” the power set is “every possible on/off choice.” This is where problems become exponential because you’re not picking one item, you’re deciding for each item whether it’s included. +> **First step:** Decide whether you must list every subset or only count or optimize over them. Listing costs $\Theta(n \cdot 2^n)$ and is realistic up to about $n = 20$. If you only need a count or a best value, look for a formula or a dynamic program instead. + The power set $\mathcal{P}(A)$ is the set of all subsets of $A$. For `A={a,b,c}`: @@ -157,10 +159,59 @@ To enumerate all subsets, you must output $2^n$ subsets: The big idea: output size dominates. If you’re generating all subsets, you’re not “being slow”, you’re paying the bill for the number of results you asked to print. +### Subsets as Bitmasks + +Number the elements $0$ to $n-1$. Every integer `mask` from $0$ to $2^n - 1$ then names exactly one subset: element $i$ is in the subset when bit $i$ of `mask` is set, which you test with `(mask >> i) & 1`. Looping over all masks is the iterative answer to LeetCode 78 Subsets and needs no recursion at all. + +A second iterative style starts with `[[]]` and, for each new element, appends a copy of every existing subset with that element added. Both produce all $2^n$ subsets. The recursive take/skip version lives in the backtracking note. + +```python +def subsets(nums): + n = len(nums) + result = [] + for mask in range(1 << n): + result.append([nums[i] for i in range(n) if (mask >> i) & 1]) + return result +``` + +The same loop solves LeetCode 784 Letter Case Permutation (a mask over the letter positions picks which letters are uppercase, giving $2^{\text{letters}}$ strings) and LeetCode 1863 Sum of All Subset XOR Totals. For 1863 there is also a shortcut: any bit that appears in at least one element is set in the XOR of exactly half of the subsets, so the answer is (OR of all elements) $\cdot\, 2^{n-1}$. + +Time is $\Theta(n \cdot 2^n)$. The most common mistake is writing `2^n` in Python or C, where `^` is XOR; write `1 << n`. + +### Enumerating Submasks + +Sometimes you already have a set `mask` and need every subset of it, for example in DP over subsets where a state is split into two parts. The trick `sub = (sub - 1) & mask` jumps straight to the next smaller submask and skips every integer that has a bit outside `mask`. + +```python +sub = mask +while True: + process(sub) + if sub == 0: + break + sub = (sub - 1) & mask +``` + +A mask with $b$ set bits has $2^b$ submasks. Running this for every mask of $n$ bits costs $3^n$ in total, because each element is either outside the mask, in the mask but not the submask, or in both. The common mistake is the loop condition: `while sub > 0` silently skips the empty submask, and forgetting the break loops forever because `(0 - 1) & mask` is `mask` again. + +### Gray Code Order + +LeetCode 89 Gray Code asks for all $2^n$ masks ordered so that neighbours differ in exactly one bit. The binary-reflected Gray code has a closed form: the $i$-th code is `i ^ (i >> 1)`. + +The reflect construction explains why it works. Take the list for $n-1$ bits, append its mirror image with the top bit set, and both the seam in the middle and the wraparound at the end change only that top bit. + +```python +def gray_code(n): + return [i ^ (i >> 1) for i in range(1 << n)] +``` + +Time is $\Theta(2^n)$. The usual mistake is shifting the wrong way, `i ^ (i << 1)`, which produces values outside the $n$-bit range. + ## Combinations Combinations are how you take the power set and say: “Okay, cool, but I only want subsets of a specific size.” This is a common “make the search space manageable” move: instead of every subset, you focus on one level of the lattice. +> **First step:** Confirm that order really does not matter and that each element can be picked at most once. Then decide whether you need the actual groups or only how many there are, and check how large $\binom{n}{k}$ can get before choosing an integer type or a modulus. + A $k$-combination is a subset of size $k$. The set of all $k$-subsets of $A$ is: @@ -216,10 +267,74 @@ Common methods: One practical tip: do pick your generation method based on what you need downstream (lexicographic order? Streaming? Constant extra memory?), and don’t assume “combinations are always small”, $\binom{n}{k}$ can still be huge around $k\approx n/2$. +### Generating Combinations in Order + +LeetCode 77 Combinations is usually solved by backtracking with a start index: after choosing position `i`, the next choice starts at `i + 1`, so no element is reused and no group appears in two orders. Prune by stopping the loop once too few elements remain to fill the group. The general backtracking template is in the backtracking note. + +The iterative alternative works on a sorted list of $k$ indices and produces the next combination in lexicographic order. Find the rightmost index that can still move up, increase it by one, and reset everything to its right to consecutive values. This is exactly what LeetCode 1286 Iterator for Combination needs, with the indices mapped to characters. + +```python +def next_combination(c, n): + k = len(c) + i = k - 1 + while i >= 0 and c[i] == n - k + i: # c[i] is already at its maximum + i -= 1 + if i < 0: + return False # c was the last combination + c[i] += 1 + for j in range(i + 1, k): + c[j] = c[j - 1] + 1 + return True +``` + +Each step costs $O(k)$. The common mistake in the recursive version is passing `i` instead of `i + 1` to the next call, which allows reuse and turns combinations into multisets. + +### Computing Binomial Coefficients Safely + +Computing $n!$ and then dividing overflows fixed-width integers long before $\binom{n}{k}$ itself does. There are three safer routes, and which one fits depends on the size of $n$ and whether a modulus is involved. + +Pascal's rule $\binom{n}{k} = \binom{n-1}{k-1} + \binom{n-1}{k}$ builds the triangle row by row using only additions. LeetCode 118 Pascal's Triangle asks for the rows, and LeetCode 119 Pascal's Triangle II asks for a single row in $O(k)$ memory, which you get by updating one array from right to left. + +```python +def pascal_row(n): + row = [1] * (n + 1) + for i in range(1, n + 1): + for j in range(i - 1, 0, -1): # right to left keeps old values + row[j] += row[j - 1] + return row +``` + +The multiplicative formula computes a single value in $O(k)$ time: start from `r = 1` and for $i = 1, \ldots, k$ set `r = r * (n - k + i) // i`. After step $i$ the value is $\binom{n-k+i}{i}$, so every division is exact. Use $k = \min(k, n-k)$ to keep the loop short. LeetCode 62 Unique Paths is this formula in disguise: a robot in an $m \times n$ grid makes $m+n-2$ moves, of which $m-1$ go down, so the answer is $\binom{m+n-2}{m-1}$. + +When the problem says "return the answer modulo $10^9+7$", you cannot divide after reducing. Because the modulus $p$ is prime, Fermat's little theorem gives the inverse $a^{-1} \equiv a^{p-2} \pmod p$ for any $a$ not divisible by $p$. Precompute factorials and inverse factorials once, and every query becomes three multiplications. + +```python +MOD = 10**9 + 7 + +def build(n): + fact = [1] * (n + 1) + for i in range(1, n + 1): + fact[i] = fact[i - 1] * i % MOD + inv = [1] * (n + 1) + inv[n] = pow(fact[n], MOD - 2, MOD) # Fermat inverse + for i in range(n, 0, -1): + inv[i - 1] = inv[i] * i % MOD + return fact, inv + +def ncr(n, r, fact, inv): + if r < 0 or r > n: + return 0 + return fact[n] * inv[r] % MOD * inv[n - r] % MOD +``` + +Pascal's triangle costs $O(nk)$, the multiplicative formula $O(k)$, and the factorial tables $O(n)$ to build plus $O(1)$ per query. The most common mistake is writing `fact[n] // (fact[r] * fact[n - r]) % MOD` after the factorials were already reduced, which gives a wrong answer because integer division does not survive reduction modulo $p$. Fermat inverses also require a prime modulus and $n < p$. + ## Permutations Permutations are what happens when you stop treating a chosen set as a bag of items and start treating it like a sequence. In algorithm terms, you’ve moved from “which items?” to “in what order?” +> **First step:** Decide whether you need every ordering, only the next one, or only the $k$-th one, and whether the input has repeated values. Listing all $n!$ orderings is only realistic for $n$ up to about 8 to 10, so "next" and "$k$-th" questions should be answered by construction. + There are two closely related ideas: ### (A) Permutations of all n elements @@ -302,10 +417,84 @@ Classic algorithms: In-place swapping limits working memory, but explicitly producing every permutation still takes factorially many outputs. Estimate the output size before choosing enumeration, and use streaming when results can be processed one at a time. +### Permutations with Duplicates + +LeetCode 46 Permutations uses the standard backtracking with a `used` array, described in the backtracking note. LeetCode 47 Permutations II adds repeated values, and the number of distinct results drops to $N!/(m_1! \cdots m_r!)$, where $m_i$ is how many times value $i$ appears. + +To generate each distinct ordering once, sort the input and, at each depth, skip a value that equals its left neighbour when that neighbour is not currently used. This forces equal values to be placed in their original left-to-right order, so swapping two equal copies never creates a second copy of the same result. LeetCode 996 Number of Squareful Arrays uses the same skip rule. + +```python +def permute_unique(nums): + nums.sort() + used, path, out = [False] * len(nums), [], [] + def go(): + if len(path) == len(nums): + out.append(path[:]) + return + for i in range(len(nums)): + if used[i] or (i > 0 and nums[i] == nums[i - 1] and not used[i - 1]): + continue + used[i] = True + path.append(nums[i]) + go() + path.pop() + used[i] = False + go() + return out +``` + +Time is $O(n)$ per result plus pruned branches. The usual mistake is forgetting to sort, which leaves equal values apart so the neighbour check never fires. + +### Next Permutation + +LeetCode 31 Next Permutation asks for the next arrangement in lexicographic order, in place. Scanning from the right, the longest non-increasing suffix is already the largest possible ordering of its items, so nothing inside it can grow. The change must happen just before it. + +The algorithm has three steps. First, find the pivot: the largest index $i$ with $a[i] < a[i+1]$. If there is none, the array is the last permutation and you reverse it all to get the first. Second, find the rightmost $j > i$ with $a[j] > a[i]$ and swap $a[i]$ and $a[j]$. Third, reverse the suffix after $i$, which turns it from largest to smallest. + +For `1 3 5 4 2`, the pivot is `3`, the rightmost larger value in the suffix is `4`, swapping gives `1 4 5 3 2`, and reversing the suffix gives `1 4 2 3 5`. + +```python +def next_permutation(a): + i = len(a) - 2 + while i >= 0 and a[i] >= a[i + 1]: + i -= 1 + if i >= 0: + j = len(a) - 1 + while a[j] <= a[i]: + j -= 1 + a[i], a[j] = a[j], a[i] + a[i + 1:] = reversed(a[i + 1:]) +``` + +It runs in $O(n)$ time and $O(1)$ extra space, and the same routine solves LeetCode 556 Next Greater Element III. The most common mistake is using strict comparisons (`>` and `<`) in the two scans, which picks the wrong pivot or swap target when values repeat. + +### The k-th Permutation + +LeetCode 60 Permutation Sequence asks for the $k$-th permutation of $1, \ldots, n$. Generating $k$ permutations would be far too slow, but the answer can be read off directly. Among the $n!$ orderings, the first $(n-1)!$ start with the smallest element, the next $(n-1)!$ with the second smallest, and so on. + +So with a 0-based rank $k$, the first element is the one at index $\lfloor k/(n-1)! \rfloor$ among the remaining elements, then $k$ becomes $k \bmod (n-1)!$ and you repeat with $(n-2)!$. The sequence of indices is the representation of $k$ in the factorial number system (also called the Lehmer code). For $n = 4$ and 1-based $k = 9$, the 0-based rank is 8: $8 = 1 \cdot 3! + 1 \cdot 2! + 0 \cdot 1! + 0 \cdot 0!$, which picks `2`, then `3`, then `1`, then `4`, giving `2314`. + +```python +from math import factorial + +def kth_permutation(n, k): + items = list(range(1, n + 1)) + k -= 1 # switch to a 0-based rank + out = [] + for size in range(n, 0, -1): + idx, k = divmod(k, factorial(size - 1)) + out.append(items.pop(idx)) + return "".join(map(str, out)) +``` + +This is $O(n^2)$ because of the list removals, which is nothing for $n \le 9$. Running the same idea backwards gives the rank of a given permutation. The most common mistake is forgetting to convert $k$ from 1-based to 0-based, which shifts the answer by one permutation. + ## Selection with Repetition Subsets and permutations without repetition use each element at most once. Cartesian products and powers allow the same value in different positions. If reuse is allowed, the counting flips in a really clean way: “no repetition” tends to create factorials and falling factorials; “repetition allowed” tends to create powers and stars-and-bars. +> **First step:** Ask whether the things being placed are identical or distinguishable, and whether the slots or bins are ordered. Distinct choices in ordered positions with reuse give a plain power, while identical items dropped into distinct bins give stars and bars. + ### Cartesian product becomes “strings of length k” If you choose $k$ positions and each position can be any of $n$ symbols, you get: @@ -343,6 +532,33 @@ n^k This use of “permutations with repetition” means unrestricted sequences. Arranging a fixed multiset is a different problem: if $N$ items have multiplicities $m_1,\ldots,m_r$ with sum $N$, the number of distinct orderings is $N!/(m_1!\cdots m_r!)$. For example, `aab` has three orderings: `aab`, `aba`, and `baa`. +A useful variant of stars and bars requires every bin to get at least one item. Put one item in each of the $n$ bins first, then distribute the remaining $k - n$ freely, which gives $\binom{k-1}{n-1}$ ways for $k \ge n \ge 1$. + +## Catalan Numbers + +> **First step:** If a problem counts nested or recursive structures (balanced brackets, tree shapes, ways to split a sequence around a root), compute the first few answers by hand. If you see 1, 1, 2, 5, 14, 42, you are looking at Catalan numbers. + +The Catalan numbers count structures that split into a left part and a right part around one chosen piece. They start at $C_0 = 1$ and follow the recurrence + +$$C_{n+1} = \sum_{i=0}^{n} C_i \, C_{n-i}$$ + +with the closed form + +$$C_n = \frac{1}{n+1}\binom{2n}{n} = \binom{2n}{n} - \binom{2n}{n+1}$$ + +LeetCode 96 Unique Binary Search Trees is the recurrence directly: pick root $r$ among $n$ keys, the left subtree holds $r-1$ keys and the right subtree $n-r$, and the shapes multiply. LeetCode 22 Generate Parentheses lists the $C_n$ balanced strings with $n$ pairs, and LeetCode 95 Unique Binary Search Trees II lists the trees. The same numbers count polygon triangulations and grid paths that never cross the diagonal. + +```python +def num_trees(n): + dp = [1] + [0] * n + for nodes in range(1, n + 1): + for root in range(1, nodes + 1): + dp[nodes] += dp[root - 1] * dp[nodes - root] + return dp[n] +``` + +The recurrence costs $O(n^2)$ and the closed form $O(n)$, while listing all structures costs about $n \cdot C_n$, and $C_n$ grows roughly like $4^n / n^{3/2}$. For 22, the most common mistake is allowing a closing bracket whenever fewer than $n$ have been used; the correct rule is to add `)` only while the number of closes is below the number of opens. + ## Viewing the Constructions as Functions This section is the “snap everything into place” moment. If you ever feel lost, switching to the function viewpoint usually makes the counting obvious: how many ways can I assign labels to inputs? @@ -405,6 +621,94 @@ This is why many problems that ask you to “try all subsets” are exponential: One last do/don’t that matters in real projects: do treat these counts like early warning signs (a design review for your algorithm), and don’t wait until you’ve implemented everything to realize you built an $n!$ machine. +## Inclusion-Exclusion + +> **First step:** Look for the words "at least one", "or", or "divisible by any of". Decide whether it is easier to count the union directly with inclusion-exclusion or to count the complement ("none of them") and subtract from the total. + +Adding the sizes of overlapping sets counts the overlaps more than once. Inclusion-exclusion corrects this by alternately subtracting and adding the intersections: + +$$|A \cup B| = |A| + |B| - |A \cap B|$$ + +$$|A \cup B \cup C| = |A| + |B| + |C| - |A \cap B| - |A \cap C| - |B \cap C| + |A \cap B \cap C|$$ + +A small example: among $1, \ldots, 30$, there are 15 multiples of 2, 10 multiples of 3, and 5 multiples of 6, so $15 + 10 - 5 = 20$ numbers are divisible by 2 or 3. For divisibility, the intersection of "multiples of $a$" and "multiples of $b$" is the multiples of $\operatorname{lcm}(a, b)$, not of $a \cdot b$. + +LeetCode 1201 Ugly Number III combines this with binary search. Count the numbers up to $x$ divisible by $a$, $b$ or $c$, then search for the smallest $x$ whose count reaches $n$. + +```python +from math import lcm + +def nth_ugly_number(n, a, b, c): + ab, ac, bc, abc = lcm(a, b), lcm(a, c), lcm(b, c), lcm(a, b, c) + def count(x): + return (x // a + x // b + x // c + - x // ab - x // ac - x // bc + x // abc) + lo, hi = 1, 2 * 10**9 + while lo < hi: + mid = (lo + hi) // 2 + if count(mid) >= n: + hi = mid + else: + lo = mid + 1 + return lo +``` + +With $m$ sets, loop over every nonempty mask of the $m$ sets and add the intersection size when the mask has an odd number of bits, subtracting it when even. That is $2^m$ terms, fine for $m$ up to about 20. The most common mistake is using the product $a \cdot b$ instead of $\operatorname{lcm}(a, b)$ when the numbers share a factor. + +## Sets as Bits: XOR and Bit Tricks + +> **First step:** Check whether the universe is small (at most 32 or 64 distinct items) or whether the problem is about values that "appear twice" or "appear once". Small universes fit in an integer used as a set, and pairing problems usually collapse under XOR. + +An integer whose bit $i$ means "item $i$ is present" is a set, and the bitwise operators are the set operations. `a | b` is the union, `a & b` the intersection, `a & ~b` the difference, and `a ^ b` the symmetric difference. Add item $i$ with `a | (1 << i)`, remove it with `a & ~(1 << i)`, and complement within $n$ bits with `a ^ ((1 << n) - 1)`. + +Two expressions show up constantly. `x & (x - 1)` removes the lowest set bit, and `x & -x` isolates it. LeetCode 231 Power of Two is `n > 0 and n & (n - 1) == 0`. LeetCode 191 Number of 1 Bits repeats `n &= n - 1` and counts the steps, so it runs once per set bit. LeetCode 338 Counting Bits fills a table with `bits[i] = bits[i >> 1] + (i & 1)`, because shifting right drops only the last bit. + +XOR acts like toggling membership: $x \oplus x = 0$, $x \oplus 0 = x$, and the order does not matter. XOR all values in LeetCode 136 Single Number and every pair cancels, leaving the single value. LeetCode 268 Missing Number XORs the indices $0, \ldots, n$ with the values, so every present number cancels and only the missing one remains. LeetCode 137 Single Number II, where the others appear three times, counts each bit position modulo 3 instead (in Python, reinterpret bit 31 as the sign at the end). + +LeetCode 260 Single Number III has two unique values $a$ and $b$. XOR of everything is $a \oplus b$, which is nonzero, so pick any set bit of it; $a$ and $b$ differ there. Split all numbers by that bit and XOR each group separately. + +```python +def single_number_iii(nums): + x = 0 + for v in nums: + x ^= v + low = x & -x # a bit where the two answers differ + a = 0 + for v in nums: + if v & low: + a ^= v + return [a, x ^ a] +``` + +All of these run in $O(n)$ time and $O(1)$ extra space, or $O(1)$ per bit operation. The most common mistake is precedence in C and C++, where `n & n - 1 == 0` parses as `n & ((n - 1) == 0)` (Java rejects it outright), so always write `(n & (n - 1)) == 0`. + +## Set Operations with Hash Sets + +> **First step:** Decide whether duplicates matter. A plain set answers "is it present", while a counter (multiset) answers "how many times", and problems like intersection come in both flavours. + +When the universe is large, a hash set gives $O(1)$ average membership tests and turns many sorting solutions into linear ones. LeetCode 217 Contains Duplicate is `len(set(nums)) < len(nums)`. LeetCode 349 Intersection of Two Arrays is a set intersection, while LeetCode 350 Intersection of Two Arrays II keeps multiplicities, which in Python is `(Counter(a) & Counter(b)).elements()` and takes the minimum count of each value. + +LeetCode 128 Longest Consecutive Sequence needs $O(n)$ time. Put all values in a set and only start counting from values $x$ where $x - 1$ is missing, since those are the starts of runs. Each value is then visited at most twice. + +```python +def longest_consecutive(nums): + s = set(nums) + best = 0 + for x in s: + if x - 1 not in s: # x starts a run + y = x + while y + 1 in s: + y += 1 + best = max(best, y - x + 1) + return best +``` + +LeetCode 41 First Missing Positive asks for the same idea in $O(1)$ extra space. The answer is always in $1, \ldots, n+1$, so the array itself can serve as the set: swap each value $v$ in that range into index $v - 1$, repeating at each position until it holds the right value or a duplicate, then the first index $i$ with $a[i] \ne i + 1$ gives the answer $i + 1$. Compute the target index before swapping, because `a[i], a[a[i] - 1] = a[a[i] - 1], a[i]` in Python assigns `a[i]` first and then reads the wrong slot. + +Set relationships also drive counting problems such as LeetCode 2306 Naming a Company. Group the suffixes by first letter; for two letters with suffix sets $S$ and $T$, the valid pairs number $(|S| - |S \cap T|)(|T| - |S \cap T|)$, doubled for the two orders. Iterating over the 26 by 26 letter pairs keeps it fast. + +Hash-set operations run in $O(n)$ average time and space. The most common mistake in 128 is looping over `nums` instead of the set, which repeats the inner walk for duplicated run starts and can degrade toward quadratic time. + ## Choosing a Counting Formula Ask two questions: @@ -429,6 +733,25 @@ Order doesn't matter C(n,k)=n!/(k!(n-k)!) C(n+k-1,k) Use the table by deciding first whether order matters and then whether elements can be reused. Those two assumptions distinguish the four counting problems. +### More Counting Formulas + +Many interview counting problems reduce to one of a short list of formulas. Identify the structure first, then look up the count. + +| Situation | Count | Example problem | +| --- | --- | --- | +| Sequences of length $k$ from $n$ symbols, reuse allowed | $n^k$ | PIN codes of length $k$ | +| All subsets of $n$ items | $2^n$ | 78 Subsets | +| Orderings of $n$ distinct items | $n!$ | 46 Permutations | +| Ordered picks of $k$ from $n$, no reuse | $n!/(n-k)!$ | arranging $k$ of $n$ people in a row | +| Unordered picks of $k$ from $n$ | $\binom{n}{k}$ | 77 Combinations | +| Multisets of size $k$ from $n$ types | $\binom{n+k-1}{k}$ | $k$ scoops from $n$ ice cream flavours | +| $k$ identical items into $n$ bins, none empty | $\binom{k-1}{n-1}$ | splitting $k$ into $n$ positive parts | +| Orderings of a multiset with counts $m_1, \ldots, m_r$ summing to $N$ | $N!/(m_1! \cdots m_r!)$ | 47 Permutations II | +| Right and down paths in an $m \times n$ grid | $\binom{m+n-2}{m-1}$ | 62 Unique Paths | +| Balanced strings of $n$ bracket pairs, BST shapes on $n$ keys | $\frac{1}{n+1}\binom{2n}{n}$ | 22 Generate Parentheses, 96 Unique BSTs | +| (mask, submask) pairs over $n$ bits | $3^n$ | DP over subsets | +| Nonempty subsets of $m$ sets in inclusion-exclusion | $2^m - 1$ | 1201 Ugly Number III | + ## Why these appear in algorithms - Power set / combinations show up in: subset-sum, knapsack variants, feature selection, graph vertex subsets, brute-force optimization. @@ -452,3 +775,28 @@ Sets (A) | +--> order them ---> k-permutations = C(n,k)·k! = P(n,k) ``` + +## Reading the Problem Statement + +Most problems in this area announce their structure in a few stock phrases. Matching those phrases to a technique before writing any code saves you from building the wrong search. + +| If the problem says | What it means for your code | +| --- | --- | +| "order does not matter", "choose", "groups", "teams" | Combinations: generate with a start index, count with $\binom{n}{k}$ | +| "arrangements", "orderings", "order matters" | Permutations: track used items, count with $n!$ or $n!/(n-k)!$ | +| "all possible subsets" or "the power set" | Loop masks from $0$ to $2^n - 1$ or recurse take/skip | +| "each element may be used unlimited times" | Recurse with the same index, not the next one; counts become powers or stars and bars | +| "may contain duplicates" and "no duplicate answers" | Sort and skip equal neighbours at the same depth; counts divide by repeated factorials | +| "return the answer modulo $10^9+7$" | Reduce after every addition and multiplication, and replace division with a modular inverse | +| $n$ up to 20 | $2^n$ is about a million, so bitmask enumeration or bitmask DP is fine | +| $n$ up to 15 or so with "split into groups" | Submask enumeration at $3^n$ total is affordable | +| $n$ up to 8 to 10 | Listing all permutations is feasible | +| $n$ up to $10^5$ or more and "number of ways" | Use a formula with precomputed factorials, never enumeration | +| "the $k$-th permutation" or "the $k$-th in lexicographic order" | Build it directly with the factorial number system | +| "the next greater arrangement" or "next lexicographic order" | Next permutation: find pivot, swap with rightmost larger, reverse suffix | +| "every element appears twice except one" | XOR everything so the pairs cancel | +| "every element appears three times except one" | Count each bit position modulo 3 | +| "adjacent values differ by exactly one bit" | Gray code, `i ^ (i >> 1)` | +| "divisible by a, b or c" or "at least one of" | Inclusion-exclusion with lcm, or count the complement | +| "valid parentheses" or "structurally unique BSTs" | Catalan numbers, by recurrence or closed form | +| "O(n) time" on unsorted values, or "O(1) extra space" with values in 1 to n | Hash set, or use the array itself as the set by index | diff --git a/notes/searching.md b/notes/searching.md index fd16e34..09e8daf 100644 --- a/notes/searching.md +++ b/notes/searching.md @@ -91,6 +91,8 @@ Text search is its own world because matching strings has structure you can expl Linear search is the baseline: it requires no assumptions and no preprocessing, which is exactly why it still matters. If your data is tiny, frequently changing, or you only search once, linear search is often “good enough” and the simplest correct solution. +> **First step:** Check whether the data is unsorted, tiny, or searched only once. If it is, a single left-to-right scan is the right answer, and the only decision left is whether you need the first match, the last match, or all of them. + ### Linear Search Scan the list from left to right, comparing the target with each element until you either find a match (return its index) or finish the list (report “not found”). @@ -292,6 +294,8 @@ Scan: Divide-and-conquer search algorithms assume structure, almost always sorted order, and then exploit it to throw away most of the search space quickly. The “why you should care” is simple: when $n$ gets big, cutting the problem in half repeatedly is the difference between instant and unbearable. +> **First step:** Confirm that the data is sorted, or that some yes or no question about it flips exactly once across the range, and that you can jump to any index in constant time. Then default to binary search and switch to a variant only when one of the specific conditions below applies. + ### Binary Search On a sorted array, repeatedly halve the search interval by comparing the target to the middle element until found or the interval is empty. @@ -393,6 +397,8 @@ For inclusive bounds, compute `mid = low + (high - low) // 2` to avoid adding tw A common refinement in real work is “find the first/last occurrence.” The trick is not to stop when you see the target, keep going left (or right) while preserving correctness. Same structure, slightly different termination condition. +The section Binary Search Patterns for Interview Problems, after Interpolation Search, gives code for this and for the other boundary searches that most interview problems need. + ### Ternary Search For sorted-array lookup, ternary search splits the interval using two midpoints. A related but distinct optimization technique finds an extremum of a unimodal function by comparing function values; it does not use the target-comparison rules shown below. @@ -713,10 +719,312 @@ At this point, an early-stop check already tells us `target (45) < A[low] (50)` - Very fast when value-to-index mapping is close to linear (e.g., near-uniform numeric keys). - Requires careful handling when A[high] = A[low] (avoid division by zero); also sensitive to integer rounding in discrete arrays. +## Binary Search Patterns for Interview Problems + +Most binary search problems on LeetCode are not "find this value in a sorted array". They ask for the first index where something becomes true, the smallest capacity that still works, or the minimum of an array that was rotated. All of them reduce to one move: find a yes or no question whose answers across the range look like no, no, no, yes, yes, and locate the point where they flip. + +The subsections below give one template and then the problem families that plug into it. Problem numbers refer to LeetCode. + +> **First step:** Write the predicate that is false for a prefix of the range and true for the rest, and say in words which index or value you want. Choose whether lo and hi are inclusive or half-open, and decide what to return when the predicate is never true. + +### One Template and Its Loop Invariant + +The template searches the half-open range [lo, hi) for the first index where pred is true. Its invariant is that every index below lo is known to be false and every index at or above hi is known to be true, or is the out-of-range end. Each step keeps that true, so when the loop stops with lo equal to hi, that index is the boundary. + +```python +def first_true(lo, hi, pred): + # pred must be False...False True...True on [lo, hi) + while lo < hi: + mid = lo + (hi - lo) // 2 + if pred(mid): + hi = mid # mid may be the answer, keep it + else: + lo = mid + 1 # mid is not the answer, drop it + return lo # equals the original hi if pred is never true +``` + +Two details make it terminate. With floor rounding, mid is always strictly less than hi, so `hi = mid` shrinks the range, and `lo = mid + 1` always shrinks it too. A version that sets `lo = mid` on success, used to find the last true value, must round mid up with `lo + (hi - lo + 1) // 2`. With floor rounding, a two-element range would pick mid equal to lo forever. + +The closed form `while lo <= hi` with `lo = mid + 1` and `hi = mid - 1` is fine for exact-match search, as in the Binary Search section above, because it returns the moment it sees the target. For boundary questions the half-open form is easier to get right, because the answer is wherever lo ends up and no separate "best so far" variable is needed. + +In C++, Java, or Go, `(lo + hi) / 2` can overflow when both indices are near $2^{31}$, so write `lo + (hi - lo) / 2`. That form also floors correctly when the range contains negative numbers, while `(lo + hi) / 2` truncates toward zero there and can hang the loop. Python integers do not overflow, and `//` always floors. + +Several classic problems are this template with a one-line predicate. In 278 First Bad Version, call it with lo = 1, hi = n and pred = isBadVersion, since version n is known to be bad. In 69 Sqrt(x), search [0, x + 1) for the first m with m * m > x and subtract one; 367 Valid Perfect Square finds the same root and checks whether its square equals the input. Use 64-bit products or compare m > x / m in fixed-width languages. In 658 Find K Closest Elements, search window starts with lo = 0, hi = n - k and pred(i) = x - a[i] <= a[i + k] - x, then return a[lo : lo + k]. + +Each search costs $O(\log(hi - lo))$ predicate calls, times the cost of one call. The most common mistake is a predicate that is not monotone, such as `a[mid] == target`, which is true in the middle and false on both sides. + +### Lower Bound and Upper Bound + +Two boundary searches answer almost every question about a target in a sorted array with duplicates. lower_bound(a, t) is the first index whose value is at least t, and upper_bound(a, t) is the first index whose value is greater than t. Both return len(a) when no such index exists, which is also where t would be inserted. + +Everything else is read off those two numbers. The first occurrence of t is lower_bound, provided that index is in range and holds t. The last occurrence is upper_bound minus one, and the count of t is upper_bound minus lower_bound. The largest value below t sits at lower_bound minus one, and the largest value at most t sits at upper_bound minus one. + +```python +def lower_bound(a, t): # first i with a[i] >= t + lo, hi = 0, len(a) + while lo < hi: + mid = (lo + hi) // 2 + if a[mid] >= t: hi = mid + else: lo = mid + 1 + return lo + +def upper_bound(a, t): # first i with a[i] > t + lo, hi = 0, len(a) + while lo < hi: + mid = (lo + hi) // 2 + if a[mid] > t: hi = mid + else: lo = mid + 1 + return lo + +def search_range(a, t): # 34 + first = lower_bound(a, t) + if first == len(a) or a[first] != t: + return [-1, -1] + return [first, upper_bound(a, t) - 1] +``` + +704 Binary Search is lower_bound plus the same in-range check. 35 Search Insert Position is lower_bound returned as is. 34 Find First and Last Position of Element in Sorted Array is search_range above. 744 Find Smallest Letter Greater Than Target is upper_bound on the letters, wrapping around to index 0 when the result equals the length. + +Each bound costs $O(\log n)$, so 34 stays logarithmic even when every element equals the target. The most common mistake is reading a[first] before checking that first is less than len(a). + +### Library Binary Search: bisect and lower_bound + +In Python, `bisect.bisect_left(a, t)` is lower_bound and `bisect.bisect_right(a, t)` is upper_bound. Both take optional lo and hi arguments to search part of a list without slicing it, and since Python 3.10 they also take a key function. In C++, `std::lower_bound` and `std::upper_bound` return iterators (subtract `v.begin()` for an index), `std::equal_range` returns both at once, and `std::binary_search` only reports whether the value exists. Java's `Arrays.binarySearch` returns an arbitrary index among duplicates and `-(insertion point) - 1` when the value is absent, so write lower_bound by hand in Java when duplicates matter. + +Two common problems take a few lines with bisect. In 981 Time Based Key-Value Store, timestamps for each key arrive strictly increasing, so appending keeps each list sorted, and a get at time t reads the entry just before bisect_right(times, t). In 300 Longest Increasing Subsequence, tails[L] holds the smallest possible last value of an increasing subsequence of length L + 1. Each new value replaces the first tail that is not smaller than it, or extends the list, and the final length is the answer. + +```python +from bisect import bisect_left, bisect_right + +def time_map_get(times, values, t): # 981, lists for one key + i = bisect_right(times, t) - 1 # last timestamp <= t + return values[i] if i >= 0 else "" + +def length_of_lis(nums): # 300 + tails = [] + for x in nums: + i = bisect_left(tails, x) # first tail >= x + if i == len(tails): + tails.append(x) + else: + tails[i] = x + return len(tails) +``` + +Each library call is $O(\log n)$, so 981 answers a get in $O(\log n)$ and the LIS method runs in $O(n \log n)$. The most common mistake in LIS is using bisect_right, which lets equal values extend a run and solves the non-decreasing version instead. + +### Binary Search on the Answer + +Some problems never hand you a sorted array. They ask for the smallest speed, capacity, or largest part sum that still makes a plan work. If a value works, every larger value also works, so a feasibility check is false and then true across the candidate answers. Binary search that range with the template and run the check at each mid. + +The real work is choosing the range and the check. In 875 Koko Eating Bananas the speed runs from 1 to max(piles), and speed k is feasible when the sum of (p + k - 1) // k over all piles is at most h. In 1011 Capacity to Ship Packages Within D Days the capacity runs from max(weights) to sum(weights), and the check packs greedily from left to right, starting a new day whenever the next package would overflow. 410 Split Array Largest Sum is the same problem with days renamed to subarrays. In 1482 Minimum Number of Days to Make m Bouquets the candidate is a day, the check counts runs of k adjacent bloomed flowers, and the answer is -1 when m times k exceeds the number of flowers. + +```python +def ship_within_days(weights, days): # 1011, and 410 with days = k + def feasible(cap): + used, load = 1, 0 + for w in weights: + if load + w > cap: # start a new day + used += 1 + load = 0 + load += w + return used <= days + + lo, hi = max(weights), sum(weights) # hi always works + while lo < hi: + mid = (lo + hi) // 2 + if feasible(mid): + hi = mid + else: + lo = mid + 1 + return lo +``` + +When the question asks to maximize a minimum, the check runs the other way: true for small values and false for large ones. In 1552 Magnetic Force Between Two Balls, sort the positions, and call a gap d feasible if greedy placement from the left fits m balls with neighbors at least d apart. Search for the last true value with `lo = mid` on success, `hi = mid - 1` on failure, and mid rounded up as `(lo + hi + 1) // 2`. + +The total cost is $O(n \log R)$, where $R$ is the width of the answer range, so even answers up to $10^9$ need only about 30 checks. The most common mistake is a lower end that is too small, for example starting 1011 at 1, where this greedy check would accept a capacity smaller than the heaviest package. + +### Rotated Sorted Arrays + +A rotated sorted array such as [4, 5, 6, 7, 0, 1, 2] is two sorted runs, and every value in the first run is larger than every value in the second. Compare a[mid] with the last element of the range. If a[mid] is larger, mid sits in the first run and the minimum lies strictly to its right; otherwise mid sits in the second run and the minimum is at mid or to its left. That is 153 Find Minimum in Rotated Sorted Array. + +For 33 Search in Rotated Sorted Array, one clean approach finds the rotation point and then runs an ordinary lower_bound on whichever run can hold the target. The one-pass version you will also see relies on the fact that at least one of the halves a[lo..mid] and a[mid..hi] is sorted. If a[lo] <= a[mid] the left half is sorted, so check whether the target lies inside its range and move accordingly; otherwise apply the same test to the right half. + +```python +from bisect import bisect_left + +def find_min_index(a): # 153, distinct values + lo, hi = 0, len(a) - 1 + while lo < hi: + mid = (lo + hi) // 2 + if a[mid] > a[hi]: + lo = mid + 1 # the drop is right of mid + else: + hi = mid # mid may be the minimum + return lo + +def search_rotated(a, t): # 33 + k = find_min_index(a) # a[k:] and a[:k] are each sorted + lo, hi = (k, len(a)) if t <= a[-1] else (0, k) + i = bisect_left(a, t, lo, hi) + return i if i < hi and a[i] == t else -1 +``` + +Duplicates break this reasoning. In 81 Search in Rotated Sorted Array II and 154 Find Minimum in Rotated Sorted Array II, a[mid] equal to a[hi] does not say which run mid is in, as [1, 1, 1, 0, 1] and [1, 0, 1, 1, 1] show. The safe move is to shrink by one, hi -= 1 for the minimum, or lo += 1 and hi -= 1 in 81 once a[mid] is known not to be the target. + +With distinct values both problems run in $O(\log n)$, and with duplicates the worst case becomes $O(n)$. The most common mistake is comparing a[mid] with a[lo] when looking for the minimum, which sends the search right on an array that was never rotated. + +### Peak Finding and Mountain Arrays + +162 Find Peak Element asks for any index whose value beats both neighbors, with the ends treated as negative infinity and no two neighbors equal. Compare a[mid] with a[mid + 1]. If the slope goes up, either the values come down again later or the right edge is a peak, so a peak exists right of mid. If the slope goes down, mid or something left of it is a peak. The invariant is "a peak exists in [lo, hi]", not sortedness, which is why this works on an unsorted array. + +852 Peak Index in a Mountain Array uses the same code, and there the predicate a[mid] > a[mid + 1] really is false then true. 1095 Find in Mountain Array builds on it: find the peak, binary search the increasing part [0, peak], and only if that fails, binary search the decreasing part [peak + 1, n - 1] with the comparisons reversed. Searching the left part first returns the smaller index and stays within the limit of 100 calls to get. + +```python +def find_peak(a): # 162 and 852 + lo, hi = 0, len(a) - 1 + while lo < hi: + mid = (lo + hi) // 2 # mid < hi, so a[mid + 1] exists + if a[mid] < a[mid + 1]: + lo = mid + 1 # uphill, a peak lies to the right + else: + hi = mid # downhill, mid or a left index is a peak + return lo +``` + +Peak finding takes $O(\log n)$ comparisons. The most common mistake is writing `while lo <= hi` around this body, which reads a[mid + 1] past the end once lo and hi both reach n - 1. + +### Hidden Monotone Predicates + +Some problems hide the false-then-true pattern in a less obvious place. In 540 Single Element in a Sorted Array, every value appears twice except one. Before the single element each pair starts at an even index, so a[i] == a[i + 1] for even i. From the single element onward the pairs start at odd indices and that test fails. Search only even positions for the first even i where a[i] != a[i + 1]. + +In 287 Find the Duplicate Number the array holds n + 1 values from 1 to n and may not be modified. Count how many values are at most mid. If no repeated value is at most mid, that count is at most mid, and once mid reaches the duplicate the count exceeds mid by the pigeonhole principle. Search the value range [1, n] for the first mid whose count is above mid. Floyd's cycle detection on the index graph also solves it, in $O(n)$. + +```python +def single_non_duplicate(a): # 540 + lo, hi = 0, len(a) - 1 # both stay even + while lo < hi: + mid = (lo + hi) // 2 + if mid % 2 == 1: + mid -= 1 # align to the start of a pair + if a[mid] == a[mid + 1]: + lo = mid + 2 # pairs intact up to here + else: + hi = mid # the break is at mid or earlier + return a[lo] + +def find_duplicate(nums): # 287 + lo, hi = 1, len(nums) - 1 + while lo < hi: + mid = (lo + hi) // 2 + if sum(x <= mid for x in nums) > mid: + hi = mid + else: + lo = mid + 1 + return lo +``` + +540 runs in $O(\log n)$, and the counting version of 287 runs in $O(n \log n)$ with $O(1)$ extra space. The most common mistake in 540 is skipping the even alignment of mid, which compares elements across a pair boundary. + +### Searching a Sorted 2D Matrix + +74 Search a 2D Matrix promises that each row is sorted and each row starts above the previous row's last value. Read row by row, the matrix is one sorted list of length rows times cols, and flat index i lives at row i // cols, column i % cols. Run ordinary binary search on the flat index. + +240 Search a 2D Matrix II only promises that rows and columns are each sorted, so rows can overlap and flattening fails. Start at the top-right corner instead. If that value is larger than the target, the whole column below it is larger too, so step left; if it is smaller, the whole row to its left is smaller, so step down. Every comparison discards a row or a column. + +```python +def search_flat(M, t): # 74 + rows, cols = len(M), len(M[0]) + lo, hi = 0, rows * cols - 1 + while lo <= hi: + mid = (lo + hi) // 2 + v = M[mid // cols][mid % cols] + if v == t: return True + if v < t: lo = mid + 1 + else: hi = mid - 1 + return False + +def search_staircase(M, t): # 240, also correct for 74 + r, c = 0, len(M[0]) - 1 + while r < len(M) and c >= 0: + if M[r][c] == t: return True + if M[r][c] > t: c -= 1 # column below is all bigger + else: r += 1 # row to the left is all smaller + return False +``` + +The flat search costs $O(\log(mn))$ and the staircase costs $O(m + n)$ for an m by n matrix. The most common mistake is dividing the flat index by the number of rows instead of columns, which only works on square matrices. + +### K-th Smallest by Counting + +When the k-th smallest item of a large implicit set is needed, binary search over values instead of positions. For a candidate x, count the items that are at most x. That count never decreases as x grows, so the answer is the first x whose count reaches k. That first x is always a real item, because the count only rises at values that are present. + +In 378 Kth Smallest Element in a Sorted Matrix the count comes from a staircase walk from the bottom-left corner in $O(n)$. In 719 Find K-th Smallest Pair Distance, sort the numbers, search distances from 0 to max minus min, and count pairs at distance at most d with a sliding window. For each right index, advance left while nums[right] - nums[left] > d, then add right - left. + +```python +def kth_smallest(M, k): # 378, n by n matrix + n = len(M) + def count_le(x): + r, c, cnt = n - 1, 0, 0 # start bottom-left + while r >= 0 and c < n: + if M[r][c] <= x: + cnt += r + 1 # M[0..r][c] are all <= x + c += 1 + else: + r -= 1 + return cnt + + lo, hi = M[0][0], M[n - 1][n - 1] + while lo < hi: + mid = (lo + hi) // 2 # Python floors negatives too + if count_le(mid) >= k: + hi = mid + else: + lo = mid + 1 + return lo +``` + +378 costs $O(n \log R)$ and 719 costs $O(n \log n + n \log R)$, where $R$ is the value range. The most common mistake is returning mid as soon as the count equals k, which can produce a value that is not in the matrix. + +### Median of Two Sorted Arrays + +4 Median of Two Sorted Arrays asks for $O(\log(m + n))$ time, so merging is out. Instead, choose how many items i the shorter array A contributes to the left half of the combined order. The longer array B then contributes j = half - i, with half = (m + n + 1) // 2. The cut is correct when every left item is at most every right item, which comes down to A[i - 1] <= B[j] and B[j - 1] <= A[i]. + +If A[i - 1] > B[j], A gave too many items, so move i left. If B[j - 1] > A[i], A gave too few, so move i right. Missing neighbors at the edges count as minus or plus infinity. Once the cut is right, the median is the largest left value for an odd total, or the mean of the largest left and smallest right values for an even total. + +```python +def find_median(A, B): # 4 + if len(A) > len(B): + A, B = B, A # search the shorter array + m, n = len(A), len(B) + half = (m + n + 1) // 2 + lo, hi = 0, m # i ranges over 0..m inclusive + while True: + i = (lo + hi) // 2 + j = half - i + a_left = A[i - 1] if i > 0 else float("-inf") + a_right = A[i] if i < m else float("inf") + b_left = B[j - 1] if j > 0 else float("-inf") + b_right = B[j] if j < n else float("inf") + if a_left > b_right: + hi = i - 1 + elif b_left > a_right: + lo = i + 1 + else: + left_max = max(a_left, b_left) + if (m + n) % 2 == 1: + return left_max + return (left_max + min(a_right, b_right)) / 2 +``` + +The search takes $O(\log \min(m, n))$ steps. The most common mistake is searching the longer array, which can push j below zero or past the end of the other array. + ## Hash-based Search Hash-based search is for when you want direct access by key. Instead of narrowing down ranges, you compute a location in (roughly) constant time. The cost is setup complexity: you need a hash function, a collision strategy, and resizing rules to keep performance stable. +> **First step:** Decide what the key is and what you need back for it. A plain yes or no needs a set, while an index, count, or stored value needs a map; in interview code you use the built-in dict, HashMap, or unordered_map, and the collision schemes below explain why those stay fast. + - Separate chaining: Easiest deletions, steady $O(1)$ with α≈1; good when memory fragmentation isn’t a concern. - Open addressing (double hashing): key-dependent probe steps reduce clustering, but scattered probes usually have poorer locality than linear probing. Choose a load threshold appropriate to the implementation. - Open addressing (linear/quadratic): Simple and fast at low α; watch clustering and tombstones. @@ -1393,10 +1701,39 @@ FOUND in T₂ at index 4 - Deletions are easy (remove key); no tombstones, but ensure invariants remain. - Sensitive to hash quality; poor hashes increase cycle risk. +### Pair Lookup: Hash Complements and Two Pointers + +The most common hash-table move in interviews is the complement lookup in 1 Two Sum. Walk the array once, and for each value x ask whether target - x has already been seen. If it has, the stored index and the current index are the answer; if not, record x with its index. Checking before inserting keeps an element from pairing with itself while still handling [3, 3] with target 6. + +When the array is already sorted, as in 167 Two Sum II - Input Array Is Sorted, two pointers do the same job in constant space. Start at both ends. If the sum is too small, the left value cannot pair with anything, because it already faces the largest remaining value, so move left inward. If the sum is too large, the right value is useless for the same reason, so move right inward. The same scan, run once per fixed first element, drives 15 3Sum after sorting. + +```python +def two_sum(nums, target): # 1 + seen = {} # value -> index + for i, x in enumerate(nums): + if target - x in seen: + return [seen[target - x], i] + seen[x] = i + return [] + +def two_sum_sorted(a, target): # 167, answer is 1-indexed + l, r = 0, len(a) - 1 + while l < r: + s = a[l] + a[r] + if s == target: return [l + 1, r + 1] + if s < target: l += 1 + else: r -= 1 + return [] +``` + +Both run in $O(n)$ time; the hash version uses $O(n)$ extra space and the two-pointer version $O(1)$. The most common mistake is storing x before looking up its complement, which returns the same index twice when the target equals 2x. + ## Probabilistic Membership Filters Probabilistic membership filters answer a very specific question: “Is this item in the set?” but with a twist, they allow uncertainty in one direction. They can say “definitely not” with full confidence, or “maybe” when the filter thinks the item could be present. This trade lets them be extremely space-efficient and extremely fast. +> **First step:** Ask whether a wrong "maybe" is acceptable and whether items must ever be removed. If a false positive can be confirmed against a slower exact store, use a Bloom filter for insert-only sets and a counting Bloom or cuckoo filter when deletions are needed. + The main rule to remember is: - No false negatives: if the filter says “definitely not,” the item is not in the set. @@ -1891,6 +2228,8 @@ A good mental model: cuckoo filters are like a coat check where every coat has e Searching in strings is about finding a pattern of length `m` inside a text of length `n`. The naive method works by checking every alignment, but smarter algorithms exploit structure, especially the fact that mismatches can tell you how far you can skip ahead. +> **First step:** Write down the text length $n$, the pattern length $m$, and whether there is one pattern or many. For a single pattern the built-in find or a KMP prefix function is enough, and Rabin–Karp pays off when many windows or patterns are compared by hash. + ### Naive String Search Slide the pattern across the text one position at a time; at each position, compare characters until mismatch or full match. @@ -1925,3 +2264,30 @@ Rabin–Karp uses a rolling hash. Hash the pattern and each length-`m` window of - Great for multiple patterns (hash them all) or streaming windows. Why it matters: the rolling hash lets you update the window hash in constant time, which is powerful when you’re scanning huge text or many patterns. + +## Reading the Problem Statement + +Search problems usually announce their technique in a few telltale phrases. The table maps the common ones to what they mean for the code you write. + +| If the problem says | What it means for your code | +| --- | --- | +| "sorted array" or "sorted in non-decreasing order" | Binary search for lookups, or two pointers for pairs; aim for $O(\log n)$ per query. | +| "rotated sorted array" | Binary search that first decides which half is sorted, or finds the minimum and then searches one run. | +| "may contain duplicates" together with rotation | When a[mid] equals the ends you cannot pick a side; shrink by one and accept an $O(n)$ worst case. | +| "first occurrence" or "last occurrence" or "starting and ending position" | lower_bound for the first, upper_bound minus one for the last, and check that the target is really there. | +| "insert position" or "where it would be inserted" | lower_bound, and the answer may equal n. | +| "smallest element greater than target" | upper_bound; handle the case where it returns n, or wrap to index 0 if the problem says so. | +| "must run in O(log n)" | A strong hint for binary search, even when the input does not look sorted at first. | +| "minimum possible maximum" or "smallest value such that" | Binary search on the answer with a greedy feasibility check; lo is the smallest plausible value, hi one that surely works. | +| "maximize the minimum distance" | Binary search on the answer for the last feasible value; set lo = mid on success and round mid up. | +| n up to $10^5$ and values up to $10^9$ | About 30 iterations of an $O(n)$ check over the value range fit easily. | +| "find any peak" or "return any of them" | Compare a[mid] with a[mid + 1] and walk uphill; you do not need the global maximum. | +| "mountain array" | Find the peak first, then binary search each side with the comparison direction matching that side. | +| "k-th smallest" in a sorted matrix or among pair distances | Binary search over values with a count of items at most mid; return the first value whose count reaches k. | +| "rows and columns are sorted" | Staircase from the top-right corner in $O(m + n)$; if each row starts above the previous one, flatten the index instead. | +| "median of two sorted arrays" | Binary search the cut position in the shorter array. | +| "timestamps are strictly increasing" and "value at or before time t" | Append per key and use bisect_right minus one. | +| unsorted input and "does it exist" or "find a pair" | Hash set or hash map with expected $O(1)$ lookups. | +| sorted input and "pair that sums to target" | Two pointers from both ends in $O(1)$ extra space. | +| "n + 1 integers in the range 1 to n" and "do not modify the array" | Binary search on the value range with counting, or Floyd's cycle detection. | +| "longest increasing subsequence" with large n | Keep a tails array and place each value with bisect_left for $O(n \log n)$. | diff --git a/notes/sorting.md b/notes/sorting.md index 956a858..f92f1e8 100644 --- a/notes/sorting.md +++ b/notes/sorting.md @@ -13,6 +13,8 @@ What makes sorting worth caring about is that it quietly powers “everyday” f ## Stability in Sorting +> **First step:** Ask whether equal keys carry other data whose original order matters. If they do, use a stable sort or add the original index as the last tie-breaker. + Stability, in the context of sorting, refers to preserving the relative order of equal items in the sorted output. In simple terms, when two items have equal keys: - In an unstable sorting algorithm, their order might be reversed in the sorted output. @@ -81,8 +83,50 @@ A quick rule of thumb: And because many real systems sort records, stability is one of those “small” choices that saves you from weird bugs later. +## Sort Keys, Tie-Breakers, and Comparators + +> **First step:** Write the full ordering in words before writing code, including what happens on ties. Then decide whether it fits in a key tuple or truly needs a pairwise comparator. + +Many sorting problems are not about the sorting algorithm at all. They are about the order. 1636 Sort Array by Increasing Frequency, 451 Sort Characters By Frequency, 791 Custom Sort String and 1122 Relative Sort Array all come down to one library sort with the right key. + +In Python, a key function maps each item to something that compares naturally, usually a tuple. Tuples compare entry by entry, so the first entry is the primary key and later entries break ties. To reverse one numeric component, negate it. For 451 the key is `(-count[c], c)`, and for 791 it is the character's position in the order string. + +Some orders depend on both items at once. In 179 Largest Number, `a` belongs before `b` when the string `a + b` is larger than `b + a`, which no single-item key expresses directly. Write a comparator that returns a negative, zero or positive number and wrap it with `functools.cmp_to_key`. + +406 Queue Reconstruction by Height is a sort followed by inserts. Sort by height descending and then by `k` ascending, and insert each person at index `k`. Everyone already placed is at least as tall, so index `k` leaves exactly `k` such people in front, and later shorter people never change that count. + +```python +# 1636: frequency ascending, then value descending +cnt = Counter(nums) +nums.sort(key=lambda x: (cnt[x], -x)) + +# 1122: values from arr2 in arr2's order, the rest ascending at the end +rank = {v: i for i, v in enumerate(arr2)} +arr1.sort(key=lambda x: (0, rank[x]) if x in rank else (1, x)) + +# 179: a goes first when a + b is the larger string +def cmp(a, b): + return -1 if a + b > b + a else (1 if a + b < b + a else 0) +s = "".join(sorted(map(str, nums), key=cmp_to_key(cmp))) +answer = "0" if s[0] == "0" else s + +# 406: tallest first, then by k, insert at index k +people.sort(key=lambda p: (-p[0], p[1])) +queue = [] +for p in people: + queue.insert(p[1], p) +``` + +In C++, `std::sort` requires a comparator that is a strict weak ordering. Above all, `comp(a, a)` must be false. Writing `return a <= b;` breaks this, because for equal items both `comp(a, b)` and `comp(b, a)` are true. That is undefined behavior: the output can be wrong, and some implementations read past the end of the range and crash when there are many duplicates. Use `<`, and for several keys compare `std::tie(a.k1, a.k2) < std::tie(b.k1, b.k2)`. + +Stability also differs by language. Python's `sorted` and `list.sort` are stable, so the multi-pass trick from the previous section works. C++ `std::sort` is not stable; use `std::stable_sort` or add the original index as the last key. + +Complexity is $O(n\log n)$ comparisons times the cost of one comparison, which is $O(L)$ for the string concatenations in 179. The most common mistake is a missing tie-breaker, such as forgetting that 1636 wants larger values first among equal frequencies. + ## Bubble Sort +> **First step:** In practice, call the library sort instead. Bubble sort is worth knowing because every swap it makes fixes exactly one inversion, so its swap count equals the number of inversions in the input. + Bubble sort is one of the simplest sorting algorithms. It is often used as an introductory algorithm because it is easy to understand, even though it is not efficient for large datasets. The name comes from the way larger elements "bubble up" to the top (end of the list), just as bubbles rise in water. @@ -217,6 +261,8 @@ Implementation: ## Selection Sort +> **First step:** Check whether writes are the expensive operation or whether you only need the smallest few items. Selection sort does at most $n-1$ swaps, and stopping after $k$ passes gives the $k$ smallest items in $O(nk)$ time. + Selection sort is another simple sorting algorithm, often introduced right after bubble sort because it is equally easy to understand. Instead of repeatedly "bubbling" elements, selection sort works by repeatedly selecting the smallest (or largest) element from the unsorted portion of the array and placing it into its correct position. @@ -356,6 +402,8 @@ Implementation: ## Insertion Sort +> **First step:** Check how close to sorted the input already is. Insertion sort runs in $O(n + I)$ time, where $I$ is the number of inversions, so it is excellent on nearly sorted or tiny inputs and poor otherwise. + Insertion sort is a simple, intuitive sorting algorithm that works the way people often sort playing cards in their hands. It builds the sorted portion one element at a time, by repeatedly taking the next element from the unsorted portion and inserting it into its correct position among the already sorted elements. @@ -477,6 +525,8 @@ Implementation: ## Merge Sort +> **First step:** Decide whether you need what merge sort uniquely gives: a stable order, a guaranteed $O(n\log n)$, a sort of a linked list, or a chance to count something across the two halves during the merge. + Merge sort divides the input into two halves, sorts each half recursively, and merges the two sorted results. Unlike quicksort, its split does not depend on the values, so the recursion stays balanced. To merge, compare the first unconsumed item from each half and copy the smaller one to the output. When one half is exhausted, append the remainder of the other half. Each item is processed once during a merge. @@ -507,8 +557,65 @@ Implementations: - [C++](https://github.com/djeada/Algorithms-And-Data-Structures/tree/master/src/sorting/cpp/merge_sort) - [Python](https://github.com/djeada/Algorithms-And-Data-Structures/tree/master/src/sorting/python/merge_sort) +## Merging from the Back and Sorting Linked Lists + +> **First step:** Look at where the output has to live. If it shares memory with an input, merge in the direction where the free space is; if the input is a linked list, split with slow and fast pointers instead of indexing. + +88 Merge Sorted Array gives `nums1` with `m` real values followed by `n` empty slots. Merging from the front would overwrite values that have not been read yet. Merge from the back instead: compare the last unread values of both arrays and write the larger into the last free slot. When `nums2` runs out, the rest of `nums1` is already in place; when `nums1` runs out, copy what is left of `nums2`. + +148 Sort List asks for $O(n\log n)$ on a singly linked list, which rules out quicksort's random access and makes merge sort the natural fit. Merging two lists needs no extra buffer, because nodes are relinked rather than copied. + +```python +def sort_list(head): + if head is None or head.next is None: + return head + slow, fast = head, head.next + while fast and fast.next: + slow, fast = slow.next, fast.next.next + mid, slow.next = slow.next, None # cut the list in two + left, right = sort_list(head), sort_list(mid) + dummy = tail = ListNode() + while left and right: + if left.val <= right.val: # <= keeps it stable + tail.next, left = left, left.next + else: + tail.next, right = right, right.next + tail = tail.next + tail.next = left or right + return dummy.next +``` + +Both run in linear time per merge, so 88 is $O(m+n)$ and 148 is $O(n\log n)$ with $O(\log n)$ recursion stack (a bottom-up version gets $O(1)$ extra space). The most common mistake in 148 is starting `fast` at `head`, which on a two-node list leaves the right half empty and recurses forever. + +## Counting Pairs with Merge Sort + +> **First step:** Check whether the question counts pairs $i < j$ that satisfy a comparison between `nums[i]` and `nums[j]`. If it does, every such pair is split by exactly one merge step, so you can count it there. + +When merge sort combines two sorted halves, every element of the left half came before every element of the right half in the original array. So any pair with $i$ on the left and $j$ on the right can be counted using only the two sorted halves, and pairs inside one half were already counted deeper in the recursion. Because both halves are sorted, a single pointer that only moves forward does the counting in linear time. + +The classic case is inversion counting, pairs with $i < j$ and `nums[i] > nums[j]`. 493 Reverse Pairs counts `nums[i] > 2 * nums[j]`, which is a different test from the merge comparison, so count in a separate pass before merging. 315 Count of Smaller Numbers After Self needs a count per element: sort `(value, index)` pairs, and whenever a left element is taken during the merge, add the number of right elements already taken to `counts[index]`. Taking the left element on ties keeps equal values out of the count. + +```python +def sort_and_count(a): + if len(a) <= 1: + return a, 0 + mid = len(a) // 2 + left, x = sort_and_count(a[:mid]) + right, y = sort_and_count(a[mid:]) + count, j = x + y, 0 + for v in left: # left is sorted, so j never moves back + while j < len(right) and v > 2 * right[j]: # 493; use v > right[j] for inversions + j += 1 + count += j + return merge(left, right), count # the usual linear merge +``` + +This is $O(n\log n)$ time and $O(n)$ extra space. A Fenwick tree over coordinate-compressed values, scanned from right to left, solves the same problems in the same time. The most common mistake is in C++, where `2 * nums[j]` overflows a 32-bit int; compute it in 64 bits. + ## Quick Sort +> **First step:** Choose the pivot at random (or shuffle once) and decide how keys equal to the pivot are handled. A fixed first or last pivot is quadratic on sorted input, and the tests for 912 Sort an Array are known to punish that. + Quick Sort is a divide-and-conquer algorithm. Unlike bubble sort or selection sort, which work by repeatedly scanning the whole array, Quick Sort works by partitioning the array into smaller sections around a "pivot" element and then sorting those sections independently. It is one of the fastest sorting algorithms in practice, widely used in libraries and systems. @@ -651,8 +758,67 @@ Implementation: - [C++](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/src/sorting/cpp/quick_sort/src/quick_sort.cpp) - [Python](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/src/sorting/python/quick_sort/src/quick_sort.py) +## Three-Way Partition (Dutch National Flag) + +> **First step:** Count how many distinct groups the output needs. Two groups is a plain partition with one write pointer; three groups (less than, equal to, greater than) is the Dutch national flag with three pointers. + +75 Sort Colors asks to sort an array of 0s, 1s and 2s in place in one pass. Keep three regions: `a[:lo]` holds 0s, `a[lo:mid]` holds 1s, `a[hi+1:]` holds 2s, and `a[mid:hi+1]` is still unknown. Each step looks at `a[mid]` and grows one region. + +The same routine with "less than pivot, equal, greater than pivot" is the fix for quicksort and quickselect on inputs with many duplicates, since the whole equal block is excluded from further recursion. 905 Sort Array By Parity is the two-group version: one pointer marks where the next even number goes. 324 Wiggle Sort II builds on the same idea. The simple $O(n\log n)$ answer sorts a copy, fills the even indices with the smaller half reversed and the odd indices with the larger half reversed, and the reversal keeps copies of the median apart. + +```python +def sort_colors(a): + lo, mid, hi = 0, 0, len(a) - 1 + while mid <= hi: + if a[mid] == 0: + a[lo], a[mid] = a[mid], a[lo] + lo += 1 + mid += 1 + elif a[mid] == 1: + mid += 1 + else: + a[mid], a[hi] = a[hi], a[mid] + hi -= 1 # do not move mid: the new a[mid] is unread +``` + +This is $O(n)$ time and $O(1)$ space. The most common mistake is advancing `mid` after swapping with `hi`, which skips the value that just arrived from the right end. + +## Quickselect for the k-th Element + +> **First step:** Convert the request into a 0-based index in ascending order. The k-th largest of $n$ items sits at index $n-k$, and the k closest points are the first $k$ after selecting by distance. + +Quickselect is quicksort that recurses into one side only. After a partition the pivot is at its final index `p`. If `p` equals the target index you are done; otherwise continue only in the side that contains the target. With a random pivot each round discards a constant fraction of the range on average, so the total work is a geometric series. + +It fits 215 Kth Largest Element in an Array and 973 K Closest Points to Origin (select on squared distance; the first $k$ items are the answer in any order). It also finds the median that the linear-time version of 324 Wiggle Sort II needs. In C++, `std::nth_element` does the same job. + +```python +def select(a, k): # value that would be at index k after sorting + lo, hi = 0, len(a) - 1 + while True: + r = random.randint(lo, hi) + a[r], a[hi] = a[hi], a[r] + store = lo + for i in range(lo, hi): + if a[i] < a[hi]: + a[i], a[store] = a[store], a[i] + store += 1 + a[store], a[hi] = a[hi], a[store] + if store == k: + return a[store] + if store < k: + lo = store + 1 + else: + hi = store - 1 + +kth_largest = select(nums, len(nums) - k) +``` + +Expected time is $O(n)$ with $O(1)$ extra space, but the worst case is $O(n^2)$. The most common mistake is a partition like this one on input with many equal values: every equal item lands on one side and each round shrinks the range by one, so switch to a three-way partition when duplicates are likely. + ## Heap Sort +> **First step:** Ask whether you need the full order or only the top $k$ items. A full heap sort costs $O(n\log n)$, while a heap kept at size $k$ answers top $k$ questions in $O(n\log k)$. + Heap Sort is a comparison-based sorting algorithm that uses a special data structure called a binary heap. It is efficient, with guaranteed $O(n \log n)$ performance, and sorts in-place (no extra array needed). @@ -816,8 +982,30 @@ Implementation: - [C++](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/src/sorting/cpp/heap_sort/src/heap_sort.cpp) - [Python](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/src/sorting/python/heap_sort/src/heap_sort.py) +## Partial Sorting with a Heap (Top k) + +> **First step:** Decide which end you are keeping and pick the opposite kind of heap. To keep the $k$ largest, use a min-heap of size $k$ so the weakest kept item sits on top and is the one evicted. + +Many problems ask for a few items from the top of an order: 215 Kth Largest Element, 347 Top K Frequent Elements, 973 K Closest Points to Origin, and 692 Top K Frequent Words. Sorting everything costs $O(n\log n)$ and does more work than needed. A heap that never grows past $k$ costs $O(n\log k)$ and also works on a stream. + +Python's `heapq` is a min-heap. To keep the $k$ smallest items instead, push negated keys, for example `(-dist, x, y)` in 973. For counts, push `(count, value)` pairs so the heap orders by count. For one-off queries `heapq.nlargest(k, items, key=...)` does this for you, and C++ has `std::partial_sort` and `std::priority_queue`. + +```python +def k_largest(nums, k): + heap = [] + for x in nums: + heapq.heappush(heap, x) + if len(heap) > k: + heapq.heappop(heap) # drop the smallest of the k + 1 + return heap # heap[0] is the k-th largest +``` + +This runs in $O(n\log k)$ time and $O(k)$ space. The most common mistake is using the heap in the wrong direction, a max-heap of size $k$ for the $k$ largest, which evicts exactly the items you wanted to keep. + ## Radix Sort +> **First step:** Count the digits $d$ in the largest key and pick the base $b$. Radix sort pays off when $d(n+b)$ is clearly smaller than $n\log n$, and negative numbers need a shift or separate handling. + Radix Sort is a non-comparison-based sorting algorithm. Instead of comparing elements directly, it processes numbers digit by digit, from either the least significant digit (LSD) or the most significant digit (MSD), using digit-based distribution. LSD radix sort requires a stable intermediate sort, commonly counting sort; MSD radix sort recursively sorts digit buckets and may use an unstable in-place partition. @@ -927,6 +1115,8 @@ Implementation: ## Counting Sort +> **First step:** Find the value range $k$ and compare it with $n$. If $k$ is small (letters, scores up to 1000, frequencies up to $n$), a count array replaces sorting entirely. + Counting Sort is a non-comparison-based sorting algorithm that works by counting occurrences of each distinct element and then calculating their positions in the output array. It is especially efficient when: @@ -1018,6 +1208,33 @@ Implementation: - [C++ (The Algorithms)](https://github.com/TheAlgorithms/C-Plus-Plus/blob/master/sorting/counting_sort.cpp) - [Python (The Algorithms)](https://github.com/TheAlgorithms/Python/blob/master/sorts/counting_sort.py) +## Bucket Sort for Frequencies and Gaps + +> **First step:** Look for a natural bound on the keys. Frequencies never exceed $n$, letters fit in 26 slots, and $n$ values spread over a range force at least one large gap; any of these lets buckets replace comparisons. + +A frequency is an integer between 1 and $n$, so items can be bucketed by how often they occur instead of sorted by it. That gives 347 Top K Frequent Elements and 451 Sort Characters By Frequency in linear time: count, put each value into `buckets[count]`, and read the buckets from high to low. + +274 H-Index uses the same bound. Any citation count above $n$ can be capped at $n$, so count papers per capped value, then walk $h$ from $n$ down while keeping a running total of papers with at least $h$ citations; the first $h$ where that total reaches $h$ is the answer. 242 Valid Anagram and 49 Group Anagrams also fit here: a tuple of 26 letter counts is an $O(L)$ key, where sorting each word costs $O(L\log L)$. + +164 Maximum Gap asks for linear time. With $n$ values from `lo` to `hi`, the largest gap is at least the average $(hi - lo)/(n-1)$. Use buckets of width $b = \max(1, \lfloor (hi-lo)/(n-1) \rfloor)$. Two values in one bucket differ by less than $b$, so the answer always spans two buckets. Keep only each bucket's minimum and maximum and compare each nonempty bucket's minimum with the previous nonempty bucket's maximum. LSD radix sort followed by a scan also works. + +```python +def top_k_frequent(nums, k): + cnt = Counter(nums) + buckets = [[] for _ in range(len(nums) + 1)] + for value, c in cnt.items(): + buckets[c].append(value) + out = [] + for c in range(len(nums), 0, -1): + for value in buckets[c]: + out.append(value) + if len(out) == k: + return out + return out +``` + +These run in $O(n)$ time plus the size of the bucket array. The most common mistake is sizing the bucket array by the values instead of by the bound, for example allocating up to the largest citation count in 274 instead of capping at $n$. + ## Comparison Table Space means auxiliary storage, including the recursion stack. Average-case bounds require input or randomization assumptions; the table does not predict elapsed time. @@ -1038,3 +1255,97 @@ Use complexity bounds together with stability, memory requirements, and input ch | LSD Radix Sort | $O(d(n+b))$ | $O(d(n+b))$ | $O(d(n+b))$ | $O(n+b)$ | Yes | $d$ digits, radix $b$ | Use insertion sort for tiny or nearly-sorted arrays, quick sort for general high speed (with good pivot strategy), heap sort for worst-case guarantees, and counting/radix when your keys make it possible. + +## Interval Problems + +> **First step:** Decide whether you are combining intervals or choosing among them. Combining (merge, insert, count overlaps) sorts by start; choosing the most compatible intervals or the fewest points that hit them sorts by end. Also settle whether touching endpoints overlap. + +56 Merge Intervals sorts by start and walks once. Each interval either overlaps the last merged one, which then extends to the larger of the two ends, or starts a new merged interval. 57 Insert Interval gets input that is already sorted, so it needs no sort: copy the intervals that end before the new one starts, absorb every interval that starts at or before its end, then copy the rest. 252 Meeting Rooms sorts by start and checks each neighbor pair for overlap. + +253 Meeting Rooms II asks for the maximum number of meetings running at once. Sort the start times and the end times separately and walk both, opening a room when a meeting starts before the earliest unfinished one ends. A min-heap of end times gives the same answer and is easier to extend when you also need to know which room is used. + +435 Non-overlapping Intervals and 452 Minimum Number of Arrows to Burst Balloons sort by end. Keeping the interval that finishes first leaves the most room for the rest, and the exchange argument behind this is in the [greedy algorithms notes](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/notes/greedy_algorithms.md). In 435 the answer is $n$ minus the number kept. In 452 shoot each arrow at the current end and fire a new one only when a balloon starts strictly after it, since touching balloons share an arrow. 1288 Remove Covered Intervals sorts by start ascending and end descending, so an interval is covered exactly when its end does not exceed the largest end seen so far. + +```python +def merge_intervals(intervals): # 56 + intervals.sort(key=lambda iv: iv[0]) + out = [] + for s, e in intervals: + if out and s <= out[-1][1]: + out[-1][1] = max(out[-1][1], e) + else: + out.append([s, e]) + return out + +def min_rooms(intervals): # 253 + starts = sorted(s for s, e in intervals) + ends = sorted(e for s, e in intervals) + rooms, j = 0, 0 + for s in starts: + if s < ends[j]: + rooms += 1 # no meeting has ended yet: open a room + else: + j += 1 # reuse the room freed at ends[j] + return rooms +``` + +All of these are $O(n\log n)$ for the sort and $O(n)$ for the scan, except 57, which is $O(n)$. The most common mistake in 56 is setting the merged end to the new end instead of the maximum, which breaks when one interval contains the next, as in `[1, 10]` followed by `[2, 3]`. + +## Sorting as Preprocessing + +> **First step:** Ask whether the answer depends on the original positions. If it does not, sorting first costs only $O(n\log n)$ and turns a pair or triple search into two pointers or binary search. + +Once an array is sorted, a pair sum that is too small can only grow by moving the left pointer right, and one that is too large can only shrink by moving the right pointer left. This turns the $O(n^2)$ pair search into $O(n)$. 15 3Sum fixes each first element and runs that two-pointer scan on the rest, for $O(n^2)$ overall; 16 3Sum Closest and 18 4Sum follow the same shape. Sorting also puts duplicates next to each other, which is how 3Sum avoids repeated triplets. + +Sorted data also supports binary search, for example counting how many values are at most a target with `bisect_right`; the [searching notes](https://github.com/djeada/Algorithms-And-Data-Structures/blob/master/notes/searching.md) cover the variants. 945 Minimum Increment to Make Array Unique sorts and then keeps `need`, the smallest value the next item may take: each item `x` costs `max(0, need - x)` moves and sets `need = max(need, x) + 1`. 49 Group Anagrams uses the sorted string of each word as a dictionary key. + +```python +def three_sum(nums): + nums.sort() + res = [] + for i in range(len(nums) - 2): + if i > 0 and nums[i] == nums[i - 1]: + continue # same first value, same triplets + lo, hi = i + 1, len(nums) - 1 + while lo < hi: + total = nums[i] + nums[lo] + nums[hi] + if total < 0: + lo += 1 + elif total > 0: + hi -= 1 + else: + res.append([nums[i], nums[lo], nums[hi]]) + lo, hi = lo + 1, hi - 1 + while lo < hi and nums[lo] == nums[lo - 1]: + lo += 1 + return res +``` + +The cost is $O(n\log n)$ for the sort plus the scan, $O(n^2)$ in total for 3Sum. The most common mistake is sorting when the problem asks for original indices, as in 1 Two Sum; sort `(value, index)` pairs or use a hash map instead. + +## Reading the Problem Statement + +Certain phrases in a problem statement point straight at a sorting technique. The table maps the common ones to what they usually mean for your code. + +| If the problem says | What it means for your code | +| --- | --- | +| Merge overlapping intervals | Sort by start, extend the last merged interval with the larger end | +| Keep the most non-overlapping intervals, or remove the fewest | Sort by end and greedily keep the earliest finish | +| Minimum rooms, or the most events at the same time | Sort starts and ends separately, or sweep with a min-heap of end times | +| k-th largest or k-th smallest | Quickselect with a random pivot or a heap of size $k$, not a full sort | +| Top k, k closest, k most frequent | Heap of size $k$ in $O(n\log k)$, or buckets by frequency | +| Values in a small range, or only lowercase letters | Counting sort or a count array | +| Sort in place, only three distinct values | Dutch national flag with three pointers in one pass | +| Move all items with a property to the front | Two-pointer partition in $O(n)$ time and $O(1)$ space | +| Relative order of equal items must be kept | Stable sort, or the original index as the last key | +| Sort by frequency, then by value | Key tuple with the tie-breaker written out | +| Order defined by another string or array | Rank dictionary as the key, or counting in that order | +| Arrange numbers to form the largest number | Custom comparator on a + b against b + a | +| Anagrams, or the same letters in a different order | Sorted string or 26-count tuple as the key | +| Count pairs i < j where nums[i] is larger than c times nums[j] | Merge sort counting, or a Fenwick tree over compressed values | +| For each element, count smaller elements to its right | Merge sort over (value, index) pairs, or a Fenwick tree | +| Sort a linked list in $O(n\log n)$ | Merge sort, split with slow and fast pointers | +| Merge into the first array, which has spare room at the end | Fill from the back with three pointers | +| Linear time even though values are large | Bucket (pigeonhole) or radix sort | +| n up to $10^5$ or $10^6$ | $O(n\log n)$ sorting fits comfortably | +| Triplets summing to a target without duplicates | Sort, then two pointers, skipping equal neighbors | From a15e358ac41f2534227cad9f9ae6c1a1db22cefe Mon Sep 17 00:00:00 2001 From: Adam Djellouli Date: Thu, 12 Nov 2026 15:26:27 +0100 Subject: [PATCH 2/2] Fix math rendering in searching, DP, matrices and basic concepts notes Use \log and \sqrt commands, escape set braces inside display math, and remove padded dollar delimiters that some renderers skip. Co-Authored-By: Claude Opus 5.5 --- notes/basic_concepts.md | 2 +- notes/dynamic_programming.md | 4 +++- notes/matrices.md | 2 +- notes/searching.md | 28 ++++++++++++++-------------- 4 files changed, 19 insertions(+), 17 deletions(-) diff --git a/notes/basic_concepts.md b/notes/basic_concepts.md index 0451620..d4a6316 100644 --- a/notes/basic_concepts.md +++ b/notes/basic_concepts.md @@ -370,7 +370,7 @@ Big O notation is a practical tool for comparing growth bounds. The examples bel Use these growth rates to explain the work an algorithm performs. Nested loops may be quadratic, but their actual iteration bounds matter; repeated halving usually introduces a logarithm. Derive the bound from the work rather than from the visual shape of the code. - The time complexity $O(1)$, known as constant time complexity, means that regardless of the input size, the algorithm performs its task in a fixed amount of time. A common example of this is retrieving an item by its index from an array or, under suitable hashing and load assumptions, an expected constant-time hash-table lookup. -- When an algorithm has $O(log n)$ time complexity, it operates logarithmically, meaning the time taken increases logarithmically with input size. As the input size doubles, the time taken only increases marginally. Binary search and operations on balanced binary trees are typical examples. +- When an algorithm has $O(\log n)$ time complexity, it operates logarithmically, meaning the time taken increases logarithmically with input size. As the input size doubles, the time taken only increases marginally. Binary search and operations on balanced binary trees are typical examples. - An algorithm with $O(n)$ time complexity exhibits linear behavior, where the running time scales directly with the input size. This is seen in simple, single-pass processes like iterating over an array or a linked list. - In cases of $O(n log n)$ time complexity, also called log-linear complexity, the running time grows both linearly and logarithmically with the input size. Merge sort and heap sort have this worst-case bound; randomized quicksort has this expected bound but can take quadratic time in the worst case. - With $O(n^2)$ time complexity, the running time increases quadratically, often due to nested loops. Algorithms like Bubble Sort and Insertion Sort fall into this category. diff --git a/notes/dynamic_programming.md b/notes/dynamic_programming.md index ab657a4..9cac963 100644 --- a/notes/dynamic_programming.md +++ b/notes/dynamic_programming.md @@ -30,7 +30,9 @@ Mathematical Representation: Consider a problem where we want to find the optimal value $V(n)$ for a given parameter $n$. If there exists a function $f$ such that: -$ V(n) = \min_{1 \le k < n} f(V(k), V(n-k)), \qquad n \ge 2 $ +$$ +V(n) = \min_{1 \le k < n} f(V(k), V(n-k)), \qquad n \ge 2 +$$ then this is one possible form of an optimal-substructure recurrence, provided the state is sufficient and the combination accounts for every valid choice and its cost. It is not a universal DP formula: many problems need several state parameters or different transitions. Base cases must be defined separately. diff --git a/notes/matrices.md b/notes/matrices.md index 6fb5676..2358b7e 100644 --- a/notes/matrices.md +++ b/notes/matrices.md @@ -517,7 +517,7 @@ Rotate each ring by cycling 4 positions. How it works: -For layer $\ell$ with bounds $ [\ell..n-1-\ell]$, for each offset move: +For layer $\ell$ with bounds $[\ell..n-1-\ell]$, for each offset move: ``` top ← left, left ← bottom, bottom ← right, right ← top diff --git a/notes/searching.md b/notes/searching.md index 09e8daf..4a1cbf4 100644 --- a/notes/searching.md +++ b/notes/searching.md @@ -388,7 +388,7 @@ Active range: indices 4..4 FOUND at index 4 - Requires a sorted array (assume ascending here). -- Time: $O(log n)$; Space: $O(1)$ iterative. +- Time: $O(\log n)$; Space: $O(1)$ iterative. - Returns any one matching index by default; “first/last occurrence” is a small, common refinement. - Robust, cache-friendly, and a building block for many higher-level searches. - Beware of off-by-one errors when shrinking bounds. @@ -463,7 +463,7 @@ Since `4 < 9 < 12`, continue in indices `2..3`. The next midpoints are 2 and 3; - Also assumes a sorted array. - For discrete sorted arrays, it does not beat binary search asymptotically; it performs more comparisons per step. - Most valuable for searching the extremum of a unimodal function on a continuous domain; for arrays, prefer binary search. -- Complexity: $O(log n)$ steps but with larger constant factors than binary search. +- Complexity: $O(\log n)$ steps but with larger constant factors than binary search. ### Jump Search @@ -531,7 +531,7 @@ Block [16 ][25 ] The element $25$ is found at index 4. - Works on sorted arrays; pick jump ≈ √n for good balance. -- Time: $O(√n)$ comparisons on average; Space: $O(1)$. +- Time: $O(\sqrt{n})$ comparisons on average; Space: $O(1)$. - Useful when binary search isn’t desirable: binary search has poor locality (random jumps); jump search scans a block sequentially, which is more cache-friendly and has better branch prediction. - Degrades gracefully to “scan block then stop.” @@ -713,7 +713,7 @@ A = [ 10 ][ 20 ][ 30 ][ 40 ][ 50 ][ 60 ][ 70 ] At this point, an early-stop check already tells us `target (45) < A[low] (50)` → cannot exist in `A[4..6]` → not found. -- Best on uniformly distributed sorted data; expected time $O(log log n)$. +- Best on uniformly distributed sorted data; expected time $O(\log \log n)$. - Worst case can degrade to $O(n)$, especially on skewed or clustered values. - Space: $O(1)$. - Very fast when value-to-index mapping is close to linear (e.g., near-uniform numeric keys). @@ -1041,7 +1041,7 @@ Example 1 $$ \text{Table size: } m = 7, -\quad \text{Keys stored: } {10, 24, 31}, +\quad \text{Keys stored: } \{10, 24, 31\}, \quad \text{Target: } 24 $$ @@ -1053,7 +1053,7 @@ Example 2 $$ \text{Table size: } m = 7, -\quad \text{Keys stored: } {10, 24, 31}, +\quad \text{Keys stored: } \{10, 24, 31\}, \quad \text{Target: } 18 $$ @@ -1151,7 +1151,7 @@ Example 1 $$ m = 10, -\quad \text{Stored keys: } {12, 22, 32}, +\quad \text{Stored keys: } \{12, 22, 32\}, \quad \text{Target: } 22 $$ @@ -1163,7 +1163,7 @@ Example 2 $$ m = 10, -\quad \text{Stored keys: } {12, 22, 32}, +\quad \text{Stored keys: } \{12, 22, 32\}, \quad \text{Target: } 42 $$ @@ -1234,7 +1234,7 @@ Example 1 $$ m = 11 (\text{prime}), -\quad \text{Stored keys: } {22, 33, 44}, +\quad \text{Stored keys: } \{22, 33, 44\}, \quad \text{Target: } 33 $$ @@ -1250,7 +1250,7 @@ Example 2 $$ m = 11 (\text{prime}), -\quad \text{Stored keys: } {22, 33, 44}, +\quad \text{Stored keys: } \{22, 33, 44\}, \quad \text{Target: } 55 $$ @@ -1348,7 +1348,7 @@ Example 1 $$ m = 11, -\quad \text{Stored keys: } {22, 33, 44}, +\quad \text{Stored keys: } \{22, 33, 44\}, \quad \text{Target: } 33 $$ @@ -1377,7 +1377,7 @@ Example 2 $$ m = 11, -\quad \text{Stored keys: } {22, 33, 44}, +\quad \text{Stored keys: } \{22, 33, 44\}, \quad \text{Target: } 55 $$ @@ -1487,7 +1487,7 @@ $$ Keys stored: $$ -{12, 22, 7, 3, 14} +\{12, 22, 7, 3, 14\} $$ Bucket contents after hashing: @@ -1830,7 +1830,7 @@ $$ Inserted set: $$ -{"cat", "dog"} +\{\text{"cat"}, \text{"dog"}\} $$ Example 1