Loops: Advanced

Apache Spark's Python API uses lazy iteration and generator expressions to process datasets that are larger than available RAM, letting data scientists at Netflix run machine learning jobs on petabytes of viewing history on standard hardware without loading the entire dataset at once. Generators produce one value at a time on demand, so a pipeline processing a billion rows uses only as much memory as a single row. The advanced iteration patterns in this lesson, including itertools and generator expressions, are the foundation of that memory-efficient approach.

Multiple Pointers

Daily Life
Interviews

Solve pair problems in a single pass

The two-pointer technique uses two index variables to traverse a sequence simultaneously. Rather than nested loops that check every pair (O(n²) complexity), two pointers can solve many problems in a single pass (O(n) complexity). This technique is especially powerful with sorted sequences.

Pointers from Both Ends

The most common pattern uses one pointer at the start and one at the end, moving them toward each other. This efficiently solves problems like finding pairs, checking palindromes, and partitioning arrays:
1# Find if any two numbers sum to target
2def has_pair_with_sum(sorted_nums, target):
3 left = 0
4 right = len(sorted_nums) - 1
5
6 while left < right:
7 current_sum = sorted_nums[left] + sorted_nums[right]
8 if current_sum == target:
9 return (sorted_nums[left], sorted_nums[right])
10 elif current_sum < target:
11 # Need larger sum
12 left += 1
13 else:
14 # Need smaller sum
15 right -= 1
16
17 return None
18
19# Test with sorted list
20numbers = [1, 3, 5, 7, 9, 11, 15]
21print(has_pair_with_sum(numbers, 12))
22print(has_pair_with_sum(numbers, 20))
23print(has_pair_with_sum(numbers, 50))
>>>Output
(1, 11)
(5, 15)
None
Because the list is sorted, we know that if the sum is too small, we need a larger left value. If too large, we need a smaller right value. This lets us eliminate many possibilities with each comparison.

Palindrome Check

Checking if a string is a palindrome is a classic two-pointer problem. Compare characters from both ends, moving inward:
1def is_palindrome(text):
2 # Clean: lowercase, letters only
3 cleaned = "".join(c.lower() for c in text if c.isalnum())
4
5 left = 0
6 right = len(cleaned) - 1
7
8 while left < right:
9 if cleaned[left] != cleaned[right]:
10 return False
11 left += 1
12 right -= 1
13
14 return True
15
16# Test cases
17print(is_palindrome("racecar"))
18print(is_palindrome("A man, a plan, a canal: Panama"))
19print(is_palindrome("hello"))
20print(is_palindrome("Was it a car or a cat I saw"))
>>>Output
True
True
False
True

Fast and Slow Pointers

Another pattern uses two pointers moving at different speeds. The fast pointer explores ahead while the slow pointer tracks a different position. This solves problems like removing duplicates in-place:
1def remove_duplicates(sorted_list):
2 """Remove duplicates in-place."""
3 if not sorted_list:
4 return 0
5
6 slow = 0
7
8 for fast in range(1, len(sorted_list)):
9 if sorted_list[fast] != sorted_list[slow]:
10 slow += 1
11 sorted_list[slow] = sorted_list[fast]
12
13 # Elements 0 through slow are unique
14 return slow + 1
15
16# Test
17nums = [1, 1, 2, 2, 2, 3, 4, 4, 5]
18new_length = remove_duplicates(nums)
19print("Unique count:", new_length)
20print("Unique values:", nums[:new_length])
>>>Output
Unique count: 5
Unique values: [1, 2, 3, 4, 5]
TIP
The fast-slow pointer pattern is also called the "runner" technique. The fast pointer scans ahead while the slow pointer marks where to place the next valid element. This enables in-place modifications without extra memory.

Three Pointers

Some problems require three pointers. The classic example is partitioning an array into three sections, like the Dutch National Flag problem for sorting colors:
1def dutch_flag_sort(arr):
2 """Sort array of 0s, 1s, 2s in single pass"""
3 low = 0
4 mid = 0
5 high = len(arr) - 1
6
7 while mid <= high:
8 if arr[mid] == 0:
9 arr[low], arr[mid] = arr[mid], arr[low]
10 low += 1
11 mid += 1
12 elif arr[mid] == 1:
13 mid += 1
14 else:
15 arr[mid], arr[high] = arr[high], arr[mid]
16 high -= 1
17
18 return arr
19
20# Test
21colors = [2, 0, 1, 2, 1, 0, 0, 2, 1]
22print("Before:", colors.copy())
23print("After:", dutch_flag_sort(colors))
>>>Output
Before: [2, 0, 1, 2, 1, 0, 0, 2, 1]
After: [0, 0, 0, 1, 1, 1, 2, 2, 2]
The two-pointer technique applies to many classic problems. Each variant uses a different pointer movement strategy:
Pair sum in sorted arrays
Pair sum in sorted arrays
Left and right pointers converge toward each other to find target sums
Merge two sorted arrays
Merge two sorted arrays
One pointer per array, advancing whichever has the smaller current value
Remove duplicates in place
Remove duplicates in place
Fast pointer scans ahead while slow pointer marks the write position
Partition arrays
Partition arrays
Three-way partitioning sorts elements into regions in a single pass

Sliding Window Pattern

Daily Life
Interviews

Analyze contiguous subsequences in O(n)

The sliding window pattern maintains a "window" over a contiguous portion of a sequence. The window slides through the data, adding elements at one end and removing them from the other. This efficiently solves problems involving contiguous subarrays or substrings.

Fixed-Size Window

The simplest form uses a fixed window size. Rather than recalculating from scratch for each position, we update the window incrementally:
1def max_sum_subarray(nums, k):
2 """Max sum of k consecutive."""
3 if len(nums) < k:
4 return None
5
6 # Calculate sum of first window
7 window_sum = sum(nums[:k])
8 max_sum = window_sum
9
10 # Slide the window
11 for i in range(k, len(nums)):
12 # Add new element, remove old element
13 window_sum += nums[i] - nums[i - k]
14 max_sum = max(max_sum, window_sum)
15
16 return max_sum
17
18# Test
19values = [1, 4, 2, 10, 2, 3, 1, 0, 20]
20print("Max sum of 3:", max_sum_subarray(values, 3))
21print("Max sum of 4:", max_sum_subarray(values, 4))
>>>Output
Max sum of 3: 23
Max sum of 4: 24

Instead of recalculating sum() for each window position (O(n×k) time), we update incrementally by adding the new element and subtracting the element that left the window (O(n) time). For large k, this is dramatically faster.

The fixed-size sliding window follows a simple four-step recipe on every iteration:
01
Initialize window
Compute the aggregate for the first k elements as your starting state
02
Add new element
Include the element entering the window at the right edge
03
Remove old element
Exclude the element leaving the window at the left edge
04
Update best result
Compare the current window value against the running optimum

Moving Average

A practical application is calculating moving averages, common in time series analysis and financial data:
1def moving_average(data, window_size):
2 """Calculate moving average for each position"""
3 if len(data) < window_size:
4 return []
5
6 result = []
7 window_sum = sum(data[:window_size])
8 result.append(window_sum / window_size)
9
10 for i in range(window_size, len(data)):
11 window_sum += data[i] - data[i - window_size]
12 result.append(window_sum / window_size)
13
14 return result
15
16# Stock prices over days
17prices = [100, 102, 104, 103, 105, 107, 106, 108, 110]
18ma3 = moving_average(prices, 3)
19print("3-day moving averages:")
20for i, avg in enumerate(ma3):
21 print(" Day", i + 3, ":", round(avg, 2))
>>>Output
3-day moving averages:
Day 3 : 102.0
Day 4 : 103.0
Day 5 : 104.0
Day 6 : 105.0
Day 7 : 106.0
Day 8 : 107.0
Day 9 : 108.0

Variable-Size Window

More complex problems use a variable-size window that expands and contracts based on conditions. This is powerful for finding optimal subarrays:
1def longest_subarray_sum(nums, max_sum):
2 """Find length of longest subarray with sum <= max_sum"""
3 left = 0
4 current_sum = 0
5 max_length = 0
6
7 for right in range(len(nums)):
8 current_sum += nums[right]
9
10 # Shrink window while sum exceeds limit
11 while current_sum > max_sum and left <= right:
12 current_sum -= nums[left]
13 left += 1
14
15 max_length = max(max_length, right - left + 1)
16
17 return max_length
18
19values = [1, 2, 3, 4, 5]
20print("Max length with sum <= 6:", longest_subarray_sum(values, 6))
21print("Max length with sum <= 9:", longest_subarray_sum(values, 9))
22print("Max length with sum <= 15:", longest_subarray_sum(values, 15))
>>>Output
Max length with sum <= 6: 3
Max length with sum <= 9: 4
Max length with sum <= 15: 5

The right pointer expands the window, and the left pointer contracts it when the constraint is violated. This finds the optimal window in O(n) time.

Unique Character Substrings

A classic interview problem is finding the longest substring without repeating characters. The variable window tracks character counts:
1def longest_unique_substring(s):
2 """Find longest substring with all unique characters"""
3 char_index = {}
4 left = 0
5 max_length = 0
6 max_start = 0
7
8 for right, char in enumerate(s):
9 # If char was seen and is in current window
10 if char in char_index and char_index[char] >= left:
11 left = char_index[char] + 1
12
13 char_index[char] = right
14
15 if right - left + 1 > max_length:
16 max_length = right - left + 1
17 max_start = left
18
19 return s[max_start:max_start + max_length]
20
21print(longest_unique_substring("abcabcbb"))
22print(longest_unique_substring("bbbbb"))
23print(longest_unique_substring("pwwkew"))
>>>Output
abc
b
wke
Python Quiz

> Compute the sum of the first three elements as a sliding window start, then find the overall maximum. Pick the aggregation for the window, and the one that scans the entire list.

data = [1, 3, 5, 2, 8]
window = ___(data[:3])
print(window)
print(___(data))
max
len
sorted
sum
min
The sliding window pattern shines when the data stream is continuous and you cannot afford to reprocess earlier elements. Initializing the first window correctly is the foundation, and then each step only updates what changed.

Combining sum() for windows and max() for full scans are common patterns in data analysis. They map cleanly onto SQL aggregations like SUM() OVER and MAX(), making this mental model useful across tools.

Fixed-size windows suit problems with a known frame size, like a 7-day rolling average. Variable-size windows suit problems with a constraint, like "all elements under a budget", and are solved with the expand-shrink loop pattern.

Reverse Iteration

Daily Life
Interviews

Traverse sequences from end to start

Iterating backwards through sequences is often necessary for algorithms that build results from end to beginning, or when modifications affect indices of later elements. Python provides several ways to iterate in reverse.

The reversed() Function

The reversed() function returns an iterator that yields elements in reverse order. It works on any sequence without creating a copy:

1# Reverse iteration with reversed()
2items = ["first", "second", "third", "fourth"]
3
4for item in reversed(items):
5 print(item)
6
7print("---")
8
9# Works with strings
10for char in reversed("Python"):
11 print(char, end=" ")
12print()
13
14# Original is unchanged
15print("Original:", items)
>>>Output
fourth
third
second
first
---
n o h t y P
Original: ['first', 'second', 'third', 'fourth']

range() with Negative Step

For index-based reverse iteration, use range() with a negative step:

1# Count down from 5 to 1
2for i in range(5, 0, -1):
3 print(i)
4
5print("---")
6
7# Access list indices in reverse
8nums = [10, 20, 30, 40, 50]
9for i in range(len(nums) - 1, -1, -1):
10 print("Index", i, "=", nums[i])
>>>Output
5
4
3
2
1
---
Index 4 = 50
Index 3 = 40
Index 2 = 30
Index 1 = 20
Index 0 = 10

Note: range(len(nums) - 1, -1, -1) goes from the last index down to 0. The stop value is -1 (not included) because we want to include index 0.

Why Reverse Iteration

Reverse iteration is essential when modifying a list based on indices. Deleting from the beginning shifts all later indices, but deleting from the end keeps earlier indices valid:
1# Remove all even numbers by iterating backwards
2numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
3
4# Iterate backwards so deletions don't affect remaining indices
5for i in range(len(numbers) - 1, -1, -1):
6 if numbers[i] % 2 == 0:
7 del numbers[i]
8
9print("After removing evens:", numbers)
10
11# Another example: remove elements that match previous
12data = ["a", "a", "b", "b", "b", "c", "c"]
13for i in range(len(data) - 1, 0, -1):
14 if data[i] == data[i - 1]:
15 del data[i]
16
17print("After removing consecutive dupes:", data)
>>>Output
After removing evens: [1, 3, 5, 7, 9]
After removing consecutive dupes: ['a', 'b', 'c']
This code tries to remove even numbers while iterating forward, but a stale index causes it to skip elements. Fix the loop direction so that every even number is removed safely.
Debug Challenge

> This code tries to remove even numbers while iterating forward, but skips elements due to shifting indices. Fix the loop direction.

IndexError: list index out of range because forward deletion shifts indices

Building Results Backwards

Some algorithms naturally build results from end to beginning. Reverse iteration aligns the logic with the output order:
1def reverse_words(sentence):
2 """Reverse words in a sentence"""
3 words = sentence.split()
4 result = []
5
6 for i in range(len(words) - 1, -1, -1):
7 result.append(words[i])
8
9 return " ".join(result)
10
11print(reverse_words("Hello World"))
12print(reverse_words("The quick brown fox"))
13
14# Calculate running sum from end
15values = [1, 2, 3, 4, 5]
16suffix_sums = [0] * len(values)
17
18running = 0
19for i in range(len(values) - 1, -1, -1):
20 running += values[i]
21 suffix_sums[i] = running
22
23print("Values:", values)
24print("Suffix sums:", suffix_sums)
>>>Output
World Hello
fox brown quick The
Values: [1, 2, 3, 4, 5]
Suffix sums: [15, 14, 12, 9, 5]
•Use reversed()
  • Simple reverse iteration
  • No index needed
  • Read-only access
  • Cleaner syntax
•Use range() Negative
  • Need the index value
  • Modifying by index
  • Deleting elements
  • Complex index math

Modifying While Looping

Daily Life
Interviews

Safely change collections mid-loop

Modifying a collection while iterating over it is dangerous and often causes bugs. Elements get skipped or the iterator becomes invalid. However, with the right techniques, you can safely modify collections during iteration.
Before looking at specific techniques, here are the three safe strategies you can rely on whenever you need to change a collection mid-loop.
Three Safe Modification Strategies
  • Iterate over a copy (list[:] or list(dict.keys())) while modifying the original.
  • Loop backwards with range(len-1, -1, -1) so deletions do not shift unvisited indices.
  • Build a brand-new collection with a list comprehension and replace the old one.

The Problem

Removing elements while iterating forward causes items to be skipped because indices shift:
1# Problem: removing while iterating
2# This shows what goes wrong:
3
4nums = [2, 4, 6, 8]
5result = []
6for num in nums:
7 if num % 2 == 0:
8 result.append(num)
9
10# What happens if we tried removing:
11test = [2, 4, 6, 8]
12# Safe: iterate over a copy
13for num in list(test):
14 if num % 2 == 0:
15 test.remove(num)
16print("After unsafe pattern would give: [4, 8]")
17print("After safe pattern (copy):", test)
>>>Output
After unsafe pattern would give: [4, 8]
After safe pattern (copy): []
When you remove index 0, element at index 1 moves to index 0. But the iterator advances to index 1, skipping what is now at index 0.

Solution 1: Copy, Then Iter

Create a copy of the list and iterate over it while modifying the original:
1# Iterate over copy, modify original
2nums = [2, 4, 6, 8, 10]
3
4for num in nums[:]:
5 if num % 4 == 0:
6 nums.remove(num)
7
8print("After removing multiples of 4:", nums)
9
10# Alternative: explicit copy
11items = ["keep", "remove", "keep", "remove"]
12for item in list(items):
13 if item == "remove":
14 items.remove(item)
15
16print("After filtering:", items)
>>>Output
After removing multiples of 4: [2, 6, 10]
After filtering: ['keep', 'keep']

Solution 2: Reverse Iter

As shown earlier, iterating backwards keeps all earlier indices valid after deletion:
1# Reverse iteration for safe deletion
2nums = [1, 2, 3, 4, 5, 6, 7, 8]
3
4for i in range(len(nums) - 1, -1, -1):
5 if nums[i] % 2 == 0:
6 del nums[i]
7
8print("After removing evens:", nums)
>>>Output
After removing evens: [1, 3, 5, 7]

Solution 3: New Collection

The most Pythonic approach is often to build a new collection rather than modify the existing one:
1# Build new list (list comprehension)
2nums = [1, 2, 3, 4, 5, 6, 7, 8]
3odds = [n for n in nums if n % 2 != 0]
4print("Odds:", odds)
5
6# Or with filter
7nums = [1, 2, 3, 4, 5, 6, 7, 8]
8odds = list(filter(lambda n: n % 2 != 0, nums))
9print("Odds (filter):", odds)
10
11# Transform and filter together
12data = [" hello ", "", " world ", " ", "python"]
13cleaned = [s.strip() for s in data if s.strip()]
14print("Cleaned:", cleaned)
>>>Output
Odds: [1, 3, 5, 7]
Odds (filter): [1, 3, 5, 7]
Cleaned: ['hello', 'world', 'python']
TIP
List comprehensions are generally preferred over in-place modifications. They are clearer, create fewer bugs, and often perform better because Python optimizes list comprehension creation.

Modifying Dictionaries

Dictionary modification during iteration has the same issues. Use dict.copy() or build a new dict:
1# Removing keys while iterating
2scores = {"alice": 85, "bob": 45, "charlie": 92, "diana": 38}
3
4# Method 1: Iterate over copy of keys
5for name in list(scores.keys()):
6 if scores[name] < 50:
7 del scores[name]
8
9print("Passing scores:", scores)
10
11# Method 2: Dictionary comprehension (preferred)
12scores = {"alice": 85, "bob": 45, "charlie": 92, "diana": 38}
13passing = {k: v for k, v in scores.items() if v >= 50}
14print("Passing (comprehension):", passing)
>>>Output
Passing scores: {'alice': 85, 'charlie': 92}
Passing (comprehension): {'alice': 85, 'charlie': 92}
Choosing the right modification strategy prevents silent data corruption. Follow these guidelines to keep your iterations safe:
✓Do
  • Iterate over list[:] copy to modify original
  • Use reverse iteration for index-based deletes
  • Build new collections with comprehensions
  • Collect keys first, then modify the dict
✗Don't
  • Remove items while iterating forward
  • Add keys to a dict during iteration
  • Delete by index while looping forward
  • Assume iterator stays valid after changes
Python Quiz

> Safely remove negative numbers from a list by iterating over a copy. Pick the function that creates a snapshot of the list, and the method that deletes a specific value.

items = [1, -2, 3, -4, 5]
for x in ___(items):
    if x < 0:
        items.___(x)
print(items)
print(len(items))
list
tuple
remove
pop
set

Iterating over list(items) is a clear, readable pattern that signals to other developers that you intend to modify the original collection during the loop.

For bulk filtering, a list comprehension is often cleaner than the copy-and-remove pattern. Write [x for x in items if condition] instead of modifying items in a loop when readability matters most.

Dictionary modification during iteration raises RuntimeError in Python 3. Always iterate over list(d.keys()) or use a dictionary comprehension to build a new dict with the desired entries.

Using any() and all()

Daily Life
Interviews

Test conditions across entire sequences

The built-in functions any() and all() test conditions across entire sequences. They replace common loop patterns with concise, readable, and efficient expressions. Both short-circuit, meaning they stop as soon as the result is determined.

These two functions cover the vast majority of sequence-wide condition checks you will ever need:
any()all()not any()not all()
any()
At least one
True if one item is truthy
all()
Every item
True if all items are truthy
not any()
None at all
True if nothing is truthy
not all()
Some falsy
True if any item is falsy

The any() Function

any(iterable) returns True if at least one element is truthy. It stops at the first True value:

1# Basic any() usage
2print(any([False, False, True, False]))
3print(any([False, False, False]))
4print(any([]))
5
6# With conditions using generator expression
7nums = [2, 4, 6, 8, 10]
8has_odd = any(n % 2 != 0 for n in nums)
9print("Has odd number:", has_odd)
10
11nums = [2, 4, 5, 8, 10]
12has_odd = any(n % 2 != 0 for n in nums)
13print("Has odd number:", has_odd)
14
15# Check if any string is empty
16strings = ["hello", "world", "", "python"]
17has_empty = any(s == "" for s in strings)
18print("Has empty string:", has_empty)
>>>Output
True
False
False
Has odd number: False
Has odd number: True
Has empty string: True

The all() Function

all(iterable) returns True only if all elements are truthy. It stops at the first False value:

1# Basic all() usage
2print(all([True, True, True]))
3print(all([True, False, True]))
4print(all([]))
5
6# All positive numbers?
7nums = [1, 2, 3, 4, 5]
8all_positive = all(n > 0 for n in nums)
9print("All positive:", all_positive)
10
11nums = [1, 2, -3, 4, 5]
12all_positive = all(n > 0 for n in nums)
13print("All positive:", all_positive)
14
15# Validate data
16users = [
17 {"name": "Alice", "age": 30},
18 {"name": "Bob", "age": 25},
19 {"name": "Charlie", "age": 35},
20]
21all_adults = all(u["age"] >= 18 for u in users)
22all_have_names = all(u.get("name") for u in users)
23print("All adults:", all_adults)
24print("All have names:", all_have_names)
>>>Output
True
False
True
All positive: True
All positive: False
All adults: True
All have names: True

Replacing Loop Patterns

Many common loop patterns can be replaced with any() or all():

1nums = [10, 20, 35, 40, 50]
2
3# Instead of loop with break
4# found = False
5# for n in nums:
6# if n > 30:
7# found = True
8# break
9
10# Use any()
11found = any(n > 30 for n in nums)
12print("Found > 30:", found)
13
14# Instead of loop checking all
15# valid = True
16# for n in nums:
17# if n < 0:
18# valid = False
19# break
20
21# Use all()
22valid = all(n >= 0 for n in nums)
23print("All non-negative:", valid)
>>>Output
Found > 30: True
All non-negative: True
Try switching between any() and all() to see how each evaluates the same list of values differently:
Fill in the Blank

> The functions any() and all() check conditions across a list. Pick one to see how each evaluates the same data differently.

nums = [2, 4, 7, 8, 10]
result = (n % 2 == 0 for n in nums)
print(result)

Validation Examples

These functions shine in data validation scenarios:
1# Validate a form submission
2def validate_form(data):
3 required_fields = ["email", "password", "username"]
4
5 # All required fields present?
6 if not all(field in data for field in required_fields):
7 return "Missing required fields"
8
9 # All fields non-empty?
10 if not all(data.get(f) for f in required_fields):
11 return "Fields cannot be empty"
12
13 # Password requirements
14 password = data["password"]
15 if not any(c.isupper() for c in password):
16 return "Password needs uppercase"
17 if not any(c.isdigit() for c in password):
18 return "Password needs digit"
19
20 return "Valid"
21
22# Test cases
23print(validate_form({"email": "a@b.com"}))
24print(validate_form({"email": "a@b.com", "password": "", "username": "x"}))
25print(validate_form({"email": "a@b.com", "password": "weak", "username": "x"}))
26print(validate_form({"email": "a@b.com", "password": "Strong1", "username": "x"}))
>>>Output
Missing required fields
Fields cannot be empty
Password needs uppercase
Valid

Combining any() and all()

Complex conditions can combine both functions:
1# Matrix operations
2matrix = [
3 [1, 2, 3],
4 [4, 5, 6],
5 [7, 8, 9],
6]
7
8# All rows have at least one even number?
9result = all(any(n % 2 == 0 for n in row) for row in matrix)
10print("All rows have even:", result)
11
12# Any row has all numbers > 5?
13result = any(all(n > 5 for n in row) for row in matrix)
14print("Any row all > 5:", result)
15
16# File type validation
17files = ["doc.pdf", "data.csv", "image.png", "report.pdf"]
18valid_extensions = [".pdf", ".csv", ".xlsx"]
19
20all_valid = all(any(f.endswith(ext) for ext in valid_extensions) for f in files)
21print("All files valid:", all_valid)
>>>Output
All rows have even: True
Any row all > 5: True
All files valid: False
Here is a side-by-side comparison of how any() and all() evaluate elements.
•any()
  • True if ANY element is truthy
  • Stops at first True (short-circuit)
  • Empty sequence = False
  • Like OR across all elements
•all()
  • True if ALL elements are truthy
  • Stops at first False (short-circuit)
  • Empty sequence = True
  • Like AND across all elements
Advanced iteration techniques let you handle complex data processing with elegance and efficiency. Put your skills to the test with hands-on challenges in the Python Builder.
The Real-Time Log AnalyzerStep 1
>

You are the data engineer at a cloud hosting company that monitors 500 servers. Each server emits log events every second: timestamps, CPU usage, memory usage, and error codes. The operations team needs real-time alerting when a server shows sustained high CPU (above 90% for 30 consecutive seconds), memory leaks (steadily increasing memory over 5-minute windows), or error bursts (more than 10 errors in any 60-second window). The current monitoring script processes logs one at a time and is falling behind during peak hours.

server_logs
timestampserver_idcpu_pctmem_mberror_code
14:00:01srv-042922048null
14:00:02srv-042952052E-504
14:00:03srv-042912058null
May 2026
CPU Alert Logic

The first requirement is detecting when a server stays above 90% CPU for 30 consecutive seconds. With 500 servers emitting one log per second, that is 500 events per second. How should you track the consecutive high-CPU count per server?

any() and all() are the clearest way to express intent when checking conditions across a collection. They read like English and short-circuit early, making them both expressive and efficient.

Nesting any() inside all() or vice versa lets you express matrix-level conditions concisely: "every row has at least one even number" becomes all(any(n % 2 == 0 for n in row) for row in matrix).

In production monitoring systems, combining sliding windows with any() and all() checks gives you real-time alerting that scales to thousands of events per second without complex threading or external tools.

❯❯❯PUTTING IT ALL TOGETHER

> You are a senior data engineer at Snowflake processing millions of raw query event records to detect anomalous usage patterns, where loading everything into memory at once is not an option.

Multiple pointers scan the sorted event log from both ends simultaneously to find mismatched start and end timestamps without nested passes.
Sliding window maintains a fixed-size buffer of consecutive events to detect rate spikes across any contiguous time interval.
Reverse iteration with negative range() steps processes correction records in reverse chronological order, overwriting earlier entries without a list reversal copy.
any() and all() validate quality conditions across entire record batches in a single readable expression rather than explicit loops.
KEY TAKEAWAYS
Two-pointer technique uses left/right indices to traverse sorted sequences in O(n) time instead of O(n squared)
Sliding window pattern efficiently analyzes contiguous subsequences by updating incrementally
reversed() or range(len-1, -1, -1) for backward iteration
Iterate backwards when deleting elements to keep earlier indices valid
Never modify a collection while iterating forward; use a copy, reverse iteration, or build a new collection
any() returns True if at least one element is truthy; stops at first True
all() returns True only if all elements are truthy; stops at first False
Prefer list comprehensions over in-place modifications for cleaner, safer code

Algorithmic iteration techniques

Category
Python
Difficulty
advanced
Duration
34 minutes
Challenges
0 hands-on challenges

Topics covered: Multiple Pointers, Sliding Window Pattern, Reverse Iteration, Modifying While Looping, Using any() and all()

Lesson Sections

  1. Multiple Pointers (concepts: pyTwoPointer)

    Pointers from Both Ends The most common pattern uses one pointer at the start and one at the end, moving them toward each other. This efficiently solves problems like finding pairs, checking palindromes, and partitioning arrays: Because the list is sorted, we know that if the sum is too small, we need a larger left value. If too large, we need a smaller right value. This lets us eliminate many possibilities with each comparison. Palindrome Check Checking if a string is a palindrome is a classic

  2. Sliding Window Pattern (concepts: pySlidingWindow)

    The sliding window pattern maintains a "window" over a contiguous portion of a sequence. The window slides through the data, adding elements at one end and removing them from the other. This efficiently solves problems involving contiguous subarrays or substrings. Fixed-Size Window The simplest form uses a fixed window size. Rather than recalculating from scratch for each position, we update the window incrementally: The fixed-size sliding window follows a simple four-step recipe on every iterat

  3. Reverse Iteration

    Iterating backwards through sequences is often necessary for algorithms that build results from end to beginning, or when modifications affect indices of later elements. Python provides several ways to iterate in reverse. The reversed() Function range() with Negative Step Why Reverse Iteration Reverse iteration is essential when modifying a list based on indices. Deleting from the beginning shifts all later indices, but deleting from the end keeps earlier indices valid: This code tries to remove

  4. Modifying While Looping

    Modifying a collection while iterating over it is dangerous and often causes bugs. Elements get skipped or the iterator becomes invalid. However, with the right techniques, you can safely modify collections during iteration. Before looking at specific techniques, here are the three safe strategies you can rely on whenever you need to change a collection mid-loop. The Problem Removing elements while iterating forward causes items to be skipped because indices shift: When you remove index 0, eleme

  5. Using any() and all()

    These two functions cover the vast majority of sequence-wide condition checks you will ever need: The any() Function The all() Function Replacing Loop Patterns Try switching between any() and all() to see how each evaluates the same list of values differently: Validation Examples These functions shine in data validation scenarios: Combining any() and all() Complex conditions can combine both functions: Here is a side-by-side comparison of how any() and all() evaluate elements. Advanced iteration