Data Types: Intermediate
A BOOLEAN column has 3 possible values. TRUE and FALSE are recorded answers. NULL means the system never recorded an answer. Take an email_verified flag on a users table. Rows written after the verification feature shipped carry TRUE or FALSE. Rows written before it carry NULL, because nobody asked the question back then. Reading that NULL as FALSE would flag long-time customers as unverified, and reading it as TRUE would skip a security check. The database does neither. It keeps the unknown unknown, and every operator that touches it follows rules that fit in a truth table.
The truth table with a third value
| a | b | a AND b | a OR b |
|---|---|---|---|
| TRUE | TRUE | TRUE | TRUE |
| TRUE | FALSE | FALSE | TRUE |
| TRUE | NULL | NULL | TRUE |
| FALSE | TRUE | FALSE | TRUE |
| FALSE | FALSE | FALSE | FALSE |
| FALSE | NULL | FALSE | NULL |
| NULL | TRUE | NULL | TRUE |
| NULL | FALSE | FALSE | NULL |
| NULL | NULL | NULL | NULL |
| a | NOT a |
|---|---|
| TRUE | FALSE |
| FALSE | TRUE |
| NULL | NULL |
- ▸FALSE AND anything is FALSE, and TRUE OR anything is TRUE: 1 known side can settle the answer on its own.
- ▸TRUE AND NULL and FALSE OR NULL stay NULL: the unknown side is the one that decides, and its value is not known.
- ▸IS NULL and IS NOT NULL always return TRUE or FALSE, so they are the way to ask about the unknown itself.
Watching the rules run on real rows
The users table below has 6 rows. email_verified and phone_verified are its 2 flags, and across the rows they cover every combination of the 3 BOOLEAN values. The query computes 3 new BOOLEAN columns from them. Before you run it, predict the row for user 3, where email is FALSE and phone is NULL.
A row has a = NULL and b = FALSE. What does NOT (a AND b) return?
Comparisons produce BOOLEAN values
A comparison is an expression whose type is BOOLEAN. country = 'US' yields TRUE for a US row and FALSE for a Canadian one. When country is NULL the comparison yields NULL, because an unknown country might or might not be the US. That is why country = NULL never finds anything: comparing against an unknown returns unknown for every row, including the rows where the column really is NULL. The operator built for that question is IS NULL. It inspects the slot rather than the value, so it returns TRUE or FALSE and never NULL, and IS NOT NULL gives the opposite.
Account 3 returns NULL for both is_us and is_not_us, so a row can fail the US test and fail the non-US test at the same time. Only country_missing returns TRUE for it. Any time a report splits rows into 2 buckets with a comparison, a third bucket of NULL rows is sitting between them, and it stays invisible until a query tests for it by name.
NOT flag is also NULL. To get a TRUE for every row where a flag is anything other than TRUE, combine 2 tests: flag IS NULL OR NOT flag, which returns TRUE for both the FALSE rows and the NULL rows.Precision and scale
| Type | Largest value | Fits |
|---|---|---|
| DECIMAL(5,2) | 999.99 | a unit price in dollars |
| DECIMAL(10,2) | 99999999.99 | an invoice total |
| DECIMAL(15,6) | 999999999.999999 | an exchange rate |
| DECIMAL(20,0) | 20 nines | an id too wide for 64 bits |
The sum that comes out wrong
The pennies table stores 3 pairs of amounts as text, the way a CSV loader would leave them. The query adds each pair twice, once after casting to DOUBLE and once after casting to DECIMAL(10,2). Predict the first row before running.
What arithmetic does to the type
order_lines stores unit_price and tax_rate as text and quantity as a whole number. The CTE casts each column once, and the outer query builds the subtotal, the raw tax with its full 6 places, and the tax rounded to cents.
A column price is DECIMAL(6,3) and qty is DECIMAL(4,1). What is the type of price * qty?
- Prices and tax rates
- Exchange rates and fees
- Anything summed or reconciled
- Values compared for equality
- Sensor readings and coordinates
- Statistical measures
- Ratios shown to 2 or 3 places
- Very large or very small magnitudes
A TIMESTAMP is a reading off a wall clock with the clock's location left out. 2024-03-15 09:00:00 says a clock showed 9 in the morning, and nothing about which clock. A warehouse in Tokyo and a warehouse in New Jersey both record 2024-03-15 09:00:00 for a morning pickup, and those 2 pickups are 13 hours apart. TIMESTAMP WITH TIME ZONE, which this section's title and many engineers shorten to TIMESTAMPTZ, keeps the clock's location with the reading: 2024-03-15 09:00:00 Asia/Tokyo. With the zone attached, the value names one instant on the global timeline, and 2 such values can be ordered or subtracted no matter where they were recorded.
| TIMESTAMP | TIMESTAMP WITH TIME ZONE | |
|---|---|---|
| Literal | TIMESTAMP '2024-03-15 09:00:00' | TIMESTAMP '2024-03-15 09:00:00 Asia/Tokyo' |
| What it records | A clock reading | A clock reading plus the clock's zone |
| Names one instant | No | Yes |
| Good for | Store hours, birthdays, a 9am alarm | Logins, payments, pickups, deploys |
Same digits, different instants
The shipments table records every pickup twice. pickup_local is what the warehouse clock showed. pickup_stamped is the same reading with the warehouse's zone appended. Both are stored as text, so each query casts the column it needs. The 2 tabs below differ in which column they cast and sort by. Pick the order you expect from each before you switch.
| Zone | Offset from UTC on 2024-03-15 | Why |
|---|---|---|
| America/Los_Angeles | -07:00 | daylight time since March 10 |
| America/Chicago | -05:00 | daylight time since March 10 |
| America/New_York | -04:00 | daylight time since March 10 |
| Europe/London | +00:00 | summer time starts March 31 |
| Europe/Berlin | +01:00 | summer time starts March 31 |
| Asia/Kolkata | +05:30 | no daylight time |
| Asia/Tokyo | +09:00 | no daylight time |
| Australia/Sydney | +11:00 | daylight time until April 7 |
Every example in this lesson and the next section uses a date in March 2024, so the offsets above are the ones to use when predicting a result.
A table has 2 rows, 2024-03-15 20:00:00 Europe/London and 2024-03-15 15:00:00 America/New_York, both cast to TIMESTAMP WITH TIME ZONE. How do the 2 instants compare?
What each type stores
- ▸A store opens at 09:00 local time, whichever zone the store is in.
- ▸A birthday or anniversary that repeats on a calendar day.
- ▸A schedule exported as text where the zone is carried in another column.
Shipment 1 is 2024-03-15 10:00:00 Asia/Tokyo, shipment 2 is 2024-03-15 02:00:00 Europe/Berlin and shipment 3 is 2024-03-14 20:00:00 America/New_York. Which row does this query return first?
AT TIME ZONE changes the reading and keeps the instant
AT TIME ZONE 'UTC' takes a TIMESTAMP WITH TIME ZONE and returns the same instant expressed in the zone named, so the digits change and the point on the timeline stays where it was. The logins table below holds 4 readings from 4 zones as text. The animation casts each one and converts it to UTC; watch how the row with the latest clock reading on March 15, Sydney at 23:40, becomes the earliest instant of the 4.
SELECTuser_id,login_at,CAST(login_at AS TIMESTAMP WITH TIME ZONE) AT TIME ZONE 'UTC' AS login_utcFROM loginsORDER BY login_utc
Converting back for the reader
Stored in UTC, an event converts to any zone on demand, and the same AT TIME ZONE does the job in the other direction. The payments table keeps paid_at in UTC. The query shows each payment as the Chicago finance team and the Tokyo team would see it. Payment 2 is the one to watch: it moves to a different calendar day in Chicago.
A payment is stored as 2024-03-16 02:30:00 UTC. What does AT TIME ZONE America/Los_Angeles return for it, given Los Angeles is 7 hours behind UTC on that date?
When the zone was never recorded
A plain TIMESTAMP cannot be converted on its own, because AT TIME ZONE needs to know which zone the reading came from before it can move it. The repair takes 2 steps: attach the zone the clock was in, then convert. with_timezone does the first step. You give it a TIMESTAMP and a zone name, and it returns a TIMESTAMP WITH TIME ZONE with the digits unchanged. Which zone to attach is a fact about the source system, found in its configuration or its documentation, and never in the data itself.
- Store event times as TIMESTAMP WITH TIME ZONE in UTC
- Convert to a local zone in the final SELECT, for display
- Name zones by region, such as Europe/Berlin
- Write down the source zone of every legacy feed
- Add or subtract fixed hours to shift a zone
- Store a local reading without its zone
- Use an offset such as +01:00 as if it were a zone
- Assume a column holds UTC from its name alone
Mixing 2 types in 1 expression forces the engine to pick a common type, and for numbers it does so without being asked. An INTEGER added to a DOUBLE becomes a DOUBLE first. An INTEGER multiplied by a DECIMAL is treated as a DECIMAL with scale 0, so the CAST(quantity AS DECIMAL(5,0)) in the tax example could have been left out and the engine would have promoted the INTEGER the same way. These silent promotions are implicit casts. They run toward the type that can hold the wider range, and range is a different property from exactness: a BIGINT, the 64-bit whole-number type, loses its last digits above 2^53 when promoted to DOUBLE, and a DECIMAL mixed with a DOUBLE becomes a DOUBLE, so DECIMAL 0.1 plus DOUBLE 0.2 gives 0.30000000000000004. A single DOUBLE column in a money calculation turns the whole result into an approximation. The promotion that bites people most often is the one that never happens: 2 INTEGER values divided give an INTEGER, and the fractional part is discarded.
Campaign spring_sale has 37 clicks and 120 views. The integer ratio is 0, because 37 divided by 120 is 0 with 37 left over, and INTEGER keeps the whole part. Once either side is DOUBLE the whole expression is DOUBLE and the result is 0.308333 with a tail of digits. The DECIMAL version rounds to the 4 places of its scale, 0.3083, so the type you cast to decides the digits a reader sees. Campaign retarget with 600 clicks over 400 views comes out 1, 1.5 and 1.5000: the integer version keeps the 1 and drops the half.
Where implicit casts stop
Text never promotes. '5' + 1 fails with a type error, because the engine will not guess whether the text holds a number, and the same error appears when a VARCHAR column meets an INTEGER in a comparison. That refusal protects you from a column of ids where most rows look numeric and 1 row holds N/A. You already know CAST. The other explicit tool, TRY_CAST, has the same syntax and 1 different behavior: when a value cannot become the target type, TRY_CAST returns NULL for that row and the query keeps running.
TRY_CAST on a messy feed
raw_signups is a feed where every field arrived as text and some rows were typed by hand. Each column below is converted to the type it should have had. Rows 2 and 3 carry the damage; predict which of their cells come back NULL and why. The last column applies IS NULL to the converted age to flag a value that failed.
Row 2 loses every conversion. thirty holds no digits, February 30 is a date that does not exist, maybe is not on the short list BOOLEAN accepts (the words true and false or their first letters in any case, plus 1 and 0), and 1200.00 needs 6 digits where DECIMAL(5,2) allows 5. Row 3 is subtler. 27.5 fails as an INTEGER because the conversion demands a whole number and refuses to round, and 03/15/2024 fails because the TIMESTAMP parser expects the date written year first with dashes between the parts. Its 1 does convert, to TRUE. Row 4 shows that an empty string becomes NULL as an INTEGER, while the same empty string as a VARCHAR is a real value that holds no characters.
What does TRY_CAST(12.0 AS INTEGER) return?
The cost of not crashing
- Fails the whole query on 1 bad value
- Right when the data is already clean
- Surfaces a broken upstream feed at once
- Syntax: CAST(x AS type)
- Returns NULL for the bad value and keeps going
- Right for raw feeds and hand-typed input
- Needs a rejected flag so failures stay visible
- Syntax: TRY_CAST(x AS type)
> Shopify's finance team closes the books on a weekend flash sale that merchants ran across North America, Europe and Asia. Order amounts arrive from the payments feed as text, order times are stamped by each merchant's local clock, and a refund flag is NULL for every order placed before the refund feature launched.
Exact money, pinned moments, and conversions that survive bad input.
Where pennies vanish and NULLs defy
- Category
- SQL
- Difficulty
- intermediate
- Duration
- 19 minutes
- Challenges
- 15 hands-on challenges
Topics covered: BOOLEAN and NULL logic, DECIMAL precision and scale, TIMESTAMP vs TIMESTAMPTZ, Time zones and UTC handling, TRY_CAST and implicit casts
Lesson Sections
- BOOLEAN and NULL logic (concepts: Three-Valued Logic)
The truth table with a third value Read AND as 'both sides must be TRUE'. When one side is FALSE the answer is settled no matter what the other side holds, so FALSE AND NULL is FALSE: whatever the unknown turns out to be, both sides cannot be TRUE. When one side is TRUE, the result depends entirely on the other side, so TRUE AND NULL stays NULL. OR works as the mirror image. TRUE OR NULL is TRUE because one TRUE is enough, and FALSE OR NULL is NULL because the unknown side decides. NOT swaps TRU
- DECIMAL precision and scale (concepts: Numeric Precision)
A DOUBLE stores a number as a binary fraction in 64 bits. Binary fractions represent a half or an eighth exactly, but nothing whose denominator has a factor of 5, so 0.1 is stored as the nearest value the format can reach, which begins 0.1000000000000000055511151231257827. The error sits far past the digits anyone looks at, which is how it stays hidden until a ledger that adds millions of such values comes out a few cents off. DECIMAL stores digits, the way a bank statement does, so 0.1 means 0.
- TIMESTAMP vs TIMESTAMPTZ (concepts: Timezones)
Same digits, different instants With zones, shipment 4 drops from first to second: its 07:30 in Berlin is 06:30 UTC, and Tokyo's 09:00 is 00:00 UTC, which puts the Tokyo pickup 6 and a half hours earlier. Without zones the engine has no basis for that move, because the values it compares are the digits 07:30 and 09:00 and nothing more. Neither cast is wrong. The wall-clock version answers 'which pickups happened before the morning break', and the zoned version answers 'which pickup happened firs
- Time zones and UTC handling (concepts: Timezones)
A system that runs in several regions collects clock readings from several zones, and a report that groups them by day needs every reading on 1 scale. The scale is UTC: the reference time with no daylight saving and no political changes, which every zone is defined as an offset from. The working rule is to convert to UTC on the way into storage, keep every stored event in UTC, and convert back to a local zone only at the moment a human reads the number. AT TIME ZONE changes the reading and keeps
- TRY_CAST and implicit casts (concepts: Safe Casting)
Where implicit casts stop TRY_CAST on a messy feed The cost of not crashing TRY_CAST trades a loud failure for a quiet NULL, and a quiet NULL can hide a broken feed for weeks. After the conversion, the NULLs from bad input are identical to the NULLs from values that were missing to begin with: row 2's age and a row whose age was never filled in look the same. Keep the original text column next to the converted one, so a NULL can be traced to the value that produced it. Then compute a rejected fl