Bitwise Operators on Integers

Concepts covered: Ternary Expression

An int is stored as bits too, one per power of 2. bin(13) returns '0b1101': 8 + 4 + 1. Bitwise operators work on those columns one position at a time, with no carrying between them, which lets an integer hold many independent yes or no answers.

Packing flags into one integer

Give each yes or no fact its own bit, written 1 << n. A single integer then holds all of them. Discord's permission system works this way: its developer documentation assigns VIEW_CHANNEL to 1 << 10 and SEND_MESSAGES to 1 << 11, with ATTACH_FILES going to 1 << 15, so a role's permissions travel as a single number. | grants a flag and & ~ revokes it. ^ toggles the flag, and & tests it.

VIEW_CHANNEL = 1 << 10
SEND_MESSAGES = 1 << 11
ATTACH_FILES = 1 << 15

perms = 0
perms |= VIEW_CHANNEL
perms |= SEND_MESSAGES | ATTACH_FILES
print(perms, bin(perms))
can_attach = (perms & ATTACH_FILES) != 0
perms &= ~ATTACH_FILES
can_attach_now = (perms & ATTACH_FILES) != 0
perms ^= SEND_MESSAGES
print(perms, bin(perms))
print(can_attach, can_attach_now)

35840 0b1000110000000000
1024 0b10000000000
True False

Watch the test lines: perms & ATTACH_FILES keeps only that bit, so the result is either 0 or the flag's own value, 32768. Comparing with != 0 turns it into a bool, and by the truthiness rules bool(perms & ATTACH_FILES) gives the same answer.

What does this print?

perms = 6
print(perms & 4 == 4, 1 << 2 + 1)

• 0 5
• True 5
• 0 8
• True 8

^ also diffs 2 snapshots. An audit job that stores a role's permissions each day can compute before ^ after: every bit that changed comes out as 1, and every unchanged bit as 0. Masking the diff with & after keeps the grants, and masking with & before keeps the revokes.

before = (1 << 10) | (1 << 11)
after = (1 << 10) | (1 << 15)
changed = before ^ after
print(bin(changed))
print(changed & after == 1 << 15)
print((changed & before) == 1 << 11)

Output:

0b1000100000000000
True
True

Bit 15 comes out as the grant of ATTACH_FILES and bit 11 as the revoke of SEND_MESSAGES. The middle line has no parentheses and still prints True, because & runs before ==; the last line keeps them only so a reader sees the grouping at a glance.

Reading fields out of a packed ID

Shifts and masks pull apart a number built from several fields. Discord documents its snowflake IDs as 64 bits in 4 fields: the top 42 bits count milliseconds since the Discord epoch, 1420070400000 (the first second of 2015), bits 17 to 21 hold an internal worker ID, bits 12 to 16 an internal process ID, and the low 12 bits an increment. That puts 22 bits below the timestamp, so id >> 22 drops them and leaves the timestamp. id & 0xFFF keeps the lowest 12, because 0xFFF is 12 ones in binary.

from datetime import datetime, timezone

snowflake = 175928847299117063
DISCORD_EPOCH_MS = 1420070400000
created_ms = (snowflake >> 22) + DISCORD_EPOCH_MS
print(created_ms)
print(datetime.fromtimestamp(created_ms / 1000, tz=timezone.utc))
print(snowflake & 0xFFF)
as_float = int(float(snowflake))
print(as_float == snowflake, as_float & 0xFFF)

Output:

1462015105796
2016-04-30 11:18:25.796000+00:00
7
False 0

The last line joins this section to the first: the ID passes through float, its low bits round away, and the increment reads 0 where the real value is 7. The timestamp survives this time only because it sits in the high bits, which the float keeps.

About this lesson

This section is part of the Python Expressions: Advanced lesson on DataDriven, the home of data engineering. Lessons cover SQL, Python, Spark, data modeling and pipeline architecture, and each section pairs its explanation with worked examples and practice you run as you go.