New Caesar
We found a brand new type of encryption, can you break the secret code? (Wrap with picoCTF{}) fegdeogdgecoeocgcgchcfcffccfca
Download the Python code.
Let’s walk through the encryption step by step.
1. The custom “base16” encoding
1
ALPHABET = "abcdefghijklmnop"
Instead of using hexadecimal digits 0-9a-f, the code uses:
1
2
3
4
5
0 -> a
1 -> b
2 -> c
...
15 -> p
The function:
1
2
3
4
5
6
7
def b16_encode(plain):
enc = ""
for c in plain:
binary = "{0:08b}".format(ord(c))
enc += ALPHABET[int(binary[:4], 2)]
enc += ALPHABET[int(binary[4:], 2)]
return enc
takes each character and:
Converts it to an 8-bit binary value.
Splits it into two 4-bit halves (nibbles).
Converts each nibble to a letter
a-p.
Example: 'A'
ASCII:
1
A = 65
Binary:
1
01000001
Split:
1
0100 0001
Convert each nibble:
1
2
0100 = 4 = e
0001 = 1 = b
So:
1
'A' -> "eb"
2. The shift function
1
2
3
4
def shift(c, k):
t1 = ord(c) - ord('a')
t2 = ord(k) - ord('a')
return ALPHABET[(t1 + t2) % 16]
This is a Vigenère/Caesar shift on the alphabet a-p.
Example with key 'd':
1
2
3
4
5
6
c = e (4)
k = d (3)
(4 + 3) mod 16 = 7
result = h
So:
1
shift('e','d') = 'h'
3. Encryption process
1
2
3
4
b16 = b16_encode(flag)
for i, c in enumerate(b16):
enc += shift(c, key[i % len(key)])
Because:
1
assert len(key) == 1
the same key character is used for every position.
So encryption is:
1
2
3
4
5
6
7
flag
↓
custom base16 (a-p)
↓
Caesar shift by key value
↓
ciphertext
Example
Suppose:
1
2
flag = "A"
key = "c"
Step 1: b16 encode
1
2
3
A -> 01000001
-> 0100 0001
-> e b
Result:
1
"eb"
Step 2: shift by key c
1
c = 2
Shift each character:
1
2
e(4) + 2 = g
b(1) + 2 = d
Ciphertext:
1
"gd"
Why decryption is easy
Since:
1
len(key) == 1
there are only 16 possible keys:
1
a b c d e f g h i j k l m n o p
Trying all 16 keys is trivial. For each key:
- Reverse the shift:
1
(original - key) mod 16
- Convert pairs of letters back to bytes.
That’s why this scheme is very weak cryptographically—it is effectively a Caesar cipher over a 16-character alphabet after a reversible encoding step.
The decryptor simply reverses the two operations performed during encryption, in reverse order.
Encryption recap
The original code does:
1
2
3
4
5
6
7
8
9
Plaintext
↓
b16_encode()
↓
letters a-p
↓
shift by key
↓
Ciphertext
So decryption must do:
1
2
3
4
5
6
7
8
9
Ciphertext
↓
reverse shift
↓
letters a-p
↓
b16_decode()
↓
Plaintext
Step 1: Reverse the shift
Encryption used:
1
2
def shift(c, k):
return ALPHABET[(t1 + t2) % 16]
where:
1
2
t1 = value of plaintext character
t2 = value of key character
For example:
1
2
3
4
5
e (4)
key = c (2)
(4 + 2) mod 16 = 6
→ g
To recover the original character, subtract instead of add:
1
2
3
4
def unshift(c, k):
t1 = ord(c) - ord('a')
t2 = ord(k) - ord('a')
return ALPHABET[(t1 - t2) % 16]
Example:
1
2
3
4
5
cipher = g (6)
key = c (2)
(6 - 2) mod 16 = 4
→ e
So:
1
g → e
Step 2: Reverse the custom base16 encoding
Recall that:
1
2
3
4
5
a = 0
b = 1
c = 2
...
p = 15
During encryption, each byte became two letters.
Example:
1
'A'
ASCII:
1
65
Binary:
1
01000001
Split:
1
0100 0001
Convert:
1
2
0100 → 4 → e
0001 → 1 → b
Result:
1
"A" → "eb"
To decode:
1
2
3
4
high = ALPHABET.index('e') # 4
low = ALPHABET.index('b') # 1
byte = (high << 4) | low
Calculation:
1
2
3
4
5
(4 << 4) | 1
=
64 + 1
=
65
ASCII 65:
1
'A'
Recovered successfully.
The b16 decoder
1
2
3
4
5
6
7
8
9
10
11
def b16_decode(enc):
out = ""
for i in range(0, len(enc), 2):
high = ALPHABET.index(enc[i])
low = ALPHABET.index(enc[i + 1])
byte = (high << 4) | low
out += chr(byte)
return out
What happens?
Suppose:
1
enc = "eb"
Loop:
1
2
high = 4
low = 1
Rebuild byte:
1
byte = (4 << 4) | 1
Result:
1
65
Convert back:
1
chr(65)
Result:
1
'A'
Complete decryptor
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
import string
LOWERCASE_OFFSET = ord("a")
ALPHABET = string.ascii_lowercase[:16] # abcdefghijklmnop
def unshift(c, k):
t1 = ord(c) - LOWERCASE_OFFSET
t2 = ord(k) - LOWERCASE_OFFSET
return ALPHABET[(t1 - t2) % len(ALPHABET)]
def b16_decode(enc):
out = ""
for i in range(0, len(enc), 2):
high = ALPHABET.index(enc[i])
low = ALPHABET.index(enc[i + 1])
byte = (high << 4) | low
out += chr(byte)
return out
def decrypt(ciphertext, key):
# Reverse Vigenère shift
b16 = ""
for i, c in enumerate(ciphertext):
b16 += unshift(c, key[i % len(key)])
# Decode base16-like encoding
return b16_decode(b16)
# Example usage
ciphertext = "fegdeogdgecoeocgcgchcfcffccfca"
for key in ALPHABET:
try:
print(f"{key}: {decrypt(ciphertext, key)}")
except:
pass
print(decrypt(ciphertext, key))
Why brute force works
The original program contains:
1
assert len(key) == 1
So the key is only one character from:
1
abcdefghijklmnop
That’s only:
1
16 possible keys
Therefore you can simply try them all:
1
2
for key in ALPHABET:
print(key, decrypt(ciphertext, key))
Look for readable English or a recognizable flag among the outputs to identify the recovered plaintext.
Flag: picoCTF{et_tu?_77866c61}
