Crack Me Challenge: Final Edition

Part 6

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

The code for logical segment 6 is as follows:

0040181F |. B9 10000000 mov ecx,10

00401824 |. 8DB424 C0000000 lea esi,dword ptr ss:[esp+C0]sta

0040182B |. 8D7C24 10 lea edi,dword ptr ss:[esp+10]

0040182F |. F3:A5 rep movs dword ptr es:[edi],dword ptr ds>

00401831 |. 8B75 10 mov esi,dword ptr ss:[ebp+10]

00401834 |. 33C0 xor eax,eax

00401836 |. 85F6 test esi,esi

00401838 |. 7E 1E jle short main.00401858

0040183A |. 8D9B 00000000 lea ebx,dword ptr ds:[ebx]

00401840 |> 8B55 0C /mov edx,dword ptr ss:[ebp+C]

00401843 |. 8A1410 |mov dl,byte ptr ds:[eax+edx]

00401846 |. 8BC8 |mov ecx,eax

00401848 |. 83E1 3F |and ecx,3F

0040184B |. 30540C 10 |xor byte ptr ss:[esp+ecx+10],dl

0040184F |. 8D4C0C 10 |lea ecx,dword ptr ss:[esp+ecx+10]

00401853 |. 40 |inc eax

00401854 |. 3BC6 |cmp eax,esi

00401856 |.^7C E8 jl short main.00401840

The [esp+C0] points to the address 0x0012EB08, which contains the whirlpool hash of the Name value plus ESETNOD32@ string, like this:

AAAAAAD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32 (continued on next line)

@ESETNOD32@ESETNOD32@ESETNOD32@

In the above code we're reading the whirlpool hash from [esp+C0] and writing it to the address [esp+10] with the rep instruction. The [ebp+10] stack value contains the length of the Key 1 value. We're clearing the eax register to zero testing whether the length of the Key 1 value is 0, which it isn't, so the jump isn't taken. Then we're moving the value of [ebx] into ebx, which contains exactly the Name input argument. The following instruction reads some memory address from stack and puts it into register edx:

00401840 |> 8B55 0C /mov edx,dword ptr ss:[ebp+C]

The ebp+C register constitutes the input argument that was passed to this function. If we look at the stack, we can see that function parameters were passed into the current function residing at address 0x00401760 from the addresses right before the address 0x00401B2D. The stack memory that contains those values can be seen in the picture below:

The highlighted value is the memory address being passed to the function 0x00401760 when being called from the address 0x00401B2B. The program execution is then returned to the address 0x00401B2D. The important memory address on the above picture being passed into the function as a parameter is 0x00B98660. Thus the edx register will contain the address 0x00B98660 which we'll look into more detail.

Let's disassemble the instructions right before the address 0x00401B2D:

00401AF8 . 52 push edx ; Arg1

00401AF9 . E8 04410F00 call main.004F5C02 ; Function Call

00401AFE . 8BD8 mov ebx,eax

00401B00 . 83C4 04 add esp,4

00401B03 . 85DB test ebx,ebx

00401B05 . 74 7C je short main.00401B83

00401B07 . 8BCB mov ecx,ebx

00401B09 . 8BD6 mov edx,esi

00401B0B . 8D85 F0FAFFFF lea eax,dword ptr ss:[ebp-510]

00401B11 . E8 AAF7FFFF call main.004012C0

00401B16 . 8B85 ECFAFFFF mov eax,dword ptr ss:[ebp-514]

00401B1C . 8B40 F4 mov eax,dword ptr ds:[eax-C]

00401B1F . 57 push edi

00401B20 . 53 push ebx

00401B21 . 50 push eax

00401B22 . 8D8D F0FEFFFF lea ecx,dword ptr ss:[ebp-110]

00401B28 . E8 33FCFFFF call main.00401760

00401B2D . 53 push ebx

We also need to disassemble the first few instructions at address 0x00401760:

00401760 /$ 55 push ebp

00401761 |. 8BEC mov ebp,esp

00401763 |. 83E4 F8 and esp,FFFFFFF8

00401766 |. 81EC FC000000 sub esp,0FC]

We can see that in the function 0x00401760 we're pushing another value onto the stack before we're taking care of the function frame. Therefore the value at [ebp+C] is exactly the value which is getting set by the push ebx instruction right before the function call at address 0x00401B20. The ebp register is set at the address 0x00401AFE when eax is copied into it. And eax is set by the function call at address 0x004F5C02 that happens one instruction before.

The memory at the address returned by the function looks as follows:

00B98660 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA

00B98670 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA

00B98680 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA

00B98690 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA

00B986A0 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA

00B986B0 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA

But that memory region is changed in the duration of the program. The instruction at 0x00401840 memory region at address 0x00B98660 contains the following values:

00B98660 69 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A 69

00B98670 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A 69 A6

00B98680 9A 69 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A

00B98690 69 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A 69

00B986A0 A6 9A 69 A6 9A 69 A6 9A 69 A6 9A 00 0D F0 AD BA

00B986B0 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA 0D F0 AD BA

        

We can quickly confirm that the computed values are in fact calculated with a function residing at the address 0x004012C0, which we've already encountered. The algorithm looks like this:

1. eax = [0x5414C0 + key1[i]]
2. ebx = [0x5414C0 + key1[i+1]]
3. al = al * 2
4. bl = bl shift right by 4
5. al = al * 2
6. al = al xor bl

The key1 is the value that was inputed into the field Key 1. But the length is not the same as with the previous algorithm, where the length was 64 bytes. The length of the computed values can be determined by looking at the following instructions:

00401348 . 3BF2 cmp esi,edx

0040134A .^7C 94 jl short main.004012E0

The esi register starts at 0 and increases by 4 each iteration, whereas the edx register is the argument into the function. The edx register contains the length of the Key 1 input value. By now we've figured out what the dl register at the end of the loop in the above code section should hold.

What follows is the xor instruction that computes the XOR of the whirlpool hash byte and the dl register one byte at a time. It writes the result into the address pointed to by [esp+10] where the copied whirlpool hash is. Afterwards we're increasing the eax register by 1 and comparing it to esi, which contains the length of the Key 1 value. Once the loop is finished the program execution can continue. Whirlpool hash is only 64 bytes long, so why are we copying the number of bytes that equal the length of the Key 1 input value? We don't know just yet.

But let's try to think for a minute. We're computing some 'random' values (we don't know if the values above have any meaning yet) on the stack. We're starting at stack address [esp+ecx+10] and continuing as many times as is the length of the Key 1 input value. The register ecx depends upon register eax, which starts with a value 0 and is being increased by 1 every loop. So the values in ecx are computed like this:

ecx = eax

ecx = ecx AND 00111111

That is why the concrete values of ecx are known in advance, since only the last 6 bits are kept; the other bits are being rewritten to zero. The following section holds the representation of the values in eax and their appropriated counterpart values in ecx register.

EAX ECX

00000000 00000000

00000001 00000001

00000002 00000002

00000003 00000003

00111111 00111111

01000000 00000000

01000001 00000001

01111111 00111111

This is an effective algorithm to overwrite only the first 0x40 bytes from the address [esp+10] onwards. The whole scheme how the stack values are computed can be seen in the picture below:

We can see that first we're copying the whirlpool hash located at the address [esp+C0] into a temporary stack address H on the right: H0, H1, ..., H63. The stack addresses S: S0, S1, ..., S63 right below the stack addresses H: H0, H1, ..., H63 contain exactly the same value; they are present just for a clearer picture of what happens in each step. After the whirlpool hash is copied to the temporary stack address H or S (doesn't really matter, since they are the same), we're applying the algorithm in the frame box to it. The algorithm works by taking each byte from the Data Structure and xoring it with the copied whirlpool hash and saving the value in the temporary stack location where the whirlpool hash is located. The loop is repeated for each byte of the Key 1 input value. Since the whirlpool hash contains only 64 bytes, the stack addresses will be xored more than once if the length of the Key 1 input field is longer than that.Logical Code Segment 7

The code for the logical segment 7 is presented below:

00401858 |> 33F6 xor esi,esi

0040185A |. 33C9 xor ecx,ecx

0040185C |. 8D6424 00 lea esp,dword ptr ss:[esp]

00401860 |> 8A5C34 10 /mov bl,byte ptr ss:[esp+esi+10]

00401864 |. 0FB6C3 |movzx eax,bl

00401867 |. 99 |cdq

00401868 |. BF 50000000 |mov edi,50

0040186D |. F7FF |idiv edi

0040186F |. 83C6 04 |add esi,4

00401872 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

00401877 |. 03C1 |add eax,ecx

00401879 |. 99 |cdq

0040187A |. 8BCF |mov ecx,edi

0040187C |. F7F9 |idiv ecx

0040187E |. 0FB6CA |movzx ecx,dl

00401881 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

00401885 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

00401889 |. 8A5C34 0D |mov bl,byte ptr ss:[esp+esi+D]

0040188D |. 884434 0C |mov byte ptr ss:[esp+esi+C],al

00401891 |. 0FB6C3 |movzx eax,bl

00401894 |. 99 |cdq

00401895 |. F7FF |idiv edi

00401897 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

0040189C |. 03C1 |add eax,ecx

0040189E |. 99 |cdq

0040189F |. 8BCF |mov ecx,edi

004018A1 |. F7F9 |idiv ecx

004018A3 |. 0FB6CA |movzx ecx,dl

004018A6 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

004018AA |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018AE |. 8A5C34 0E |mov bl,byte ptr ss:[esp+esi+E]

004018B2 |. 884434 0D |mov byte ptr ss:[esp+esi+D],al

004018B6 |. 0FB6C3 |movzx eax,bl

004018B9 |. 99 |cdq

004018BA |. F7FF |idiv edi

004018BC |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

004018C1 |. 03C1 |add eax,ecx

004018C3 |. 99 |cdq

004018C4 |. 8BCF |mov ecx,edi

004018C6 |. F7F9 |idiv ecx

004018C8 |. 0FB6CA |movzx ecx,dl

004018CB |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

004018CF |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018D3 |. 8A5C34 0F |mov bl,byte ptr ss:[esp+esi+F]

004018D7 |. 884434 0E |mov byte ptr ss:[esp+esi+E],al

004018DB |. 0FB6C3 |movzx eax,bl

004018DE |. 99 |cdq

004018DF |. F7FF |idiv edi

004018E1 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

004018E6 |. 03C1 |add eax,ecx

004018E8 |. 99 |cdq

004018E9 |. 8BCF |mov ecx,edi

004018EB |. F7F9 |idiv ecx

004018ED |. 0FB6CA |movzx ecx,dl

004018F0 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

004018F4 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018F8 |. 884434 0F |mov byte ptr ss:[esp+esi+F],al

004018FC |. 83FE 40 |cmp esi,40

004018FF |.^0F8C 5BFFFFFF jl main.00401860

First we're initializing registers esi and ecx to zero. The next lea instruction doesn't actually do anything, so we can skip it.

In the loop, we're reading the previously computed values from [esp+10] onwards into the register bl. We're interested in instructions that we can use to overwrite certain values on the stack. Those instructions are the following:

00401885 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

0040188D |. 884434 0C |mov byte ptr ss:[esp+esi+C],al

004018AA |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018B2 |. 884434 0D |mov byte ptr ss:[esp+esi+D],al

004018CF |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018D7 |. 884434 0E |mov byte ptr ss:[esp+esi+E],al

004018F4 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018F8 |. 884434 0F |mov byte ptr ss:[esp+esi+F],al

We are interested in instructions that can set the values at [esp+54] and [esp+5C]. The esi register can never reach values larger than 0x40, which dictates the number of loop iterations:

004018FC |. 83FE 40 |cmp esi,40

004018FF |.^0F8C 5BFFFFFF jl main.00401860

Therefore, only the following instructions are actually interesting to us:

00401885 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018AA |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018CF |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018F4 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

Let's mention again why are those two stack addresses are so important. Because if we overwrite those two addresses with correct values, we can solve the first challenge. The [esp+54] is so important because of what follows after the loop:

00401905 |. 817C24 54 0A0B0C0D cmp dword ptr ss:[esp+54],0D0C0B0A

0040190D |. 75 50 jnz short main.0040195F

Therefore the value in the [esp+54] must equal to 0x0D0C0B0A for the jump not to be taken. The following code is what immediately follows:

0040190F |. 8D5424 70 lea edx,dword ptr ss:[esp+70]

00401913 |. 52 push edx

00401914 |. 8D4424 14 lea eax,dword ptr ss:[esp+14]

00401918 |. 50 push eax

00401919 |. 8D4F FC lea ecx,dword ptr ds:[edi-4]

0040191C |. C74424 6C CA8A5712 mov dword ptr ss:[esp+6C],12578ACA

00401924 |. C74424 70 78B6CAEF mov dword ptr ss:[esp+70],EFCAB678

0040192C |. C74424 74 78563412 mov dword ptr ss:[esp+74],12345678

00401934 |. E8 E7160000 call main.00403020

00401939 |. 8D47 BC lea eax,dword ptr ds:[edi-44]

0040193C |. 33C9 xor ecx,ecx

0040193E |. 8BFF mov edi,edi

00401940 |> 8B540C 70 /mov edx,dword ptr ss:[esp+ecx+70]

00401944 |. 3B540C 64 |cmp edx,dword ptr ss:[esp+ecx+64]

00401948 |. 75 1D |jnz short main.00401967

0040194A |. 83E8 04 |sub eax,4

0040194D |. 83C1 04 |add ecx,4

00401950 |. 83F8 04 |cmp eax,4

00401953 |.^73 EB jnb short main.00401940

Ok, we don't really need to understand this code as much. What we must acknowledge is that the cmp instruction at 00401944, which jumps directly to address 0x00401967, which in turn jumps to whatever value is contained on the stack at address [esp+5C]:

00401967 |> FF5424 5C call dword ptr ss:[esp+5C]

Therefore if we manage to somehow write the correct values on the stack addresses [esp+54] and [esp+5C] we would solve the first challenge. The value at [esp+54] must be 0x0D0C0B0A and the value at [esp+5C] must contain an address 0x00401720, which would jump directly to the Congratulations message box.

Therefore we must inspect the loop a little bit further to cause it to write the correct values on the correct memory addresses. We must look at the register bf in the mentioned mov instruction that actually overwrites the stack with some value.Mov Instruction

The code for the first mov instruction is as follows:

00401860 |> 8A5C34 10 /mov bl,byte ptr ss:[esp+esi+10]

00401864 |. 0FB6C3 |movzx eax,bl

00401867 |. 99 |cdq

00401868 |. BF 50000000 |mov edi,50

0040186D |. F7FF |idiv edi

0040186F |. 83C6 04 |add esi,4

00401872 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

00401877 |. 03C1 |add eax,ecx

00401879 |. 99 |cdq

0040187A |. 8BCF |mov ecx,edi

0040187C |. F7F9 |idiv ecx

0040187E |. 0FB6CA |movzx ecx,dl

00401881 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

00401885 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

00401889 |. 8A5C34 0D |mov bl,byte ptr ss:[esp+esi+D]

0040188D |. 884434 0C |mov byte ptr ss:[esp+esi+C],al

We can immediately see that we're reading one value bl from the stack address 0x00401860 and writing it to another stack address 0x00401885; thus effectively permuting the values on the stack. Because we can influence the values on the stack from the previous code section and we're not changing them in this section, it's obvious that we have the total control of the stack values. We just need to figure out a way to actually permute the values.

Move the value in register bl into eax:

00401864 |. 0FB6C3 |movzx eax,bl

From 32-bit value make 64-bit value by sign extending the value in register eax into registers eax and edx (since values in eax are very small, this effectively sets edx register to all zeros):

00401867 |. 99 |cdq

Move a constant value of 0x50 into register edi:

00401868 |. BF 50000000 |mov edi,50

Divide the 64-bit value in registers edx:eax by the operand edi that contains a constant 0x50. The operand is put in register eax whereas the remainder is put in register edx.

0040186D |. F7FF |idiv edi

Move a byte value at [esp+0+10] into the register eax. This instruction doesn't have an effect since we're overwriting the value in eax register in the next instruction.

00401872 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

Add the value in ecx and eax together and save it in register eax. Then sign extend the value in eax into edx:eax. Copy the value in edi into ecx and delete the 64-bit value in edx:eax by ecx and put the operand in eax and remained in edx.

00401877 |. 03C1 |add eax,ecx

00401879 |. 99 |cdq

0040187A |. 8BCF |mov ecx,edi

0040187C |. F7F9 |idiv ecx

Save the remained of the divide operation into register ecx, which is later used to permute the stack values:

0040187E |. 0FB6CA |movzx ecx,dl

The whole code section can be truncated into the following formula:

ecx = (bl + ecx) % 80

At the end we have the following code:

00401881 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

00401885 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

00401889 |. 8A5C34 0D |mov bl,byte ptr ss:[esp+esi+D]

0040188D |. 884434 0C |mov byte ptr ss:[esp+esi+C],al

If we set the computed value in ecx a variable x, then we can write our equations as follows:

al = S[i+x]

S[i+x] = bl

bl = S[i+y+D]

S[i+y+C] = al

Mov Instruction

The code is presented below:

00401891 |. 0FB6C3 |movzx eax,bl

00401894 |. 99 |cdq

00401895 |. F7FF |idiv edi

00401897 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

0040189C |. 03C1 |add eax,ecx

0040189E |. 99 |cdq

0040189F |. 8BCF |mov ecx,edi

004018A1 |. F7F9 |idiv ecx

004018A3 |. 0FB6CA |movzx ecx,dl

004018A6 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

004018AA |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018AE |. 8A5C34 0E |mov bl,byte ptr ss:[esp+esi+E]

004018B2 |. 884434 0D |mov byte ptr ss:[esp+esi+D],al

This essentially does the same as the first mov instruction.

eax = S[i + bl % 80]

eax = eax + x

eax = eax / 80

ecx = eax % 80

al = S[i+x]

S[i+x] = bl

bl = S[i+y+E]

S[i+y+D] = al

Mov Instruction

The code is presented below:

004018B6 |. 0FB6C3 |movzx eax,bl

004018B9 |. 99 |cdq

004018BA |. F7FF |idiv edi

004018BC |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

004018C1 |. 03C1 |add eax,ecx

004018C3 |. 99 |cdq

004018C4 |. 8BCF |mov ecx,edi

004018C6 |. F7F9 |idiv ecx

004018C8 |. 0FB6CA |movzx ecx,dl

004018CB |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

004018CF |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018D3 |. 8A5C34 0F |mov bl,byte ptr ss:[esp+esi+F]

004018D7 |. 884434 0E |mov byte ptr ss:[esp+esi+E],al

The code is exactly the same as before, except that the constant F is used instead of the E. The rules are:

al = S[i+x]

S[i+x] = bl

bl = S[i+y+F]

S[i+y+E] = al

urth Mov Instruction

The code is presented below:

004018DB |. 0FB6C3 |movzx eax,bl

004018DE |. 99 |cdq

004018DF |. F7FF |idiv edi

004018E1 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]

004018E6 |. 03C1 |add eax,ecx

004018E8 |. 99 |cdq

004018E9 |. 8BCF |mov ecx,edi

004018EB |. F7F9 |idiv ecx

004018ED |. 0FB6CA |movzx ecx,dl

004018F0 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]

004018F4 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl

004018F8 |. 884434 0F |mov byte ptr ss:[esp+esi+F],al

Again, this is the same code as before, except that the following rules are used:

al = S[i+x]

S[i+x] = bl

S[i+y+F] = al

Part 7

At the end of the function there's only the following code left, which is located right after the previous permutation occurs.

00401905 |. 817C24 54 0A0B>cmp dword ptr ss:[esp+54],0D0C0B0A

0040190D |. 75 50 jnz short main.0040195F

0040190F |. 8D5424 70 lea edx,dword ptr ss:[esp+70]

00401913 |. 52 push edx

00401914 |. 8D4424 14 lea eax,dword ptr ss:[esp+14]

00401918 |. 50 push eax

00401919 |. 8D4F FC lea ecx,dword ptr ds:[edi-4]

0040191C |. C74424 6C CA8A>mov dword ptr ss:[esp+6C],12578ACA

00401924 |. C74424 70 78B6>mov dword ptr ss:[esp+70],EFCAB678

0040192C |. C74424 74 7856>mov dword ptr ss:[esp+74],12345678

00401934 |. E8 E7160000 call main.00403020

00401939 |. 8D47 BC lea eax,dword ptr ds:[edi-44]

0040193C |. 33C9 xor ecx,ecx

0040193E |. 8BFF mov edi,edi

00401940 |> 8B540C 70 /mov edx,dword ptr ss:[esp+ecx+70]

00401944 |. 3B540C 64 |cmp edx,dword ptr ss:[esp+ecx+64]

00401948 |. 75 1D |jnz short main.00401967

0040194A |. 83E8 04 |sub eax,4

0040194D |. 83C1 04 |add ecx,4

00401950 |. 83F8 04 |cmp eax,4

00401953 |.^73 EB jnb short main.00401940

00401955 C74424 5C 2017>mov dword ptr ss:[esp+5C],main.00401720

0040195D |. EB 08 jmp short main.00401967

0040195F |> C74424 5C 4017>mov dword ptr ss:[esp+5C],main.00401740

00401967 |> FF5424 5C call dword ptr ss:[esp+5C]

0040196B |. 8B8C24 0401000>mov ecx,dword ptr ss:[esp+104]

When permuting the values, we needed to set two stack variables:

1. esp+54: which should contain the value 0x0D0C0B0A for a jump at address 0x0040190D not to be taken.
2. esp+5C: which should contain the value 0x00401720, which would jump to the congratulations message box.

But in the above code we can see that it's trying to compare the values at addresses [esp+70] and [esp+64] to be equal. This means that the whole stack region needs to contain the same 4-byte values, which cannot happen in a real program. This is why we need to overwrite the address [esp+5C] with the address of the congratulations message box - 0x00401720. We know that the compare instruction at the address 0x00401944 will fail and program execution will be transferred to the address 0x00401967. At this point the value in the address [esp+5C] should contain the relevant stack address where the program execution will be redirected. In our case this is the congratulations message box.

The picture below presents the whole process of crackme.exe logic.

The steps presented in the above picture are as follows:

1. Copy the Name part of input value together with string ESETNOD32@ into the [esp+70] stack address region.
2. Calculate the whirlpool hash cryptographic value from the string contained in the [esp+70] and save it into the [esp+C0] stack address region.
3. Change the data region at [esp+70] with the specified algorithm; the input argument is the Key 1 input value.
4. Compare the memory regions at [esp+70] and [esp+C0], which must be the same.
5. Copy the hash from the address [esp+C0] into the [esp+10] stack memory region.
6. Change the data region at [0x00B985D8] with specified algorithm.
7. Change the data region at [esp+10] using the specified algorithm.
8. Permute the data region at [esp+10].

   Two important values located at [esp+54] and [esp+5C] need to contain specific values after the permutation process. Those two values are marked with pink color in the picture above.

We've seen that we can control the whole stack the way we want. There are a lot of xor operations, hashing and permutations, but it can be done. There's quite a lot of mathematical background involved, but we've seen that since we can save arbitrary values into the stack, we can control what gets executed.

You may ask: how can we input arbitrary values onto the stack? The answer is quite simple, we can control all the important stack values that are used to guide the program through its execution with our input fields. Since we can put arbitrary input values into Name and Key 1, we can effectively control the whole stack and bypass the first stage.

Therefore we can continue with the second stage.

The Length of the Key 2

When running the code, we can quickly determine that the first problem is the following code:

00401CF2 . 83FA 0C cmp edx,0C

00401CF5 0F85 9D000000 jnz main.00401D98

The register edx contains the length of the input argument, which is compared to the value 0x0C. If the values are not the same, we're jumping to the address 0x00401D98 that essentially displays our failure message box. This address holds the following code:

00401D98 > 57 push edi

00401D99 . 6A 30 push 30

00401D9B . 68 0C1A5400 push main.00541A0C

00401DA0 > E8 4B5C0000 call main.004079F0

First we're pushing three arguments to the stack:

• edi
• 0x30
• a pointer to a string "Invalid key value."

Afterwards we're calling the message box which displays the message "Invalid key value.". Ok, so we must avoid jumping to the address 0x00401D98. We can do that by entering exactly 12 characters into the input field. Thus we have our first predicate:

len(key2) = 0xC

The Allowed Characters in Key 2

Afterwards we stumble upon the code presented below:

00401D00 > 8A440C 24 mov al,byte ptr ss:[esp+ecx+24]

00401D04 . 3C 41 cmp al,41

00401D06 . 7C 04 jl short main.00401D0C

00401D08 . 3C 46 cmp al,46

00401D0A . 7E 10 jle short main.00401D1C

00401D0C > 3C 30 cmp al,30

00401D0E . 0F8C 84000000 jl main.00401D98

00401D14 . 3C 39 cmp al,39

00401D16 . 0F8F 7C000000 jg main.00401D98

00401D1C > 41 inc ecx

00401D1D . 83F9 0C cmp ecx,0C

00401D20 .^72 DE jb short main.00401D00

That code can be constructed into the following higher-level language:

i = 0;

while(i<12) {

if(a[i] < 0x41) {

goto label30;

}

else if(a[i] <= 0x46) {

goto end;

}

else if(a[i] < 0x30) {

label30:

printf("Incorrect Key 2. Try again.");

}

else if(a[i] > 0x39) {

printf("Incorrect Key 2. Try again.");

}

end:

i++;

}

We can see that what the code actually does, is take the input string of 12 characters, and compare each byte at a time with the constants: 0x41 = A, 0x46 = F, 0x30 = 0 and 0x39 = 9. This gives us five different options of formatting our input. We can enter the following characters:

• 0x29: Invalid.
• 0x38: Ok.
• 0x40: Invalid.
• 0x45: Ok.
• 0x47: Invalid.

Therefore only two ranges of numbers are allowed to be inputed: 0x30−0x39 = 1−9 and 0x41−0x46 = A−F. So if we enter twelve A's (0x41) as input, this filter should let it through and not print an error.

How to Solve Stage 2

The rest of the code is as follows:

00401D7C . 50 push eax

00401D7D . E8 AE060000 call main.00402430

00401D82 . 6A 00 push 0

00401D84 . 6A 30 push 30

00401D86 . 85C0 test eax,eax

00401D88 . 74 07 je short main.00401D91

00401D8A . 68 201A5400 push main.00541A20

00401D8F . EB 0F jmp short main.00401DA0

00401D91 > 68 481A5400 push main.00541A48

00401D96 . EB 08 jmp short main.00401DA0

00401D98 > 57 push edi

00401D99 . 6A 30 push 30

00401D9B . 68 0C1A5400 push main.00541A0C

00401DA0 > E8 4B5C0000 call main.004079F0

Here, we're pushing the value in register eax to stack as an argument to the function at address 0x00402430. That function must return the value in eax that is not zero. This would set appropriate flags when the test eax,eax instruction is being executed, which would not jump on the je instruction. When the jump is not taken, the push 00541A20 is pushed onto the stack and a jump to an address 0x00401DA0 is taken. On that address another function is called that displays the message box, either that we succeeded in solving the problem or that we failed. It depends on the argument pushed onto the stack. If the argument is 00541A20, then the message box should say that we succeeded in solving the problem. If the argument is 00541A48 then the message box will say that we failed.

Therefore we must convince the function called at the address 0x00402430 to save 1 into the register eax and not 0.

Part 8

First, the complete code of the function is in order. The code is presented below:

00402430 $ 55 push ebp

00402431 . 8BEC mov ebp,esp

00402433 . 83EC 28 sub esp,28

00402436 . 33C0 xor eax,eax

00402438 . 53 push ebx

00402439 . 56 push esi

0040243A . 8945 E0 mov dword ptr ss:[ebp-20],eax

0040243D . 8945 E4 mov dword ptr ss:[ebp-1C],eax

00402440 . 8945 F4 mov dword ptr ss:[ebp-C],eax

00402443 . 57 push edi

00402444 . B8 28015600 mov eax,main.00560128

00402449 > C645 FF 00 mov byte ptr ss:[ebp-1],0

0040244D . 8945 F8 mov dword ptr ss:[ebp-8],eax

00402450 . C745 F0 080000>mov dword ptr ss:[ebp-10],8

00402457 . EB 0A jmp short main.00402463

00402459 . 8DA424 0000000>lea esp,dword ptr ss:[esp]

00402460 > 8B45 F8 mov eax,dword ptr ss:[ebp-8]

00402463 > 8B18 mov ebx,dword ptr ds:[eax]

00402465 . 8B40 04 mov eax,dword ptr ds:[eax+4]

00402468 . C745 EC 010000>mov dword ptr ss:[ebp-14],1

0040246F . 33FF xor edi,edi

00402471 . 8945 DC mov dword ptr ss:[ebp-24],eax

00402474 > B8 01000000 mov eax,1

00402479 . 33D2 xor edx,edx

0040247B . 8BCF mov ecx,edi

0040247D . E8 3E840F00 call main.004FA8C0

00402482 . 8B75 DC mov esi,dword ptr ss:[ebp-24]

00402485 . 8BCB mov ecx,ebx

00402487 . 23C8 and ecx,eax

00402489 . 23F2 and esi,edx

0040248B . 0BCE or ecx,esi

0040248D . 74 0C je short main.0040249B

0040248F . 8B4D 08 mov ecx,dword ptr ss:[ebp+8]

00402492 . 2301 and eax,dword ptr ds:[ecx]

00402494 . 2351 04 and edx,dword ptr ds:[ecx+4]

00402497 . 0BC2 or eax,edx

00402499 . 74 0B je short main.004024A6

0040249B > 47 inc edi

0040249C . 83FF 40 cmp edi,40

0040249F .^7C D3 jl short main.00402474

004024A1 . 8A45 EC mov al,byte ptr ss:[ebp-14]

004024A4 . EB 02 jmp short main.004024A8

004024A6 > 33C0 xor eax,eax

004024A8 > 0045 FF add byte ptr ss:[ebp-1],al

004024AB . 8345 F8 08 add dword ptr ss:[ebp-8],8

004024AF . FF4D F0 dec dword ptr ss:[ebp-10]

004024B2 .^75 AC jnz short main.00402460

004024B4 . 0FB64D FF movzx ecx,byte ptr ss:[ebp-1]

004024B8 . 81E1 01000080 and ecx,80000001

004024BE . 79 05 jns short main.004024C5

004024C0 . 49 dec ecx

004024C1 . 83C9 FE or ecx,FFFFFFFE

004024C4 . 41 inc ecx

004024C5 > 84C9 test cl,cl

004024C7 . 74 15 je short main.004024DE

004024C9 . 8B4D F4 mov ecx,dword ptr ss:[ebp-C]

004024CC . B8 01000000 mov eax,1

004024D1 . 33D2 xor edx,edx

004024D3 . E8 E8830F00 call main.004FA8C0

004024D8 . 0945 E0 or dword ptr ss:[ebp-20],eax

004024DB . 0955 E4 or dword ptr ss:[ebp-1C],edx

004024DE > 8B45 F8 mov eax,dword ptr ss:[ebp-8]

004024E1 . FF45 F4 inc dword ptr ss:[ebp-C]

004024E4 . 3D 28115600 cmp eax,main.00561128

004024E9 .^0F8C 5AFFFFFF jl main.00402449

004024EF . 817D E0 32EFA0>cmp dword ptr ss:[ebp-20],23A0EF32

004024F6 . 75 17 jnz short main.0040250F

004024F8 . 817D E4 8EECBD>cmp dword ptr ss:[ebp-1C],55BDEC8E

004024FF . 75 0E jnz short main.0040250F

00402501 . B8 01000000 mov eax,1

00402506 . 5F pop edi

00402507 . 5E pop esi

00402508 . 5B pop ebx

00402509 . 8BE5 mov esp,ebp

0040250B . 5D pop ebp

0040250C . C2 0400 retn 4

0040250F > 5F pop edi

00402510 . 5E pop esi

00402511 . 33C0 xor eax,eax

00402513 . 5B pop ebx

00402514 . 8BE5 mov esp,ebp

00402516 . 5D pop ebp

00402517 . C2 0400 retn 4

This function takes one input argument at the address [ebp+8]. That address holds the value 0x0012EF70 that points to our [esp+18] and [esp+1C] values on the stack. Therefore the following code effectively loads and uses those values:

0040248F . 8B4D 08 mov ecx,dword ptr ss:[ebp+8]

00402492 . 2301 and eax,dword ptr ds:[ecx]

00402494 . 2351 04 and edx,dword ptr ds:[ecx+4]

The function must set the eax register to something that is not zero. This happens at the end of function when the following code is being executed:

004024EF . 817D E0 32EFA0>cmp dword ptr ss:[ebp-20],23A0EF32

004024F6 . 75 17 jnz short main.0040250F

004024F8 . 817D E4 8EECBD>cmp dword ptr ss:[ebp-1C],55BDEC8E

004024FF . 75 0E jnz short main.0040250F

00402501 . B8 01000000 mov eax,1

00402506 . 5F pop edi

00402507 . 5E pop esi

00402508 . 5B pop ebx

00402509 . 8BE5 mov esp,ebp

0040250B . 5D pop ebp

0040250C . C2 0400 retn 4

0040250F > 5F pop edi

00402510 . 5E pop esi

00402511 . 33C0 xor eax,eax

00402513 . 5B pop ebx

00402514 . 8BE5 mov esp,ebp

00402516 . 5D pop ebp

00402517 . C2 0400 retn 4

We need to force the program to execute the return statement at 0x0040250C. This must happen because a few commands back there's an instruction mov eax,1, which sets eax register to 1. If the latter retn instruction is called, it sets the register to 0 with xor eax, eax. This is why both compare conditions need to hold - the value at address [ebp-20] must hold 0x23A0EF32 and the [ebp-1C] must contain 0x55BDEC8E.

The bytes at addresses [esp-20] and [esp-1C] are set a few instructions before:

004024D3 . E8 E8830F00 call main.004FA8C0

004024D8 . 0945 E0 or dword ptr ss:[ebp-20],eax

004024DB . 0955 E4 or dword ptr ss:[ebp-1C],edx

And at the beginning of the function we're zeroing out those two addresses to start with a clean slate:

00402436 . 33C0 xor eax,eax

00402438 . 53 push ebx

00402439 . 56 push esi

0040243A . 8945 E0 mov dword ptr ss:[ebp-20],eax

0040243D . 8945 E4 mov dword ptr ss:[ebp-1C],eax

Afterwards there are two loops we need to take a look at. The inner loop is as follows:

00402474 > B8 01000000 mov eax,1

00402479 . 33D2 xor edx,edx

0040247B . 8BCF mov ecx,edi

0040247D . E8 3E840F00 call main.004FA8C0

00402482 . 8B75 DC mov esi,dword ptr ss:[ebp-24]

00402485 . 8BCB mov ecx,ebx

00402487 . 23C8 and ecx,eax

00402489 . 23F2 and esi,edx

0040248B . 0BCE or ecx,esi

0040248D . 74 0C je short main.0040249B

0040248F . 8B4D 08 mov ecx,dword ptr ss:[ebp+8]

00402492 . 2301 and eax,dword ptr ds:[ecx]

00402494 . 2351 04 and edx,dword ptr ds:[ecx+4]

00402497 . 0BC2 or eax,edx

00402499 . 74 0B je short main.004024A6

0040249B > 47 inc edi

0040249C . 83FF 40 cmp edi,40

0040249F .^7C D3 jl short main.00402474

We're setting the register eax to 1 and zeroing register edx. We're also incrementing the edi register from 0 to 64 that can be seen here:

0040246F . 33FF xor edi,edi

0040249B > 47 inc edi

0040249C . 83FF 40 cmp edi,40

Therefore the ecx register contains values from 0 to 63 when the loop is being executed. The next subsection contains a detailed explanation of that loop.

I haven't had the time to program the exact version of the high-level Python version of the function, but I've been able to come up with pretty close function definition. It still contains some bugs, as it doesn't return 0x1 in variable a as it should when the right input arguments are passed to it. The code is appended here:

# Local variables:

# l0 = [ebp - 4]

# l1 = [ebp - 8]

# l2 = [ebp - C]

# l3 = [ebp - 10]

# l4 = [ebp - 14]

# l5 = [ebp - 18]

# l6 = [ebp - 1C]

# l7 = [ebp - 20]

# l8 = [ebp - 24]

# l9 = [ebp - 28]

def funct402430(s1, s2):

a = 1

l7 = a

l6 = a

l2 = a

a = 0x00560128

l0 = 0x10

# start loop: eax < 0x561128

while True:

#mov byte ptr ss:[ebp-1],0

l0 = ((l0 << 2) >> 2)

l1 = a

l3 = 8

# start loop: 00402460

while l3 != 0:

# read from [eax] in [eax+4]

b = memory(a)

a = memory(a+4)

l4 = 1

di = 0

l8 = a

temp = 0

for di in range(0,0x40):

a = 1

d = 0

c = di

a,c,d = funct4FA8C0(a,c,d)

si = l8

c = b

c = c & a

si = si & d

if c == si:

t = 0

else:

t = 1

c = c | si

if t != 0:

a = s1

d = s2

if a == d:

t1 = 0

else:

t1 = 1

a = a | d

if t1 == 0:

temp = 1

break

# 004024A1 mov al,byte ptr ss:[ebp-14]

if temp == 0:

a = (a >> 8) << 8

t = (l4 << 24) >> 24

a = a + t

else:

a = 0

print "AAA: "+str(hex(a))

# add byte ptr ss:[ebp-1],al

a1 = ((a << 24) & 0xFFFFFFFF)

l0 = l0 + a1

print "L0: "+str(hex(l0))

l1 = l1 + 8

l3 = l3 - 1

# moved at the end from the start of the loop

a = l1

# end loop: 00402460

# 004024B4: movzx ecx,byte ptr ss:[ebp-1]

c = (l0 >> 24)

# 0x004024BE

t = c

c = c & 0x80000001

if t == 0x80000001:

c = c - 1

c = c | 0xFFFFFFFE

c = c + 1

# 0x004024C5

else:

c = l2

a = 1

d = 0

a,c,d = funct4FA8C0(a,c,d)

print "ABC: "+str(hex(a))+" "+str(hex(c))+" "+str(hex(d))

l7 = l7 | a

l6 = l6 | d

print "LL: "+str(hex(l6))+" "+str(hex(l7))

# 0x004024DE

a = l1

l2 = l2 + 1

# end loop: eax < 0x561128

if a >= 0x00561128:

break

# outside any loop

if l7 == 0x23A0EF32 and l6 == 0x55BDEC8E:

a = 1

else:

a = 0

return a

s1 = 0xFD7C0372

s2 = 0x19473B90

a = funct402430(s1, s2)

print "A: "+str(hex(a))

Memory at Address 0x00560128

The relevant instructions that access the memory outside of the scope of the input arguments, input registers, or stack local variables are as follows:

00402444 . B8 28015600 mov eax,main.00560128

00402463 > 8B18 mov ebx,dword ptr ds:[eax]

00402465 . 8B40 04 mov eax,dword ptr ds:[eax+4]

004024E4 . 3D 28115600 cmp eax,main.00561128

From the above instructions we can see that we initialize the eax register to a value 0x00560128 and continue to increase its value until 0x00561128 after which the loop is terminated. While the loop is still executing, the values being accessed are on the addresses [eax] and [eax+4], which are the addresses in range: 0x00560128 − 0x00561128. This is why we need to introduce a new function that will hold all the static values of the described part of the memory and return the relevant value.

That part of the memory is presented here:

00560128 04 00 00 00 00 00 00 00 91 00 00 00 00 00 00 00

00560138 F9 00 00 00 00 00 00 00 48 00 00 00 00 00 00 00

...

00561118 08 D1 B5 67 81 4B 64 0E E7 90 56 42 A6 FE CD 01

00561128 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

For simplicity's sake it would be best to keep the whole memory region in an array variable that can easily be accessed. The Python program that returns appropriate values of this array as seen in the function at address 0x00402430 is appended below:

arr = [ 0x04,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x91, ... ]

def memory(addr):

index = addr - 0x560128

v = 0x0

v += arr[index+3]

v = v << 8

v += arr[index+2]

v = v << 8

v += arr[index+1]

v = v << 8

v += arr[index]

return v & 0xFFFFFFFF

a = memory(0x560138)

print hex(a)

We input the memory address on a range from 0x560138−0x561138 and the function memory will return a 4-byte value of memory residing at that location.

Input Arguments

The function takes 1 parameter as argument at address [ebp+8]. This can be concluded because of the retn 4 instruction at the end. But the following instruction in the function prologue also indicates that the function takes one parameter:

00401D78 > 8D4424 14 lea eax,dword ptr ss:[esp+14]

00401D7C . 50 push eax

00401D7D . E8 AE060000 call main.00402430

The [esp+14] resolves directly into the address 0x0012EF70 on the stack, which holds the value 0xB7780FE3. The function uses that value like this:

0040248F . 8B4D 08 mov ecx,dword ptr ss:[ebp+8]

00402492 . 2301 and eax,dword ptr ds:[ecx]

00402494 . 2351 04 and edx,dword ptr ds:[ecx+4]

00402497 . 0BC2 or eax,edx

We can see that it loads the address 0x0012EF70 into register ecx and reads the value 0xB7780FE3 from the stack into eax. Then it reads the value 0x3B47E4E1 from the stack address 0x0012EF74 into register edx. Afterwards the operation XOR is applied to registers eax and edx. The values 0xB7780FE3 and 0x3B47E4E1 are different regarding the inputed value Key 2. The above values were computed when we inputed 12 A's into the input field.

Therefore the function takes 1 parameter that constitutes of a 8-byte (64-bit) value accessible at address 0x0012EF70. But how are those two values computed? In the function that starts at address 0x00401C10 we can see that the values are first initialized to some predefined constants:

00401C52 . C74424 14 8077>mov dword ptr ss:[esp+14],B11B7780

00401C5A . C74424 18 71B4>mov dword ptr ss:[esp+18],7A83B471

Afterwards the only place those two values are changed is in those two functions:

00401D67 . E8 54060000 call main.004023C0

00401D6C . E8 9F040000 call main.00402210

Part 9

Function at Address 0x004023C0

The function at address 0x004023C0 has the following code:

004023C0 /$ 55 push ebp

004023C1 |. 8BEC mov ebp,esp

004023C3 |. 83EC 08 sub esp,8

004023C6 |. 53 push ebx

004023C7 |. 33DB xor ebx,ebx

004023C9 |. 57 push edi

004023CA |. BF 01000000 mov edi,1

004023CF |. 895D FC mov dword ptr ss:[ebp-4],ebx

004023D2 |> B8 01000000 /mov eax,1

004023D7 |. 33D2 |xor edx,edx

004023D9 |. 8BCB |mov ecx,ebx

004023DB |. E8 E0840F00 |call main.004FA8C0

004023E0 |. 2345 08 |and eax,dword ptr ss:[ebp+8]

004023E3 |. 2355 FC |and edx,dword ptr ss:[ebp-4]

004023E6 |. 8BCF |mov ecx,edi

004023E8 |. 0BC2 |or eax,edx

004023EA |. B8 01000000 |mov eax,1

004023EF |. 74 0E |je short main.004023FF

004023F1 |. 33D2 |xor edx,edx

004023F3 |. E8 C8840F00 |call main.004FA8C0

004023F8 |. 0906 |or dword ptr ds:[esi],eax

004023FA |. 0956 04 |or dword ptr ds:[esi+4],edx

004023FD |. EB 10 |jmp short main.0040240F

004023FF |> 33D2 |xor edx,edx

00402401 |. E8 BA840F00 |call main.004FA8C0

00402406 |. F7D0 |not eax

00402408 |. 2106 |and dword ptr ds:[esi],eax

0040240A |. F7D2 |not edx

0040240C |. 2156 04 |and dword ptr ds:[esi+4],edx

0040240F |> 83C7 1D |add edi,1D

00402412 |. 81E7 3F000080 |and edi,8000003F

00402418 |. 79 05 |jns short main.0040241F

0040241A |. 4F |dec edi

0040241B |. 83CF C0 |or edi,FFFFFFC0

0040241E |. 47 |inc edi

0040241F |> 43 |inc ebx

00402420 |. 83FB 10 |cmp ebx,10

00402423 |.^7C AD jl short main.004023D2

00402425 |. 5F pop edi

00402426 |. 5B pop ebx

00402427 |. 8BE5 mov esp,ebp

00402429 |. 5D pop ebp

0040242A . C2 0400 retn 4

We can see that this function accepts one parameter in [ebp+8], that is used only in bitwise and in operation at address 0x004023E0. It's also evident that the esi register isn't being initialized anywhere, but is used nevertheless. The esi register contains a value of 0x0012EF70, which points to our [esp+14] and [esp+18] stack address, which are being overwritten by the function.

We've written the following piece of code that essentially does the same as the above assembly piece of code. We're inputing the argument arg as the first and only parameter into the function. But we're also inputting arguments s1 and s2 that represent the [esi] and [esi+4] values accordingly. The code written in Python programming language is as follows:

#!/usr/bin/python

def funct4FA8C0(a,c,d):

#a = 1

#d = 0

if c >= 64:

a = 0

d = 0

return a,c,d

elif c >= 32:

d = a

a = 0

c = c & 31

d = d << c

return a,c,d

else:

t1 = d

t1 = t1 << c

t2 = a

t2 = t2 >> (32-c)

d = t1|t2

a = a << c

return a,c,d

# arg = a value 0x12CC used for bitwise AND; s == pointer to our two stack addresses

def funct4023C0(arg, s1, s2):

b = 0

di = 1

# local var

l = 0

for b in range(0,0x10):

a = 1

d = 0

c = b

# call 004FA8C0

a,c,d = funct4FA8C0(a,c,d)

a = a & (arg & 0xFFFFFFFF)

d = d & (l & 0xFFFFFFFF)

c = di

if a == d:

t = 0

else:

t = 1

a = a | d

a = 1

# je short main.004023FF

if t != 0:

d = 0

a,c,d = funct4FA8C0(a,c,d)

s1 = s1 | a

s2 = s2 | d

else:

d = 0

a,c,d = funct4FA8C0(a,c,d)

a = ~a & 0xFFFFFFFF

s1 = s1 & a

d = ~d & 0xFFFFFFFF

s2 = s2 & d

di = di + 0x1D

t = di

di = di & 0x8000003F

if t == 0x8000003F:

di = di - 1

di = di | 0xFFFFFFC0

di = di + 1

#b = b + 1

return s1,s2

s1 = 0xB11B7780

s2 = 0x7A83B471

arg = 0x12CC

print "Input values : "+str(hex(s1))+" : "+str(hex(s2))

s1,s2 = funct4023C0(arg,s1,s2)

print "Output values: "+str(hex(s1))+" : "+str(hex(s2))

We're loading the initialization values 0xB11B7780 and 0x7A83B471 into the stack values s1 and s2. We're also loading the init value 0x12CC into the arg argument. Then calling the function funct4023C0, which does the same as the function in assembly that resides at address 0x004023C0. We're also printing the values before and after the function call. A successful function call looks like this:

Input values : 0xb11b7780 : 0x7a83b471

Output values: 0xb11b77c0 : 0x7a83b471

We can see that the input values of s1 and s2 were 0xb11b7780 and 0x7a83b471, which were then changed into the function to values 0xb11b77c0 and 0x7a83b471, which is correct. We can verify the correctness of the function with the pictures below. On the picture below, we can see that we're about to execute the function 0x004023C0.

The input argument arg has just been pushed to the stack and contains the value 0x12CC. The important stack values are located at the stack addresses [esp+18] and [esp+1C] and contain our input values 0xb11b7780 and 0x7a83b471.

Right after the function call, our EIP is located on the call instruction to the function 0x00402210 like presented on the picture below:

And the stack memory addresses at [esp+18] and [esp+1C] have been changed into values 0xb11b77c0 and 0x7a83b471. We can see that on the picture below:

Thus we've verified that our function returned the correct values. If we debug the program a little more, we can quickly see that the function returns correct values throughout the application, whenever the function at address 0x004023C0 is called.

Up until now, we've used the constant values we've inputted into our function, but in the program it isn't exactly like that. The used constants 0xB11B7780 and 0x7A83B471 are really constants and are initialized at the addresses 0x00401C52 and 0x00401C5A:

00401C52 . C74424 14 8077>mov dword ptr ss:[esp+14],B11B7780

00401C5A . C74424 18 71B4>mov dword ptr ss:[esp+18],7A83B471

The input argument arg that we've represented with a constant 0x12CC is a different matter. This value isn't represented by some constant, but is calculated dynamically. The code below represents the instructions used to compute the value of arg:

00401D44 > 8D4424 14 lea eax,dword ptr ss:[esp+14]

00401D48 . E8 43050000 call main.00402290

00401D4D . BA 01000000 mov edx,1

00401D52 . 8ACB mov cl,bl

00401D54 . D3E2 shl edx,cl

00401D56 . 85D0 test eax,edx

00401D58 . 74 06 je short main.00401D60

00401D5A . F7D2 not edx

00401D5C . 23C2 and eax,edx

00401D5E . EB 02 jmp short main.00401D62

00401D60 > 0BC2 or eax,edx

00401D62 > 50 push eax

We can see that the stack address of [esp+14] is loaded into register eax and the function at address 0x00402290 is called. Let's take a look what the function at address 0x00402290 actually does.

Function at Address 0x00402290

This function is represented by the code below:

00402290 /$ 55 push ebp

00402291 |. 8BEC mov ebp,esp

00402293 |. 83EC 08 sub esp,8

00402296 |. 53 push ebx

00402297 |. 8B58 04 mov ebx,dword ptr ds:[eax+4]

0040229A |. 56 push esi

0040229B |. 57 push edi

0040229C |. 8B38 mov edi,dword ptr ds:[eax]

0040229E |. C745 FC 000000>mov dword ptr ss:[ebp-4],0

004022A5 |. BE 01000000 mov esi,1

004022AA |. C745 F8 020000>mov dword ptr ss:[ebp-8],2

004022B1 |> B8 01000000 /mov eax,1

004022B6 |. 33D2 |xor edx,edx

004022B8 |. 8BCE |mov ecx,esi

004022BA |. E8 01860F00 |call main.004FA8C0

004022BF |. 23C7 |and eax,edi

004022C1 |. 23D3 |and edx,ebx

004022C3 |. 0BC2 |or eax,edx

004022C5 |. 74 15 |je short main.004022DC

004022C7 |. 8B4D F8 |mov ecx,dword ptr ss:[ebp-8]

004022CA |. 83C1 FE |add ecx,-2

004022CD |. B8 01000000 |mov eax,1

004022D2 |. 33D2 |xor edx,edx

004022D4 |. E8 E7850F00 |call main.004FA8C0

004022D9 |. 0945 FC |or dword ptr ss:[ebp-4],eax

004022DC |> 83C6 1D |add esi,1D

004022DF |. 81E6 3F000080 |and esi,8000003F

004022E5 |. 79 05 |jns short main.004022EC

004022E7 |. 4E |dec esi

004022E8 |. 83CE C0 |or esi,FFFFFFC0

004022EB |. 46 |inc esi

004022EC |> B8 01000000 |mov eax,1

004022F1 |. 33D2 |xor edx,edx

004022F3 |. 8BCE |mov ecx,esi

004022F5 |. E8 C6850F00 |call main.004FA8C0

004022FA |. 23C7 |and eax,edi

004022FC |. 23D3 |and edx,ebx

004022FE |. 0BC2 |or eax,edx

00402300 |. 74 13 |je short main.00402315

00402302 |. 8B4D F8 |mov ecx,dword ptr ss:[ebp-8]

00402305 |. 49 |dec ecx

00402306 |. B8 01000000 |mov eax,1

0040230B |. 33D2 |xor edx,edx

0040230D |. E8 AE850F00 |call main.004FA8C0

00402312 |. 0945 FC |or dword ptr ss:[ebp-4],eax

00402315 |> 83C6 1D |add esi,1D

00402318 |. 81E6 3F000080 |and esi,8000003F

0040231E |. 79 05 |jns short main.00402325

00402320 |. 4E |dec esi

00402321 |. 83CE C0 |or esi,FFFFFFC0

00402324 |. 46 |inc esi

00402325 |> B8 01000000 |mov eax,1

0040232A |. 33D2 |xor edx,edx

0040232C |. 8BCE |mov ecx,esi

0040232E |. E8 8D850F00 |call main.004FA8C0

00402333 |. 23C7 |and eax,edi

00402335 |. 23D3 |and edx,ebx

00402337 |. 0BC2 |or eax,edx

00402339 |. 74 12 |je short main.0040234D

0040233B |. 8B4D F8 |mov ecx,dword ptr ss:[ebp-8]

0040233E |. B8 01000000 |mov eax,1

00402343 |. 33D2 |xor edx,edx

00402345 |. E8 76850F00 |call main.004FA8C0

0040234A |. 0945 FC |or dword ptr ss:[ebp-4],eax

0040234D |> 83C6 1D |add esi,1D

00402350 |. 81E6 3F000080 |and esi,8000003F

00402356 |. 79 05 |jns short main.0040235D

00402358 |. 4E |dec esi

00402359 |. 83CE C0 |or esi,FFFFFFC0

0040235C |. 46 |inc esi

0040235D |> B8 01000000 |mov eax,1

00402362 |. 33D2 |xor edx,edx

00402364 |. 8BCE |mov ecx,esi

00402366 |. E8 55850F00 |call main.004FA8C0

0040236B |. 23C7 |and eax,edi

0040236D |. 23D3 |and edx,ebx

0040236F |. 0BC2 |or eax,edx

00402371 |. 74 13 |je short main.00402386

00402373 |. 8B4D F8 |mov ecx,dword ptr ss:[ebp-8]

00402376 |. 41 |inc ecx

00402377 |. B8 01000000 |mov eax,1

0040237C |. 33D2 |xor edx,edx

0040237E |. E8 3D850F00 |call main.004FA8C0

00402383 |. 0945 FC |or dword ptr ss:[ebp-4],eax

00402386 |> 83C6 1D |add esi,1D

00402389 |. 81E6 3F000080 |and esi,8000003F

0040238F |. 79 05 |jns short main.00402396

00402391 |. 4E |dec esi

00402392 |. 83CE C0 |or esi,FFFFFFC0

00402395 |. 46 |inc esi

00402396 |> 8B45 F8 |mov eax,dword ptr ss:[ebp-8]

00402399 |. 83C0 04 |add eax,4

0040239C |. 8945 F8 |mov dword ptr ss:[ebp-8],eax

0040239F |. 83C0 FE |add eax,-2

004023A2 |. 83F8 10 |cmp eax,10

004023A5 |.^0F8C 06FFFFFF jl main.004022B1

004023AB |. 8B45 FC mov eax,dword ptr ss:[ebp-4]

004023AE |. 5F pop edi

004023AF |. 5E pop esi

004023B0 |. 5B pop ebx

004023B1 |. 8BE5 mov esp,ebp

004023B3 |. 5D pop ebp

004023B4 . C3 retn

The function again takes care of its own stack frame. It takes register eax as input value. Register eax contains the stack address 0x0012EF70, which points to our value 0xB11B7780 on the stack. The eax register is used to reference the values at stack addresses [esp+14] and [esp+18]. The function also reserves space for two local variables, which lie at l0 = [ebp−4] and l1 = [ebp−8].

The whole function can be reprogrammed into the following high-level python code:

def funct402290(s1,s2):

b = s2

di = s1

l0 = 0

si = 1

l1 = 2

a = 0

while(a < 0x10):

a = 1

d = 0

c = si

a,c,d = funct4FA8C0(a,c,d)

a = a & di

d = d & b

if a == d:

t = 0

else:

t = 1

a = a | d

if t != 0:

c = l1

c = c - 2

a = 1

d = 0

a,c,d = funct4FA8C0(a,c,d)

l0 = l0 | a

si = si + 0x1D

t = si

si = si & 0x8000003F

if t == 0x8000003F:

si = si - 1

si = si | 0xFFFFFFC0

si = si + 1

a = 1

d = 0

c = si

a,c,d = funct4FA8C0(a,c,d)

a = a & di

d = d & b

if a == d:

t = 0

else:

t = 1

a = a | d

if t != 0:

c = l1

c = c - 1

a = 1

d = 0

a,c,d = funct4FA8C0(a,c,d)

l0 = l0 | a

si = si + 0x1D

t = si

si = si & 0x8000003F

if t == 0x8000003F:

si = si - 1

si = si | 0xFFFFFFC0

si = si + 1

a = 1

d = 0

c = si

a,c,d = funct4FA8C0(a,c,d)

a = a & di

d = d & b

if a == d:

t = 0

else:

t = 1

a = a | d

if t != 0:

c = l1

a = 1

d = 0

a,c,d = funct4FA8C0(a,c,d)

l0 = l0 | a

si = si + 0x1D

t = si

si = si & 0x8000003F

if t == 0x8000003F:

si = si - 1

si = si | 0xFFFFFFC0

si = si + 1

a = 1

d = 0

c = si

a,c,d = funct4FA8C0(a,c,d)

a = a & di

d = d & b

if a == d:

t = 0

else:

t = 1

a = a | d

if t != 0:

c = l1

c = c + 1

a = 1

d = 0

a,c,d = funct4FA8C0(a,c,d)

l0 = l0 | a

si = si + 0x1D

t = si

si = si & 0x8000003F

if t == 0x8000003F:

si = si - 1

si = si | 0xFFFFFFC0

si = si + 1

a = l1

a = a + 4

l1 = a

a = a - 2

a = l0

return a,c,d

The function takes stack values as input arguments, and computes the corresponding registers eax, ecx and edx. The value in the register eax is later used as an input into the function residing at address 0x004023C0.

We won't go into too much detail what the function actually does, since the Python code should be fairly readable; it does something mathematically expensive and complicated.

Part 10

The function at address 0x00402210 has the following code:

00402210 /$ 55 push ebp

00402211 |. 8BEC mov ebp,esp

00402213 |. 83EC 10 sub esp,10

00402216 |. 53 push ebx

00402217 |. 8B5E 04 mov ebx,dword ptr ds:[esi+4]

0040221A |. 57 push edi

0040221B |. 8B3E mov edi,dword ptr ds:[esi]

0040221D |. 897D F0 mov dword ptr ss:[ebp-10],edi

00402220 |. 895D F4 mov dword ptr ss:[ebp-C],ebx

00402223 |. 81E7 D13E72FB and edi,FB723ED1

00402229 |. 81E3 45C71A62 and ebx,621AC745

0040222F |. 33C0 xor eax,eax

00402231 |. 8945 F8 mov dword ptr ss:[ebp-8],eax

00402234 |. 8945 FC mov dword ptr ss:[ebp-4],eax

00402237 |> 8B4D FC /mov ecx,dword ptr ss:[ebp-4]

0040223A |. B8 01000000 |mov eax,1

0040223F |. 33D2 |xor edx,edx

00402241 |. E8 7A860F00 |call main.004FA8C0

00402246 |. 23C7 |and eax,edi

00402248 |. 23D3 |and edx,ebx

0040224A |. 0BC2 |or eax,edx

0040224C |. 74 03 |je short main.00402251

0040224E |. FF45 F8 |inc dword ptr ss:[ebp-8]

00402251 |> 8B45 FC |mov eax,dword ptr ss:[ebp-4]

00402254 |. 40 |inc eax

00402255 |. 8945 FC |mov dword ptr ss:[ebp-4],eax

00402258 |. 83F8 40 |cmp eax,40

0040225B |.^7C DA jl short main.00402237

0040225D |. 8B45 F0 mov eax,dword ptr ss:[ebp-10]

00402260 |. 8B4D F4 mov ecx,dword ptr ss:[ebp-C]

00402263 |. 0FA4C1 01 shld ecx,eax,1

00402267 |. 03C0 add eax,eax

00402269 |. 8906 mov dword ptr ds:[esi],eax

0040226B |. 8A45 F8 mov al,byte ptr ss:[ebp-8]

0040226E |. 24 01 and al,1

00402270 |. 0FB6C0 movzx eax,al

00402273 |. 99 cdq

00402274 |. 0906 or dword ptr ds:[esi],eax

00402276 |. 5F pop edi

00402277 |. 894E 04 mov dword ptr ds:[esi+4],ecx

0040227A |. 0956 04 or dword ptr ds:[esi+4],edx

0040227D |. 5B pop ebx

0040227E |. 8BE5 mov esp,ebp

00402280 |. 5D pop ebp

00402281 . C3 retn

We can immediately see that the function is talking care of its own stack frame all by itself. Apparently it doesn't take any input arguments, but uses the esi register that is inputted from an outer function. The esi register contains a value of 0x0012EF70, which points to our [esp+14] and [esp+18] stack address and changes them accordingly. From the instruction sub esp,10, it's evident that the function uses four local variables. Since the constant 0x10 in hexadecimal can be represented by 16 in decimal form, we can gather the function reserved 16 bytes for its use. Local variables can then be represented by variables l0, where l0 = [ebp−4], l1, where l1 = [ebp−8], l2, where l2 = [ebp−C] and l3, where l3 = [ebp−C].

We've written a python representation of the code that does exactly the same as the code above:

# Input arguments s1,a2

# Local arguments: l1 = [ebp-8], l2 = [ebp-C], l3 = [ebp-C]

def funct402210(s1,s2):

b = s2

di = s1

l3 = di

l2 = b

di = di & 0xFB723ED1

b = b & 0x621AC745

a = 0

l1 = a

l0 = a

# the loop

while(a < 0x40):

c = l0

a = 1

d = 0

a,c,d = funct4FA8C0(a,c,d)

a = a & di

d = d & b

if a == d:

t = 0

else:

t = 1

a = a | d

if t != 0:

l1 = l1 + 1

a = l0

a = a + 1

l0 = a

# after the loop

a = l3

c = l2

# shld ecx,eax,1

t1 = c

t1 = t1 << 1

t2 = a

t2 = t2 >> 31

c = t1|t2

a = a + a

s1 = a

# mov al,byte ptr ss:[ebp-8]

# and al,1

# movzx eax,al

# cdq

al = l1

al = al & 0x1

a = 0x00000011

a = a & al

s1 = s1 | a

s2 = c

s2 = s2 | d

# return only 32-bits, discard the overflow

#return (s1 % 2**32),s2

return s1 & ((1<<32)-1), s2 & ((1<<32)-1)

s1 = 0xb11b77c0

s2 = 0x7a83b471

print "Input values: "+str(hex(s1))+" : "+str(hex(s2))

s1,s2 = funct402210(s1,s2)

print "Output values: "+str(hex(s1))+" : "+str(hex(s2))

When we run the program, we get the output below:

Input values: 0xb11b77c0 : 0x7a83b471

Output values: 0x6236ef80 : 0xf50768e3

The two values 0xb11b77c0 and 0x7a83b471 being inputted into the function are exactly the values that the function at address 0x004FA8C0 has calculated. To verify that the values are correct, let's take a look at the stack when the function is done executing. First we must ensure that we set a breakpoint on the instruction after the function call, like we can see in the picture below:

The corresponding stack is as follows:

The important values lie at the stack addresses [esp+14] and [esp+18], which hold the values 0x6236ef80 and 0xf50768e3, which are also the same values that we've gotten back from our function. Therefore, our function must calculate the values correctly.

Complete Python Program

Complete Python program can be written with the following Python code (where the functions that were already presented in the previous article are accessible):

s1 = 0xB11B7780

s2 = 0x7A83B471

strr = "111111111111"

s1, s2 = funct(strr, s1, s2)

print "Input: "+strr+" | Output: ["+str(hex(s1))+" , "+str(hex(s2))+"]"

Appropriate functions are named after their addresses in the crackme.exe program, so there is no confusion. The function named funct is the actual code that takes the Key 2 input value and uses it in computing the values at stack addresses [esp+18] and [esp+1C]. All the other functions are just helper function used to carry out the dirty work for the main function.

The Inner Loop

The function 0x00402430 calls this function, which contains the following instructions and effectively sets the registers eax and edx that are later saved into appropriate values [ebp-20] and [ebp-1C]:

004FA8C0 |$ 80F9 40 cmp cl,40

004FA8C3 |. 73 15 jnb short main.004FA8DA

004FA8C5 |. 80F9 20 cmp cl,20

004FA8C8 |. 73 06 jnb short main.004FA8D0

004FA8CA |. 0FA5C2 shld edx,eax,cl

004FA8CD |. D3E0 shl eax,cl

004FA8CF |. C3 retn

004FA8D0 |> 8BD0 mov edx,eax

004FA8D2 |. 33C0 xor eax,eax

004FA8D4 |. 80E1 1F and cl,1F

004FA8D7 |. D3E2 shl edx,cl

004FA8D9 |. C3 retn

004FA8DA |> 33C0 xor eax,eax

004FA8DC |. 33D2 xor edx,edx

004FA8DE . C3 retn

This can be represented by the following code:

void function(char cl) {

if(cl >= '@') {

eax = 0

edx = 0

return

}

else if(cl >= ' ') {

edx = eax

eax = 0

and cl,00011111

shl edx,cl

return

}

else {

shld edx,eax,cl

shl eax,cl

return

}

}

The function uses eax, ecx and edx registers. It uses ecx register to determine what code block is being executed, and it sets the eax and edx registers effectively. The eax register inputted into the function always contains the value 1 and the edx register is always 0. And ecx register contains values from 0-63.

This is why we can write a program in python that can compute the same values as the function above. The python code for this function can be something like this:

#!/usr/bin/python

def funct(c):

a = 1

d = 0

if c >= 64:

a = 0

d = 0

return a,c,d

elif c >= 32:

d = a

a = 0

c = c & 31

d = d << c

return a,c,d

else:

t1 = d

t1 = t1 << c

t2 = a

t2 = t2 >> (32-c)

d = t1^t2

a = a << c

return a,c,d

print "-------------------"

print "COUNT: |EAX|ECX|EDX|"

for c in range(0,64):

a,b,d = funct(c)

print str(c)+": |"+str(hex(a)[2:])+"|"+str(hex(b)[2:])+"|"+str(hex(d)[2:])+"|"

print "-------------------"

When we run the program, we get the following output, which prints exact values as the assembly program above:

-------------------

COUNT: |EAX|ECX|EDX|

0: |1|0|0|

1: |2|1|0|

2: |4|2|0|

3: |8|3|0|

4: |10|4|0|

5: |20|5|0|

6: |40|6|0|

7: |80|7|0|

8: |100|8|0|

9: |200|9|0|

10: |400|a|0|

11: |800|b|0|

12: |1000|c|0|

13: |2000|d|0|

14: |4000|e|0|

15: |8000|f|0|

16: |10000|10|0|

17: |20000|11|0|

18: |40000|12|0|

19: |80000|13|0|

20: |100000|14|0|

21: |200000|15|0|

22: |400000|16|0|

23: |800000|17|0|

24: |1000000|18|0|

25: |2000000|19|0|

26: |4000000|1a|0|

27: |8000000|1b|0|

28: |10000000|1c|0|

29: |20000000|1d|0|

30: |40000000|1e|0|

31: |80000000|1f|0|

32: |0|0|1|

33: |0|1|2|

34: |0|2|4|

35: |0|3|8|

36: |0|4|10|

37: |0|5|20|

38: |0|6|40|

39: |0|7|80|

40: |0|8|100|

41: |0|9|200|

42: |0|a|400|

43: |0|b|800|

44: |0|c|1000|

45: |0|d|2000|

46: |0|e|4000|

47: |0|f|8000|

48: |0|10|10000|

49: |0|11|20000|

50: |0|12|40000|

51: |0|13|80000|

52: |0|14|100000|

53: |0|15|200000|

54: |0|16|400000|

55: |0|17|800000|

56: |0|18|1000000|

57: |0|19|2000000|

58: |0|1a|4000000|

59: |0|1b|8000000|

60: |0|1c|10000000|

61: |0|1d|20000000|

62: |0|1e|40000000|

63: |0|1f|80000000|

-------------------

We could use better formatting rules to make the output prettier, but the only thing that's important is that it works. In the first column we can see the ECX being increased from 0 to 63.Loop

The code that calculates the right stack values and uses all above functions is as follows:

00401D26 > 8A5C3C 24 mov bl,byte ptr ss:[esp+edi+24]

00401D2A . 8D4B BF lea ecx,dword ptr ds:[ebx-41]

00401D2D . 80F9 05 cmp cl,5

00401D30 . 77 05 ja short main.00401D37

00401D32 . 80EB 37 sub bl,37

00401D35 . EB 0D jmp short main.00401D44

00401D37 > 8D43 D0 lea eax,dword ptr ds:[ebx-30]

00401D3A . B2 09 mov dl,9

00401D3C . 3AD0 cmp dl,al

00401D3E . 1ADB sbb bl,bl

00401D40 . F6D3 not bl

00401D42 . 22D8 and bl,al

00401D44 > 8D4424 14 lea eax,dword ptr ss:[esp+14]

00401D48 . E8 43050000 call main.00402290

00401D4D . BA 01000000 mov edx,1

00401D52 . 8ACB mov cl,bl

00401D54 . D3E2 shl edx,cl

00401D56 . 85D0 test eax,edx

00401D58 . 74 06 je short main.00401D60

00401D5A . F7D2 not edx

00401D5C . 23C2 and eax,edx

00401D5E . EB 02 jmp short main.00401D62

00401D60 > 0BC2 or eax,edx

00401D62 > 50 push eax

00401D63 . 8D7424 18 lea esi,dword ptr ss:[esp+18]

00401D67 . E8 54060000 call main.004023C0

00401D6C . E8 9F040000 call main.00402210

00401D71 . 47 inc edi

00401D72 . 3B7C24 20 cmp edi,dword ptr ss:[esp+20]

00401D76 .^72 AE jb short main.00401D26

We can see that the loop is repeated 0xC times, which is exactly the length of the Key 2 value. The edi register is used for iterating from value 0 to 12. At the end of the loop we're comparing edi to a value that is stored on the stack address [esp+20], which holds the length of the Key 2 - the value 0xC.

The first instruction is used to iterate over the input value:

00401D26 > 8A5C3C 24 mov bl,byte ptr ss:[esp+edi+24]

The stack address at [esp+24] contains exactly 12 characters of the input key. The loop counter is used to iterate over certain values of the input key, therefore we're actually accessing values from [esp+24] to [esp+2F]. If we enter the key that contains exactly 12 A's into the Key 2 input field, the stack will look like the picture below:

Therefore we load the value of 0x41 into the bl register at the start of each iteration in a loop. Then we're subtracting the value of 0x41 from each of the input characters and storing the result in register ecx:

00401D2A . 8D4B BF lea ecx,dword ptr ds:[ebx-41]

If the remained is greater than 0x5, the jump is taken. But since the input characters can only be in ranges from 0x30−0x39 and 0x41−0x46, the jump is taken on the input characters 0x30−0x39, since the subtraction gives us a negative number. When the character is from a range 0x41−0x46, only the sub bl,37 instruction is executed.

5. Bruteforcing the Values: Program Itself

I guess the first impression is that we can reverse engineer the entire logic of the function, instruction by instruction, but that would take considerably too much time and effort. Rather than doing that, we can bruteforce the right values the function is using as input. The function doesn't use traditional arguments being passed into the function; the arguments are not pushed onto the stack, which the function then reads out and uses in its computation logic. That is why we must determine what 'input' arguments the function does use. To do that, we can change various values on the stack right before the function is called on the address 0x00401D7D:

00401D7D . E8 AE060000 call main_cal.00402430

We should also set a breakpoint on the following line in the function itself:

004024EF . 817D E0 32EFA0>cmp dword ptr ss:[ebp-20],23A0EF32

This should enable us to observe if the value at the address ebp-20 on the stack actually gets changed when we change arbitrary stack values before the function is called. If the value at address ebp-20 is changed, it should give us a clear indication that the value we changed and ebp-20 are correlated, revealing the input argument. The input arguments are located at values esp+14 and esp+18 before the function is called (and right before the push eax instruction). Therefore we change the code around the function call to repeatedly call the function with those two values being increased in each iteration. The code could look like this:

00401D5E . 33C0 xor eax,eax

00401D63 83F8 00 cmp eax,0

00401D66 75 2A jnz short main.00401D92

00401D68 834424 14 01 add dword ptr ss:[esp+14],1

00401D6D 90 nop

00401D6E 90 nop

00401D6F 90 nop

00401D70 90 nop

00401D71 . 47 inc edi

00401D72 83F8 00 cmp eax,0

00401D75 90 nop

00401D76 90 nop

00401D77 90 nop

00401D78 8D4424 14 lea eax,dword ptr ss:[esp+14]

00401D7C 50 push eax

00401D7D . E8 AE060000 call main.00402430

00401D82 8B5C24 14 mov ebx,dword ptr ss:[esp+14]

00401D86 83FB FF cmp ebx,-1

00401D89 ^75 D8 jnz short main.00401D63

00401D8B 834424 18 01 add dword ptr ss:[esp+18],1

00401D90 C74424 14 0000>mov dword ptr ss:[esp+14],0

00401D98 90 nop

00401D99 ^EB C8 jmp short main.00401D63

00401D9B 90 nop

00401D9C 90 nop

00401D9D 90 nop

00401D9E F4 hlt

00401D9F 90 nop

The beginning xor eax,eax is important to set the eax to 0 when we start with the loop, otherwise we're immediately finishing our loop, since the eax is not initialized to zero when the loop starts.

We're also setting the values at esp+14 and esp+18 at zero on the beginning. Then the loop should add 1 to the value stored at that address effectively bruteforcing the values that would return eax, which would not equal to 0. This is a nested loop whereas the value at esp+18 is changed first from 0x00000000 to 0xFFFFFFFF, then set to zero back again. When the eax which is not equal to zero is returned, the loop should jump to hlt instruction, which should effectively stop the program execution. Then we could loop at the stack values esp+18 and esp+14 to see which values where used to invoke the right function return. We should also mention that this will take quite a long time to complete, so it's probably not the best solution - a way better solution would be to actually try to reverse engineer the mathematical algorithm used by the function itself.

But we can try to fasten the bruteforce by trying to use just one loop at a time, thus making the two values on the address esp+18 and esp+14 not dependent upon each other. This should decrease the input space dramatically, but is prone to errors - what if the two values calculated by the function are dependent upon each other? Then of course, this won't work. First we must fill the following two lines with nops to actually compare just the first value:

004024F8 . 817D E4 8EECBD>cmp dword ptr ss:[ebp-1C],55BDEC8E

004024FF . 75 0E jnz short main.0040250F

The first loop should be around in no time - it should take a couple of hours on a decent machine. This really isn't that long compared to what it would take if a nested loop would be used. After that we need to use the adapted version of a loop, like the assembly code below:

00401D60 83F8 00 cmp eax,0

00401D63 75 1F jnz short main.00401D84

00401D65 834424 14 01 add dword ptr ss:[esp+14],1

00401D6A 90 nop

00401D6B 90 nop

00401D6C 90 nop

00401D6D 90 nop

00401D6E 90 nop

00401D6F 90 nop

00401D70 90 nop

00401D71 90 nop

00401D72 90 nop

00401D73 90 nop

00401D74 90 nop

00401D75 90 nop

00401D76 90 nop

00401D77 90 nop

00401D78 > 8D4424 14 lea eax,dword ptr ss:[esp+14]

00401D7C . 50 push eax

00401D7D . E8 AE060000 call main.00402430

00401D82 ^EB DC jmp short main.00401D60

00401D84 F4 hlt

00401D85 90 nop

After the function returns a value 0x1 in register eax, the following code gets executed:

00401D86 . 85C0 test eax,eax

00401D88 74 07 je short main.00401D91

00401D8A . 68 201A5400 push main.00541A20

00401D8F . EB 0F jmp short main.00401DA0

00401D91 > 68 481A5400 push main.00541A48

00401D96 . EB 08 jmp short main.00401DA0

First, the test instruction on register eax is used to set the right flags in the eflags register. If the eax register contains a value 0, the first jump is taken, giving us the failure message. But if the eax contains a value 1, the second jump is taken, which displays the congratulations window, which we can see in the picture below:

To actually solve the challenge, we can use above Python program that bruteforces the right values on the stack that will cause the function to return the value 1 in register eax after it is called.

The values [esp+18] and [esp+1C] that force the function 0x00402430 to return the value 1 in register eax are the following:

ESP+18 > FD7C0372

ESP+1C > 19473B90

The bruteforcing of the values should be fairly long process, but at the end it's worth it. We've used the following program to bruteforce the actual values:

s1 = 0xB11B7780

s2 = 0x7A83B471

for item in itertools.product("123456789ABCDEF", repeat=12):

v1,v2 = funct(item, s1, s2)

if v1 == 0xfd7c0372 and v2 == 0x19473b90:

print "Value: "+item+" "+str(hex(v1))+"-"+str(hex(v2))

exit(1)

After the program finishes execution, it will find the input value that solves the secondary challenge.

We've looked at what the ESET NOD32 have prepared for us this year. We can see that the challenge was not as innocent as it may seem at a first glance, but we managed to describe it thoroughly. We can see that we didn't actually provide a program that would solve the first challenge, since we don't have the time just now, but we've presented the program that can be used to bruteforce the second challenge. We could have applied some intense mathematical logic to solve the secondary challenge, but that is outside of the reverse engineering scope - it should be for people with good mathematical background.

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

I think that ESET should not apply such intense mathematics into their challenges, because the challenge is to reverse engineer the program and not to solve its mathematical predicates. There should be less mathematics involved and more anti-reversing techniques that would complicate the reversing process considerably.