Analysis of a 64-bit Windows PE executable, tracing back step by step from the entry point to understand how the program processes user input, using assembly language, the internal workings of PE sections, and deliberately discreet validation logic, revealing a simple but clever mechanism typical of small reverse engineering challenges.

Windows PE Reverse Engineering – Crackme #1

Infos:

  • Executable name: crack_me.exe
  • Format: PE32+
  • Architecture: x86-64 Windows
  • Sections: 3 sections
  • Executable: Console

Techniques/Concepts used

  • RIP-relative addressing to access data without an absolute address
  • Data stored directly in the .text section (inline data)
  • Mapping table indexed via a bitwise mask (AND 0x7)
  • Verification loop combining user input and an internal table

I use PE-bear, which analyzes files in PE (Portable Executable) format.

  • RCX, RDX, R8, R9 -> function arguments
  • RAX -> return value
  • The stack (rsp) is used for local buffers

Note that the three sections are clearly present: .text, .pdata, and .idata. The binary’s entry point (EP) is located at offset 0x410 in the .text section.

Screen

Here is the main part starting from offset 0x410 because that is where our entry point is located.

Screen

We will examine what the different function calls correspond to in order to better understand the program.

call 0x4004eb corresponds to a call to the puts function. For the second call 0x400493, PE-bear sends me to address 0x400493, which contains the following code:

400493:	56                   	push   rsi
400494:	57                   	push   rdi
400495:	53                   	push   rbx
400496:	48 83 ec 30          	sub    rsp,0x30
40049a:	48 89 ce             	mov    rsi,rcx
40049d:	48 8d 5c 24 58       	lea    rbx,[rsp+0x58]
4004a2:	48 89 13             	mov    QWORD PTR [rbx],rdx
4004a5:	4c 89 43 08          	mov    QWORD PTR [rbx+0x8],r8
4004a9:	4c 89 4b 10          	mov    QWORD PTR [rbx+0x10],r9
4004ad:	48 89 5c 24 28       	mov    QWORD PTR [rsp+0x28],rbx
4004b2:	b9 01 00 00 00       	mov    ecx,0x1
4004b7:	e8 3b 00 00 00       	call   0x4004f7

What interests us most are the push at the beginning and the sub rsp,0x30, which corresponds to the prologue of a function, called the calling convention on Windows x64. Then, at the end, we see a mov ecx,0x1 and a call to the api-ms-win-crt-stdio-l1-1-0.dll DLL and the __acrt_iob_func function. The __acrt_iob_func function returns the standard FILE* stdin, stdout, and stderr. So the mov ecx,0x1 allows us to understand that it returns to stdout. Then for the third call 0x4004f1, we see that it calls the function gets_s.

So we know that the beginning of the user input corresponds to lea rcx,[rsp+0x20] and then there is a mov edx,0x80, so the call to gets_s is done like this gets_s(buffer, 128) because 0x80 corresponds to 128.

With objdump we see:

40044b:	4c 8d 05 10 fe ff ff 	lea    r8,[rip+0xfffffffffffffe10]        # 0x400262

rip+0xfffffffffffffe10 corresponds to a negative offset, but objdump gives me the address of the loaded strings, which is 0x400262, which I can confirm via PE-bear. If we convert this, we get 0xfffffffffffffe10 = -0x1F0 = -496, so rip - 496. Screen So the strings loaded in r8 correspond to BGOTHXIY.

The part that interests us most in finding the password is this one:

400452:	8a 54 04 20          	mov    dl,BYTE PTR [rsp+rax*1+0x20]
400456:	0f b6 0c 06          	movzx  ecx,BYTE PTR [rsi+rax*1]
40045a:	83 e1 07             	and    ecx,0x7
40045d:	42 3a 14 01          	cmp    dl,BYTE PTR [rcx+r8*1]
400461:	75 10                	jne    0x400473
400463:	48 ff c0             	inc    rax
400466:	48 83 f8 0d          	cmp    rax,0xd
40046a:	75 e6                	jne    0x400452
40046c:	80 7c 24 2d 00       	cmp    BYTE PTR [rsp+0x2d],0x0
400471:	74 17                	je     0x40048a
  • rax is the index for traversing all strings.
  • mov dl,BYTE PTR [rsp+rax*1+0x20] stores each character of the user input in the dl register.
  • movzx ecx,BYTE PTR [rsi+rax*1] one byte from the program’s secret table
  • and ecx,0x7 performs a logical & between ecx and 0x7.
  • cmp dl,BYTE PTR [rcx+r8*1] compares the password character with a character in the string BGOTHXIY at index ecx,0x7.
  • jne 0x400473 if it is not equal, we jump to 0x400473 which exits to -> Don’t think you have the slightest clue about debugging.
  • inc rax increments the index rax.
  • cmp rax,0xd compares the index rax with 0xd, which is 13 in decimal.
  • jne 0x400452 if rax is not equal to 0xd, it goes back to the beginning of the loop, which confirms that the secret table must be 13 bytes long.
  • cmp BYTE PTR [rsp+0x2d],0x0 compares the last bytes of the user input with a \0, 0x2d = 45 and 0x20 + 0xd make 0x2d. Let’s not forget that RSP + 0x20 is the start of the user input.

Let’s go back to searching for the secret table. Using movzx ecx,BYTE PTR [rsi+rax*1], we can find it. It’s at the beginning of rsi, so let’s go back to the entry point after push rsi. We have a lea rax,[rip+0xfffffffffffffff9], so it loads the address rip, which is the address of the next instruction, i.e. 40041f + the offset 0xfffffffffffffff9, which is -7 in decimal. So 0x40041f - 0x7 = 0x400418, so it loads its own address via lea.

The table therefore starts at address 0x400418. 48 8d 05 f9 ff ff ff 48 89 c6 48 8d 0d

Now we need to write some code that calculates the secret for us.

#include <stdio.h>
#include <stdint.h>

int main(void)
{
    char str[] = "BGOTHXIY";
    uint8_t tab[] = { 0x48, 0x8d, 0x05, 0xf9, 0xff, 0xff, 0xff, 0x48, 0x89, 0xc6, 0x48, 0x8d, 0x0d };

    int size = 13;
    char password[14];

    for (int i = 0; i < size; i++)
    {
        int idx = tab[i] & 0x07;
        password[i] = str[idx];
    }
    password[size] = '\0';

    printf("Password = %s\n", password);
    return (0);
}

Screen