x86 Function Call Conventions

Hi Fellow,

At this bloc article I am going to write about the difference call standard in x86 assembly.

#include <stdio.h>

int test(int first, int second, int third) {

    printf("FUnction Called");
    return first+second+third;
}

int main()
{
    int result = test(1, 2, 3);
    printf("Rio Asmara Suryadi %d", result);
}

; Attributes: bp-based frame

; int __cdecl main(int argc, const char **argv, const char **envp)
_main proc near

var_CC= byte ptr -0CCh
var_8= dword ptr -8
argc= dword ptr  8
argv= dword ptr  0Ch
envp= dword ptr  10h

push    ebp
mov     ebp, esp
sub     esp, 0CCh
push    ebx
push    esi
push    edi
lea     edi, [ebp+var_CC]
mov     ecx, 33h
mov     eax, 0CCCCCCCCh
rep stosd
mov     ecx, offset unk_41C003
call    j_@__CheckForDebuggerJustMyCode@4 ; __CheckForDebuggerJustMyCode(x)
push    3               ; int
push    2               ; int
push    1               ; int
call    test
add     esp, 0Ch
mov     [ebp+var_8], eax
mov     eax, [ebp+var_8]
push    eax
push    offset aRioAsmaraSurya ; "Rio Asmara Suryadi %d"
call    j__printf
add     esp, 8
xor     eax, eax
pop     edi
pop     esi
pop     ebx
add     esp, 0CCh
cmp     ebp, esp
call    j___RTC_CheckEsp
mov     esp, ebp
pop     ebp
retn
_main endp

1. cdecl

As we can see above that above is using cdecl standard because we can see that the caller handle the stack clear by running this instruction add esp, 0Ch. cdecl function also push the parameter variable from right to left

C codes
int result = test(1, 2, 3);

x86 assembly codes
push    3               ; int
push    2               ; int
push    1               ; int
call    test
add     esp, 0Ch

2. stdcal

the function call with stdcal would be similar to cdecl but the stack clear will be handled by the stack itself.

3. fastcall

The fastcall calling convention will have a bit different. The first few arguments (typically two) are passed in registers with most commonly used registers being EDX and ECX where the rest of argument will be pushed from right to left. The calling function will do the stack clear.

.LC0:
        .string "FUnction Called"
test(int, int, int):
        push    rbp
        mov     rbp, rsp
        sub     rsp, 16
        mov     DWORD PTR [rbp-4], edi
        mov     DWORD PTR [rbp-8], esi
        mov     DWORD PTR [rbp-12], edx
        mov     edi, OFFSET FLAT:.LC0
        mov     eax, 0
        call    printf
        mov     edx, DWORD PTR [rbp-4]
        mov     eax, DWORD PTR [rbp-8]
        add     edx, eax
        mov     eax, DWORD PTR [rbp-12]
        add     eax, edx
        leave
        ret
.LC1:
        .string "Rio Asmara Suryadi %d"
main:
        push    rbp
        mov     rbp, rsp
        sub     rsp, 16
        mov     edx, 3
        mov     esi, 2
        mov     edi, 1
        call    test(int, int, int)
        mov     DWORD PTR [rbp-4], eax
        mov     eax, DWORD PTR [rbp-4]
        mov     esi, eax
        mov     edi, OFFSET FLAT:.LC1
        mov     eax, 0
        call    printf
        mov     eax, 0
        leave
        ret

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s