Hi Friend
Reading some article on how the EDR able to hook its API into windows API to detect the malicious interaction between application and operating system
I got a simple and easy to understand code from https://cocomelonc.github.io/tutorial/2021/11/30/basic-hooking-1.html where I will explain more detail from the perspective of assemby
The 5 Bytes
As you know that every windows API will start with the common 5 bytes of instruction mov edi, edi push ebp move ebp,esp where in hex 8b ff 55 8b ec


Hooking
API Hooking is the process of patching the first 5 bytes with another instuction to jump to our evil function. We will overwrite the 5 bytes of the API with the jump instruction to our function address by pushing the address to the stack just like the below sequence
// push myFunc memory address onto the stack
push hookedAddress
// jump to HookedMessageBox
ret
This is the C++ code to do the memory hooking. In this case we are hooking WinExec API
// hooking logic
void setMySuperHook() {
HINSTANCE hLib;
VOID* myFuncAddress;
DWORD* rOffset;
DWORD src;
DWORD dst;
CHAR patch[5] = { 0 };
// get memory address of function WinExec
hLib = LoadLibraryA("kernel32.dll");
hookedAddress = GetProcAddress(hLib, "WinExec");
// save the first 5 bytes into originalBytes (buffer)
ReadProcessMemory(GetCurrentProcess(), (LPCVOID)hookedAddress, originalBytes, 5, NULL);
// overwrite the first 5 bytes with a jump to myFunc
myFuncAddress = &myFunc;
// will jump from the next instruction (after our 5 byte jmp instruction)
src = (DWORD)hookedAddress + 5;
dst = (DWORD)myFuncAddress;
rOffset = (DWORD*)(dst - src);
// \xE9 - jump instruction
memcpy(patch, "\xE9", 1);
memcpy(patch + 1, &rOffset, 4);
printf("prepare for patching");
WriteProcessMemory(GetCurrentProcess(), (LPVOID)hookedAddress, patch, 5, NULL);
}
We can see the hookedAddress is now pointing to the Kernel32_WinExec. Keep your eye on the hookedAddress because we are going to overwrite this address

We see after the execution of hooking API code the hookedAddress has been overwritten to point to our evil function
The idea of the below code is simple where we push the address of the evil function and do the return where will go back to the most upper stack
// will jump from the next instruction (after our 5 byte jmp instruction)
src = (DWORD)hookedAddress + 5;
dst = (DWORD)myFuncAddress;
rOffset = (DWORD*)(dst - src);
// \xE9 - jump instruction
memcpy(patch, "\xE9", 1);
memcpy(patch + 1, &rOffset, 4);
printf("prepare for patching");
WriteProcessMemory(GetCurrentProcess(), (LPVOID)hookedAddress, patch, 5, NULL);

We confirmed that the address 772CDF20h is the jump of our function address

Now we can see that the call to winExec will actually be executing our evil function because the first 5 bytes have been overwrite with jmp myFunc

What is next ?
The next thing to do is to ensure you write back the original memory sequece to allow the WinExec to work normaly. Why this is required is first to ensure no recursive loop everytime we call WinExec. Second reason is to ensure we are not being detection by AV.
// we will jump to after the hook has been installed
int __stdcall myFunc(LPCSTR lpCmdLine, UINT uCmdShow) {
printf("Hooked API is called and executing Calc\n");
// unhook the function: rewrite original bytes
WriteProcessMemory(GetCurrentProcess(), (LPVOID)hookedAddress, originalBytes, 5, NULL);
// return to the original function and modify the text
return WinExec("calc", uCmdShow);
}