How to call machine code stored in char array?

One first problem might be that the location where the prog data is stored is not executable.

On Linux at least, the resulting binary will place the contents of global variables in the "data" segment or here, which is not executable in most normal cases.

The second problem might be that the code you are invoking is invalid in some way. There's a certain procedure to calling a method in C, called the calling convention (you might be using the "cdecl" one, for example). It might not be enough for the called function to just "ret". It might also need to do some stack cleanup etc. otherwise the program will behave unexpectedly. This might prove an issue once you get past the first problem.

You need to call memprotect in order to make the page where prog lives executable. The following code does make this call, and can execute the text in prog.

#include <unistd.h>#include <stdio.h>#include <malloc.h>#include <stdlib.h>#include <errno.h>#include <sys/mman.h>char prog[] = {   0x55,             // push   %rbp   0x48, 0x89, 0xe5, // mov    %rsp,%rbp   0xf2, 0x0f, 0x10, 0x05, 0x00, 0x00, 0x00,       //movsd  0x0(%rip),%xmm0        # c <x+0xc>   0x00,   0x5d,             // pop    %rbp   0xc3,             // retq};int main(){    long pagesize = sysconf(_SC_PAGE_SIZE);    long page_no = (long)prog/pagesize;    int res = mprotect((void*)(page_no*pagesize), (long)page_no+sizeof(prog), PROT_EXEC|PROT_READ|PROT_WRITE);    if(res)    {        fprintf(stderr, "mprotect error:%d\n", res);        return 1;    }    typedef double (*dfunc)(void);    dfunc d = (dfunc)(&prog[0]);    double x = (*d)();    printf("x=%f\n", x);    fflush(stdout);    return 0;}

As everyone already said, you must ensure prog[] is executable, however the proper way to do it, unless you're writing a JIT compiler, is to put the symbol in an executable area, either by using a linker script or by specifying the section in the C code if the compiler allows , e.g.:

const char prog[] __attribute__((section(".text"))) = {...}