pwndbg: 'Could not allocate dynamic translator buffer' Error when execute "start" command

@iiwwnnaa Can I ask you to test out this PR on your Oracle box? #1546

Or @wtdcode you on ur environment?

I don’t have a test binary for that but I assume it will work. Note it looks like you forget to unmap the test page?

Cool, so this is indeed comming from Unicorn. Now, let’s try to check out why.

If we look into the code_gen_alloc code, it prints the "Could not allocate dynamic translator buffer" error and exits if the alloc_code_gen_buffer(uc) call returns NULL. This call seems to be used to allocate the buffer to store the generated code and it has three different implementations: one for windows, one for static buffer and one is “all else”. Let’s look at the first and last one.

The first one seems to do sth with RWX memory mappings, but it also aborts on failures:

    if (qemu_mprotect_rwx(buf, size)) {
        abort();
    }

The last one allocates a RWX memory page like this:

    int prot = PROT_WRITE | PROT_READ | PROT_EXEC;
    int flags = MAP_PRIVATE | MAP_ANONYMOUS;
    size_t size = tcg_ctx->code_gen_buffer_size;
    void *buf;
#ifdef USE_MAP_JIT
    flags |= MAP_JIT;
#endif
    buf = mmap(NULL, size, prot, flags, -1, 0);
    if (buf == MAP_FAILED) {
        return NULL;
    }

Triggering an abort would not show us the error print. So it seems we are hitting the other implementation and that in your case the call to mmap with RWX protections is failing? It is either that you don’t have enough (virtual) memory for the allocation – which I doubt it would be the case – or maybe your kernel/VM does not allow you to allocate RWX memory pages.

Can you somehow confirm this? One way to do it would be:

strace -e mmap,mmap2 gdb --quiet --batch --ex 'start' --ex 'quit' /bin/ls 2>&1

Or even:

strace -e mmap,mmap2 gdb --quiet --batch --ex 'start' --ex 'quit' /bin/ls 2>&1 | grep PROT_EXEC | grep PROT_WRITE | grep PROT_READ

And I bet there will be a call from Unicorn that fails.

In my case, where it works, it is:

$ strace -e mmap,mmap2 gdb --quiet --batch --ex 'start' --ex 'quit' /bin/ls 2>&1 | grep PROT_EXEC | grep PROT_WRITE
mmap(NULL, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1a368ac000

If this is the case that this call fails, it can be one of:

1. your kernel has a custom patch that disallows for WRITE+EXECUTE mappings
2. you are using SELinux or other Linux Security Module that disallows for write+execute if not specifically allowed
3. you don’t have enough memory for the allocation, but given that it seems to be only 4kB, it’s very unlikely

@disconnect3d Yea, Exception handled. Thx

I can perfectly reproduce this btw:

ubuntu@instance-20210917-1603 ~>
python3 -c "from unicorn import *; uc=Uc(UC_ARCH_X86, UC_MODE_64);uc.reg_write(0, 0)"
Could not allocate dynamic translator buffer
ubuntu@instance-20210917-1603 ~ [1]> free -h
              total        used        free      shared  buff/cache   available
Mem:          972Mi       376Mi       111Mi       2.0Mi       484Mi       439Mi
Swap:            0B          0B          0B
ubuntu@instance-20210917-1603 ~>
python3 -c "from mmap import *;mmap(-1, 4096, MAP_PRIVATE | MAP_ANON, PROT_WRITE | PROT_READ | PROT_EXEC)"
ubuntu@instance-20210917-1603 ~>

Maybe we should then attempt to mmap 1GB RWX and if it fails, disable unicorn with a warning “hey it may abort” 😄

That’s cool and enables the max compatibility. I will try to get rid of 1GB requirement for the next release.

@wtdcode Did you read my entire response? I wrote that the “not enough memory” case is unlikely and what is likely going on here is that the mmap fails to allocate a writable and executable page.

Unfortunately, “not enough memory” is exactly the case and I don’t think Oracle Ubuntu enables W^X.

And that’s something we likely can’t fix anyhow on our side and we likely can’t even handle this error anyhow as its aborting the whole GDB.

Yes you are right and I’m working on propagating this error to users, which is a bit harder than I thought before because QEMU never considers this case.

@disconnect3d

you don’t have enough memory for the allocation, but given that it seems to be only 4kB, it’s very unlikely

This is probably wrong because Unicorn (QEMU) by default requests 1GB of memory on all platforms. You might observer some other allocation other than the code buffer.

@iiwwnnaa

I’m using Oracle Cloud Virtual Instance, as i know this system built via QEMU

AFAIK, the free tier gets 1GB per x86 instance and you could try to allocate some swap spaces as a workaround.