ggml: Problems with Metal and iOS

Hi,

We have a problem with Metal in iOS:

-[MTLDebugDevice newBufferWithBytesNoCopy:length:options:deallocator:]:700: failed assertion `Buffer Validation
newBufferWith*:length 0x6692c000 must not exceed 1024 MB.

Reference: https://github.com/leejet/stable-diffusion.cpp/issues/108

Can anyone help us with this?

Thanks.

About this issue

Original URL
State: closed
Created 7 months ago
Reactions: 1
Comments: 16 (3 by maintainers)

Most upvoted comments

A fix for this has been merged in llama.cpp (https://github.com/ggerganov/llama.cpp/pull/5181). Applications that use ggml_backend_alloc_ctx_tensors will get the fix automatically without any changes, and applications that manage the allocations themselves can use ggml_backend_buft_get_max_size to obtain the maximum buffer size and split the model tensors into multiple buffers accordingly.

slaren on Jan 29, 2024