WasmEdge: bug: Can not run llama-api-server with kuasar
Summary
I’m trying to run llama-api-server with Llama-3-8B in wasm-sandboxer of Kuasar! But got some errors.
Current State
No response
Expected State
No response
Reproduction steps
- Load wasmedge plugins, which are already downloaded and installed in my computer
PluginManager::load(None).unwrap();
let mut host_options = HostRegistrationConfigOptions::default();
host_options = host_options.wasi(true);
let config = ConfigBuilder::new(CommonConfigOptions::default())
.with_host_registration_config(host_options)
.build()
.unwrap();
let vm = VmBuilder::new()
.with_config(config)
.build()
.map_err(anyhow::Error::msg)
.unwrap();
- Initialize a wasi instance:
let wasi_instance: &mut WasiInstance = vm.wasi_module_mut().ok_or(RunError::NoModule)?;
wasi_instance.initialize(
Some(args.iter().map(|s| s as &str).collect()),
Some(envs.iter().map(|s| s as &str).collect()),
Some(preopens.iter().map(|s| s as &str).collect()),
);
args: ["llama-api-server.wasm", "--prompt-template", "llama-3-chat", "--ctx-size 4096", "--model-name", "Llama-3-8B", "--log-all"]
preopens: ["/:/run/containerd/io.containerd.runtime.v2.task/k8s.io/93f3bfa0c945317105ea245062e59a6d26890b96ce9165e4dfc4aa40a6a86ccd/rootfs"]
and /run/containerd/io.containerd.runtime.v2.task/k8s.io/xxxxx/rootfs is the rootfs path of a container:
/run/containerd/io.containerd.runtime.v2.task/k8s.io/af59f9e042cc72bcedd76360cde7b97666c60e2de6ad1d70ae494232d8896ab1/rootfs:
total 5608312
drwxr-xr-x 5 root root 4096 Mar 25 15:17 chatbot-ui
-rw-r--r-- 1 root root 6895613 Jul 27 16:28 llama-api-server.wasm
-rw-r--r-- 1 root root 3010900 Jul 27 16:37 llama-chat.wasm
-rw-r--r-- 1 root root 5732987104 Jul 27 16:34 Meta-Llama-3-8B-Instruct-Q5_K_M.gguf
- Initialize the wasi_nn plug-in with the preloads
let nn_preload = "default:GGML:AUTO:/run/containerd/io.containerd.runtime.v2.task/k8s.io/af59f9e042cc72bcedd76360cde7b97666c60e2de6ad1d70ae494232d8896ab1/rootfs/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf";
PluginManager::nn_preload(vec![
wasmedge_sdk::plugin::NNPreload::from_str(&nn_preload).unwrap()
]);
- Registry module and run the function
let vm = vm
.register_module_from_file("main", "/run/containerd/io.containerd.runtime.v2.task/k8s.io/af59f9e042cc72bcedd76360cde7b97666c60e2de6ad1d70ae494232d8896ab1/rootfs/llama-api-server.wasm")
.map_err(RunError::WasmEdge)?;
vm.run_func(Some("main"), "_start", params!())
.map_err(RunError::WasmEdge)?;
Screenshots
Any logs you want to share for showing the specific issue
Finally, I got this error
[2024-07-27 17:49:11.684] [error] instantiation failed: unknown import, Code: 0x62
[2024-07-27 17:49:11.684] [error] When linking module: "wasi_ephemeral_nn" , function name: "set_input"
[2024-07-27 17:49:11.684] [error] At AST node: import description
[2024-07-27 17:49:11.684] [error] At AST node: import section
[2024-07-27 17:49:11.684] [error] At AST node: module
Components
Rust SDK
WasmEdge Version or Commit you used
wasmEdge lib 0.13.5, wasmedge-sdk 0.13.2
Operating system information
CentOS-like OS
Hardware Architecture
x86_64
Compiler flags and options
No response
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 19 (6 by maintainers)
I got it. And I also add “wasi_logging” plugin:
But the result is also failed: