rhai: Cannot reach a function that defined in non-entry module from Rust

You’re correct. Some systems simply store the the function itself with the closure, so it can, sort of, “always find” it whenever you need to call it. However, this doesn’t resolve you from having to make available other functions that this closure may call, which essentially is the same problem.

It is much easier to do this without multiple nested layers of modules that import each other, each with proper encapsulation. Encapsulated modules mean that you’re not supposed to affect what’s inside a module from the outside. However, by storing a closure to call later on, if it depends on something within a deeply-nested module, then there are basically only a few options:

1. Make all those functions accessible outside the module system (essentially that means making them all global)
2. Store the entire tree of modules together with the closure
3. Have some way to recreate that tree of modules when you call the closure

Of course, if you are not loading dynamic modules (e.g. you’re using a system where the modules are pre-loaded into the engine), then it obviously is not a problem.

The problem always surfaces when: 1) you have a closure, 2) that depends on some function, 3) within a dynamically-loaded module that’s deeply-nested, 4) and modules are generally encapsulated. Take away one of them and you don’t have a problem.

Ok, I successfully handled this issue. I can call closures that are not defined in root module now! But, I think, it is very very counterintuitive, and very tricky to implement - I have written module caching logics and many other things to overcome this issue. Moreover, it’s very inefficient because this solution introduces extra overhead. And I have to allocate some dynamic length of vectors every time to call a Rhai function/closure.

Can it be better somehow? If not, it’s ok for now, but I will have to consider to use other languages 😦

Anyway, thank you 😃

I have tried it and successfully recreated the NativeCallContext simply by storing all the fields:

// Store fields for later use
let fn_name = context.fn_name().to_string();
let source = context.source().map(|s| s.to_string());
let global = context.global_runtime_state().unwrap().clone();
let pos = context.position();
let call_level = context.call_level();

// Store the paths of the stack of call modules up to this point.
let modules_list: Vec<String> =
    context.iter_namespaces()
           .map(|m| m.id().unwrap_or(""))
           .filter(|id| !id.is_empty())
           .map(|id| id.to_string())
           .collect();

// ... store the fields somewhere ...

// ... do something else ...

// Recreate the 'NativeCallContext' - requires the 'internals' feature
let mut libraries = Vec::<Shared<Module>>::new();

for path in modules_list {
    // Recreate the stack of call modules by resolving each path with
    // the module resolver.
    let module = engine.module_resolver().resolve(engine, None, &path, pos)?;

    libraries.push(module);
}

let lib: Vec<&Module> = libraries.iter().map(|m| m.as_ref()).collect();

let new_context = NativeCallContext::new_with_all_fields(
                        &engine,
                        &fn_name,
                        src,
                        &global,
                        &lib,
                        pos,
                        call_level
                  );

In this case, you don’t even need to keep the AST around, as all your functions/closures reside in modules. All you need is the path to the correct module for each closure. But of course you’d need it for the resolver anyway.

Seems like closures are somehow not kept in loaded modules. That’s a good catch. I’ll fix it.

And I think it’s very common to register a script function/closure to the host side, and call them back later. The Rhai should support those features, as the Rhai is script language, in my opinion.

My feeling is that any language you embed is going to give you the exact same problem because eventually you’re going to need to do essentially the same thing

Well, at least Lua doesn’t have is problem. Normally, there’s not state to management, to call closures. Closures are always callable, because that closure holds captures in their hand(closure = function + captures).

But I don’t know the Rhai’s implementation well. Maybe the Rhai need context to call closures, to restore/reconstruct an environment to emulate capturing where the closures are defined.

Anyway, As I said, it’s good for now. I hope there’s no further problems… thank you as always 😃

it is very very counterintuitive, and very tricky to implement

It really is, because you’re really forcing Rhai to do something that it wasn’t designed to do – which is to suspend an execution state and resume it later on (probably on another thread).

My feeling is that any language you embed is going to give you the exact same problem because eventually you’re going to need to do essentially the same thing – which is to save the execution state (up to that point) as a struct… then restore that state (as much as possible) when you’re ready to resume.

Some languages may give you a pre-built way to store the execution state and you might save some keystrokes, but essentially, in the background, it’ll all be done in the same way.

For example, if you do it in JavaScript, and you have multiple layers of modules (like you do), and you want to save a closure and call it later on in some other context (without those layers of modules), then you’re going to find yourself in exactly the same situation. The simple solution, of course, is how JavaScript started off with – forget about moduling and put everything in a global shared namespace (called window).

Ok, let me try. I’ll report here when it’s done.

BTW, I think you’d want the first module in the list, not the last. Anyhow, check the id of the module to see which one is the right path. I think it should be the first one.

Actually I have edited my message just 5 minutes before you replied. So here is my idea of a solution:

You call NativeCallContext:;iter_namespaces() and get the last namespace (which should be the calling environment). In fact, with this, you can recreate the entire chain of modules.

The last module obviously should contain your closure.

You check Module::id() which should contain the path of that module.

You resolve that module. Currently you have to do it via a new resolver, but I’ll add a method to Engine that lets you just resolve modules from it.

Once you get that module, you recreate the list: &[ast.as_ref(), module] which you can then use to create your new NativeCalContext. After that, Bob’s your Uncle.