deno: Deno.openPlugin resolution is not inline with the rest of Deno

I don’t think is import is good for this. Different platforms use different plugins, so dynamic import makes more sense.

Also, I support making openPlugin accepts http URL and caching plugins to $DENO_DIR/plugins.

Resolving and loading plugins like modules would be nice, but there are some difficulties here. The underlying c function dlopen that supports the entire plugin functionality can only accept file paths.

I have considered emulating the underlying function of dlopen(load file contents to ram, inform kernel this memory space is executable, dynamically link resources from the memory namespace, etc). This might be a valid option ,but would add a significant amount of complexity and would require specific support on our end for each kernel.

Maybe a simpler approach here is to cache the plugin file like we do compiled JavaScript. Either way downloading a file and loading directly into memory as executable presents a lot of security concerns.

A potential third approach of adding custom compiler support and building a custom cargo/rust compiler might be a good solution here as well. I like this solution the most, since a custom compiler could actually generate TypeScript bindings to go on top of said plugins. This also avoids downloading a compiled binary and loading directly to memory. When I first started working on plugins I thought that it would be cool if we could just import a cargo.toml file and compile like we do TypeScript, but I quickly realized how complex that would be(especially for http/https imports). A solution like that would have been almost as much complexity as the rest of deno at the time for a feature that isn’t even the main function of deno. It makes a lot more sense for something like this to be it’s own project.

The existing plugin system is intended to be very simple it’s primary goal is to enable calling custom native code from the deno runtime. It’s not a framework. Just a simple way to call into custom rust code. While I would like to be able to resolve plugins like imports it’s not possible to determine the caller from the rust side like a referrer in a import resolution. For now I think the best we can really expect is some tooling in std to make this a little easier and maybe some tweaks to resolution to allow absolute full paths(file:///some/path). I have found using import.meta.url plus join to be a pretty good solution right now. I have some working code using what I would consider standard practices in the form of deno_in_deno(check out the use_new_plugin_api branch for more up to date examples).

It’s not a JS module, module resolution is its own thing. While URL plugins are interesting 😄, I think it’s properly consistent with other FS ops.

I don’t see why those shouldn’t support the file: protocol, though.

I was against applying module semantics when plugins are brought in with a function call.

Since this is maybe going in that direction, I’m perfectly okay with treating them as modules if they are actually brought in using the import syntax: import { some_op as someOp } from "./plugins/libsome_op.so" assuming there are no other issues with that.

I think we should go for the approach where we download into a folder in DENO_DIR and then also load them from there. I think manually loading the contents into memory, marking it as exec and directly calling the dlopen syscall is too complicated.

If we build remote resolution of plugins right into the ‘secure’ part of Deno then this makes the experience of using plugins simpler for users. We can use the same caching algorithms as JS imports, we can respect the --no-remote flag, we can respect the --reload flag, and all of the cached deps (js, ts, wasm and plugins) can all be stored in the same dir - DENO_DIR. Implementing this in std would just require us to expose all of these flags to the user via ops. It seems nicer to just use the existing rust code. We might even be able to integrate this into the lock file which would be very nice.

I don’t think there are any more security implications in running with --allow-plugin than there are using --allow-run. --allow-run can also execute arbitrary binaries. Specifying either of these permissions gives you full system access and all of the Deno security promises are gone. I understand loading a dynamic lib is more scary as you are right inside of the Deno process, but ey, that’s just one of the tradeoffs you have with plugins - all security is gone.