opentelemetry-js: Auto-instrumentation doesn't work when using Yarn 2 with PnP

Removing the bug label as behavior with a non-standard dependency linker is not yet defined

Just fyi, this isn’t a PnP issue. It’s a Node issue that PnP highlights more than node_modules can do. The problem is that because you’re calling require on something that isn’t listed as a dependency, you’re subject to the hoisting whims. For example, take the following case. There’s nothing special, just two incompatible telemetry versions used in two different deps:

root → dep-a → opentelemetry@1
             → opentelemetry-http@1
     → dep-b → opentelemetry@2
             → opentelemetry-http@2

Package managers will tend to hoist them to the top. Now, there’s no rule around hoisting, except that the dependency contract must be respected (ie a package doing require on one of its listed dependency must obtain a version compatible with the range it requested). Because you didn’t list explicit relationships, the following is possible:

root → opentelemetry@2
     → opentelemetry-http@1
     → dep-a → opentelemetry@1
     → dep-b
             → opentelemetry-http@2

When opentelemetry@2 will make the require call to retrieve its http plugin, due to the hoisting, it will get the v1 from this plugin, not the v2. The hoisting is invalid, but because of the lack of explicit relationship, package managers don’t have the information needed to prevent it.

---

In non-plugin scenarios, you’d list those dependencies as (optional?) peer dependencies. This would be enough to fix the problem, as the additional constraint would prevent package managers from generating hoisting like the above. In your case however, you likely don’t know what plugins will be used - after all, maybe your users will develop their own!

In this case, I’d suggest to do the same thing as Webpack, Babel, ESLint, …: load your plugins on behalf of someone else. For example, if your require call is like this:

const http = require(pluginName)

Instead require it using this:

const {createRequire} = require('module');

const req = createRequire(pathToConfigFile);
const http = req(pluginName);

Or, another option:

const httpPath = require.resolve(pluginName, {paths: [pathToConfigFile]});
const http = require(pluginName);

In both cases, the resolution is done on behalf of the package that actually lists the plugin as dependencies. How you know its path is up to you, but it can come from anything: the configuration file path (Babel, ESLint), a basedir parameter in the initializer (Webpack), … it’s the most flexible option.

Hi there, did any one succeed yet with Instrumentation using yarn 2 (with PnP) and webpack? I could only see modules like dns, tcp or http working out of the box. Modules like express, graphql or mongodb are not working.

Do I need to externalize some of the deps?

It seems like dependencies should be declared in the packages that have the dependencies. If consumers of a package are left to figure out which packages (and versions) to puzzle together, you’re kind of missing the target.

By the way you can workaround this by adding overrides to .yarnrc.yml

packageExtensions:
  "@opentelemetry/node@^0.14.0":
    dependencies:
      "@opentelemetry/plugin-http": "^0.14.0"
      "@opentelemetry/plugin-https": "^0.14.0"
      "@opentelemetry/plugin-pg-pool": "^0.12.0"
      "@opentelemetry/plugin-pg": "^0.12.0"
      "@opentelemetry/plugin-ioredis": "^0.12.0"
      "@opentelemetry/plugin-mongodb": "^0.12.0"
  "@opentelemetry/plugin-ioredis@^0.12.1":
    dependencies:
      "@opentelemetry/semantic-conventions": "^0.14.0"
  "@opentelemetry/plugin-mongodb@^0.12.1":
    dependencies:
      "@opentelemetry/semantic-conventions": "^0.14.0"
pnpMode: strict

We don’t have any overrides for opentelemetry in our .yarnrc.yaml any more, I think they added the necessary peerDependencies at some point.

Not sure about that error, it looks like a tricky one to debug. I guess you could look and see if that file exists in the package zip file, or configure yarn to unzip that module and see if it makes a difference. Some debugging to do there, more than what I can help with from here.

Hi @dyladan, any update on this one?

We’re having the exact same problem (with webpack), auto-instrumentation doesn’t work.

I prefer to build the lower layers as explicitly as possible, then build the “magic” (like auto-detecting/auto-installing plugins) into wrappers around it. That way you can always drop down to the explicit version in the case that you have some unexpected failure.

If we’re going to make this change, it needs to be done before GA in my opinion as changing the plugin interface to allow manual plugin installation would be quite the breaking change.

Thanks for this additional context. Makes sense.

I’ve run into similar problems when using bundlers (e.g. esbuild or webpack). I think this is a common enough scenario/pattern in modern applications that it’s worth considering the change you’ve outlined.