browserify: checking for the existence of Buffer causes it to exist

Some modules check for the existence of Buffer but don’t require it to work. Because the identifier Buffer is seen by browserify, it becomes bundled. Buffer addes quite a lot of code to a bundle that often isn’t needed. (About 40kB)

An example of this is if you require clone, which checks if Buffer exists, and therefore causes it to be added to a browserified bundle.

It would be handy to be able to explicitly tag that your module does not require Buffer in the package.json (even if you use the Buffer identifier in your source) to allow for it to be not added if possible.

About this issue

  • Original URL
  • State: open
  • Created 9 years ago
  • Comments: 27 (12 by maintainers)

Most upvoted comments

I’m in favor of adding a special check for Buffer.isBuffer() that doesn’t include all of Buffer. This should live in insert-module-globals.

Very nice

In the case of the clone, it creates a new Buffer with new Buffer

It doesn’t. That was the point of this thread. I came here with the same issue. I’ll try the noParse: ['clone'] option.

@substack is there a way to completely omit node specific things from the package? Like if a module could work in both browser and node and it has something like typeof Buffer != 'undefined' && Buffer.isBuffer(bla) and I see bunch of things get included, like EventEmitter, TypedArrays, Streams, process object (btw, --ignore process doesn’t help).

Thanks.

“I don’t think anything can be done there”

Why would a flag that works like peer-deps not work?

New in insert-module-deps@6.6.0 (you get this for free on a clean browserify install):

echo "Buffer.isBuffer('whatever')" | browserify - | wc -c
1223

In the case where Buffer.isBuffer() is used but not Buffer, insert-module-globals will inline is-buffer. Otherwise, the old behavior with the entire Buffer implementation kicks in.

In the case of the clone, it creates a new Buffer with new Buffer, so I don’t think anything can be done there. The typeof check is a different matter.

Updated to check window instead, but really I’d rather the pr wasn’t required at all.

This is exactly why a package.json flag would be handy.

@KoryNunn I looked at https://github.com/pvorb/node-clone/issues/46. Are you sure that works? There’s no GLOBAL on the browser and browserify only transforms global, not GLOBAL.

FYI: if you rely on global, you’re relying on the same mechanism that was adding Buffer, which means nobody will be able to noParse that module. Nor will they be able to detectGlobals: false, their build (I do this sometimes as an optimization).

Nice, works well in the meantime, thanks @zertosh

@KoryNunn if you noParse: ['clone'], it’ll skip the “insert globals” transform and browserify won’t add Buffer or anything. From a cursory look at “clone”, it shouldn’t be a problem to “noParse” it - it doesn’t have any requires and it doesn’t use any other node globals.

I guess this could be assumed by browserify by hunting for typeof Buffer, which would remove the requirement for explicit control, and remove the requirement for plugins, but I’m a fan of configuration over magic.

  • I’m not sure how one pull request for a module is more annoying than needing to install a plugin for every app you use that module in.
  • In terms of complexity, I’m not sure how something like if(!packageJson.ignoreBuffer && source.needsBuffer) could be a deal breaker.
  • In terms of file-size, 40kB is an enormous amount of dead code. In my case, it’s 25% of the whole application.
  • I can’t see how this would be a niche problem, since almost no clientside code needs Buffer, but many clientside projects need to clone things, or use cross-compatible libs that might need to know how to handle Buffer.

Here is a pr for clone that could solve the problem in the meantime, but it isn’t pretty: https://github.com/pvorb/node-clone/pull/46/files

I don’t think you can really expect modules authors to set a flag in all their package.json files just to optimize away a few kb’s in browserify apps, especially considering many modules are written by Node/Webpack/etc devs. And if any one of the potentially dozens of modules you require hasn’t set that flag, you’ll end up getting Buffer anyways.

This is really specific to your use-case; adding it as a core feature seems like it would add a deal of complexity and frustration for the average module dev (e.g. getting PRs for adding some browserify-specific flag to package.json).

The theoretical browserify plugin would act on a user-supplied list of modules. This way you have control (no need to PR/bug authors) and also things won’t break if you forget to list a module.

bundler.plugin(BufferStripper([ 'clone', 'other-module' ]))

Just my 2c. 😄

Clone works fine without Buffer, it checks for its existence here: https://github.com/pvorb/node-clone/blob/51a79bbb0b1b781503cf2497ab1ea083bfaba49f/clone.js#L35

Making it a flag per project is a really bad idea, because you never know if/when an npm up could cause your project to break. Having a flag per module allows those like clone to say “Hey I don’t need Buffer, but I can deal with them if other modules pass them to me”.

It seems like clone would not work if it doesn’t include Buffer.

Since this is app-specific, maybe it’s better to keep it as an optimizing step at the application level. e.g. If you’re certain Buffer isn’t needed in your app, you could use a plugin which strips Buffer from your bundle, or replaces it with stubs.