prisma: Serverless Framework project fails to deploy because of archive size
Bug description
Generated bundle of the Serverless project has too big size and exceeds allowed by AWS Lambda 250 MB.
It happens because of big sizes of downloaded @prisma/client, which, for some reasons, downloads engines folder that has unzipped size of 96 MB. And the other big files are connected to generated using command prisma generate schema.
Dependency versions
@prisma/client: “^2.20.0” prisma (dev) : “^2.20.1” serverless (dev): “^2.32.1” serverless-webpack: “^5.4.0” webpack: “^5.11.0” @types/webpack: “^4.41.25”
Webpack config
import nodeExternals from 'webpack-node-externals';
import slsw from 'serverless-webpack';
import { TsconfigPathsPlugin } from 'tsconfig-paths-webpack-plugin';
import CopyPlugin from 'copy-webpack-plugin';
const isOffline = !!slsw.lib.webpack.isLocal;
if (isOffline) {
// eslint-disable-next-line no-console
console.log('Offline mode detected!');
}
// Add plugins for the offline mode here
const offlinePlugins: NonNullable<Configuration['plugins']> = [];
const config: Configuration = {
target: 'node',
entry: slsw.lib.entries,
externals: [nodeExternals()],
mode: isOffline ? 'development' : 'production',
devtool: 'source-map',
module: {
rules: [
{
test: /\.ts$/,
use: [
'source-map-loader',
{
loader: 'ts-loader',
options: {
onlyCompileBundledFiles: true,
},
},
],
},
{
test: /\.graphql$/,
use: ['raw-loader'],
},
],
},
plugins: [
...(isOffline ? offlinePlugins : []),
new CopyPlugin({
patterns: [
{ from: './src/database/prisma/schema.prisma', to: './schema.prisma' },
],
})],
resolve: {
plugins: [
new TsconfigPathsPlugin(),
],
extensions: ['.ts', '.js'],
},
optimization: {
minimize: !isOffline,
},
stats: 'minimal',
};
export = config;
Serverless webpack options
webpackConfig: webpack.config.ts
includeModules: true
packager: npm
packagerOptions:
scripts:
- prisma generate
Environment
& setup
- OS: Mac OS Big Sur 11.1
- Database: PostgreSQL
- Node.js version: 12
- Prisma version: 2.20.1
Background
This is on the first deploy of Prisma to our existing Lambda stack. We have not been able to get Lambda working yet. We bundle our functions through webpack before deploying.
I have looked at the webpack bundle and confirm that @prisma/client is being bundled with the lambda function before deploying.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 7
- Comments: 60 (31 by maintainers)
Commits related to this issue
- Deploying to AWS Lambda: Add note about Package Pattern to exclude binaries To avoid problems like https://github.com/prisma/prisma/issues/6644, we want to filter out all binaries but the required on... — committed to prisma/docs by janpio 3 years ago
- Update package patterns for prisma 3 https://github.com/prisma/prisma/issues/6644#issuecomment-953359344 — committed to jsw/docs by jsw 3 years ago
- AWS Lambda, serverless Framework package pattern: Update for prisma 3 (#2477) * Update package patterns for prisma 3 https://github.com/prisma/prisma/issues/6644#issuecomment-953359344 * Update... — committed to prisma/docs by jsw 3 years ago
- AWS Lambda, serverless Framework package pattern: Update for prisma 3 (#2477) * Update package patterns for prisma 3 https://github.com/prisma/prisma/issues/6644#issuecomment-953359344 * Update... — committed to prisma/docs by jsw 3 years ago
@janpio I created a minimum repro at https://github.com/jsw/prisma-issue-6644
Also, I fixed part of the issue by moving the
packages.patternsfrom function scope to file scope, though both are supposed to work if I’m correctly following the serverless docs. Take a look at myserverless.yml. And@prisma/enginesandprismaare still included unless your workaround above is applied.Quick update: I can confirm that we upload both binaries, the native and the
rhelone, in standard Serverless Framework projects. No Webpack required even. If your own application code is big, this can of course lead to upload (50MB limit) or unpack (150MB limit) problems. I’ll look for a solution and let you know again.As my development environment is Windows 10. After using @jsw solution, my package size is still 62MB. I have updated my serverless package patterns to the following, and now my package size is 19MB.
For those using serverless platform, since upgrading to prisma 3, I have updated my serverless package patterns to the following, which seems to cover everything
The prisma documentation seems incomplete.
https://www.prisma.io/docs/guides/deployment/deployment-guides/deploying-to-aws-lambda#package-pattern-in-serverlessyml
says to use:
https://www.prisma.io/docs/guides/deployment/deployment-guides/caveats-when-deploying-to-aws-platforms#deleting-prisma-engines-that-are-not-required
says that
node_modules/@prisma/enginescan be excluded.And finally, as mentioned in an earlier comment, there are engine libraries in
node_modules/prismathat can be excluded.cc @janpio
@AndriiSherman That is weird, as you are explicitly asking Serverless Framework to exclude that path, right? Can you please open a new issue? Then we can debug this in detail.
@allanitis The explicit
excludeFilesis ignored, or are there other engines in the archive? Optimally also create a new issue and we can debug this individually.@NikitaHarzha In your project I am observing Serverless Framewor including all devDependencies, even when Prisma and Prisma Client are not installed. Can you confirm this behavior? The bundled file via
sls packageshould in theory only include dependencies and their dependencies in thenode_modulesfolder.I will come up with a more general
package.patternsentry that should include all files from Prisma CLI and Prisma Client later, to work around Serverless Framework somehow, sometimes not excluding devDependencies. But this will unfortunately only help for Prisma itself. I fear you are only noticing the Prisma inclusion because of the file size (which of course makes it most pressing and relevant), but it looks to me that Serverless Framework is failing to exclude unneeded dependencies in general somehow.Thanks, I opened a PR from your patch at https://github.com/prisma/docs/pull/5726
As written before, we have reduced the size of our engines a lot and as far as we know, this solved this problem for most our user and their use cases. Hence I am closing this issue for now.
If you and your project are still having problems because of the remaining size of our engines, please open a new issue and describe your situation - we are happy to take a look and hopefully fix that. Thanks!
In the case of
serverless-webpack, dependencies are installed in a separate directory and files are combined during packaging.If you use the plugin below, it automatically performs
prisma generateafter installingserverless-webpackdependency and automatically deletes unnecessary engines.The
@prisma/enginesandprismadependencies should not be included in the first place at all. We will need to investigate why that is happening. Can you maybe share your project or a minimal reproduction repository of the relevant bits?I guess the following additional patterns should fix it as a workaround:
@Jolg42 Thanks so much for looking into this. I opened new case #7016 as suggested. I am not sure how I can control
node_modules/@prisma/enginesas I only have the following prisma dependencies inpackages.json(showing the currently working case). When upgrading those two packages to > “2.19.0” I run into the AWS lambda size limits on Vercel.There is more detailed info on the new case.
If this helps shed any light on the issue, I’m having size issues because serverless will not remove the binaries in node_modules/.prisma/client via:
@janpio sure, no problems!
Ah yes, I have that of course, I copy pasted your configuration from above - that is what is confusing me.
But I’ll continue the investigation into the size first before trying to fix this now.
@janpio Resources file just contains CORS settings
Resources file
src/lambdas/test.handleris an exact path to the lambda in my project Lambda function just has test of the setup of Prisma, just one test requestTest lambda function
But it has my custom types for the event and response, you will need to use similar types from
aws-sdkto make lambda work as expectedOooookay, seems you are bundling the CLI as well here - I overlooked that before. Sorry.
Where do these files live? Do you need the CLI in the built project?
Otherwise, if you put theUpdate: You already have what I describe here. I’ll need to figure out what is going on here.prismadependency into thedevDependenciesof your project this would automatically be excluded from the package completely.@janpio Thanks for your help! Unfortunately, it didn’t worked for me. Files are still included in a package and package size is too big. I tried to implement this in different ways, but still I am not able to deploy my changes
@janpio Here is my
serverless.yamlfileProject dependencies and scripts
Prisma config