verdaccio: aws-s3-storage Publish fails when optional accessKeyId & secretAccessKey are not provided
Describe the bug aws-s3-storage - Publish fails when optional accessKeyId & secretAccessKey are not provided. This does not happen with Verdaccio V3 and it’s S3 storage plugin on our Cloud Set up.
Looking in the Verdaccio log the plugin is reported as installed however throws the errors below
warn --- config file - /etc/verdaccio/config.yaml
warn --- Plugin successfully loaded: verdaccio-aws-s3-storage
error-=- s3: [_getData] err: no such package available
error-=- s3: [_getData] err 404 create new database
warn --- Plugin successfully loaded: verdaccio-htpasswd
warn --- Plugin successfully loaded: verdaccio-audit
warn --- http address - http://0.0.0.0:4873/ - verdaccio/4.0.0
(node:702) [DEP0106] DeprecationWarning: crypto.createDecipher is deprecated.
Despite this the S3 Bucket is access and the verdaccio-s3-db.json is uploaded to the bucket inside the prefixed folder specified
When I try to publish the verdaccio log produces this output whether I log in or not; this particular output I have logged in using npm via ssh
http <-- 201, user: phil(212.58.232.179 via 127.0.0.1), req: 'PUT /-/user/org.couchdb.user:phil', byt
es: 127/73
http <-- 201, user: phil(212.58.232.179 via 127.0.0.1), req: 'PUT /-/user/org.couchdb.user:phil', byt
es: 127/73
error-=- s3: [S3PackageManager _getData] aws The specified key does not exist.
error-=- s3: [S3PackageManager _getData] The specified key does not exist.
error-=- s3: [S3PackageManager readPackage] no such package available
error-=- error reading package.json: no such package available
http <-- 500, user: phil(212.58.232.179 via 127.0.0.1), req: 'PUT /s3-contacts-upload-demo', error: I
nternal Server Error
http <-- 500, user: phil(212.58.232.179 via 127.0.0.1), req: 'PUT /s3-contacts-upload-demo', error: I
nternal Server Error
To Reproduce
- Add verdaccio-aws-s3-storage to an EC2 inside a vpn behind a network load balancer with permissions to write to the bucket.
- The Bucket has ‘Block all public access’ turn on
- Access is via the bucket owner’s Canonical ID
- The bucket policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::193231631354:role/int-ise-sandbox-verdaccio-infrastructure-EC2Role-V88WA4WD8IMB"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::int-bbc-npm-storage-poc3",
"arn:aws:s3:::int-bbc-npm-storage-poc3/*"
]
}
]
}
- No CORS configuration
- Upload is done via ssh and developer certificates
npm config set registry 'int-DCA-verdaccio-v4-nlb-53aae13bfe866c8a.elb.eu-west-1.amazonaws.com'
npm set ca null
alias npm="npm --registry https://int-dca-verdaccio-v4-nlb-53aae13bfe866c8a.elb.eu-west-1.amazonaws.com/ --cert=\"$(cat /etc/pki/certificate.pem)\" --key=\"$(cat /etc/pki/certificate.pem)\" --cafile=/etc/pki/tls/certs/ca-bundle.crt"
npm publish
Expected behavior I would expect to be able to publish as I can with verdaccio v3 and the s3 plugin for that version
Configuration File (cat ~/.config/verdaccio/config.yaml) I have used both storage: ./storage & storage: verdaccio/storage with the same results; this is the same config I use for the Verdaccio v3 with the appropriate s3 plugin for that version. Only difference is the plugin name under ‘store’
storage: ./storage
store:
aws-s3-storage:
bucket: int-bbc-npm-storage-poc3
keyPrefix: ISE
region: eu-west-1 # optional, will use aws s3's default behavior if not specified
s3ForcePathStyle: false # optional, will use path style URLs for S3 objects
packages:
'@ise-release-internal/*':
access: bbc
publish: bbc
proxy: npmjs
'@ise-release--external/*':
access: $all
publish: bbc
proxy: npmjs
'**':
access: $all
publish: $all
proxy: npmjs
uplinks:
npmjs:
url: https://registry.npmjs.org/
cache: true
logs:
- {type: stdout, format: pretty, level: http}
web:
enable: true
title: BBC CDA NPM Registry
scope:
primary_color: "#323232"
listen:
0.0.0.0:4873
max_body_size: 100mb
auth:
htpasswd:
file: /usr/lib/verdaccio/htpasswd
middlewares:
audit:
enabled: true
Additional context Add any other context about the problem here.
About this issue
- Original URL
- State: open
- Created 7 years ago
- Comments: 15 (3 by maintainers)
So my last comment was based on the errors that people had noted, but I have managed to get v5 29.2 working with an assumed role via our kubernetes setup. No errors and 26th using the guidelines for version 5 migration, got it working with no errors.
Thanks for the update!
Know what you mean about too many request!
P.
On 20 Feb 2020, at 15:45, Juan Picado @jotadeveloper notifications@github.com wrote:
Not much from my side, sorry have to many request, hopefully someone else can look into this or well, just wait a bit more I clean up my queue.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/verdaccio/monorepo/issues/321?email_source=notifications&email_token=AEBW5JTNOHGZIY7BUCOHWNTRD2QQJA5CNFSM4KRNBD7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMO2PHA#issuecomment-589146012, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEBW5JSPREUB4N36NCI2Z3LRD2QQJANCNFSM4KRNBD7A.
Hey @juanpicado hope this is more readable
Describe the bug aws-s3-storage - Publish fails when optional accessKeyId & secretAccessKey are not provided. This does not happen with Verdaccio V3 and it’s S3 storage plugin on our Cloud Set up.
Looking in the Verdaccio log the plugin is reported as installed however throws the errors below
Despite this the S3 Bucket is access and the verdaccio-s3-db.json is uploaded to the bucket inside the prefixed folder specified
When I try to publish the verdaccio log produces this output whether I log in or not; this particular output I have logged in using npm via ssh
To Reproduce
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::193231631354:role/int-ise-sandbox-verdaccio-infrastructure-EC2Role-V88WA4WD8IMB" }, "Action": "s3:", "Resource": [ "arn:aws:s3:::int-bbc-npm-storage-poc3", "arn:aws:s3:::int-bbc-npm-storage-poc3/" ] } ] }Expected behavior I would expect to be able to publish as I can with verdaccio v3 and the s3 plugin for that version
Configuration File (cat ~/.config/verdaccio/config.yaml) I have used both storage: ./storage & storage: verdaccio/storage with the same results; this is the same config I use for the Verdaccio v3 with the appropriate s3 plugin for that version. Only difference is the plugin name under ‘store’