goreleaser: packagecloud.io http 500 error on RPMs built with v0.120.x

Describe the bug

Since the new RPM code in nfpm and goreleaser 0.120.x was introduced it is no longer possible to push RPMs to packagecloud.io.

Packagecloud support response:

This package is missing a lot of required RPM headers that must appear in the YUM metadata so that YUM can find this package and install it from a repository.

I would suggest that the maintainer of this package spend some time reviewing the RPM documentation and/or requesting the tool they use for generating the RPM automatically all required RPM headers. There’s lots of possible malformed RPM packages that will make our system 500; we catch a lot of the cases, but we do miss a few and will still 500 in some rare, isolated cases (like this package, which is significantly malformed).

To Reproduce

Steps to reproduce the behavior:

It should be possible to reproduce this bug locally from the https://github.com/pantheon-systems/pauditd project:

git clone git@github.com:pantheon-systems/pauditd.git
cd pauditd

curl -sL https://git.io/goreleaser | bash -s -- --snapshot --rm-dist

push to an existing packagecloud.io repo:

$ package_cloud push xxx/xxx/el/8 dist/*rpm

Looking for repository at xxx/xxx... success!
Pushing dist/pauditd-v1.1.11-next-1572710235-x86_64.rpm... Traceback (most recent call last):
        23: from /usr/local/bin/package_cloud:22:in `<main>'
        22: from /usr/local/bin/package_cloud:22:in `load'
        21: from /usr/local/lib/ruby/gems/2.6.0/gems/package_cloud-0.3.05/bin/package_cloud:4:in `<top (required)>'
        20: from /usr/local/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
        19: from /usr/local/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
        18: from /usr/local/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
        17: from /usr/local/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
        16: from /usr/local/lib/ruby/gems/2.6.0/gems/package_cloud-0.3.05/lib/package_cloud/cli/entry.rb:108:in `push'
        15: from /usr/local/Cellar/ruby/2.6.5/lib/ruby/2.6.0/benchmark.rb:293:in `measure'
        14: from /usr/local/lib/ruby/gems/2.6.0/gems/package_cloud-0.3.05/lib/package_cloud/cli/entry.rb:168:in `block in push'
        13: from /usr/local/lib/ruby/gems/2.6.0/gems/package_cloud-0.3.05/lib/package_cloud/cli/entry.rb:168:in `each'
        12: from /usr/local/lib/ruby/gems/2.6.0/gems/package_cloud-0.3.05/lib/package_cloud/cli/entry.rb:178:in `block (2 levels) in push'
        11: from /usr/local/Cellar/ruby/2.6.5/lib/ruby/2.6.0/benchmark.rb:293:in `measure'
        10: from /usr/local/lib/ruby/gems/2.6.0/gems/package_cloud-0.3.05/lib/package_cloud/cli/entry.rb:182:in `block (3 levels) in push'
         9: from /usr/local/lib/ruby/gems/2.6.0/gems/package_cloud-0.3.05/lib/package_cloud/cli/entry.rb:242:in `create_package'
         8: from /usr/local/lib/ruby/gems/2.6.0/gems/package_cloud-0.3.05/lib/package_cloud/repository.rb:52:in `create_package'
         7: from /usr/local/lib/ruby/gems/2.6.0/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute'
         6: from /usr/local/lib/ruby/gems/2.6.0/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute'
         5: from /usr/local/lib/ruby/gems/2.6.0/gems/rest-client-2.0.2/lib/restclient/request.rb:715:in `transmit'
         4: from /usr/local/Cellar/ruby/2.6.5/lib/ruby/2.6.0/net/http.rb:920:in `start'
         3: from /usr/local/lib/ruby/gems/2.6.0/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in `block in transmit'
         2: from /usr/local/lib/ruby/gems/2.6.0/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in `process_result'
         1: from /usr/local/lib/ruby/gems/2.6.0/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in `return!'
/usr/local/lib/ruby/gems/2.6.0/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in `exception_with_response': 500 Internal Server Error (RestClient::InternalServerError)

Additional context

Workaround is to use goreleaser v0.119.3, the last release before the nfpm rpm refactor.

Apologies I cannot provide more details at this time. If I am able to determine anything helpful I’ll update this thread.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 28 (19 by maintainers)

Commits related to this issue

Most upvoted comments

🚀

Awesome! This one was hard hahaha

Thanks for all the help Joe! Will release and close this 👍

@caarlos0 victory!! (master @ 333d834b496fa5da99fc88d4db0c5889f785a4d6)

$ package_cloud push xx/yy/el/8 dist/*rpm
Using https://packagecloud.io with token:******xxx
Looking for repository at xx/yy... success!
Pushing dist/pauditd-v1.1.11-next-1573052483-x86_64.rpm... success!

That last link in the previous comment (https://refspecs.linuxfoundation.org/LSB_1.3.0/gLSB/gLSB.html#AEN19637) had some interesting rpm tags that were not mentioned in the fedora link from a couple days ago in this thread.

That led me to poking a bit more at the “good” rpm from goreleaser 0.119 and a “bad” one:

I’m not sure if this is the cause but it might be interesting to look into:

script are marked optional in the doc, but there are differences between the two RPMs:

$ rpm -qp --scripts *GOOD.rpm
preinstall scriptlet (using /bin/sh):
# noop
postinstall scriptlet (using /bin/sh):
# noop
preuninstall scriptlet (using /bin/sh):
# noop
postuninstall scriptlet (using /bin/sh):
# noop

$ rpm -qp --scripts *BAD.rpm
$
$ rpm -qp --requires *GOOD.rpm
/bin/sh
/bin/sh
/bin/sh
/bin/sh
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1

$ rpm -qp --requires *BAD.rpm
$
$rpm -qp --provides *GOOD.rpm
pauditd = 1.1.11_next-1

$ rpm -qp --provides *BAD.rpm
pauditd = 1.1.11-next
<div>Linux Standard Base Specification 1.3</div>

Got it, it happens if we call nfpm binary, but using as an API it doesn’t, because the check is on the yaml parser func.

Will work on a proper fix tomorrow.

yep, seems like!

will require a PR on rpmpack as well, already WIP

thanks for reporting this 😃

There is a HEADER TAGS section of this doc which has a list of required -vs- optional header tags: https://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-package-structure.html

Constant Value Type Required?
RPMTAG_NAME 1000 STRING Yes
RPMTAG_VERSION 1001 STRING Yes
RPMTAG_RELEASE 1002 STRING Yes
RPMTAG_SUMMARY 1004 I18NSTRING Yes
RPMTAG_DESCRIPTION 1005 I18NSTRING Yes
RPMTAG_BUILDTIME 1006 INT32 Optional
RPMTAG_BUILDHOST 1007 STRING Optional
RPMTAG_SIZE 1009 INT32 Yes
RPMTAG_LICENSE 1014 STRING Yes
RPMTAG_GROUP 1016 I18NSTRING Yes
RPMTAG_OS 1021 STRING Yes
RPMTAG_ARCH 1022 STRING Yes
RPMTAG_SOURCERPM 1044 STRING Optional
RPMTAG_FILEVERIFYFLAGS 1045 INT32 Optional
RPMTAG_ARCHIVESIZE 1046 INT32 Optional
RPMTAG_RPMVERSION 1064 STRING Optional
RPMTAG_CHANGELOGTIME 1080 INT32 Optional
RPMTAG_CHANGELOGNAME 1081 STRING_ARRAY Optional
RPMTAG_CHANGELOGTEXT 1082 STRING_ARRAY Optional
RPMTAG_COOKIE 1094 STRING Optional
RPMTAG_OPTFLAGS 1122 STRING Optional
RPMTAG_PAYLOADFORMAT 1124 STRING Yes
RPMTAG_PAYLOADCOMPRESSOR 1125 STRING Yes
RPMTAG_PAYLOADFLAGS 1126 STRING Yes
RPMTAG_RHNPLATFORM 1131 STRING Deprecated
RPMTAG_PLATFORM 1132 STRING Optional

A few required tags that might be missing based on the rpm -qpi output:

  • Group
  • Summary
  • Description
<div>Chapter 24. RPM Package File Structure</div>

I do not. I will try poking around the RPM metadata from a pre-0.120 version for comparison and let you know if I find anything.

On Mon, Nov 4, 2019 at 6:05 PM Carlos Alexandro Becker < notifications@github.com> wrote:

any idea of which fields are missing? I don’t believe the rpm is significantly malformed as I manually checked some on fedora when we were still working on the migration…

cc/ @djgilcrease https://github.com/djgilcrease as well

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/goreleaser/goreleaser/issues/1229?email_source=notifications&email_token=AAC4GA2GSIW53R7MIF2OUDLQSDIFBA5CNFSM4JI4MHRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDBLFPY#issuecomment-549630655, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAC4GA62AWS5J3NEFWABNR3QSDIFBANCNFSM4JI4MHRA .