aurutils: aur-sync: Provide a summary of build failures instead of failing early

When performing multiple builds in an aur-sync pass, the current behavior is that the first failing build aborts the process. As a user,

  1. it is not immediately obvious which package failed, because the name of the package being build is only mentioned at the start of the build process, which could have been however many lines ago,
  2. it is annoying to have to exclude the failing package from the initial command to continue with the rest of the package list before you run the command again to proceed with the remaining packages,
  3. it is easier to leave successfully built packages in the command than to dig them up from the log and remove them,
  4. if the failing package is a dependency of a package originally specified to aur-sync, it is a pain to figure out which of the specified packages pulled in the failed package and would thus need to be excluded to get at least the remaining packages to build.

As a general idea, it would be much nicer if

  1. aur-sync didn’t terminate on the first build failure and would instead continue with any builds that don’t depend on failed builds,
  2. there was a build summary at the end of an aur-sync call that mentions all packages that failed to be built and maybe also those that have been build successfully.

Disadvantages of the proposal:

  1. It wouldn’t be immediately obvious why a certain package failed to be built because it could have happened anywhere during the process, but there would be log files of the individual builds.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 31 (31 by maintainers)

Commits related to this issue

Most upvoted comments

Please file a separate issue for that and I’ll take a look.

Just tried it out. Very cool, thank you!

Indeed, this is what is I went with in the end. The common case is (presumably) a successful run, and no verbosity should be added then.

@FichteFoll @bartoszek (#350) @jendrikw (#388) call for testing (install ninja, then AUR_SYNC_USE_NINJA=1 aur sync ... --keep-going=0)

It’s a difficult call because it’s one of those situations where it’s the information you might have liked to know after the fact, when it is too late. Perhaps only display this if there was an error?

Beautiful

Nice! The console pool works as intended.

Not sure how directories work either. At least ninja -t clean does not remove them (it has the error Directory not empty). Since the path to package tarballs is variable (depending on package version, local repository path and compression) perhaps a “dummy” file could be used instead.

edit: https://ninja-build.org/manual.html#_tarball_extraction

edit2:

aurbuild_opts = -srn
rule aurbuild
  command = env -C ${directory} aur build ${aurbuild_opts} && date --rfc-3339=ns > ${out}

build wayland-git.stamp: aurbuild wayland-git/PKGBUILD |
    directory = wayland-git
    pool = console

build wayland-protocols-git.stamp: aurbuild wayland-protocols-git/PKGBUILD |
    directory = wayland-protocols-git
    pool = console

build weston-git.stamp: aurbuild weston-git/PKGBUILD | wayland-git wayland-protocols-git 
    directory = weston-git
    pool = console

now ninja -t clean works as expected, and ninja only builds the package if the corresponding PKGBUILD was modified. I’d need to check how complicated it would be to point to actual tarball paths.

ninja has a concept called pools which allow you to control the level of parallelism. I’d probably suggest using ninja -j 1 regardless, for that reason. The build targets might also benefit from being package tarballs, I’m not sure how ninja responds to directories.

As for sudo, I wonder if ninja’s console pool would help as it connects to the terminals stdin/stdout. The pool already limits parallelism to 1 so it might be quite helpful in that respect too.

Another problem is that ninja and sudo prompts don’t play nice. If a build command in ninja runs sudo, and a prompt ensues, there’s no way to

  1. abort ninja apart from SIGKILL ;
  2. enter the password in the first place;
  3. even when the password could be entered, it is shown on the console.

If #907 is implemented, sudo ninja can be used to circumvent this. However, ninja swallows all output from aur-build regardless of verbosity switches. That alone makes using build.ninja as an aur build runner useless. (I guess it’s a “modern” thing to hide all output from the user.)

ninja -t graph can be used to generate a graph, either way: graph