PowerShell: [alpine Linux] - some missing standard Modules in alpine linux APK package

Surmising from that script, those modules aren’t actually built, but rather fetched and packaged as is in the tar.gz file. While this approach is acceptable on the .NET ecosystem, it is generally not acceptable on linux ecosystems.

@ayakael Please excuse my ignorance as I am not familiar with Linux packaging best practices. What is the issue with pulling in those Microsoft created modules into the build as prebuilt rather than building from source? I bring it up as there are other DLLs included in the PowerShell package that are also pulled in prebuilt like, .NET BCL, Newtonsoft JSON, etc. I opened up the existing PowerShell Alpine package and noticed they are still signed by Microsoft or the third party meaning they weren’t built during the Alpine PowerShell build.

I can see how there is a benefit to building from source, but there is also a benefit to pulling in the prebuilt binaries as they are already signed by the creator.

No worries for the question! Indeed, those DLLs are holdouts from when I was first porting the build to Alpine. I was wanting to get back to getting these prebuilts out to reflect best practices, but alas it isn’t trivial.

As for your last question. Generally, distributions follow these principles when building:

• Limited or no network access when building This principle allows for caching of dependencies, for ensuring no prebuilts, and for verifying that every source tarball is what is expected though checksumming.
• Consistent reproducibility Reproduciblity is not only an approach to ensure secure builds, it also allows for easier debugging of issues. A build that worked last month, but not today, despite the exact same source tarball, is a hard thing to debug when, in fact, it might have been caused by a broken dependency pushed a few weeks previously. Consistent inputs and consistent outputs to a build allows one to better diagnose issues by reducing points of failure.
• Source code for everything Distributions maintain an ecosystem where every component can be built from source, allowing modifications and easier debugging from any user and developper. In cases where, for example, Newtonsoft might have an issue, the community can find and patch that issue without relying exclusively on upstream. Another aspect is that many projects are eventually abandonned, much to dismay of the communities that depend on them. Thus, communities often fork abandonned projects, and because they are libre, it is possible.

The .NET team eventually, for lack of a better expression, « saw the light » and worked to create a way to build the .NET compiler entirely from source with the .NET source-build project: https://github.com/dotnet/source-build This has been invaluable, as it allowed community developpers, like me, to better support the ecosystems we work on and work with every day.

<div> GitHub</div><div>GitHub - dotnet/source-build: A repository to track efforts to produce a source tarball of the .NET Core SDK and all its components</div><div>A repository to track efforts to produce a source tarball of the .NET Core SDK and all its components - GitHub - dotnet/source-build: A repository to track efforts to produce a source tarball of th...</div>

I did a summary investigation of what it would entail to build these modules from source:

• Most do not have buid-from-source instructions (only oneget and psreadline has instructions)
• When they do, it involves building with a psm1 script. We have a bootstrap problem when wanting to build these before pwsh is available.
• Releases aren’t properly tagged (oneget and threadjob)
• There is a warning that releases might not be available on Github in the future (PSReadLine)

This doesn’t bode well from a maintainer perspective, I’m afraid.

@kilasuit

Echoeing @the-hotmann, the situation hasn’t changed as it relates to my observations done last august:

* Most do not have buid-from-source instructions (only oneget and psreadline has instructions)
* When they do, it involves building with a psm1 script. We have a bootstrap problem when wanting to build these before pwsh is available.
* Releases aren't properly tagged (oneget and threadjob)
* There is a warning that releases might not be available on Github in the future (PSReadLine)

I’ll open issues in the relevant repos to see what those teams can do to facilitate source-building.

@richlander to not make this topic stall I hope this screenshot helps you:

Left: Official + Prebuilds Right: Alpine sourcebuild

Official + Prebuilds Alpine Sourcebuild Source needed? Microsoft.PowerShell.Archive ✓ Microsoft.PowerShell.Host Microsoft.PowerShell.Host ✗ Microsoft.PowerShell.Management Microsoft.PowerShell.Management ✗ Microsoft.PowerShell.Security Microsoft.PowerShell.Security ✗ Microsoft.PowerShell.Utility Microsoft.PowerShell.Utility ✗ PackageManagement ✓ PowerShellGet ✓ PSReadLine ✓ ThreadJob ✓

Here again in a list, all modules which can not be build by source:

* `Microsoft.PowerShell.Archive`

* `PackageManagement`

* `PowerShellGet`

* `PSReadLine`

* `ThreadJob`

Thanks for helping!

@ayakael @the-hotmann - All those in that list marked as needed are already open sourced here on GitHub mostly under the PowerShell Organisation in their own Repos, other than PackageManagement which is under the Oneget repo. so should all be able to be built from their sources, if for any of those you find they can’t be built from the source can you please raise an issue in their repo and link back to this issse for us please.

I have created an APKBUILD file to create apk containing the missing modules.

PowerShell Modules for Alpine

I am using alpine 3.18.4 and it uses

$ apk list | grep powershell-7
powershell-7.3.4-r1 aarch64 {powershell} (MIT) [installed]

The APKBUILD uses sub-packages to put each module in its own apk.

Thank you! Unfortunately, this repackages prebuilt binaries. As it is, it wont be possible to integrate this in the official aports. I missed the 3.19 release to fix this due to other obligations taking my time, but I will eventually fix this. Hopefully by then building these from source will be easier.

I can get back to investigating this in a few weeks. Right now I’m tied up with other obligations. I made some stabs at it a few weeks ago, but stalled. I’m targetting for this to be fixed before the next release of Alpine in november

@the-hotmann Thank you for the list.

Is there a list (perhaps I missed it) of the pre-builts that need to be source-built?

The .NET team eventually, for lack of a better expression, « saw the light » and worked to create a way to build the .NET compiler entirely from source with the .NET source-build project

Ha ha! Yes, you are welcome. Also, we had a lot of help, particularly from Red Hat.

I am sorry for offtop, can be used alpine tar.gz for musl void linux?

I encourage you to try, and see. Normally, portable packages targetting Alpine Linux actually targets linux-musl-x64 which is also Void Linux

If you run Get-Module -ListAvailable does the PowerShellGet module show up?