longhorn: [BUG] Longhorn does not launch on systems with a non-standard path

Updated helm file example:

repositories:
  - name: longhorn
    url: https://charts.longhorn.io
  - name: kyverno
    url: https://kyverno.github.io/kyverno
  - name: incubator
    url: https://charts.helm.sh/incubator
releases:
  - name: longhorn
    namespace: longhorn-system
    chart: longhorn/longhorn
    version: 1.3.2
  - name: kyverno
    namespace: kyverno
    chart: kyverno/kyverno
    version: 2.6.1
  - name: longhorn-admission-hooks
    namespace: longhorn-system
    chart: incubator/raw
    needs:
      - kyverno/kyverno
    values:
      - resources:
          - apiVersion: v1
            kind: ConfigMap
            metadata:
              name: longhorn-custom-path
              namespace: longhorn-system
            data:
              PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/run/wrappers/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin
          - apiVersion: kyverno.io/v1
            kind: ClusterPolicy
            metadata:
              name: add-host-path-to-longhorn
              annotations:
                policies.kyverno.io/title: Add Environment Variables from ConfigMap
                policies.kyverno.io/subject: Pod
                policies.kyverno.io/category: Other
                policies.kyverno.io/description: >-
                  Longhorn invokes executables on the host system, and needs
                  to be aware of the host systems PATH. This modifies all
                  deployments such that the PATH is explicitly set to support
                  NixOS based systems.
            spec:
              rules:
                - name: add-env-vars
                  match:
                    resources:
                      kinds:
                        - Pod
                      namespaces:
                        - longhorn-system
                  mutate:
                    patchStrategicMerge:
                      spec:
                        initContainers:
                          - (name): "*"
                            envFrom:
                              - configMapRef:
                                  name: longhorn-custom-path
                        containers:
                          - (name): "*"
                            envFrom:
                              - configMapRef:
                                  name: longhorn-custom-path

I’m currently using this as a workaround:

systemd.tmpfiles.rules = [
  "L+ /usr/local/bin - - - - /run/current-system/sw/bin/"
];

Given that /usr/local/bin is not part of NixOS’ default PATH, I reckon there are no side effects.

We currently do not have the capacity to test against non standard operating systems, for a list of our recommended operating systems have a look here: https://longhorn.io/docs/1.1.0/best-practices/#software

We do want to have another look at this issue once we are closer to the 1.2 release.

I’ll happily take a look into putting some PRs together! Should have some time to delve it at the end of this week, otherwise hopefully next week.

Will we receive a complete solution for this issue, or will the workaround(helm) remain semi-permanent?

For me it was two things for nixos. The most important was @joaojacome 's input but I also was missing openiscsi on the nodes. So every node that is in my cluster now has this additional configuration

systemd.tmpfiles.rules = [
  "L+ /usr/local/bin - - - - /run/current-system/sw/bin/"
];

services.openiscsi = {
  enable = true;
  name = "<some-name>";
};

So far this seems to work. Will post updates if I discover more. But this also seems to point at @gnufied s question that nsenter is used for iscsiadm at least that was what my logs suggested as well

Does anyone have a working version of the above for kustomize? And of course having this issue fixed instead of a workaround would be a major improvement