vscode: Workaround for machines that do not have `glibc` >= 2.28

Dear developpers,

Thank you so much for your work; I lead a team of cancer researchers and part of a consortium of ~100 people with many proudly using now VSCode for data analysis and software development. Our academic systems (HPC of research centers hosting thousands of people) do not satisfy the new minimum requirements and this new update has become absolutely disruptive for us. I hope that you understand that many HPCs, especially in academic contexts, cannot easily update their systems. We beg to please re-consider this decision or add an alternative workaround, which is different than being left behind.

Thank you

I’d also like to state our use case. We have various OS environments we must develop and being in that environment directly speeds up development significantly. They range from CentOS 5 all the way to Debian Bookworm. Currently we are unable to use VSCode remote-server in CentOS 5 and CentOS 6 environments, and losing CentOS 7 will be big loss as it creates a fragmented development process between the various systems we support (it isn’t feasible to upgrade all the OS’s as they are part of long life equipment).

However, if vscode-server was installed either as a static binary, installed with all of it’s required libraries in a virtual environment, or had the ability to determine where to find its required glibc libraries via an environment variable or config, that would not only ensure we are able to continue doing remote development in a CentOS 7 environment, but it would also allow us to open up CentOS 6 and CentOS 5. This allows us to unify our development teams’ process around VSCode and the Remote SSH extension.

If vscode-server was installed on a remote machine bundled with it’s required libraries, that would reduce the dependencies on the remote system’s OS and make a glibc change no longer mark an entire remote OS as unsupported.

Here’s the workaround I came up with as the update today broke my whole development team (we use amazonlinux2 as well). We’ve been dealing with various projects moving on past the glibc level of this distro and we build our own node v18 as a result.

On our wsl instances and lima-vm.io vm’s we can do:

cd ~/.vscode-server/bin/05047486b6df5eb8d44b2ecd70ea3bdf775fd937
mv node node-vscode
ln -sf /path/to/our/custom/build/of/node .
touch /tmp/vscode-skip-server-requirements-check

this works great with limitations like the terminal doesn’t work. Seems like it would be a much more simple workaround for people to just build their own node v18 binary than try to manage an alternate glibc installation which is a nightmare. Being able to npm install the dependencies like node-pty (which has a binary component that can be built on the system) would be really nice too.

Several of our users at @U-S-NRL-Marine-Meteorology-Division are reporting issues with Version 1.86 that was just released with our RHEL 7 systems.

Worth noting that Ubuntu 18 is still supported for 5 more years if you pay for security updates.

My team is in the same situation. I work in academia and my team and I develop software for operations in an astronomical observatory. All our servers are running on CentOS 7 and while we plan to update some of the external-facing gateways after EOL, I don’t expect that we’ll update most machines until the project is over in a couple years. My team heavily uses VSCode to develop directly on those servers and there won’t be a good way for us to do this after the 1.86 release.

I appreciate the need to drop support in certain situations, but I it would be great to have a solution that allows these old system to continue working even if it’s with limited support or new features.

I also work remotely at a company still using some CentOS 7 machines. VSCode remote stopped working on those unfortunately. However, CentOS 7 is only marked as end-of-life in June 2024, so it caught us off guard to have VSCode drop support before the EOL date!

I’m a postdoc doing computational materials science, I have been using VS Code with remote-SSH for several years and am very appreciative of the efforts of the VS Code team.

However, I am in a similar position as others - my work happens on a university owned cluster that runs CentOS 7. I have very little/no say on the choice of OS on the cluster, so this change is rather unfortunate for me and many of my colleagues. Being able to do my development work directly on the cluster with VS Code has been huge productive boost for me. I do everything from writing new code, submitting jobs to the queueing system, doing analysis in notebooks directly where the output data is stored, to writing papers in LateX on the cluster. I connect to the server in regular terminal perhaps once a month, usually because htop looks better to me there.

I have been (kindly) teasing the emacs/vim users in my department and encouraging them to just try VS Code. Yet unless this issue is resolved in a not excessively technical way I guess they might have been right to stick to their ancient ways.

Really hope a solution can be found and thanks again for all the effort.

Hi VS Code PM here 👋

From my understanding you all work for Amazon @wlovo, @rexm @ShawnCarnell @bookest. If there is someone here not from Amazon then can you please share why this issue is important for your use case - thank you 🙏

Amazon folk - can you please send me an email to inikolic@microsoft.com and we are happy to meet with you next week to understand the blockers you hit while trying to do the workaround @deepak1556 provided https://code.visualstudio.com/docs/remote/faq#_can-i-run-vs-code-server-on-older-linux-distributions

I am not from Amazon, the issue affected lots of developers here who still use RHEL 7.x machines. RHEL 7.9 is still supported by Red Hat, you should consider updating this kind of requirement after major Linux vendors ending the support to the OSs.

My org (not amazon) is on a RHEL7 compatible OS for the time being. It’s EOL’d later this year, but not for several months if i understand correctly. I’m sure there are many people out there in a similar situation.

This thread and the other bugs make it sound like this is not going to be fixed, so people using EL7 will be stranded on 1.85 until they switch OSs (not an easy or quick thing to do), sounds like. EL7 can’t build glibc 2.28 from source (needs an updated gcc as well, and i’m sure many other dependencies will need updating). It’d be good to have some path forward that doesn’t require big teams to switch OS’s.

FYI This update has blocked all our development teams across the board. Looking for solution.

Here’s a funner workaround I’ve built for myself and my dev teams that are on amazlinux2. So far everything is working for me, including the terminal, run targets, etc. Maybe it can help you out? It requires a locally build node v18 - make sure you change the path for mynode.

#!/bin/bash

# This is a script to make the vscode installation on amazonlinux2 work with a locally built node v18
# by swapping out some binary components for versions built locally
set -e

vscodever="05047486b6df5eb8d44b2ecd70ea3bdf775fd937"
vscodetop="${HOME}/.vscode-server"
vscodedir="${vscodetop}/bin/${vscodever}"
mynode="/path/to/your/node/v18/binary/build/node"

# this dependency is different - it bundles it's own builds, so lets copy it's own package binaries
# If we can't find it, exit out before doing anything
sysarch=$(arch)
if [ "${sysarch}" = "x86_64" ]; then
    watcherpkg="@parcel/watcher@2.1.0"
elif [ "${sysarch}" = "aarch64" ]; then
    watcherpkg="@parcel/watcher-linux-arm64-glibc@2.2.0"
else
    echo "Unsupported architecture for @parcel/watcher"
    exit 1
fi

# point vscode to our node build
echo "Using system node v18"
cd ${vscodedir}
ln -sf ${mynode} .
touch /tmp/vscode-skip-server-requirements-check

# rebuild vscode npm dependencies with binary builds. The list was generated by
# doing a npm ls and then looking for all installs that had a build/Release directory
echo "Building npm binary dependencies"
mkdir -p ~/.vscode-localbuilds
cd ~/.vscode-localbuilds

export PATH=$(dirname ${mynode}):$PATH

npm install --no-save \
    kerberos@2.0.1 \
    native-watchdog@1.4.2 \
    node-pty@1.1.0-beta5 \
    ${watcherpkg} \
    @vscode/spdlog@0.14.0

cp node_modules/kerberos/build/Release/kerberos.node \
    ${vscodedir}/node_modules/kerberos/build/Release/
cp node_modules/native-watchdog/build/Release/watchdog.node \
    ${vscodedir}/node_modules/native-watchdog/build/Release/
cp node_modules/node-pty/build/Release/pty.node \
    ${vscodedir}/node_modules/node-pty/build/Release/
cp node_modules/@vscode/spdlog/build/Release/spdlog.node \
    ${vscodedir}/node_modules/@vscode/spdlog/build/Release/

# custom logic for the watcher package
if [ "${sysarch}" = "x86_64" ]; then
    cp node_modules/@parcel/watcher/prebuilds/linux-x64/node.napi.glibc.node \
        ${vscodedir}/node_modules/@parcel/watcher/build/Release/watcher.node
elif [ "${sysarch}" = "aarch64" ]; then
    cp node_modules/@parcel/watcher-linux-arm64-glibc/watcher.node \
        ${vscodedir}/node_modules/@parcel/watcher/build/Release/watcher.node
fi

echo "Complete - restart vscode and connect to your wsl/ssh amazonlinux2 remote"

If there is someone here not from Amazon then can you please share why this issue is important for your use case - thank you 🙏

Following up here as well. I work in research at MIT, and it looks like our cluster is on CentOS 7. Most research faculty and staff don’t have access to/any say in what the servers run, and similar to @simozacca and some others, a system-wide upgrade doesn’t seem to be on the immediate horizon—possibly in the late summer when MGHPCC shuts down.

Hi VS Code PM here 👋

From my understanding you all work for Amazon @wlovo, @rexm @ShawnCarnell @bookest. If there is someone here not from Amazon then can you please share why this issue is important for your use case - thank you 🙏

Amazon folk - can you please send me an email to inikolic@microsoft.com and we are happy to meet with you next week to understand the blockers you hit while trying to do the workaround @deepak1556 provided https://code.visualstudio.com/docs/remote/faq#_can-i-run-vs-code-server-on-older-linux-distributions

We have documented the proper workaround at https://code.visualstudio.com/docs/remote/faq#_can-i-run-vs-code-server-on-older-linux-distributions which is also included in the error message from the requirements check.

There previously was a workaround for those who relied on such needs, but it seems the new script introduced to check requirements is rendering them useless.

If the alternate glibc is in the ld cache then the check should pick it up, can you please share simple repro steps I can use to repro the issue. It is possible to fix the check if there is a bug in the detection.