go: dl: golang.org/dl installer shouldn't create a new subdirectory of $HOME

+1 on not liking $HOME/sdk, but I expect my $GOBIN to hold binaries, not whole trees. $GOPATH/sdk makes the most sense to me, even if it ends up being back to $HOME/sdk for me).

My suggestion is to use ~/gosdk instead of ~/sdk as the default directory, and allowing the user to change the default directory using the GOSDK environment variable.

Please note that I’m not proposing to add support for GOSDK in the go tool.

@RalphCorderoy, this is supposed to be the easy way to install Go. No required arguments.

If people feel strongly that there be an optional flag to set the location, maybe. But then you have to store its location somewhere in a known location (probably in $HOME/.config, for which we don’t have a helper like https://tip.golang.org/pkg/os/#UserCacheDir) so the “go1.N” binary can find it.

So I’m kinda meh on this bug. The tool is opinionated for simplicity. If you don’t like its opinions, install it the traditional way wherever you like? shrug

I would highly appreciate if the downloader would put new releases in a subdirectory of the existing ~/go directory, instead of messing up my home directory with new folders that really don’t fit there.

In 2018 I wrote:

$GOPATH is going away, so let’s not overload that and give it new reasons to exist.

At the time it wasn’t clear whether Go 1.8’s implicit GOPATH was going to stick around.

But it has stuck around (or better or worse) and still works, and it’s used heavily for the module cache:

$ echo $GOPATH

$ go env GOPATH
/home/bradfitz/go
$ du -h $(go env GOPATH)/pkg/mod | tail -1
5.4G    /home/bradfitz/go/pkg/mod

So we could put it under $(go env GOPATH)/sdk or /dl or /version.

(what’s another 100 MB or so drop in the bucket?)

@RalphCorderoy, @bcmills, @FiloSottile, @dmitshur, shall we move it from top-level ~/sdk/ to $(go env GOPATH)/something? We can make it still look at the old locations and use the old directory if it exists, but it won’t write new directories there.

I too find it surprising that my home directory was polluted. I agree that we shouldn’t tie it to GOPATH directly. GOCACHE (the build cache) exists so one can throw it away whenever, as rebuilding it tends to not be too expensive. So anywhere under GOCACHE is likely not a good idea either.

cmd/go already has a solution for this - the directory where the module cache is placed. Right now, that happens to be GOPATH/pkg/mod, but I presume it would get moved somewhere else once GOPATH is gone.

Why not put the release download cache there too? For example, we already have pkg/mod/cache/vcs, so one could imagine pkg/mod/cache/sdk. This might require x/dl to be able to ask cmd/go something like “where is the module cache?”, but I presume we can make that work.

The only other complication is that the sdk download cache might get large, and it would be somewhat hidden. A solution could be for each goX.Y wrapper program to have a very simple goX.Y uninstall command, which would delete the cached SDK, and perhaps its own binary too.

Regarding $XDG_DATA_HOME, it’s analogous to /usr/share, thus its default of $HOME/.local/share, and things reading from it are meant to also continue the search through the colon-separated paths in $XDG_DATA_DIRS. My $XDG_DATA_HOME does have a few executables, typically ‘contrib’ scripts.

The 612 MiB go1.11beta2 dwarfs the 1-2 MiB directories in $XDG_DATA_HOME here, though size isn’t a criteria in the spec. It’s content doesn’t change, but I suppose it’s not a .cache because work is required to bring it back after delete.

The audience is small. Would an optional $GOSDK to override the use of homedir()/“sdk” in version.go be acceptable? (I’d have it set to outside the area that’s backed up.)

That old perillo changeset allows a GOSDK environment variable to override the existing location. It’s unclear that has been decided upon as the solution. Alternatives above include $(go env GOPATH)/something being the default in the first place rather than the savvy user overriding it. Given there are folks here who have been caught out by the home directory being polluted on their first go, learning there’s an environment variable to override afterwards, say by properly reading the release note, seems like a bolted horse.

Changing the default doesn’t have this problem and the alternative above has, as I said, the benefit of ‘a user who doesn’t want ~/go will already be setting $GOPATH, and it will be clear what’s to blame for the space consumption’.

I installed the Go 1.18 beta and ran into this issue. Being able to move the hardcoded ~/sdk directory would be a nice feature. Making it a subdirectory of $GOPATH feels like an intuitive solution.

I’d be fine with $(go env GOPATH)/something as it won’t be a new $HOME directory, a user who doesn’t want ~/go will already be setting $GOPATH, and it will be clear what’s to blame for the space consumption.

IMO it’s kinda gross to put large files in dot directories

I would argue that it’s kinda gross to put large files any place the user didn’t specify explicitly, except perhaps a subdirectory of the current working directory (which go1.11beta2 download presumably can’t use because that might be inside $GOPATH/src). But maybe I’m old-fashioned that way?

And Windows surely has something else.

Presumably %ProgramFiles%\go1.N, or perhaps %APPDATA%\Local\go1.N? (@alexbrainman might have a suggestion.)

Perhaps we should use $XDG_DATA_HOME¹ instead?

That seems to be the standard place to stuff long-lived data files that the user otherwise doesn’t want to think about.

¹https://specifications.freedesktop.org/basedir-spec/latest/ar01s03.html

If go1.11beta2/bin/go and everything else don’t care where they are, using argv[0] to hone in, then why not insist the user state the location with go1.11beta2 download $root.

I agree that $HOME/sdk is a strange place to put the binary by default. @bradfitz chose it in https://golang.org/cl/34385, on the theory that many users already have a $HOME/sdk directory.

I do not have such a directory, and frankly I find it pretty surprising too: when I install per-language SDKs, I usually put them in ${HOME}/${NAME_OF_SDK}, like ~/go/, with symlinks in ~/bin if I don’t want to include the whole install directory in $PATH.

I would argue that $GOBIN (default $GOPATH/bin) is a better default for Go users in general, since we need that directory to exist anyway and its use is already documented.¹

¹https://tip.golang.org/cmd/go/#hdr-GOPATH_environment_variable

(CC @andybons @FiloSottile @dmitshur)

+1 to https://go-review.googlesource.com/c/dl/+/310009

@bradfitz @bcmills can you perform a review of the changeset from @perillo

@dmitshur, I think even if we put those commands in some subdirectory of GOPATH you can still use a different GOPATH with them for testing.

(The go command looks for its own resources in GOROOT, not GOPATH, and presumably the go1.Y.Z binaries would continue to apply their own GOROOT settings like they do today.)

Something to consider, I often use goX.Y.Z commands at around the same time I’m doing some investigation where one of the things I do to have a clean environment for reproduction is export GOPATH=$(mktemp -d) (i.e., I don’t want to reuse the module cache, etc.). It would be unfortunate not to be able to use an empty GOPATH directory for such testing.

Would an optional $GOSDK to override the use of homedir()/“sdk”

I’m neutral. I’ll let others here decide.

And for Mac and Windows? $HOME/.local/share isn’t idiomatic on those platforms. Mac would be ~/Library/ something. And Windows surely has something else.

But even on Unix, IMO it’s kinda gross to put large files in dot directories. But I suppose we already do that for ~/.cache. I suppose I’m neutral on XDG_DATA_HOME.