go: time: LoadLocation doesn't work on windows if Go is not installed

I created 4d63.com/tz because I find it more convenient to import a package than manually including a file to ride alongside an executable. It embeds Go’s zoneinfo.zip and exports tz.LoadLocation(name) to load locations from the embedded tzdata.

zoneinfo.zip is a mere 366776 bytes, a small number amongst friends. I think of a typical go binary as a 10MB +/- 5MB affair, and I would happily increase that by 366KB (3% of the mean, but small compared to the standard deviation) to have my binaries be portable to windows.

I don’t see how we can fix this, at least not without bloating the size of almost every Go binary on Windows.

Providing the tzdata ourselves in the binary if a specific package is imported (e.g. time/zone) seems less worse to me than requiring every Windows user to install Go, which is much larger to download and is inconsistent with the static binary.

Could we duplicate the time.LoadLocation function to another package which also contains the tzdata, e.g. zone.LoadLocation. This new function would operate the same as time.LoadLocation, but would load from a tzdata file built into a new zone package. This would only introduce bloat to binaries that imported the zone package, and applications could choose to use time.LoadLocation or zone.LoadLocation. We could mark time.LoadLocation as deprecated, or at least mention in a comment on it about it’s limitations and about the alternative. Having the basic time code and the zone code in separate packages would mean the bloat is only introduced if applications specifically need it.

This would probably also resolve issues #20629 and #20969.

The alternative would be to reimplement large parts of this package, to use Windows time APIs directly instead of parsing a bundled tzdata.

I’m not sure if it’s 100% feasible, but, clearly these APIs do exist:

• .NET has a copy of tzdata for System.TimeZoneInfo, and Edge/IE also ship a copy of tzdata for the Intl implementation. But these might not be accessible from Go
• boost::time_zone handles this somehow in C++
• Windows 10 1709 added ucal_getDefaultTimeZone, ucal_openTimeZones etc in icuin.dll
• WinRT/UWP has Windows::Globalization::Calendar().GetTimeZone() available, that can be called by COM
• Use the classic Windows GetTimeZoneInformation C API and bundle in the (much smaller) mapping from IANA<–>Windows zone names

Actually while researching this, I discovered the %WINDIR%\Globalization\Time Zone\timezones.xml file seems to be a complete version of the tzdata database, using IANA names, in XML format ⭐

From https://data.iana.org/time-zones/tz-link.html#CLDR i think this file was introduced in Windows 8.1. (As of January 2020 that will be the minimum Microsoft supported Windows version.)

I think adding support for parsing this file in time package would resolve this issue,

Awesome @leighmcculloch. That should really become part of the standard library (or rather just be the implementation of time.LoadLocation on Windows), but while waiting for that, its just great to have it available. In the meantime, all calls to time.LoadLocation, in order to not crash on windows/OS without zoneinfo, need to become calls to tz.LoadLocation() after import "4d63.com/tz".

I was pretty astonished by this. I can trivially build and deploy binaries to Windows just like I can everywhere else, but as soon as I try to work with a timezone, my application blows up. Go’s tooling lulls me into a false sense of ease upon which I grow to rely, but it’s a false promise. I’d much rather a larger binary than an… ahem… time-bomb like this ticking away.

@mappu opened the new issue at #38453. Thanks.

What do people think of the approach in https://golang.org/cl/224588? With that CL, any program that imports the new time/tzdata package (as import _ "time/tzdata") will get an embedded copy of the tzdata database. This will let the program work if the system does not have any timezone information. The program will first check the timezone information on the system, as they may be more up to date. If it doesn’t find anything, it will use the embedded copy. This increases the size of the program by about 800K.

Add me to the list of people inconvenienced by this; philosophically my vote is for baking tz data into the binary given that the binaries are already relatively large. Whatever the fix is, I am surprised this issue was reported in 2017 and still remains for such a seemingly big issue (affecting tz use on entire platform).

With that CL, any program that imports the new time/tzdata package (as import _ "time/tzdata") will get an embedded copy of the tzdata database. … This increases the size of the program by about 800K.

This will effectively means that all my programs will become 800K larger. Some developers of some package I use in my program will decide to include “time/tzdata”, and I have to carry this useless extra disk space. This does not scale.

Alex

adding useless static data to every go program on windows (especially when it’s not required) isn’t a great solution either.

I must vehemently disagree. This is the preferred solution. My Go binaries are 50MB. My gosh. The timezone data is a tiny file, less than 400KB. Just add it. Good grief. Every program of any complexity will import the time package, and want to not crash on windows.

Sample fix, needs the niceties of review and convention applied, but for those of us who needed a solution yesterday (merges 4d63.com/tz into the time package):

https://github.com/glycerine/go/tree/timezone_windows_fix

I’ve filed #38017 as a proposal for a time/tzdata package. Please comment or emoji vote there about this idea. Thanks.

Some developers of some package I use in my program will decide to include “time/tzdata”, and I have to carry this useless extra disk space.

No, you are not impotent in this scenario. Simply delete the import from the library.

I doubt this will pacify these voices, but let’s add to the time/tzdata package documentation the convention that only the main package should be importing time/tzdata.

That way anyone can feel free to delete such an import if they find it in a library. No guilt.

There are various ways to address this that are described above.

The question is: what should the default behavior be? I agree that the current default is a poor choice. But it’s not obvious to me that growing the size of essentially every Windows Go binary by 800K is a better choice. So perhaps we need a way to make the default be to make the binary larger but have some way to disable that. But I don’t know what the disabling mechanism would be.

The solution is to ship the tzdata yourself (linked into your binary or elsewhere) and then call https://golang.org/pkg/time/#LoadLocationFromTZData.

Or set the %ZONEINFO% environment variable to point to your shipped zoneinfo.zip.

Including zoneinfo.zip in every Go binary can be done but it’s not a five minute job. The time package is at the bottom of the import hierarchy, and there are very few packages that it can import. In particular, it can’t import archive/zip. So that code will have to be repurposed and slimmed down for use in the time package.

(And people do regularly complain about binary sizes, though perhaps not on Windows.)

This issue is still open. There is a comment above suggesting that we can read the data from Windows. What needs to happen now is that somebody needs to write that code.

I don’t understand how that would help anybody. The great majority of Go programs use the time package, but very few need timezone information for any timezone other than the one they are in.

Thanks. It does look to me as though the Windows data has all the information that the time package cares about. If that data is available on all or most Windows systems, it seems worthwhile to add support to the time package for reading it. As @bradfitz says above, the time package cannot use the encoding/xml package, but the data seems easy enough to parse.

Unfortunately, the fact that it is all in one big file means that fetching the information for a specific timezone will be slower than when reading the Unix-style tzdata files. But that is probably still a good tradeoff for Go programs running on Windows.

I favor using the identical zonefile on all builds. I don’t want to deal with Heizenbugs that only affect windows. The windows .xml files don’t have the Daylight saving time (DST) rules, for example.

This is the quickie solution I used yesterday to get moving again: https://gist.github.com/shabbyrobe/b9a7a56e25bd8e441b7b3fe39dbb04fa

It relies on something not unlike the now-infamous https://github.com/jteeuwen/go-bindata to generate a go file with that tzfiles map referenced in the gist, which contains the base64-encoded zoneinfo.zip file I grabbed from the GOROOT of another Windows machine that was allowed to have Go installed on it (not always an option).

The code was dashed out under the duress of extreme time pressure so it’s not exactly nice (and comes with absolutely no warranty whatsoever) but hopefully it can help someone else get out of the same jam.