hugo: hugo creating invalid timestamps in folder public

I came across this problem a few days ago. I did a bit of digging and found where the problem occurs but sadly a full solution was not immediately obvious given my knowledge of the hugo source code. But here is what I found. The 1754 date arises because the hugofs layer uses an empty time.Time structure for ModTime in dirNameOnlyFileInfo. Specifically hugofs/fileinfo.go

func (fi *dirNameOnlyFileInfo) ModTime() time.Time {
	return time.Time{}
}

If I replace return time.Time() with return time.Now() then the folder gets the current date rather than one in the past.

It isn’t obvious to me what the correct fix for that would be given that dirNameFileInfo only stores the leaf name so does not have the path to the original folder to read information from. Indeed whilst I did not unwrap the full layering of file systems that is used for this it may be that the correct fix may lie in one of the readOnly/union file system layers that hugo implements.

This bug actually prevents hugo from working on illumos, where I suppose we are somewhat more stringent with the timestamp values we accept in the file system:

$ hugo
Total in 70 ms
Error: Error copying static files: chtimes /var/tmp/BLAG/public/: value too large for defined data type

Digging in, the call that’s failing for us is utimensat(2):

$ truss -f -v utimensat -o /tmp/sigh.out /ws/hugo/thrice/hugo/hugo
Total in 129 ms
Error: Error copying static files: chtimes /var/tmp/BLAG/public/: value too large for defined data type

$ grep -A2 OVERFLOW /tmp/sigh.out 
61343/9:	utimensat(AT_FDCWD, "/var/tmp/BLAG/public/", 0xC0008AB810, 0) Err#79 EOVERFLOW
61343/9:		at = Aug 30 22:43:41 UTC 1754  [ 18446744066914187037.128654848 ]
61343/9:		mt = Aug 30 22:43:41 UTC 1754  [ 18446744066914187037.128654848 ]

Using DTrace I was able to get a more complete stack for that call:

$ pfexec dtrace -w -n 'syscall::utimesys:entry /arg0 == 1/ { stop(); ustack(); }'
CPU     ID                    FUNCTION:NAME
  2   8105                   utimesys:entry
   libc.so.1`_syscall6+0x1b
   hugo`runtime.asmsysvicall6+0x5a
   hugo`syscall.utimensat+0xeb
   hugo`os.Chtimes+0x182
   hugo`github.com/spf13/afero.(*OsFs).Chtimes+0x9d
   hugo`github.com/spf13/fsync.(*Syncer).syncstats+0x305
   hugo`github.com/spf13/fsync.(*Syncer).sync+0x53f
   hugo`github.com/spf13/fsync.(*Syncer).syncRecover+0x87
   hugo`github.com/spf13/fsync.(*Syncer).Sync+0x110
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).copyStaticTo+0x2fe
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).copyStaticTo-fm+0x34
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).doWithPublishDirs+0x183
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).copyStatic+0x52
   hugo`github.com/gohugoio/hugo/commands.(*commandeer).fullBuild.func2+0x37
   hugo`golang.org/x/sync/errgroup.(*Group).Go.func1+0x59
   hugo`runtime.goexit+0x1

That time value looked suspicious so I spent rather a long time chasing down where it comes from, and it seems in the end to be from code in Hugo itself – albeit called through the inpenetrable web of indirection that is the afero library.

RootMappingFs.doLstat() will, under some conditions, knock together a fake os.FileInfo using newDirNameOnlyFileInfo(). That routine constructs an object, dirNameOnlyFileInfo, which will return an invalid time.Time for its ModTime() implementation:

func (fi *dirNameOnlyFileInfo) ModTime() time.Time {
        return time.Time{}
}

There would appear to be at least two simple choices for fixing this:

• return time.Now(), which will update the timestamp on the build directory and which seems useful
• return time.Unix(0, 0), which is perhaps closer to what may have potentially been intended here: a constant but obviously pretend timestamp

At any rate, this small diff appears to fix the issue for me:

diff --git a/hugofs/fileinfo.go b/hugofs/fileinfo.go
index 79d89a88..0a3ccd36 100644
--- a/hugofs/fileinfo.go
+++ b/hugofs/fileinfo.go
@@ -275,7 +275,7 @@ func (fi *dirNameOnlyFileInfo) Mode() os.FileMode {
 }
 
 func (fi *dirNameOnlyFileInfo) ModTime() time.Time {
-       return time.Time{}
+       return time.Now()
 }
 
 func (fi *dirNameOnlyFileInfo) IsDir() bool {

I had a quick look, and there are a handful of other places that end up using time.Time{}. This appears somewhat dubious in general, and if the intent is for a common epoch timestamp, perhaps time.Unix(0, 0) may be better there – at least if the date may end up in file system metadata.

I am a non-techie. My workaround for this 1754 bug (and apologies if it’s been said already) is simply: Build the site, then in my File Manager (‘Explorer’ ie Windows PC), Copy and Paste the “public” folder, and the new copy will have the correct folder dates – then just delete the original and rename the new one to “public”. Adds about 10 seconds to the process, and non-techies can do it.

I hit this bug too, after installing and testing exa. Not critical for me; just annoying. 😃

My Hugo which installed from Homebrew has the same problem: hugo Static Site Generator v0.62.0/extended darwin/amd64 BuildDate: unknown.

Which resulted in https://github.com/ogham/exa panic:

• https://github.com/ogham/exa/issues/567
• https://github.com/ogham/exa/issues/592
• https://github.com/ogham/exa/issues/596
• https://github.com/ogham/exa/issues/618

FWIW, the year 1754 appears to be a giveaway of some time.Time conversion being incorrectly defined, so there’s likely some 0001-01-01 date or something being picked up somewhere, and it’s not coming out as epoch because of an overflow:

https://codereview.appspot.com/5985059/diff/13001/src/pkg/time/time.go

Sadly I don’t know the internal logic well enough to track that down, but the various logging/debug options don’t appear to give any clues. Happy to try to step through the logic if I’m pointed in the right direction!

I have encountered the same problem; v0.57.2 linux/amd64 BuildDate: 2019-08-18T09:32:21Z. It’s only a couple of directories and a file for me, but it’s annoying because rsync kvetches about it. Looking at an example, it’s basically the mtime which is wrong:

Size: 4096      	Blocks: 8          IO Block: 4096   directory
Device: fd03h/64771d	Inode: 4470675     Links: 7
Access: (0755/drwxr-xr-x)  Uid: ( 1000/    alex)   Gid: ( 1000/    alex)
Context: unconfined_u:object_r:user_home_t:s0
Access: 1754-08-30 22:42:26.128654848 -0001
Modify: 1754-08-30 22:42:26.128654848 -0001
Change: 2019-08-25 10:04:14.602861356 +0100
Birth: 2019-08-25 10:04:14.579861765 +0100

atime gets updated once I’ve run rsync obviously, but it still doesn’t like the old mtime.

I’ve had a look at a few things and there’s no obvious reason this is happening. I’ve stat’d the various files at the leaves of these directories, and they all look correct. But then the bad mtime seems to be propagating up the hierarchy. All these files are binary, so I would expect that it’s only looking at the file metadata - maybe it’s possible that some internal content is being picked up as a bad date, and that’s then being treated as “newer” internally somehow?

I just stumbled upon this issue while Googling for the particular timestamp. It appears it has to do with overflow caused by the Unix nanoseconds of the timestamp:

emptyTime := time.Time{}
println(emptyTime.IsZero())                          // true
println(emptyTime.UnixNano())                        // -6795364578871345152
println(time.Unix(0, emptyTime.UnixNano()).IsZero()) // false
println(time.Unix(0, emptyTime.UnixNano()).String()) // 1754-08-30 22:43:41.128654848 +0000 LMT

In other words, when you ‘convert’ an zero timestamp to unix nanoseconds then convert it back to time.Time the result isn’t what you expected. I guess the solution is to manually handle converting to 0 and back when IsZero() == true. Although that might be a bit hacky since Unix epoch 0 is 01-01-1970. time.Unix(0, 0) != time.Time{}

Anyways, I hope this helps solving this issue.

This issue still exists in v0.71.1.

hugo version
Hugo Static Site Generator v0.71.1/extended darwin/amd64 BuildDate: unknown

drwxr-xr-x  62 staff   1.9K Aug 30  1754 public/

> hugo version
Hugo Static Site Generator v0.68.3/extended darwin/amd64 BuildDate: unknown

Problem still exists in the latest version. My solution until it’s fixed is to enhance deploy script:

> cat ./deploy
hugo --minify --gc && rsync -az --delete public/ USER@HOST:/var/www/your.site.com/html/
SetFile -d $(date +%d/%m/%Y) public/
SetFile -m $(date +%d/%m/%Y) public/

This fixes the date for public folder in the directory.

[edit, nope, the bug is still there after all]

version:

Hugo Static Site Generator v0.65.0-DEV linux/amd64 BuildDate: unknown

yesterday i had to do some updates and the bug is still there after all… i just have to remind myself to delete the public directory where output goes before running hugo and it seems to work. if the output directory already exists then it doesn’t work.

Thank you, @flowerbug! I’m able to reproduce with master by simply running Hugo twice.

$ hugo new site quickstart >/dev/null
$ cd quickstart
$ hugo >/dev/null
$ ls -ld public/
drwxr-xr-x 1 me users 68 Dec  8 18:57 public/
$ hugo >/dev/null
$ ls -ld public/
drwxr-xr-x 1 me users 68 Aug 30  1754 public/