tail: Cannot determine if a given line is complete or not

Describe the bug

There seems to be race condition that leads to partial line reads. That is, if a line write is not atomic, I can read a partial line, but since tail strips all newlines, I’ll never know this happened. (My code will fail further on as it finds out said line is incomplete.)

Expected behaviour

There are two ways this can be addressed:

  1. Newlines are returned in line.Text, this would be a breaking change, not really a good solution
  2. Adding a boolean flag NewlineEnding to Line, which would tell the reader if the line.Text returned actually contained a newline at the end or not. This should be fully backwards compatible.

Here’s my locally tested patch (it’s just a draft): https://github.com/kokes/tail/commit/519502e8fb339b24d380c41214a82f8ae92fd066

To Reproduce

Write a simple tailer:

A cli wrapper for this library
package main

import ( “fmt” “log” “os” “strings”

"github.com/nxadm/tail"

)

func main() { if err := run(); err != nil { log.Fatal(err) } }

func run() error { fmt.Printf(“PID: %v\n”, os.Getpid()) logfile := os.Args[1] t, err := tail.TailFile(logfile, tail.Config{Follow: true, ReOpen: true}) if err != nil { return err }

for line := range t.Lines {
	fmt.Printf("line: %v; newline ending: %v\n", line.Text, strings.HasSuffix(line.Text, "\n"))
}
return nil

}

And run this against a random text file, go run repro.go tmp.log and then write to it:

echo -n "foo" >> tmp.log
echo "foo" >> tmp.log
echo "foofoo" >> tmp.log

This creates two lines, both containing foofoo, but we can’t tell from the reader’s point of view:

$ go run repro.go tmp.log
PID: 57959
2021/03/29 11:17:07 Waiting for tmp.log to appear...
line: foo; newline ending: false
line: foo; newline ending: false
line: foofoo; newline ending: false

System information

  • tail version v1.4.8
  • OS: macOS
  • Arch: amd64

Additional context

This was reported back in 2014 in the original repo #35 and “fixed” without tests in #40, which deadlocked the process if the tailed file didn’t end with a newline. It was then reverted in this repo by merging in #126.

My proposal doesn’t affect the library’s behaviour, just adds a flag to inform the user of partial rows. I wasn’t super sure how to handle splitting or large lines, so please review my code.

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 4
  • Comments: 17 (12 by maintainers)

Most upvoted comments

Working on the new release now and bumped fsnotiy. Checking what old bugs it fixes.

I merged the PR, thank you. I am waiting on fsnotify for creating a new release. In the meantime, please keep testing it 😃

Great to hear. Just a message to let you know I am not ignoring the issue, just a little busy at the moment. Your effort is appreciated!