tail: Cannot determine if a given line is complete or not
Describe the bug
There seems to be race condition that leads to partial line reads. That is, if a line write is not atomic, I can read a partial line, but since tail
strips all newlines, I’ll never know this happened. (My code will fail further on as it finds out said line is incomplete.)
Expected behaviour
There are two ways this can be addressed:
- Newlines are returned in
line.Text
, this would be a breaking change, not really a good solution - Adding a boolean flag
NewlineEnding
toLine
, which would tell the reader if theline.Text
returned actually contained a newline at the end or not. This should be fully backwards compatible.
Here’s my locally tested patch (it’s just a draft): https://github.com/kokes/tail/commit/519502e8fb339b24d380c41214a82f8ae92fd066
To Reproduce
Write a simple tailer:
A cli wrapper for this library
package main
import (
“fmt”
“log”
“os”
“strings”
"github.com/nxadm/tail"
)
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
func run() error {
fmt.Printf(“PID: %v\n”, os.Getpid())
logfile := os.Args[1]
t, err := tail.TailFile(logfile, tail.Config{Follow: true, ReOpen: true})
if err != nil {
return err
}
for line := range t.Lines {
fmt.Printf("line: %v; newline ending: %v\n", line.Text, strings.HasSuffix(line.Text, "\n"))
}
return nil
}
And run this against a random text file, go run repro.go tmp.log
and then write to it:
echo -n "foo" >> tmp.log
echo "foo" >> tmp.log
echo "foofoo" >> tmp.log
This creates two lines, both containing foofoo
, but we can’t tell from the reader’s point of view:
$ go run repro.go tmp.log
PID: 57959
2021/03/29 11:17:07 Waiting for tmp.log to appear...
line: foo; newline ending: false
line: foo; newline ending: false
line: foofoo; newline ending: false
System information
- tail version v1.4.8
- OS: macOS
- Arch: amd64
Additional context
This was reported back in 2014 in the original repo #35 and “fixed” without tests in #40, which deadlocked the process if the tailed file didn’t end with a newline. It was then reverted in this repo by merging in #126.
My proposal doesn’t affect the library’s behaviour, just adds a flag to inform the user of partial rows. I wasn’t super sure how to handle splitting or large lines, so please review my code.
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 4
- Comments: 17 (12 by maintainers)
Working on the new release now and bumped fsnotiy. Checking what old bugs it fixes.
I merged the PR, thank you. I am waiting on fsnotify for creating a new release. In the meantime, please keep testing it 😃
Great to hear. Just a message to let you know I am not ignoring the issue, just a little busy at the moment. Your effort is appreciated!