go: os: use non-blocking I/O for pollable files automatically
Issue
As Go matures as a language it is being adopted in areas beyond simple TCP/UDP servers and command line software. Some people are working with non-standard socket types (AF_NETLINK, AF_ROUTE 10565, AF_PACKET 15021, AF_CAN 16188, etc). Also, in the embedded area it is very common to receive data through TTY devices or os pipes. All of these use cases would benefit from as easy way to set timeouts without having to launch extra user-level go routines or to cancel read/write operations without having to close the socket.
The current idiomatic way to deal with a timeout read on a generic file descriptor is as follows:
fd, _ := syscall.Open(<path>, syscall.O_CLOEXEC, 0644)
buf := make([]byte, 1024)
success := make(chan bool)
go func() {
_, err := syscall.Read(fd, buf)
if err == nil {
success <- true
}
}()
select {
case <-success:
fmt.Println(string(buf))
case <-time.After(5 * time.Second)
syscall.Close(fd)
fmt.Println("timeout")
}
This has a few issues:
- You need to close the file descriptor, this is often not what you want to do
- The write to the success channel is sort of racy, it can be worked around with some clever channel work
- Since the read is a direct syscall, it not only blocks a go-routine, it blocks on os thread, and for embedded work with constrained resources this is something that should be avoided as much as possible
Proposal
The Go runtime has a single epoll/kqueue/iocp thread setup to deal with exactly these issues for standard TCP/UDP/unix sockets. This runtime poller should be exposed for generic file descriptors with a new package called “os/poll”. The package would would allow wrapping a file descriptor with a “net.Conn”-like interface.
The proposed API looks like this:
type Descriptor struct {
// contains filtered or unexported fields
}
func Open(fd uintptr) (*Descriptor, error)
func (d *Descriptor) Close() error
func (d *Descriptor) Read(p []byte) (n int, err error)
func (d *Descriptor) Write(b []byte) (n int, err error)
func (d *Descriptor) SetDeadline(t time.Time) error
func (d *Descriptor) SetReadDeadline(t time.Time) error
func (d *Descriptor) SetWriteDeadline(t time.Time) error
func (d *Descriptor) Wait() error
func (d *Descriptor) WaitRead() error
func (d *Descriptor) WaitWrite() error
func (d *Descriptor) Cancel() error
This would provide a simple but flexible way to work with generic pollable file descriptors. The example given above would then look like this:
fd, _ := syscall.Open(<path>, syscall.O_CLOEXEC, 0644)
buf := make([]byte, 1024)
desc, _ := poll.Open(fd)
desc.SetReadDeadline(time.Now().add(5 * time.Second)
_, err := desc.Read(buf)
if err != nil {
fmt.Println("timeout")
} else {
fmt.Println(string(buf)
}
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 30
- Comments: 41 (28 by maintainers)
Commits related to this issue
- net: refactor poller into new internal/poll package This will make it possible to use the poller with the os package. This is a lot of code movement but the behavior is intended to be unchanged. Up... — committed to golang/go by ianlancetaylor 7 years ago
- os: use poller for file I/O This changes the os package to use the runtime poller for file I/O where possible. When a system call blocks on a pollable descriptor, the goroutine will be blocked on the... — committed to golang/go by ianlancetaylor 7 years ago
As revised (make os do the right thing with no new API), accepted.
Maybe we can put it under os package if the API surface is not big.
I think it is a bit weird to provide Read and Write via the poller, because the proposed APIs are working with the file descriptor. The call site already knows what the file descriptor represents, therefore how it should be consumed.
I don’t see any reason for new API.
I think os.Files should be made to just work instead. You shouldn’t have to go to extra effort for that. If you have an os.File that is poll-able, then it should use polling to avoid tying up a network thread when blocked, and maybe it should also expose timeouts.
Somewhere here there is an issue about making net.FileConn work for alternate sockets, by adding a registration function in package syscall, but I can’t find it.
Should we repurpose this proposal for making os.File “just work”? Specifically, make them not consume a thread when blocked (if possible, such as pipes, networks, and ttys) and also support deadlines (same set).
I didn’t close it earlier because the os package still does not have
SetReadDeadlineorSetWriteDeadlinemethods. But I see now that we accepted this proposal with no new API. So now I’m not sure. We use the poller now, but without a way to set a deadline it doesn’t seem fully functional to me.This does not cover the desire to multiple files/sockets/fds within the same routine via selects/poll etc, which is the case I am interested in.
Yes, I can use syscall to get access to the raw FDs etc, but you have libraries with return you connections or files which then you can’t unwrap which you might like to select on etc. I think it’d be a low hanging fruit for the stdlib returned types which would implement a Pollable interface which could be passed to something like os/poll package.
Yet I might be confusing issues, as this topic is mostly about non-standard socket types, and not having an ability to timeout operations on them. If someone has ticket which talks about polling generic pipes/sockets/files, please point me to it.
For those asking for deadlines above, wouldn’t a more generic/backwards compatible solution to have a
func (*os.File) SetContext(context.Context)be preferable? This would allow novel ways to do cancellation which the timeouts would not. Seems the internal polling would make this possible.Yes, it would have to be something like that.
If we want Read with timeout, then we need to add Read with timeout. Doing it the same way as net.Conn sounds reasonable. My point was about Wait versions which look like a C interface.
Just a quick ‘drive by’ comment but perhaps the API should allow cancellation via ‘context.Context’?
Why provide
ReadandWritemethods? Presumably people can continue to use the original descriptor.Do we really need
SetDeadline? Don’tSetReadDeadlineandSetWriteDeadlinesuffice?Note that the current poller doesn’t support
Wait. It only supportsWaitReadandWaitWrite.OpenandCloseseem like the wrong names to me, even though that is what the current runtime code uses. It’s more likeRegisterandUnregister.