tokio: tokio_stream::StreamMap allows duplicate keys
Version
List the versions of all tokio crates you are using.
This is version independent, but here you go:
$ cargo tree | grep tokio
tokio_stream--StreamMap v0.1.0 (/home/user/git/tmp/tokio_stream--StreamMap)
└── tokio-stream v0.1.9
└── tokio v1.19.2
Platform
The output of uname -a (UNIX), or version and 32 or 64-bit (Windows)
This is platform independent, but here you go:
$ uname -a
Linux archlinux 5.18.3-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 09 Jun 2022 16:14:10 +0000 x86_64 GNU/Linux
Description
[short summary of the bug]
tokio_stream::StreamMap::insert followed by tokio_stream::StreamMap::extend allows for duplicate keys.
[code sample that causes the bug]
use tokio_stream;
fn main() {
let mut streams = tokio_stream::StreamMap::new();
streams.insert("key", tokio_stream::pending::<u8>());
streams.extend(vec![
("key", tokio_stream::pending::<u8>())
]);
for (key, value) in streams.iter() {
println!("{}: {:?}", key, value);
}
}
I expected that the duplicate keys would overwrite existing ones like they are with from_iter (which calls insert, which calls remove).
However, I think it’d be best if both methods raised errors (see #4775) or at least returned a list of the replaced keys to match insert’s functionality. This way, the calling code would at least know what keys were replaced.
Instead, you can see that the StreamMap now contains duplicate keys:
key: Pending(PhantomData)
key: Pending(PhantomData)
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 21 (5 by maintainers)
Commits related to this issue
- A naive polynomial time complexity implementation that might be unworkable. However, this implementation does not allow duplicate keys and would overwrite existing ones similar to from_iter() behaviou... — committed to baum/tokio by baum a year ago
- A naive polynomial time complexity implementation that might be unworkable. However, this implementation does not allow duplicate keys and would overwrite existing ones similar to from_iter() behaviou... — committed to baum/tokio by baum a year ago
Well, on one hand, the
StreamMaptype already has some pretty bad running time complexities. On the other hand, I would prefer to not make them worse.@nashley I started working on this ticket. Will let you know if I have questions.
@nashley The change now is more complex that one I proposed. I’ll take a look into the code before confirming if I wan’t to pick this up. Is it okay if I confirm in couple of days?
I would be ok with using a
HashMap. I think you can just store theStreamdirectly in the map without the index trick. There’s no need to retain the ordering of the streams - when streams become ready at the same time, we may return them in every order.