rust-csv: serializing nested struct with #[serde(flatten)] is not supported

CI needs to be fixed. I also need to decide how to straighten out the existing flatten feature for deserialization. And fix the dependency situation as there are a few that need to be updated.

There’s probably more.

Hi @BurntSushi,

I feel like Stockly can provide solutions to the pain points highlighted. We could:

1. First provide an RFC-like document with a thorough investigation of how it would work and what kinds of use cases it would support.
2. Once you’ve validated it, implement it and feature gate under serde-flatten-unstable
3. Use it on our code base to iterate over the implementation
4. Add a tag serde-flatten on all issues from user feedback. If you keep the triage of those issues we can commit to handling them.
5. Once you feel enough time has passed without any issue we could stabilize it.

Would that work for you?

Thanks a have a nice day!

If I had the bandwidth to engage with that kind of contribution, I would love it. But as of right now, the most likely outcome is that y’all post an RFC and I’m not able to meaningfully digest it for quite some time.

You can still go forward with it, but I just want to be very clear that my bandwidth is extremely low at present. Most of my focus is on the regex crate, and I don’t anticipate that changing in the next few months. More to the point, there are several higher priority things that the csv crate needs. So if my focus were to shift to csv, it wouldn’t be on this.

So I don’t want to say “no I don’t want your contribution,” but I also don’t want to say “yes and I will be very responsive to all your work and get everything merge and be an amazing maintainer.” Does that make sense?

Aye okay, understood.

Basically what it comes down to here is that serde(flatten) has proven to be both costly to implement and result in lots of confusion on the deserialization side of things. So if we can avoid adding it at all and the “only” cost is a bit of boiler plate, I’m probably pretty happy with that trade off.

I think the thing required here is not just a simple matter of doing the work to implement this, but to do a very thorough investigation of how it would work and what kinds of use cases it would support. In theory, your serde-flatten-unstable feature would let one sidestep that, and I’m sympathetic but that looks like a different kind of headache to me. It’d be one thing if I was able to be very responsive to user feedback and iterate on it quickly to get it stabilized. But the more likely scenario is that it will sit in its unstable state for a long while and folks will come to rely on it to the point that I can’t really break it without pissing a bunch of people off.

I do appreciate the offer though.

@izolyomi There’s a chance, sure. Multiple work arounds have been discussed above. You may not like them though.

Hi @BurntSushi

Perfectly clear thanks 😊 We’ll move on with the RFC then and no pressure to review it.

As a side question, what are the higher priority things for the csv crate?

Thanks a have a nice day, Oscar

Hi! I happen to have a use case that I didn’t see mentioned anywhere until now:

I need to be able to serialize as extra CSV columns a generic struct provided by my crate’s user (order of those columns doesn’t matter in my case).

Expected output with the following example code:

a,b,custom
0,,

Here’s an example code of what would be ideal (csv version 1.1.6):

use serde::Serialize;

fn main() {
    let mut w = csv::Writer::from_writer(std::io::stdout());
    w.serialize(Data::<CustomColumns>::default()).unwrap();
}

#[derive(Debug, Serialize, Default)]
struct Data<Extra: Serialize + Default> {
	a: usize,
	b: String,
	#[serde(flatten)]
	extra: Extra,
}

#[derive(Debug, Serialize, Default)]
struct CustomColumns {
    custom: String,
}

Which, naturally, fails with:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error(Serialize("serializing maps is not supported, if you have a use case, please file an issue at https://github.com/BurntSushi/rust-csv"))', src/main.rs:5:51

If I don’t try to use #[serde(flatten)], as expected, it fails with:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error(Serialize("cannot serialize CustomColumns container inside struct when writing headers from structs"))', src/main.rs:5:51

Instead, I’m forced to workaround this situation by using a tuple struct + an intermediate inner struct for the consistent fields. This has a negative impact on ergonomics.

use serde::Serialize;

fn main() {
	let mut w = csv::Writer::from_writer(std::io::stdout());
	w.serialize(Data::<CustomColumns>::default()).unwrap();
}

#[derive(Debug, Serialize, Default)]
struct Data<Extra: Serialize + Default>(DataInner, Extra);

#[derive(Debug, Serialize, Default)]
struct DataInner {
	a: usize,
	b: String,
}

#[derive(Debug, Serialize, Default)]
struct CustomColumns {
	custom: String,
}