jackson-dataformats-binary: [Avro] Incompatibility with Avro >=1.9.0
Avro version 1.9.0 was released last month (https://mvnrepository.com/artifact/org.apache.avro/avro) and my team has run in to compatibility issues between jackson-dataformat-avro
and this latest release.
In particular, it looks to be the case that Avro has switched from Codehaus Jackson to FasterXML Jackson, which results in NoSuchMethodError
thrown when accessing utility methods.
The following test exposes the issue:
@Test
public void beanShouldBeSerializableAsAvroBinaryAndDeserializable() throws Exception {
TestBean testBean = new TestBean();
testBean.setData("Data");
AvroMapper avroMapper = new AvroMapper();
AvroSchema avroSchema = avroMapper.schemaFor(TestBean.class);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
avroMapper.writer().with(avroSchema).writeValue(outputStream, testBean);
byte[] serialized = outputStream.toByteArray();
assertNotNull(serialized);
assertTrue(serialized.length > 0);
TestBean deserialized = avroMapper.readerFor(TestBean.class).with(avroSchema).readValue(serialized);
assertEquals(testBean, deserialized);
}
public static final class TestBean {
private String data;
@Override
public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
TestBean testBean = (TestBean) o;
return Objects.equals(data, testBean.data);
}
@Override
public int hashCode() {
return Objects.hash(data);
}
public String getData() {
return data;
}
public void setData(String data) {
this.data = data;
}
}
This test passes on v1.8.2 of Avro, but fails with v1.9.0 on the following:
java.lang.NoSuchMethodError: org.apache.avro.util.internal.JacksonUtils.toObject(Lorg/codehaus/jackson/JsonNode;)Ljava/lang/Object;
at com.fasterxml.jackson.dataformat.avro.schema.RecordVisitor.schemaFieldForWriter(RecordVisitor.java:192)
at com.fasterxml.jackson.dataformat.avro.schema.RecordVisitor.optionalProperty(RecordVisitor.java:121)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.depositSchemaProperty(BeanPropertyWriter.java:839)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.acceptJsonFormatVisitor(BeanSerializerBase.java:861)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.acceptJsonFormatVisitor(DefaultSerializerProvider.java:566)
at com.fasterxml.jackson.databind.ObjectMapper.acceptJsonFormatVisitor(ObjectMapper.java:3874)
at com.fasterxml.jackson.databind.ObjectMapper.acceptJsonFormatVisitor(ObjectMapper.java:3853)
at com.fasterxml.jackson.dataformat.avro.AvroMapper.schemaFor(AvroMapper.java:96)
Note that we are using com.fasterxml.jackson.dataformat:jackson-dataformat-avro:2.9.9
About this issue
- Original URL
- State: open
- Created 5 years ago
- Comments: 37 (21 by maintainers)
FWIW, @baharclerode the issue you brought up about nested class name deserialization breaking with Avro [1.9.0, 1.10.0) may have been fixed in ~1.10.0~ (actually, looks like it may not have gotten in to 1.10.0) by this PR
Could you check if this is still the case with Apache Avro 1.9.2? Would love to help here.
@baharclerode If I understand above correctly, it sounds like…
If accurate, I think I can proceed with Jackson 2.11.0 at this point – it is not delayed by Avro module, there have been other issues, but there is some urgency to getting some of improvements, fixes out – and baseline increase wrt apache avro lib could go in 2.12.0 if issues can be resolved.
For info: @cricket007 https://issues.apache.org/jira/browse/AVRO-2687 and a link to the last discussion on the mailing list. Avro has not followed literal semantic versioning, which is often unexpected. Your (tactful and gentle) perspective would be very welcome 😄
@baharclerode : I do agree with you about the schema being an important part for backwards/forwards compatibility. The current behaviour was definitely a fix for cross-platform compatibility in 1.9.x… it’s unfortunate nobody foresaw code relying on the invalid names from 1.8.x!
I think I had a basic misunderstanding on what Jackson-Avro needed to work – I’m taking a deeper look.
Avro is planning a 1.10.x release in May (which should not be considered a minor semantic version bump). It might be worthwhile working together and targetting that to have the least-painful / most-correct fixes in both projects – maybe we can restore the “better solution” ☝️ behaviour for ReflectData at that point?