flatcc: `force_align` on vectors missing (or: how to align vectors in flatcc?)

[ Summary: flatcc should eventually support force_align on vectors - meanwhile use vectors of structs and force_align structs, or use low level builder calls to create aligned vectors access as discussed below - the problems investigated were eventually decided to be incorrect use of API - low level alignment doesn’t work if the buffer isn’t started first. ]

Hello! I have some variable-length data that must be aligned on a 16-byte boundary that I’m storing in a [uint8]:

table Buffer {
  data:[uint8];
}

At some point it looks like the google flatbuffers library started supporting force_align on these vector fields, so this works there:

table Buffer {
  data:[ubyte] (force_align: 16);
}

The current flatcc compiler doesn’t support this new usage of force_align yet though:

[build] bytecode_module_def.fbs:103:17: error: 'force_align': known attribute not expected in this context

My use requires me being able to ensure alignment for compatibility with DMA operations and SIMD code, but there are also some other big flatbuffers users that are now requiring it: https://github.com/tensorflow/tensorflow/blob/1e66b5790c93a1752fd8ea92a146aff811f2ee95/tensorflow/lite/schema/schema_v3a.fbs

If force_align on vectors could be supported in the flatcc parser/builder that’d be fantastic, but I was also wondering if there was a workaround that could be used (I suspect this isn’t the first time alignment has come up as a question 😃. I tried to directly do this by passing an alignment during construction of the vector but that doesn’t seem to do what I expected (16 byte alignment of the data in the flatbuffer):

flatcc_builder_start_vector(fbb, 1, /*align=*/16, FLATBUFFERS_COUNT_MAX(1));
uint8_t* ptr = flatbuffers_uint8_vec_extend(fbb, length);
memcpy(ptr, source_data, length);
flatbuffers_uint8_vec_end(fbb);

I’d expect the flatbuffers_uint8_vec_t when loaded from the buffer to be at a 16 byte offset (or, ignoring file header weirdness, would at least be consistently stride 16) however the pointers I get back seem rather arbitrary (4-, 8-, and 12-byte alignment). I’ve started trying to use vectors of structs (that do support force_align in flatcc) but that also doesn’t seem to do what I expect, but would love to hear if any of you have seen solutions in the past that are known to work. Also I could be doing something very wrong - I just started digging into the internals of flatcc and don’t know much yet 😃

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 37 (24 by maintainers)

Commits related to this issue

Most upvoted comments

You are welcome, and thanks for the update. It is a feature that should be added, so I’ll leave it open.