zephyr: base os: remove longstanding layering violation

Describe the bug In the Base OS, fdtable.h introduced a layering violation (in the first (?) commit) in which the POSIX header <sys/types.h> is included and the POSIX type ssize_t is used in a way that can be described as “below the line” (the line being where the POSIX API lives in the software stack). Such a layering violation creates a dependency cycle at the API level. As noted in the original commit, the wrong types seem to be added to satisfy some need of the native_posix platform.

As part of the base OS, the layering violation spread throughout the Zephyr project into the following areas (and likely more):

  • Base OS
  • Networking
  • Filesystem
  • Modem Driversd
  • Network Drivers
  • Bluetooth simulator
  • Native drivers
  • C Library
  • C++
  • POSIX API

Aside from extensive manual testing with twister and CI not catching a strange set of bugs, this layering violation was also the root cause for #75205 (as demonstrated by #75348 archived here with testing details here).

Please also mention any information which could help others to understand the problem you’re facing:

  • What target platform are you using? all
  • What have you tried to diagnose or workaround this issue? found the root cause
  • Is this a regression? Technically no, it was just always broken f484bbaa264f3a9640ca52477e3038f912ea5642

To Reproduce Steps to reproduce the behavior:

  1. See #75205

Expected behavior Developers and reviewers adding code which does not introduce dependency cycles and layering violations.

Impact This bug caused planned features to be removed from the v3.7.0 release. If it is not fixed, it will mean that the next LTS release cycle will still be quite full of these layering violations and will prevent further bugfixes from being backported to all of the aforementioned areas.

Logs and console output

Environment (please complete the following information):

  • OS: any
  • Toolchain (e.g Zephyr SDK, …): all
  • Commit SHA or Version used: c9708ff951e5915a9b68d23e6ec960fc265bb2a6

Additional context

The simpler workaround was deemed to simple. So maintainers asked for the more complicated fix. There is a PR linked to demonstrate the more complicated fix.

This bug is blocking the following (TSC-approved) features from being re-added to v3.7.0.

POSIX_FD_MGMT

POSIX_DEVICE_IO

This bug is blocking other architectural fixes that have plagued Zephyr for multiple (actually all) LTS release cycles already

This bug is also blocking numerous the stability improvements mentioned here.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 47 (46 by maintainers)

Most upvoted comments

No… it was bullying. By you and a few others.

At the risk of stating the obvious, we will not tolerate bullying and this would very much be a violation of our Code of Conduct.

I would encourage you to reach out to conduct@zephyrproject.org to report these incidents so that they can be investigated and appropriate action can be taken.

Ignoring all above, regarding the use of ssize_t, off_t, and sys/types.h in Zephyr: I acknowledge that that header and types are not part of the ISO C standard. That is, Zephyr today requires a C library which provides a non ISO C header, sys/types.h, where those 2 types are defined. This requirement means that if a user has a C library which has no or so little support for the POSIX extensions as to not have that, that user would need to add on his own the sys/types.h header with those 2 definitions. This is a relatively similar situation as with the strnlen() requirement set by Zephyr, with the difference that we are not documenting this anywhere. I see two possible approaches in that regard:

  1. A change akin to your PR, in which we remove all uses of that header and types, changing a lot of stable APIs, and forbid the use of those types. The benefit being that we do not set a requirement of the C library or its integration providing that header and types.

  2. A more pragmatic approach, where, like for strnlen(), we document that C libraries to be used with Zephyr or their integration need to provide sys/types.h incuding those 2 types definitions. The benefit in this case is that we do not need to change any API or other Zephyr code.

I simply present possible options for a problem. How we should proceed would be up to the TSC.

Couldn’t Zephyr adapt a similar strategy instead of re-inventing a very well-known set of types (in a different namespace), which people are used to working with?

The whole problem here is that these types must be defined by the POSIX layer which is built on top of Zephyr. Introducing these types inside of Zephyr creates a circular dependency where Zephyr depends on POSIX and POSIX depends on Zephyr. Linux doesn’t have a POSIX layer anywhere.

As a practical example, if you compile using picolibc in non-POSIX mode (e.g. using -std=c17 and not defining _POSIX_C_SOURCE), then these types are not defined, and so Zephyr would need to define them itself. However, code compiled in POSIX mode (e.g. using `_POSIX_C_SOURCE=200809L) would see picolibc defining these types, and those types may well conflict with the types Zephyr uses.

I’d have been completely happy if Zephyr were “natively” POSIX; either leveraging POSIX support from an underlying C library or providing an integrated POSIX layer when using a pure C library. But, I don’t think that’s an option here.