bitcoin: RAM usage regression in 26.x and master on ARM 32-bit

Is there an existing issue for this?

  • I have searched the existing issues

Current behaviour

The bitcoind -dbcache=100 -par=4 -disablewallet -reindex-chainstate command fails with OOM. First reported in https://github.com/bitcoin/bitcoin/issues/28718#issuecomment-1807197107.

Expected behaviour

RAM usage does not exceed 0.9 GB.

Steps to reproduce

100% reproducibility with Guix compiled binaries.

Relevant log output

2023-11-17T18:43:31Z [loadblk] UpdateTip: new best=000000000000000016d35e6ca77dea6f4e2caa387f245a712b1df5bbab3c559c height=314541 version=0x00000002 log2_work=80.091273 tx=44112815 date='2014-08-08T11:42:41Z' progress=0.049410 cache=105.6MiB(13710725txo)

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

master@5aa0c82ccd6ceb4a141686fc8658f679de75a787

Operating system and version

Armbian 23.8.3 jammy, 1 GB swap-file

Machine specifications

armv7l, 8 cores, 2 GB RAM

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 1
  • Comments: 41 (41 by maintainers)

Commits related to this issue

Most upvoted comments

@hebasto could you please try this commit, I believe this should fix the issue: martinus@72918e0

It works.

Here’s an excerpt from the log with the first flushing:

2023-11-19T16:17:58Z [initload] UpdateTip: new best=00000000000002711e96b4c88a8c60b97011068f0c644bf569a12bfa72b28a0d height=213239 version=0x00000002 log2_work=69.196319 tx=10230628 date='2012-12-23T00:00:41Z' progress=0.011054 cache=376.0MiB(3527856txo)
2023-11-19T16:17:58Z [initload] Cache size (394554040) exceeds total space (394371840)

The RAM usage remains as low as expected.

@martinus Thank you for the fix so much!

I’ve already tested my own similar commit. There were no log entries for it.

There should always be quite a few log lines for the map buckets, even when all works as expected

The log is flooded with “Allocate: 104 align(8), MAX_BLOCK_SIZE_BYTES=112, ALIGN_BYTES=4”

It works.

Awesome! Should I open a PR?

@hebasto Seeing your deleted comment elsewhere, I don’t think std::unordered_map::node_type is useful, as it’s just a reference to a node, it doesn’t actually contain the node data.

The log is flooded with “Allocate: 104 align(8), MAX_BLOCK_SIZE_BYTES=112, ALIGN_BYTES=4”

Ha! This is the issue right there. For whatever reason the nodes require alignment of 8 bytes, but the pool is configured to use the platform’s 4 byte instead. Thanks a lot for all the testing! I’ll try to prepare a fix

I’ve already tested my own similar commit. There were no log entries for it.

There should always be quite a few log lines for the map buckets, even when all works as expected

Testing now…

@hebasto Did the + 64 change change anything at all?

Tested https://github.com/hebasto/bitcoin/commit/80c02a9fe6607fb0a4abfca66be7be42fd744905.

No behavior change. Still OOM.

@hebasto Did the + 64 change change anything at all?

It is running… Will report soon.

Awesome! Should I open a PR?

Yes, please 😃

My suggestion would be to use @martinus’ commit above in 26.0 (if it fixes things), and then try to make sure fallback allocations get counted too in master.

TIL that on ARM 32bit an int64_t has 8 byte alignment, even when a pointer has 4 byte alignment: https://godbolt.org/z/xW3avfGef

@hebasto could you please try this commit, I believe this should fix the issue: https://github.com/martinus/bitcoin/commit/72918e0d80455ece4aa96228be92246a83c8ad50

@hebasto The guess for node size is here: https://github.com/bitcoin/bitcoin/blob/v26.0rc2/src/coins.h#L148

Can you try increasing that (say, add 64 to it), and see if that fixes things?

@hebasto could you try another patch? martinus@04fb0b3 This removes the placement new that didn’t reuse the returned pointer, which technically has always been illegal

Tested. No behavior change. Still OOM.

@hebasto could you try another patch? https://github.com/martinus/bitcoin/commit/04fb0b35cf18d5805d58595c27f027fae4428e62 This removes the placement new that didn’t reuse the returned pointer, which technically has always been illegal