wal2json: "no known snapshots" error when db rows are large
We are using wal2json with Debezium to replicate some db tables containing large rows. While streaming changes, we have encountered the following error:
2018-07-27T19:54:09.616547179Z ERROR: no known snapshots
--
2018-07-27T19:54:09.616587252Z CONTEXT: slot "debezium", output plugin "wal2json", in the change callback, associated LSN 0/16D3850
We think this is a bug with wal2json’s handling of TOAST columns, since this behavior only started happening after commit https://github.com/eulerto/wal2json/commit/ce82d7387a806bc8bacaeed80e0a0f1840733dce
I have a more detailed analysis, including steps to reproduce the issue, up on Debezium’s bugtracker: https://issues.jboss.org/browse/DBZ-842
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 4
- Comments: 25 (9 by maintainers)
Links to this issue
Commits related to this issue
- Revert include-unchanged-toast option Per discussion in issue #74, we can't rely on access unchanged TOAST data because it is not in the WAL stream. It reverts commits 947043e22accdf8276536a5878c34e... — committed to eulerto/wal2json by deleted user 6 years ago
- Add an error message to include-unchanged-toast parameter Per issue #74, parameter include-unchanged-toast wasn't safe, hence, remove it. Since it is part of release 1.0, deprecate this option for th... — committed to eulerto/wal2json by deleted user 6 years ago
- Revert include-unchanged-toast option Per discussion in issue #74, we can't rely on access unchanged TOAST data because it is not in the WAL stream. It reverts commits 947043e22accdf8276536a5878c34e... — committed to streamsets/wal2json by deleted user 6 years ago
- Add an error message to include-unchanged-toast parameter Per issue #74, parameter include-unchanged-toast wasn't safe, hence, remove it. Since it is part of release 1.0, deprecate this option for th... — committed to streamsets/wal2json by deleted user 6 years ago
- Add option format-version This option defines which format to use. Only version 1 is currently supported. New formats will be available. — committed to eulerto/wal2json by deleted user 5 years ago
- Add an error message to include-unchanged-toast parameter Per issue #74, parameter include-unchanged-toast wasn't safe, hence, remove it. Since it is part of release 1.0, deprecate this option for th... — committed to streamsets/wal2json by deleted user 6 years ago
@eulerto can you release a new tag please?
@gburanov You also asked via email. I replied there https://www.postgresql.org/message-id/20180831143637.lbxhimikooc4uk7n@alap3.anarazel.de