libelektra: testshell_markdown_tutorial_crypto: fail

Steps to Reproduce the Problem

Build libelektra while having gpgme development files. (gpgme-devel on fedora) I used

cmake -DBUILD_DOCUMENTATION=ON -DBINDINGS="ALL" -DBUILD_SHARED=ON -DBUILD_STATIC=ON -DBUILD_FULL=ON -DENABLE_COVERAGE=OFF -DENABLE_OPTIMIZATIONS=ON -DENABLE_DEBUG=ON -DENABLE_LOGGER=OFF -DBUILD_STATIC=ON -DCMAKE_INSTALL_PREFIX="$INSTALL_DIR" -DKDB_DB_SYSTEM="$SYSTEM_DIR" DCMAKE_INSTALL_PREFIX=./releaseInstallDir -DPLUGINS="ALL" -DTOOLS="ALL" ..

but not everything is relevant to the problem.

make run_all

Expected Result

Test #215: testshell_markdown_tutorial_crypto ........... Passed    2.58 sec

Actual Result

Seems that there is a problem with /tmp on tmpfs or similar? I get the problem on fedora and also on debian buster (in docker).

249/262 Test #215: testshell_markdown_tutorial_crypto ...........***Failed    2.58 sec
Input: /home/mpranj/workspace/libelektra/doc/tutorials/crypto.md
kdb mount test.ini user/tests ini
kdb set user/tests/password 1234
kdb file user/tests/password | xargs cat
kdb rm user/tests/password
kdb umount user/tests
kdb mount test.ini user/tests fcrypt "encrypt/key=$(kdb gen-gpg-testkey)" ini
kdb set user/tests/password 1234

ERROR - RET:
Return value “5” does not match “0”

kdb file user/tests/password | xargs cat

ERROR - RET:
Return value “123” does not match “0”

kdb rm user/tests/password

ERROR - RET:
Return value “11” does not match “0”

kdb umount user/tests
kdb mount test.ini user/tests fcrypt "sign/key=$(kdb gen-gpg-testkey)" ini
kdb set user/tests/password 1234

ERROR - RET:
Return value “5” does not match “0”

kdb file user/tests/password | xargs cat

ERROR - RET:
Return value “123” does not match “0”

kdb rm user/tests/password

ERROR - RET:
Return value “11” does not match “0”

kdb umount user/tests
kdb mount test.ini user/tests fcrypt "sign/key=$(kdb gen-gpg-testkey),encrypt/key=$(kdb gen-gpg-testkey)" ini
kdb set user/tests/password 1234

ERROR - RET:
Return value “5” does not match “0”

kdb file user/tests/password | xargs cat

ERROR - RET:
Return value “123” does not match “0”

kdb rm user/tests/password

ERROR - RET:
Return value “11” does not match “0”

kdb umount user/tests
kdb mount test.ini user/tests crypto_gcrypt "crypto/key=$(kdb gen-gpg-testkey)" base64 ini
kdb meta-set user/tests/password crypto/encrypt 1
kdb set user/tests/password 1234
kdb set user/tests/unencrypted "I am not encrypted"
kdb file user/tests/password | xargs cat
kdb meta-set user/tests/password crypto/encrypt 0
kdb file user/tests/password | xargs cat
kdb rm user/tests/unencrypted
kdb rm user/tests/password
kdb umount user/tests
shell_recorder /tmp/tmp.FLYIzi6Q4K RESULTS: 31 test(s) done 9 error(s).

—— Protocol ————————————————————————————————————————————————————
CMD: kdb mount test.ini user/tests ini
RET: 0

CMD: kdb set user/tests/password 1234
RET: 0
STDOUT: Create a new key user/tests/password with string "1234"

CMD: kdb file user/tests/password | xargs cat
RET: 0
STDOUT: password=1234

CMD: kdb rm user/tests/password
RET: 0

CMD: kdb umount user/tests
RET: 0

CMD: kdb mount test.ini user/tests fcrypt "encrypt/key=$(kdb gen-gpg-testkey)" ini
RET: 0

CMD: kdb set user/tests/password 1234
RET: 5
=== FAILED return value does not match expected pattern 0
STDERR: Sorry, module fcrypt issued the error C01100:
Resource: Renaming file /tmp/test.ini.2110573:1574783987.412609.tmpZJttxQ to /home/mpranj/.config/test.ini.2110573:1574783987.412609.tmp failed. Reason: Invalid cross-device link
ERROR: C01100

CMD: kdb file user/tests/password | xargs cat
RET: 123
=== FAILED return value does not match expected pattern 0
STDERR: cat: /home/mpranj/.config/test.ini: No such file or directory

CMD: kdb rm user/tests/password
RET: 11
=== FAILED return value does not match expected pattern 0
STDERR: Did not find the key

CMD: kdb umount user/tests
RET: 0

CMD: kdb mount test.ini user/tests fcrypt "sign/key=$(kdb gen-gpg-testkey)" ini
RET: 0

CMD: kdb set user/tests/password 1234
RET: 5
=== FAILED return value does not match expected pattern 0
STDERR: Sorry, module fcrypt issued the error C01100:
Resource: Renaming file /tmp/test.ini.2110783:1574783987.564306.tmpRJuvgG to /home/mpranj/.config/test.ini.2110783:1574783987.564306.tmp failed. Reason: Invalid cross-device link
ERROR: C01100

CMD: kdb file user/tests/password | xargs cat
RET: 123
=== FAILED return value does not match expected pattern 0
STDERR: cat: /home/mpranj/.config/test.ini: No such file or directory

CMD: kdb rm user/tests/password
RET: 11
=== FAILED return value does not match expected pattern 0
STDERR: Did not find the key

CMD: kdb umount user/tests
RET: 0

CMD: kdb mount test.ini user/tests fcrypt "sign/key=$(kdb gen-gpg-testkey),encrypt/key=$(kdb gen-gpg-testkey)" ini
RET: 0

CMD: kdb set user/tests/password 1234
RET: 5
=== FAILED return value does not match expected pattern 0
STDERR: Sorry, module fcrypt issued the error C01100:
Resource: Renaming file /tmp/test.ini.2111010:1574783987.747497.tmpti3bSR to /home/mpranj/.config/test.ini.2111010:1574783987.747497.tmp failed. Reason: Invalid cross-device link
ERROR: C01100

CMD: kdb file user/tests/password | xargs cat
RET: 123
=== FAILED return value does not match expected pattern 0
STDERR: cat: /home/mpranj/.config/test.ini: No such file or directory

CMD: kdb rm user/tests/password
RET: 11
=== FAILED return value does not match expected pattern 0
STDERR: Did not find the key

CMD: kdb umount user/tests
RET: 0

CMD: kdb mount test.ini user/tests crypto_gcrypt "crypto/key=$(kdb gen-gpg-testkey)" base64 ini
RET: 0

CMD: kdb meta-set user/tests/password crypto/encrypt 1
RET: 0

CMD: kdb set user/tests/password 1234
RET: 0
STDOUT: Set string to "1234"

CMD: kdb set user/tests/unencrypted "I am not encrypted"
RET: 0
STDOUT: Create a new key user/tests/unencrypted with string "I am not encrypted"

CMD: kdb file user/tests/password | xargs cat
RET: 0
STDOUT: unencrypted=I am not encrypted
#@META crypto/encrypt = 1
password=@BASE64IyFjcnlwdG8wMBEAAACCBjEzmVhqufXSsgK4VPRDUC9GyQxBhocVbgZwimonK+xHaRCSX/blNDSVdIoSRg0n

CMD: kdb meta-set user/tests/password crypto/encrypt 0
RET: 0

CMD: kdb file user/tests/password | xargs cat
RET: 0
STDOUT: unencrypted=I am not encrypted
#@META crypto/encrypt = 0
password=1234

CMD: kdb rm user/tests/unencrypted
RET: 0

CMD: kdb rm user/tests/password
RET: 0

CMD: kdb umount user/tests
RET: 0
————————————————————————————————————————————————————————————————

System Information

Elektra Version: master
Operating System: Fedora
Versions of other relevant software?

Further Log Files and Output

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 17 (17 by maintainers)

Commits related to this issue

fcrypt: try manual copy if rename () fails rename () accross file systems is not supported and causes failures. If rename () fails, fcrypt tries to perform a manual copy. See #3283 for full discussi... — committed to petermax2/libelektra by petermax2 4 years ago
cirrus: enable fcrypt on fedora again #3283 — committed to mpranj/libelektra by mpranj 4 years ago
Merge pull request #3394 from mpranj/fedorafcrypt cirrus: enable fcrypt on fedora again #3283 — committed to ElektraInitiative/libelektra by mpranj 4 years ago

Most upvoted comments

This problem might occur more often, I think. We even recommend in the fcrypt documentation to remount /tmp to a RAM-disk. So we actually recommend to trigger this error straight away.

I will try to provide a proper fix.

I will simply configure a different TMPDIR explicitly for the builds. If the issue reoccurs on target systems we can reopen and find a better solution.

This is also fine as a work-around for the build servers.

petermax2 on Apr 8, 2020

I can not reproduce the issue when building 2bc994ae8b3f3a04396303ece106ea563764e490 from scratch using your cmake command from above.

    Start 188: testshell_markdown_tutorial_crypto 
222/235 Test #188: testshell_markdown_tutorial_crypto ...........   Passed    3.16 sec

Maybe your build directory is broken somehow. Could you please try to compile and test with a new (empty) build directory?

petermax2 on Nov 26, 2019

Thank you so much for looking into it!

A simplistic solution would be to replace the rename operation by a copy + remove.

Not to replace the code but to do that if rename failed. (And to also shred the source file in that case.) Then people without RAM disc on /tmp have speed and people with RAM disc on /tmp have more security.

markus2330 on Apr 9, 2020

A simplistic solution would be to replace the rename operation by a copy + remove.

petermax2 on Apr 8, 2020

From rename docu:

[EXDEV]
[CX] [Option Start] The links named by new and old are on different file systems and the implementation does not support links between file systems. [Option End]

Edit: fails with sudo with the same error:

213: CMD: kdb set user/tests/password 1234
213: RET: 5
213: === FAILED return value does not match expected pattern 0
213: STDERR: Sorry, module fcrypt issued the error C01100:
213: Resource: Renaming file /tmp/test.ini.571145:1586371945.946855.tmpHdGEMt to /root/.config/test.ini.571145:1586371945.946855.tmp failed. Reason: Invalid cross-device link
213: ERROR: C01100

mpranj on Apr 8, 2020

The problem is not isolated to my machine. The test fails because the following rename() call fails: https://github.com/ElektraInitiative/libelektra/blob/263cbe69686a9f087204a7f3660b007d1de22da2/src/plugins/fcrypt/fcrypt.c#L231

The reason is that /tmp is a separate mountpoint on Fedora by default, which was not the case for Debian. Thus the file can not be renamed across different mountpoints.

I have verified that this is the problem by applying a (very dirty) patch. I would propose to keep it consistent with the resolver implementation and place the temp file in the same directory as the original file.

@petermax2 what do you think and would you have time to fix it with a proper patch?

mpranj on Apr 8, 2020

Are you sure this is the same problem?

No, probably several different problems.

In the long run we’ll add some fedora docker images too

Very good idea!

I edited my post from before to clarify what I meant with my analysis.

petermax2 on Nov 27, 2019

Thank you for taking a look at this issue!

I haven’t seen the other tests fail tbh. Are you sure this is the same problem? I did not even work with the installed kdb, I just ran ctest with make run_all.

In the long run we’ll add some fedora docker images too (#3227), to catch something like this earlier.

mpranj on Nov 27, 2019

kdb gen-gpg-testkey can not be found. This issue is related to #3246 (KDB_EXEC_PATH).

EDIT: The analysis is only valid for testshell_markdown_tutorial_crypto. I did not check the other tests so far.

petermax2 on Nov 27, 2019

The issue is reproducable under Fedora.

The following tests FAILED:
 41 - testshell_markdown_base64 (Failed)
 56 - testshell_markdown_csvstorage (Failed)
 76 - testshell_markdown_iconv (Failed)
 78 - testshell_markdown_ini (Failed)
 93 - testshell_markdown_mini (Failed)
116 - testmod_resolver (Failed)
181 - testshell_markdown_tutorial_crypto (Failed)

Maybe this is not a crypto-tutorial specific issue. I have to investigate.

petermax2 on Nov 27, 2019