juicefs: [MariaDB] Error 1366: Incorrect string value

What happened:

While rsync from local disk to juicefs mount, it suddenly (after several hours) stopped with

Failed to sync with 11 errors: last error was: open /mnt/juicefs/folder/file.xls: input/output error

On the jfs log, i can find these:

juicefs[187516] <ERROR>: error: Error 1366: Incorrect string value: '\xE9sa sa...' for column `jfsdata`.`jfs_edge`.`name` at row 1
goroutine 43510381 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x65
github.com/juicedata/juicefs/pkg/meta.errno({0x2df8860, 0xc0251d34a0})
        /go/src/github.com/juicedata/juicefs/pkg/meta/utils.go:76 +0xc5
github.com/juicedata/juicefs/pkg/meta.(*dbMeta).doMknod(0xc0000e0c40, {0x7fca7e165300, 0xc00ed28040}, 0x399f6f, {0xc025268160, 0x1f}, 0x1, 0x1b4, 0x0, 0x0, ...)
        /go/src/github.com/juicedata/juicefs/pkg/meta/sql.go:1043 +0x29e
github.com/juicedata/juicefs/pkg/meta.(*baseMeta).Mknod(0xc0000e0c40, {0x7fca7e165300, 0xc00ed28040}, 0x399f6f, {0xc025268160, 0x1f}, 0xc0, 0x7b66, 0x7fca, 0x0, ...)
        /go/src/github.com/juicedata/juicefs/pkg/meta/base.go:594 +0x275
github.com/juicedata/juicefs/pkg/meta.(*baseMeta).Create(0xc0000e0c40, {0x7fca7e165300, 0xc00ed28040}, 0x26b5620, {0xc025268160, 0x2847500}, 0x8040, 0xed2, 0x8241, 0xc025267828, ...)
        /go/src/github.com/juicedata/juicefs/pkg/meta/base.go:601 +0x109
github.com/juicedata/juicefs/pkg/vfs.(*VFS).Create(0xc000140640, {0x2e90348, 0xc00ed28040}, 0x399f6f, {0xc025268160, 0x1f}, 0x81b4, 0x22a4, 0xc0)
        /go/src/github.com/juicedata/juicefs/pkg/vfs/vfs.go:357 +0x256
github.com/juicedata/juicefs/pkg/fuse.(*fileSystem).Create(0xc000153900, 0xc024980101, 0xc022a48a98, {0xc025268160, 0x1f}, 0xc022a48a08)
        /go/src/github.com/juicedata/juicefs/pkg/fuse/fuse.go:221 +0xcd
github.com/hanwen/go-fuse/v2/fuse.doCreate(0xc022a48900, 0xc022a48900)
        /go/pkg/mod/github.com/juicedata/go-fuse/v2@v2.1.1-0.20210926080226-cfe1ec802a7f/fuse/opcode.go:163 +0x68
github.com/hanwen/go-fuse/v2/fuse.(*Server).handleRequest(0xc00179c000, 0xc022a48900)
        /go/pkg/mod/github.com/juicedata/go-fuse/v2@v2.1.1-0.20210926080226-cfe1ec802a7f/fuse/server.go:483 +0x1f3
github.com/hanwen/go-fuse/v2/fuse.(*Server).loop(0xc00179c000, 0x20)
        /go/pkg/mod/github.com/juicedata/go-fuse/v2@v2.1.1-0.20210926080226-cfe1ec802a7f/fuse/server.go:456 +0x110
created by github.com/hanwen/go-fuse/v2/fuse.(*Server).readRequest
        /go/pkg/mod/github.com/juicedata/go-fuse/v2@v2.1.1-0.20210926080226-cfe1ec802a7f/fuse/server.go:323 +0x534
 [utils.go:76]

Nothing was logged at MariaDB side.

Environment:

juicefs version 1.0.0-beta2+2022-03-04T03:00:41Z.9e26080
Ubuntu 20.04
MariaDB 10.4

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 21 (16 by maintainers)

Most upvoted comments

I found this bit of the code that should alter it.

If you create the table with:

MariaDB [test]> CREATE TABLE test (test varchar(255)) character set=utf8mb4;
Query OK, 0 rows affected (0.002 sec)

It works also. Note character set is the range of what it can store. Collation is its sort ordering. Focus on character set.

Per @zhijian-pro 's second comment, what is your show create table jfs_edge? You can manually alter it per the alter table from the code.

Do filesystems actually enforce a correct encoding of utf8 filenames? If not maybe varbinary(255) would be a better database choice.

grooverdan on Mar 15, 2022

@solracsf @grooverdan @mariadb-RoelVandePaar We have changed it to use varbinary(255), thanks for the feedback! For existing JuiceFS volumes, please use the latest JuiceFS from main branch and change the column manually.

davies on Apr 13, 2022

Ok I found the root cause. MariaDB sets sql_mode to:

STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,

Once set to:

ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION (so, removing STRICT_TRANS_TABLES)

After that, executing your program, you get

mysql test insert unicode char success
mariadb test insert unicode char success

But i don’t know how to solve this JuiceFS side… See https://mariadb.com/kb/en/sql-mode/

solracsf on Mar 15, 2022