tree-sitter: Main thread panicks with 'no entry found for key'

I have created a grammar which causes tree-sitter generate to fail with the following error:

$ /home/cu3po42/development/tree-sitter-minerva/node_modules/.bin/tree-sitter generate
thread 'main' panicked at 'no entry found for key', cli/src/generate/render.rs:1049:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error Command failed with exit code 101.

The verbose backtrace is as follows:

$ /home/cu3po42/development/tree-sitter-minerva/node_modules/.bin/tree-sitter generate
thread 'main' panicked at 'no entry found for key', cli/src/generate/render.rs:1049:21
stack backtrace:
   0:     0x564c8314c495 - backtrace::backtrace::libunwind::trace::h14d338b30b3ea0a7
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1:     0x564c8314c495 - backtrace::backtrace::trace_unsynchronized::h73ea91d74a3fd67f
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2:     0x564c8314c495 - std::sys_common::backtrace::_print_fmt::hd42948c952866e12
                               at src/libstd/sys_common/backtrace.rs:78
   3:     0x564c8314c495 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha8f928866ff7571e
                               at src/libstd/sys_common/backtrace.rs:59
   4:     0x564c83177f0c - core::fmt::write::he0c1e5f7426d2718
                               at src/libcore/fmt/mod.rs:1076
   5:     0x564c83144622 - std::io::Write::write_fmt::hf3afc6cfd57d0033
                               at src/libstd/io/mod.rs:1537
   6:     0x564c8314ecd0 - std::sys_common::backtrace::_print::hfc0110703f3696fd
                               at src/libstd/sys_common/backtrace.rs:62
   7:     0x564c8314ecd0 - std::sys_common::backtrace::print::h3f77c6990ddfaa22
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x564c8314ecd0 - std::panicking::default_hook::{{closure}}::heae49580a8d62d75
                               at src/libstd/panicking.rs:198
   9:     0x564c8314ea1c - std::panicking::default_hook::hecc34e3f729e213c
                               at src/libstd/panicking.rs:217
  10:     0x564c8314f313 - std::panicking::rust_panic_with_hook::he82f5d0644692441
                               at src/libstd/panicking.rs:526
  11:     0x564c8314ef0b - rust_begin_unwind
                               at src/libstd/panicking.rs:437
  12:     0x564c831768a1 - core::panicking::panic_fmt::h09c929f06bb87c98
                               at src/libcore/panicking.rs:85
  13:     0x564c83176633 - core::option::expect_failed::hb49e05615ae76cc6
                               at src/libcore/option.rs:1261
  14:     0x564c82f1e544 - tree_sitter_cli::generate::render::Generator::add_parse_table::h69cfb8cfa7f865a7
  15:     0x564c82f124f1 - tree_sitter_cli::generate::render::Generator::generate::h1a94bdd55f338bfc
  16:     0x564c82f25e4a - tree_sitter_cli::generate::render::render_c_code::h100f4f082d98533d
  17:     0x564c82ed4873 - tree_sitter_cli::generate::generate_parser_for_grammar_with_opts::h4a8f7bf507ee7555
  18:     0x564c82ed28ef - tree_sitter_cli::generate::generate_parser_in_directory::hfb424fe8b81fc43a
  19:     0x564c82e83fe6 - tree_sitter::run::h6a21535c52bdfbe2
  20:     0x564c82e811f2 - tree_sitter::main::hd24ee9c5bfe24a1c
  21:     0x564c82e882c3 - std::rt::lang_start::{{closure}}::hffb19e98173c15bc
  22:     0x564c8314f6e3 - std::rt::lang_start_internal::{{closure}}::h5d3ea623498f5f43
                               at src/libstd/rt.rs:52
  23:     0x564c8314f6e3 - std::panicking::try::do_call::hac65e71be769a440
                               at src/libstd/panicking.rs:348
  24:     0x564c8314f6e3 - std::panicking::try::hd4706e264bcf6712
                               at src/libstd/panicking.rs:325
  25:     0x564c8314f6e3 - std::panic::catch_unwind::h948a0fb4a8b3ee82
                               at src/libstd/panic.rs:394
  26:     0x564c8314f6e3 - std::rt::lang_start_internal::h72cc068ed2d0ac53
                               at src/libstd/rt.rs:51
  27:     0x564c82e882b2 - main
  28:     0x7fe65fda91e3 - __libc_start_main
  29:     0x564c82e7f839 - _start
  30:                0x0 - <unknown>
error Command failed with exit code 101.

I tried to create a minimal grammar that exhibits this error and arrived at the following:

module.exports = grammar({
  name: 'NAME',

  rules: {
    source_file: $ => "foo",

    comment: $ => $.bar,
    bar: $ => seq(
        "bar",
    ),

  },
  extras: $ => [$.comment],
});

Performing any of the following will “fix” the problem:

  • Removing extras
  • Setting source_file to $.comment
  • Replacing seq("bar") by "bar"
  • Manually inlining the reference to bar

I therefore assume that while processing the rules, the process gets to neither comment nor bar, since they are not referenced from source_file, and then when checking extras, the rule map isn’t populated in the way it should be. However, upon reviewing the code I found this, which seems to suggest that the extras rules are used as an entry point, so I’m not entirely sure what’s going on.

For now I was able to circumvent this issue by allowing comments to be used directly on the top level.

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 4
  • Comments: 21 (16 by maintainers)

Most upvoted comments

All right, I will stop being so hot-headed about it. We are all volunteers here. I will go back through the issues I’ve reported and figure out whether they’re still valid.

If you want to view the issue tracker as a liability then whatever. I won’t be reporting further issues to this project.

I can confirm I get this in 0.20.8, 0.20.9, and 0.21.0-pre-release-1.

The cause seems to be a choice rule in extras. I had:

  extras: $ => [
    ...
    $.comment,
  ],

  ...

  comment: $ => choice(
    $.line_comment,
    $.block_comment,
  ),

and changing to

  extras: $ => [
    ...
    $.line_comment,
    $.block_comment,
  ],

resolved the issue. Furthermore, the symbol in question which didn’t have a symbol_id was line_comment.

Interestingly, tree-sitter-rust seems to have a workaround for this exact issue: it has an unused comment rule, and extras contains $.line_comment and $.block_comment.

Only the ones that are super-old and I suspect it might have already been fixed. Do you think this approach is incorrect somehow?

@alanz I’ve encountered issues with an accidental duplicate rule name before. Would be nice to have a warning for that.