tree-sitter: Main thread panicks with 'no entry found for key'

I have created a grammar which causes tree-sitter generate to fail with the following error:

$ /home/cu3po42/development/tree-sitter-minerva/node_modules/.bin/tree-sitter generate
thread 'main' panicked at 'no entry found for key', cli/src/generate/render.rs:1049:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error Command failed with exit code 101.

The verbose backtrace is as follows:

$ /home/cu3po42/development/tree-sitter-minerva/node_modules/.bin/tree-sitter generate
thread 'main' panicked at 'no entry found for key', cli/src/generate/render.rs:1049:21
stack backtrace:
   0:     0x564c8314c495 - backtrace::backtrace::libunwind::trace::h14d338b30b3ea0a7
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1:     0x564c8314c495 - backtrace::backtrace::trace_unsynchronized::h73ea91d74a3fd67f
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2:     0x564c8314c495 - std::sys_common::backtrace::_print_fmt::hd42948c952866e12
                               at src/libstd/sys_common/backtrace.rs:78
   3:     0x564c8314c495 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha8f928866ff7571e
                               at src/libstd/sys_common/backtrace.rs:59
   4:     0x564c83177f0c - core::fmt::write::he0c1e5f7426d2718
                               at src/libcore/fmt/mod.rs:1076
   5:     0x564c83144622 - std::io::Write::write_fmt::hf3afc6cfd57d0033
                               at src/libstd/io/mod.rs:1537
   6:     0x564c8314ecd0 - std::sys_common::backtrace::_print::hfc0110703f3696fd
                               at src/libstd/sys_common/backtrace.rs:62
   7:     0x564c8314ecd0 - std::sys_common::backtrace::print::h3f77c6990ddfaa22
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x564c8314ecd0 - std::panicking::default_hook::{{closure}}::heae49580a8d62d75
                               at src/libstd/panicking.rs:198
   9:     0x564c8314ea1c - std::panicking::default_hook::hecc34e3f729e213c
                               at src/libstd/panicking.rs:217
  10:     0x564c8314f313 - std::panicking::rust_panic_with_hook::he82f5d0644692441
                               at src/libstd/panicking.rs:526
  11:     0x564c8314ef0b - rust_begin_unwind
                               at src/libstd/panicking.rs:437
  12:     0x564c831768a1 - core::panicking::panic_fmt::h09c929f06bb87c98
                               at src/libcore/panicking.rs:85
  13:     0x564c83176633 - core::option::expect_failed::hb49e05615ae76cc6
                               at src/libcore/option.rs:1261
  14:     0x564c82f1e544 - tree_sitter_cli::generate::render::Generator::add_parse_table::h69cfb8cfa7f865a7
  15:     0x564c82f124f1 - tree_sitter_cli::generate::render::Generator::generate::h1a94bdd55f338bfc
  16:     0x564c82f25e4a - tree_sitter_cli::generate::render::render_c_code::h100f4f082d98533d
  17:     0x564c82ed4873 - tree_sitter_cli::generate::generate_parser_for_grammar_with_opts::h4a8f7bf507ee7555
  18:     0x564c82ed28ef - tree_sitter_cli::generate::generate_parser_in_directory::hfb424fe8b81fc43a
  19:     0x564c82e83fe6 - tree_sitter::run::h6a21535c52bdfbe2
  20:     0x564c82e811f2 - tree_sitter::main::hd24ee9c5bfe24a1c
  21:     0x564c82e882c3 - std::rt::lang_start::{{closure}}::hffb19e98173c15bc
  22:     0x564c8314f6e3 - std::rt::lang_start_internal::{{closure}}::h5d3ea623498f5f43
                               at src/libstd/rt.rs:52
  23:     0x564c8314f6e3 - std::panicking::try::do_call::hac65e71be769a440
                               at src/libstd/panicking.rs:348
  24:     0x564c8314f6e3 - std::panicking::try::hd4706e264bcf6712
                               at src/libstd/panicking.rs:325
  25:     0x564c8314f6e3 - std::panic::catch_unwind::h948a0fb4a8b3ee82
                               at src/libstd/panic.rs:394
  26:     0x564c8314f6e3 - std::rt::lang_start_internal::h72cc068ed2d0ac53
                               at src/libstd/rt.rs:51
  27:     0x564c82e882b2 - main
  28:     0x7fe65fda91e3 - __libc_start_main
  29:     0x564c82e7f839 - _start
  30:                0x0 - <unknown>
error Command failed with exit code 101.

I tried to create a minimal grammar that exhibits this error and arrived at the following:

module.exports = grammar({
  name: 'NAME',

  rules: {
    source_file: $ => "foo",

    comment: $ => $.bar,
    bar: $ => seq(
        "bar",
    ),

  },
  extras: $ => [$.comment],
});

Performing any of the following will “fix” the problem:

  • Removing extras
  • Setting source_file to $.comment
  • Replacing seq("bar") by "bar"
  • Manually inlining the reference to bar

I therefore assume that while processing the rules, the process gets to neither comment nor bar, since they are not referenced from source_file, and then when checking extras, the rule map isn’t populated in the way it should be. However, upon reviewing the code I found this, which seems to suggest that the extras rules are used as an entry point, so I’m not entirely sure what’s going on.

For now I was able to circumvent this issue by allowing comments to be used directly on the top level.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 6
  • Comments: 34 (20 by maintainers)

Commits related to this issue

Most upvoted comments

If you want to view the issue tracker as a liability then whatever. I won’t be reporting further issues to this project.

All right, I will stop being so hot-headed about it. We are all volunteers here. I will go back through the issues I’ve reported and figure out whether they’re still valid.

@vzaliva apologies if you already did but did you try removing $.comment and putting $.line_comment and $.block_comment in extras directly? This is what solved my issue (see above).

If that doesn’t work you can bisect the problem. Remove some parts from your grammar and patch up the rest so that it should generate. If you get the “no entry found for key” error remove more, if you don’t get the error add some back, if you get a different error fix it and retry.

you’re gonna have to post your grammar somewhere else there’s nothing we can do

Here it is: https://github.com/rems-project/tree-sitter-core/blob/main/grammar.js

The problem started when I did more elaborate comments parsing (starting from the comment rule).

yes, GH auto closed it

I can confirm I get this in 0.20.8, 0.20.9, and 0.21.0-pre-release-1.

The cause seems to be a choice rule in extras. I had:

  extras: $ => [
    ...
    $.comment,
  ],

  ...

  comment: $ => choice(
    $.line_comment,
    $.block_comment,
  ),

and changing to

  extras: $ => [
    ...
    $.line_comment,
    $.block_comment,
  ],

resolved the issue. Furthermore, the symbol in question which didn’t have a symbol_id was line_comment.

Interestingly, tree-sitter-rust seems to have a workaround for this exact issue: it has an unused comment rule, and extras contains $.line_comment and $.block_comment.

Only the ones that are super-old and I suspect it might have already been fixed. Do you think this approach is incorrect somehow?

@alanz I’ve encountered issues with an accidental duplicate rule name before. Would be nice to have a warning for that.