nearley: Unexpected token with moo lexer

I would like to use the moo lexer together with nearley, but I keep getting the error Unexpected column token for various reasons for very simple strings, such as "description contains abc".

I’m trying to implement some kind of searching language where you can do things like username = xyz and ("full name" contains " A. " or "full name" contains " B. ").

Before moving to the moo lexer everything was working kind of nicely. Perhaps by having a look at the grammar someone knows what I am doing wrong? That would really help me πŸ˜ƒ

@preprocessor typescript

@{%
const moo = require('moo');

const lexer = moo.compile({
    ws: {match: /\s+/, lineBreaks: true},
    connector: /and|or/, // Any difference with `connector: ['and', 'or']`?
    leftParen: '(',
    rightParen: ')',
    field: /(?:[A-Za-z][A-Za-z0-9_]*)|(?:"[A-Za-z][A-Za-z0-9_ ]*?")/,
    operator: /contains|not contains|=|<|>/,
    value: /(?:[^\s()"]+)|(?:"[^\s()"]*?")/
});
%}

@lexer lexer

bexp -> bexp %ws %connector %ws bexp
bexp -> %leftParen _ bexp _ %rightParen
bexp -> pred

pred -> %field %ws %operator %ws %value

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 16

Most upvoted comments

Hooray! πŸŽ‰

I’m glad we got it sorted πŸ˜„

I agree there are a few things we could improve in the documentation; feel free to raise issues or PR these if you have time πŸ˜ƒ

It’s not a big deal to actually have keywords and forbid them as values. I’m more concern about just making it work! πŸ˜