nom: separated_list1 by multipspace0 sep failed

Hello, and thank you for submitting an issue to nom!

First, please note that, for family reasons, I have limited time to work on nom, so following the advice here will make sure I will quickly understand your problem and answer as soon as possible. Second, if I don’t get to work on your issue quickly, that does not mean I don’t consider it important or useful. Major version releases happen once a year, and a lot of fixes are done for the occasion, once I have had time to think of the right solution. So I will get back to you 😃

Prerequisites

Here are a few things you should provide to help me understand the issue:

  • Rust version : rustc 1.71.1 (eb26296b5 2023-08-03)
  • nom version :7.1.3
  • nom compilation features used: default

Test case

Please provide a short, complete (with crate import, etc) test case for the issue, showing clearly the expected and obtained results.

Example test case:

    #[test]
    fn test_separated_list1() {
        fn parser(input: &str) -> IResult<&str, Vec<&str>> {
            separated_list1(multispace0, tag("abc"))(input)
        }
        assert_eq!(parser("abcabc"), Ok(("", vec!["abc", "abc"])));
       // thread 'parser::tests::test_cypher' panicked at 'assertion failed: `(left == right)`
       // left: `Err(Error(Error { input: "abc", code: SeparatedList }))`,
       // right: `Ok(("", ["abc", "abc"]))`
    }

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 16 (9 by maintainers)

Most upvoted comments

The return type on that isn’t great, though. You’d have to map the tuple combinator to get the elements on their own. preceded or terminated are probably better options.

We should probably remove the comment that sep must be consuming…

/// # Arguments
/// * `sep` Parses the separator between list elements. Must be consuming.
/// * `f` Parses the elements of the list.

I have moved the loop check in separated_list0 and separated_list1 to cover the application of both parsers: https://github.com/rust-bakery/nom/pull/1756 As @Taywee said

Sure, that makes sense, but I think it would be more robustly handled by returning an error in the case that both sep and f both parse empty in a row, rather than one or the other. As long as one of the two isn’t parsing empty, there’s no risk of an infinite loop.

This will offer enough protection against infinite loops

if the separator does not exist the the elements are not separated and you then do not have the presence of two elements but one

That depends on what’s being parsed. In my case, I have a list of {...} blocks, which I want to be separated by any amount of whitespace (including zero whitespace). I don’t want to allow leading or trailing whitespace, and I always want at least one block. separated_list1(multispace0, block) would have been perfect for my case, but it gave an unhelpful error instead, which is how I found this issue.

Perhaps what is regarded as an optional is really an alternate combinator such as comma or space.

That’s not a problem, because if it’s not optional, it’s not optional. If somebody uses alt((tag(','), tag(' '))) or something, it can’t be skipped anyway.

separated_list1(foo, bar) is a parser that has a mandatory list of bar items separated by mandatory foo items.

separated_list1(opt(foo), bar) is a parser with bar items possibly separated by foo items.

In no situation should it be unclear whether the separator is optional or not, because you can always use a mandatory combinator if you want it to be mandatory. This is one of the major strengths of combinators, that you can build exactly the behavior you want from the building blocks. The suggestion is not making the separator optional, but allowing it to be a potentially non-consuming parser.

An optional sep in a separated list is conceptually invalid

You present this as fact when it’s really just your opinion. The difference between separated_list and a prefix/suffix per element is that the former only looks for the separator between pairs of elements, while the latter also looks for a separator at the very beginning (prefix) or end (suffix). They are not equivalent, regardless of whether the separator is optional.

Sure, that makes sense, but I think it would be more robustly handled by returning an error in the case that both sep and f both parse empty in a row, rather than one or the other. As long as one of the two isn’t parsing empty, there’s no risk of an infinite loop.