hypothesmith: Generated programs are missing spaces
I must be missing something simple. I run this program:
from hypothesis import given
import hypothesmith as hs
@given(hs.from_grammar("eval_input"))
@settings(max_examples=50)
def test(expr):
print(repr(expr))
test()
This prints examples like these:
'A'
'A\n'
'A,A,'
'A,A'
'A,'
'A,A,\n'
'A\n'
'AorA,A'
'AorA,'
'A,\n'
'lambda:A,\n'
I am pretty sure it meant A or A, not AorA. (I saw more similar example in other runs and variations of the program.)
It also occasionally prints a traceback and this error:
hypothesis.errors.FailedHealthCheck: It looks like your strategy is filtering out a lot of data. Health check fo
und 50 filtered examples but only 6 good ones. This will make your tests much slower, and also will probably dis
tort the data generation quite a lot. You should adapt your strategy to filter less. This can also be caused by
a low max_leaves parameter in recursive() calls
This is Python 3.9.2 on Windows.
hypothesis 6.13.0
hypothesmith 0.1.8
I figure Iโm doing something wrong or not understanding something?
About this issue
- Original URL
- State: open
- Created 6 years ago
- Comments: 17 (11 by maintainers)
Just to save it:
List of nodes by category (wip)
I believe I can start by adding the built-in types nodes (like integer)! edit: Ah, no, integers and floats are already registered ๐ Well, lets go with a tuple then, it should be just the same as sets.
Nevermind, Iโve taken the full list of nodes and am ordering them by categories (binary ops, unary ops, boolean ops, etc.). This is quite straightforward, I should be able to answer these questions by myself ๐
After reading your comment a second time and carefully looking at the REGISTERED list and how itโs use in the loop below, it all makes sense ๐
Since I already played with each type of node
astprovides, I believe Iโll be able to add strategies for otherlibcstnodes, and, as you suggested, start with just one in a first PR ๐An idea for following the true grammar: maybe we can augment the PEG parser generator we use for Python 3.9 and newer to spit out the grammar definition in a way thatโs consumable by hypothesmith (or maybe lark?).
.filter(compilable)check, but it might be nice for LibCST if we ran some tests without this in order to report bugs upstream.