Skip to content

Option to keep generating examples even if < 10% satisfy assumptions #4623

@ajdavis

Description

@ajdavis

I have a situation where my assume() call rejects more than 90% of example inputs. It's hard to randomly generate inputs that pass assume() in my case, so I'd like Hypothesis to just keep trying. This code stops iterating after max_examples * 10 tries, however, with the message "settings.max_examples={s.max_examples}, but < 10% of examples satisfied assumptions". The logic is not configurable. I propose to make it configurable, if that's something you're interested in? I don't yet have a proposal for what the option will be called.

(Background: I'm generating inputs to test a program which uses a SAT solver. It's hard to predict which combinations will lead to a solution or no-solution. I don't want to figure that out, especially since whatever logic I use to make solvable inputs might be overly restrictive and not fully test the program. I'd rather keep making random inputs, assume() that the SAT solver finds a solution, then assert things about the solution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementit's not broken, but we want it to be better

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions