Ax: Failed and abandoned trials: indicating trial failures and and excluding certain parameter values from future trials
I’m trying to optimize an evaluation function which consists of an optimization routine.
For certain combinations of the parameters, the routine does not converge and results in an invalid result.
I am currently using the GPEI optimizer and developer API with the SimpleExperiment class.
I’d like to be able to indicate this when this happens and also somehow exclude those values from future trials, but wasn’t sure how to do this.
I think #329 and #176 are related, but they don’t quite answer the question that I am looking for.
I couldn’t really find any documentation about how to use mark_abandoned, which seems like what I need based on the explanation.
Could I receive some suggestions for what to look at and how to implement my desired functionality?
Thanks.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 22 (12 by maintainers)
@jangkj09 you’re right actually! I was getting confused about the difference between SimpleExperiment and regular Experiment. My example is really meant more for Experiment. SimpleExperiment does the trial evaluation automatically behind-the-scenes when you call
eval, which makes it a bit harder to accomplish what you want. You probably want to use either regular Experiment or the Service API.Thanks for all your help!