ray: [tune] Cannot create an implementation based on HyperOpt or Optuna with Ray Tune

What is your question?

I’m trying to create an implementation of Raytune with Hyperopt, I have gone through the provided tutorial, however this is not relevant for how I require my hyperopt implementation. I’m optimizing a Deep Neural Net architecture and use the HyperOpt fmin function for this (this is different than from the example provided for RayTune iirc). When I do try to base me of this example from the documentation then I keep on getting an Attribute error: “metric unknown” when I try to maximize the accuracy.

Example of how I run my search space in HyperOpt: fn = create_model creates the DNN model of which the architecture is to be optimized. Space holds all the possible hyperparameters that are to be considered. best_run = fmin(fn = create_model, space = test_space, algo = tpe.suggest, max_evals = 10, trials = trials)

Regarding the Optuna library there is no example provided, although this post claims that there are 4 different search algorithm available for Optuna. Can you point me in the right direction for this?

Ray version and other system information (Python version, TensorFlow version, OS): Python Version: 3.6 Tensorflow: 1.14 Ray: 0.8.6 OS: macOS Mojave 10.14.6

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 19 (11 by maintainers)

Most upvoted comments

RE: hparams: Hmm, maybe try pip install -U tensorboardX?

RE: accuracy: please make the following change -

validation_acc = np.amax(result.history['val_acc'])
  
- tune.report(**{'loss': -validation_acc, 'status': STATUS_OK, 'model': model})
+ tune.report(**{'accuracy': validation_acc, 'status': STATUS_OK, 'model': model})

That should fix things for you!

richardliaw on Aug 19, 2020