talos: Adam is not supported by lr_normalizer

Python 3.6, Windows 10, all dependencies are up to date. I have the following code:

def the_model(x_train, y_train, x_val, y_val, params):

    model = Sequential()

    model.add(Dense(params['first_neuron'], input_dim=x_train.shape[1],
                    activation=params['activation'],
                    kernel_initializer=params['kernel_initializer']))
    
    model.add(Dropout(params['dropout']))

    model.add(Dense(params['last_neuron'], activation=params['last_activation'],
                    kernel_initializer=params['kernel_initializer']))
    
    model.compile(loss=params['losses'],
                  optimizer=params['optimizer'](lr_normalizer(params['lr'], params['optimizer'])),
                  metrics=['accuracy', talos.utils.metrics.f1score])
    
    history = model.fit(x_train, y_train, 
                        validation_data=[x_val, y_val],
                        batch_size=params['batch_size'],
                        callbacks=[early_stopper(params['epochs'], mode='moderate')], # talos.utils.live(), 
                        epochs=params['epochs'],

                        verbose=0)

    return history, model

# then we can go ahead and set the parameter space
p = {'first_neuron':[32, 64, 128],
     'lr':[1],   # Here is the learning rate
     'hidden_layers':[1, 2, 3],
     'last_neuron':[1],
     'batch_size': [32, 64, 128, 256],
     'epochs': [training_runs],
     'dropout': [0, 0.25, 0.5],
     'shapes':['brick', 'funnel', 'triangle'],
     'kernel_initializer': ['uniform','normal'],
     'weight_regulizer':[None],
     'emb_output_dims': [None],
     'optimizer': ['Adam'],
     'losses': ['binary_crossentropy'],
     'activation':['relu', 'elu'],
     'last_activation': ['sigmoid']}

# and run the experiment
t = talos.Scan(x=X_SET, y=Y_SET, 
    model=the_model, 
    params=p, 
    val_split=test_size, 
    experiment_name='stock_1_day_classifier', 
    round_limit=4, 
    clear_session='tf')

And get the error mentioned in the title. How does one get this working? What am I missing? I think I’m following the user guide as directed and the example scripts weren’t clear so I’m wondering what I might need to fix. Thanks!

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (5 by maintainers)

Most upvoted comments

I should clarify that I did make the change to Adam (from ‘Adam’) and it wasn’t working. I’m at work now but I can post my trace back for the error using Adam as the function a little later tonight. Will advise.

@mikkokotila I just discovered if I use ‘adam’ instead of ‘Adam’ it works. I looked at my previous standalone Tensorflow code and had the optimizer defined as ‘adam’ so I did that and it works. Thanks!!

UPDATE

I spoke too soon, the same issue arose, I had the LR param commented out. Anything else I could try? Thanks!