talos: Adam is not supported by lr_normalizer
Python 3.6, Windows 10, all dependencies are up to date. I have the following code:
def the_model(x_train, y_train, x_val, y_val, params):
model = Sequential()
model.add(Dense(params['first_neuron'], input_dim=x_train.shape[1],
activation=params['activation'],
kernel_initializer=params['kernel_initializer']))
model.add(Dropout(params['dropout']))
model.add(Dense(params['last_neuron'], activation=params['last_activation'],
kernel_initializer=params['kernel_initializer']))
model.compile(loss=params['losses'],
optimizer=params['optimizer'](lr_normalizer(params['lr'], params['optimizer'])),
metrics=['accuracy', talos.utils.metrics.f1score])
history = model.fit(x_train, y_train,
validation_data=[x_val, y_val],
batch_size=params['batch_size'],
callbacks=[early_stopper(params['epochs'], mode='moderate')], # talos.utils.live(),
epochs=params['epochs'],
verbose=0)
return history, model
# then we can go ahead and set the parameter space
p = {'first_neuron':[32, 64, 128],
'lr':[1], # Here is the learning rate
'hidden_layers':[1, 2, 3],
'last_neuron':[1],
'batch_size': [32, 64, 128, 256],
'epochs': [training_runs],
'dropout': [0, 0.25, 0.5],
'shapes':['brick', 'funnel', 'triangle'],
'kernel_initializer': ['uniform','normal'],
'weight_regulizer':[None],
'emb_output_dims': [None],
'optimizer': ['Adam'],
'losses': ['binary_crossentropy'],
'activation':['relu', 'elu'],
'last_activation': ['sigmoid']}
# and run the experiment
t = talos.Scan(x=X_SET, y=Y_SET,
model=the_model,
params=p,
val_split=test_size,
experiment_name='stock_1_day_classifier',
round_limit=4,
clear_session='tf')
And get the error mentioned in the title. How does one get this working? What am I missing? I think I’m following the user guide as directed and the example scripts weren’t clear so I’m wondering what I might need to fix. Thanks!
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15 (5 by maintainers)
I should clarify that I did make the change to Adam (from ‘Adam’) and it wasn’t working. I’m at work now but I can post my trace back for the error using Adam as the function a little later tonight. Will advise.
@mikkokotila I just discovered if I use ‘adam’ instead of ‘Adam’ it works. I looked at my previous standalone Tensorflow code and had the optimizer defined as ‘adam’ so I did that and it works. Thanks!!
UPDATE
I spoke too soon, the same issue arose, I had the LR param commented out. Anything else I could try? Thanks!