azure-sdk-for-python: Not able to log in parent run when using hyperdrive
- Package Name: azureml-core
- Package Version: 1.0.72
- Operating System: Windows
- Python Version: 3.7.2
Describe the bug
I’m using HyperDriveConfig
and BayesianParameterSampling
to tune my hyperparamenters. When I submit my experiment, the system creates a run (parent run) which then creates several child runs. Each child run executes my script with the different parameters from BayesianParameterSampling
. Inside my script, I have the following instructions to log the score and all the parameters in the parent run:
run_context.parent.log_row(
name=run_context.id,
metric=np.mean(scores),
learning_rate=learning_rate,
batch_size=batch_size,
epochs=epochs,
num_hidden_layers=len(layer_sizes),
first_layer_num_nodes=first_layer_num_nodes,
last_layer_num_nodes=last_layer_num_nodes
)
But when I look at the parent run in the Azure Machine Learning Studio, nothing is being logged. Logs in the child runs are recorded fine.
To Reproduce Steps to reproduce the behavior:
- Create a script that tries to log something in the parent run:
run.parent.log('test', 1)
- Create experiment
- Create an estimator
- Create a hyperparameter sampling object
BayesianParameterSampling
,RandomParameterSampling
, orGridParameterSampling
- Create HyperDriveConfig
- Submit experiment with all the above
- Go to Azure Machine Learning Studio and look for the run you just submitted and wait until some child runs are finished
- Logs recorded from the child runs should appear in the Metrics tab, but they don’t (and for me the metrics tab doesn’t even appear for the parent runs)
Expected behavior Show logs recorded in the parent run. If not, show me where I can see and download all my parameters and scores data in one central place instead of having to go into each child run
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15
@jorgeso also, if you are looking to access the hyperparameter values and metric of each child run, that can be done using the following SDK method -
https://docs.microsoft.com/en-us/python/api/azureml-train-core/azureml.train.hyperdrive.hyperdriverun?view=azure-ml-py#get-children-sorted-by-primary-metric-top-0--reverse-false--discard-no-metric-false-
we are also looking into how we can provide these details in the UI, but I wanted to send you info on how you can directly access the data in the meanwhile
@jorgeso got it. Re: the parallel co-ordinates chart, we do plan to add it to the ui in the future, but for now the only place to access this visual is in the notebook widget.
can you try to reload the run once it completes, to load the parallel coordinates visual? here is a sample snippet of code you can execute in the notebook, once the parent run completes -
hd_run_2=HyperDriveRun(experiment, <parent_run_Id>) from azureml.widgets import RunDetails RunDetails(hd_run_2).show()
@jorgeso thanks for bringing this up. we’ll look at the issue on our end and get back to you
@swatig007 those are good solutions! thanks!