Vfrdtyky

I'm trying to get the automl digit classification sample running on my Batch AI cluster. This cell appears to run correctly:

from azureml.core.compute import BatchAiCompute

from azureml.core.compute import ComputeTarget



# Choose a name for your cluster.

batchai_cluster_name = "nc12-cluster-04"



found = False

# Check if this compute target already exists in the workspace.

for ct_name, ct in ws.compute_targets().items():

    if (ct.name == batchai_cluster_name and ct.type == 'BatchAI'):

        found = True

        print('Found existing compute target: {0}'.format(ct.name))

        compute_target = ct

        break



if not found:

    print('Creating a new compute target...')

    provisioning_config = BatchAiCompute.provisioning_configuration(vm_size = "STANDARD_NC12", # for GPU, use "STANDARD_NC6"

                                                                #vm_priority = 'lowpriority', # optional

                                                                autoscale_enabled = True,

                                                                cluster_min_nodes = 0, 

                                                                cluster_max_nodes = 4)



    # Create the cluster.

    compute_target = ComputeTarget.create(ws, batchai_cluster_name, provisioning_config)



    # Can poll for a minimum number of nodes and for a specific timeout.

    # If no min_node_count is provided, it will use the scale settings for the cluster.

    compute_target.wait_for_completion(show_output = True, min_node_count = None, timeout_in_minutes = 20)



     # For a more detailed view of current Batch AI cluster status, use the 'status' property.

and I get the expected output:

Found existing compute target: nc12-cluster-04

But when I try and submit my job like this:

from azureml.core.experiment import Experiment



experiment = Experiment(ws, experiment_name)

remote_run = experiment.submit(automl_config, show_output = False)

I get this error:

~/anaconda3_501/lib/python3.6/site-packages/azureml/_restclient/operations/jasmine_operations.py in post_remote_snapshot_run(self, subscription_id, resource_group_name, workspace_name, project_name, parent_run_id, json_definition, snapshot_id, custom_headers, raw, **operation_config)

    237         if response.status_code not in [200]:

--> 238             raise HttpOperationError(self._deserialize, response)

    239 



HttpOperationError: Operation returned an invalid status code "ErrorMessage: 'BatchAI cluster nc12-cluster-04f7543302 does not exist'. Possible cause: Remote dsvm"

Notice how the cluster name now has a suffix: nc12-cluster-04f7543302 that isn't in the compute_target passed in:

vars(compute_target)



{

...

 'name': 'nc12-cluster-04',

 'provisioning_errors': None,

 'provisioning_state': 'Succeeded',

 ...

}

Any ideas?