nornir: Serialization in grouped tasks breaks when using click

Issue: Click output is not properly using serialization in a grouped task when it is in the subtask and num_worker is set to 1.   If I switch click.confirm with a custom-made input() confirmation it will properly separate the two outputs between threads.

Is there a work around for this? It is quite convenient to be able to use click for prompting like this.   Below is the result I get when utilizing the click package for prompting the user. It puts both confirmations for both switches in the filter on one line. If I answer ‘n’ it will go to a blank new line expecting the answer for the second thread and then proceed like normal. The same thing happens for all click related prompts. It’s not locking them to the thread.

2024-03-11_10-27-51

The code used is summarized below:

def create_port_lists(task: Task, data: Dict[str, Any]) -> Result:
    result = defaultdict(list)
    interfaces = data["data_collection"][task.host.name]["ttp_interfaces"]
    update = []
    ommitPorts = click.confirm(
        f"Does {task.host} have any ports to ommit?", default=False
    )
    if ommitPorts == True:
        click.echo(f"\nPorts:")
        click.echo(f"{interfaces}\n")
        ports = click.prompt(
            "Enter list of ports separated by space (Example: FiveGigabgitEthernet1/0/10 FiveGigabgitEthernet1/0/42)"
        )
        ports = ports.split()
        click.echo(f"\nOmmited Ports:")
        click.echo(f"{ports}\n")
        click.confirm(
            "Is the above list of ommited ports correct?", default=False, abort=True
        )
        for interface in ports:
            result["ommited"].append(interface)
            update.append(interface)
    interfaces = [x for x in interfaces if x not in update]
    task.host.data["ports"] = result
    return Result(host=task.host, result=result)

def examine_ports(task: Task, data: Dict[str, Any], num_worker: int) -> Result:
    task.nornir.runner.num_worker = num_worker
    task.run(task=create_port_lists, data=data)
    return Result(
        host=task.host,
        result=f"{task.host}'s configuration collected and parsed.",
    )
kwargs = {"num_workers": 1}
nr = InitNornir(config_file=NORNIR_PATH, dry_run=dryrun)
data = {}
nr.run(name="data_collection", task=data_collection)
nr.run(name="examine_ports", task=examine_ports, data=data, **kwargs)

About this issue

  • Original URL
  • State: closed
  • Created 4 months ago
  • Comments: 15 (4 by maintainers)

Most upvoted comments

Both methods work. The threading lock and the with_runner. No num_worker argument was needed thanks for the clarification on all of this.

Similar logic to what @ubaumann posted earlier, but this also should work:

<.... your basic imports and normal code >

# I assume relevant interface data is bound to the host as part of this task
result = nr.run(name="data_collection", task=data_collection)

for host_obj in nr.inventory.hosts.values():
    omit_ports(host_obj)

# Additional nr.run calls

Looping over the results should also work.

Thank you for your help. I’ll try one of these two options.

Anyways you should set it to serial for no threading (i.e. serial behavior)

The whole script from globally at the config.yaml file level? Or can I update just the runner plugin object from within the tasks themselves?

https://nornir.readthedocs.io/en/latest/plugins/execution_model.html?highlight=execution flow