skrub: UX: `ValueError: Choice and node names must be unique.` when executing some cells twice in an interactive session

I started to explore the new expressions by running examples/expressions/10_expressions_intro.py in an interactive VS Code session (ipykernel).

Here are the steps to reproduce my experience:

  • Execute all cells in order until the following cell (included):

    aggregated_products = vectorized_products.groupby("basket_ID").agg("mean").reset_index()
    baskets = baskets.merge(aggregated_products, left_on="ID", right_on="basket_ID").drop(
        columns=["ID", "basket_ID"]
    )
    
  • Now realize that you want to change some of the range values of the hparam choices in the preceding cell with the following content:

    n = skrub.choose_int(5, 15, log=True, name="n_components")
    encoder = skrub.choose_from(
        {
            "MinHash": skrub.MinHashEncoder(n_components=n),
            "LSA": skrub.StringEncoder(n_components=n),
        },
        name="encoder",
    )
    vectorizer = skrub.TableVectorizer(high_cardinality=encoder)
    

    For instance, to replace 15 by 50. Or even just re-executing the cell without changing anything.

  • Then execute again the subsequent cells in order until the cell that defines aggregated_products.

  • Then we get:

    ValueError                                Traceback (most recent call last)
    Cell In[30], line 3
          1 # %%
          2 aggregated_products = vectorized_products.groupby("basket_ID").agg("mean").reset_index()
    ----> 3 baskets = baskets.merge(
          4     aggregated_products, left_on="ID", right_on="basket_ID"
          5 ).drop(columns=["ID", "basket_ID"])
    
    File ~/code/skrub/skrub/_expressions/_expressions.py:330, in _check_return_value.<locals>.check_call_return_value(*args, **kwargs)
        328 @functools.wraps(f)
        329 def check_call_return_value(*args, **kwargs):
    --> 330     expr = f(*args, **kwargs)
        331     if "preview" not in expr._skrub_impl.results:
        332         return expr
    
    File ~/code/skrub/skrub/_expressions/_expressions.py:278, in check_expr.<locals>.checked_call(*args, **kwargs)
        276 conflicts = find_conflicts(expr)
        277 if conflicts is not None:
    --> 278     raise ValueError(conflicts["message"])
        279 if (found_df := _find_dataframe(expr, func_name)) is not None:
        280     raise TypeError(found_df["message"])
    
    ValueError: Choice and node names must be unique. The name 'encoder' was used for 2 different objects:
    first object using the name 'encoder':
    choose_from({'MinHash': MinHashEncoder(...), 'LSA': StringEncoder(...)}, name='encoder')
    second object using the name 'encoder':
    choose_from({'MinHash': MinHashEncoder(...), 'LSA': StringEncoder(...)}, name='encoder')
    

    and the only solution to fix this is to go back and re-execute all cells from the definition of the root variables.

I believe this is a spurious exception: the original encoder node should no longer be reachable from the new aggregated_products node on the second execution, so there should not be any ambiguity at this point, no?

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (17 by maintainers)

Most upvoted comments

or we can show the full message always and hope that users will be able to tell if the suggestion applies to them or not 🤔

I would start with that.

That makes sense. although I guess in some cases it could be perceived as a false positive if I consider the script is fine and I don’t intend to re-run the cell. We just discussed it IRL here in the sprint and I think for now we’re going to just improve the error message with your suggestion and add warning blocks in the documentation, and we can always revisit the warning possibility later