piranha: Process Hangs Due to Recursive Code Transformations in Rule Graphs
Thank you again for the great work. I have a question to ensure I fully understand the expectations of the rule graphs. What I’m trying to do is the following change:
before code =
from tensorflow.keras import layers
sp = layers.builder.config("config1", "5").config("config2", "5").getOrCreate()
after code =
from tensorflow.keras import layers
sp = layers.builder.config("config3", "1").config("config1", "5").config("config2", "5").getOrCreate()
Objective:
The goal is to add .config("config3", "1") to the config chain only if the import statement from tensorflow.keras import layers exists in the file. The change should not occur if the import is different, for examplefrom pytorch.ll import layers. I created a rule graph with two nodes:
- The first node checks if the import exists in the file.
- If the import is found, the second rule is executed to add
.config("config3", "1")to the chain.
Question 1: Infinite Transformation Loop When Adding .config("config3", "1")
Observation:
The process hangs when I add .config("config3", "1") because it matches the search query again. If I change the replace parameter in the extend_method_chain rule to .config("1"), the code changes successfully.
Question 2: Transformation Applied Regardless of Import Presence
Observation:
The second rule, extend_method_chain, is executed and .config("config3", "1") is added to the method chain even when from tensorflow.keras import layers is not present in the file. What I want is the change to happen only if the import exists. Additionally, I also tried to change the scope of the rule to the File, expecting to fix this error, and got below error
pyo3_runtime.PanicException: Could not create scope query for "File".
Blow is the code that hangs
from polyglot_piranha import execute_piranha, PiranhaArguments, Rule, RuleGraph, OutgoingEdges
def main():
code = """from tensorflow.keras import layers
sp = layers.builder.config("config1", "5").config("config2", "5").getOrCreate()"""
find_from_tensorflow_keras_import_layers = Rule(
name="find_from_tensorflow_keras_import_layers",
query="""(
(import_from_statement
module_name: (dotted_name
(identifier) @module_name_part
(identifier) @submodule_name_part)
name: (dotted_name
(identifier) @imported_name)
(#eq? @module_name_part "tensorflow")
(#eq? @submodule_name_part "keras")
(#eq? @imported_name "layers"))
)""", # cs indicates we are using concrete syntax
is_seed_rule=True
)
extend_method_chain = Rule(
name="extend_method_chain",
query="""(
(call
function: (attribute
object: (attribute
object: (identifier) @obj
attribute: (identifier) @builder_method
)
attribute: (identifier) @config_method
)
arguments: (argument_list
(string) @config_arg1
(string) @config_arg2
)
)
(#eq? @obj "layers")
(#eq? @builder_method "builder")
(#eq? @config_method "config")
)""", # cs indicates we are using concrete syntax
replace_node="config_method",
replace="config(\"config3\", \"1\").@config_method",
is_seed_rule=True
)
edge = OutgoingEdges("find_from_tensorflow_keras_import_layers", to=["extend_method_chain"], scope="Parent")
# Create Piranha arguments
piranha_arguments = PiranhaArguments(
code_snippet=code,
language="python",
rule_graph=RuleGraph(rules=[find_from_tensorflow_keras_import_layers, extend_method_chain], edges=[edge], )
)
# Execute Piranha and print the transformed code
piranha_summary = execute_piranha(piranha_arguments)
print(piranha_summary[0].content)
if __name__ == "__main__":
main()
Do you have any suggestions or workarounds to fix both errors? Or am I doing something unexpected?
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 20 (1 by maintainers)
The PR has been opened. Thank you once again for your help.
I agree, I think a warning could have been a better solution here ? I don’t remember the rationale for this design decision
You’re right; I had forgotten. Unfortunately, parent scope only applies to nodes that are actual parents and not their context recursively, so the current workaround is to set it to global.
As for the required changes, I would be happy to review your PRs, but I currently don’t have write access to this repository, so I can’t promise it will be merged. The change is pretty straightforward; you just have to create a scope config file and then add its relative path to the corresponding language, as shown here:
https://github.com/uber/piranha/blob/6f2e9bc4583bf5de7d86ae7b8113ac298061ccd8/src/models/language.rs#L201
In
src/cleanup_rules/python/scope_config.tomlYou may also add other scopes, like class, function, method etc