Scrapegraph-ai: [1.14.0+] pydantic ValidationError with SmartScraperGraph
Describe the bug Since version 1.14.0 (tested here on 1.15.0), SmartScraperGraph on OpenAI stopped working with a pydantic-related error:
/venv/lib/python3.12/site-packages/google_crc32c/__init__.py:29: RuntimeWarning: As the c extension couldn't be imported, `google-crc32c` is using a pure python implementation that is significantly slower. If possible, please configure a c build environment and compile the extension warnings.warn(_SLOW_CRC32C_WARNING, RuntimeWarning)
--- Executing Fetch Node ---
--- (Fetching HTML from: https://perinim.github.io/projects/) ---
--- Executing Parse Node ---
--- Executing GenerateAnswer Node ---
Traceback (most recent call last):
File "/llm_scraper/test_bug.py", line 50, in <module> result = smart_scraper_graph.run()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/scrapegraphai/graphs/smart_scraper_graph.py", line 114, in run self.final_state, self.execution_info = self.graph.execute(inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/scrapegraphai/graphs/base_graph.py", line 263, in execute return self._execute_standard(initial_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/scrapegraphai/graphs/base_graph.py", line 185, in _execute_standard raise e
File "/venv/lib/python3.12/site-packages/scrapegraphai/graphs/base_graph.py", line 169, in _execute_standard result = current_node.execute(state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/scrapegraphai/nodes/generate_answer_node.py", line 129, in execute answer = chain.invoke({"question": user_prompt})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 2878, in invoke input = context.run(step.invoke, input, config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/langchain_core/output_parsers/base.py", line 192, in invoke return self._call_with_config(
^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 1785, in _call_with_config context.run(
File "/venv/lib/python3.12/site-packages/langchain_core/runnables/config.py", line 397, in call_func_with_variable_args return func(input, **kwargs) # type: ignore[call-arg]
^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/langchain_core/output_parsers/base.py", line 193, in <lambda> lambda inner_input: self.parse_result([Generation(text=inner_input)]),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.12/site-packages/langchain_core/load/serializable.py", line 113, in __init__ super().__init__(*args, **kwargs)
File "/venv/lib/python3.12/site-packages/pydantic/v1/main.py", line 341, in __init__ raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for Generation
text
str type expected (type=type_error.str)
To Reproduce Run the example: https://github.com/ScrapeGraphAI/Scrapegraph-ai/blob/main/examples/openai/smart_scraper_schema_openai.py
I also tried to update to the latest langchain, but that didn’t help. It could be langchain-related, as I can see similar issues (but not exactly the same) on their issues list. Or it’s just a mix of pydantic versions.
Expected behavior It should succeed.
Screenshots
Desktop (please complete the following information):
- OS: MacOS
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 62 (25 by maintainers)
hi, we will fix in the next hours
@rjbks @VinciGit00 I was able to fix the error for both ChatOpenAI and GoogleGenAI (tested on gemini-1.5-pro).
I will open a pull request for this error. At the end it was related with the initialisation problem.
v1.18.1 do not contain the fixed version
It is present in v1.19.0-beta.1+ If it works there it should be fine
Can confirm the same error persists in
1.16.0:Code:
Yeah but langchain is fine with pydantic v2, and I tested without success pydantic v1.
However, I investigated and I found that there is an issue in
GenerateAnswerNode: bothself.llm_model.with_structured_outputandJsonOutputParserare used, but with_structured_output already adds an output parser: https://github.com/langchain-ai/langchain/blob/fabd3295fabb4c79fedb4dbbe725a308658ef8d8/libs/partners/openai/langchain_openai/chat_models/base.py#L1414C25-L1414C36So it’s effectively trying to parse a Pydantic object.
So you can either ask the llm class to return a structured output: https://github.com/ScrapeGraphAI/Scrapegraph-ai/blame/a96617d6f88f7370ecb7c58d0a62d3bdc0d80b31/scrapegraphai/nodes/generate_answer_node.py#L103 Or provide an output parser: https://github.com/ScrapeGraphAI/Scrapegraph-ai/blame/a96617d6f88f7370ecb7c58d0a62d3bdc0d80b31/scrapegraphai/nodes/generate_answer_node.py#L133C47-L133C47
(side note: it seems that
json_schemamode is only implemented for openai chats, not for mistral or others - but langchain will ignore it silently: https://github.com/ScrapeGraphAI/Scrapegraph-ai/blame/a96617d6f88f7370ecb7c58d0a62d3bdc0d80b31/scrapegraphai/nodes/generate_answer_node.py#L100)I get a similar error:
Did you manage to reproduce using the example?
@rjbks
https://github.com/ScrapeGraphAI/Scrapegraph-ai/pull/664#664There seem to be 2, possibly unrelated, issues here. First is boolean json representations throw an error as they are represented in the python syntax (title case like True instead of true). Second seems to be something that is most likely specific to either the max_results number of pages searched, or with a specific chunk of data extracted from a specific page in those search results in my code above. These 2 errors, and not the specific error the OP started the thread with are the issues here. I suppose this belongs in another issue/thread. Change the max_results config value to 20 and see what happens with my specific query/code above. It throws for me consistently.
It is working for me in 1.19beta with OpenAI
Hi, I wanted to let you know that this comment is still valid for version 1.17.0, the latest one on branch pre/beta.
Thanks for reporting this. I’ve had similar issues myself while working on a side project.
I managed to fix my problem by tinkering with the system prompt. I’ll see if I can do the same here, and let you know.