crawl4ai: [Bug]: RateLimitConfig Class is missing in 0.4.3

crawl4ai version

0.4.3

Expected Behavior

I want to import the RateLimitConfig class but it says it doesnt exist.

Current Behavior

It is greyed out

Is this reproducible?

No

Inputs Causing the Bug


Steps to Reproduce


Code snippets


OS

macOS

Python version

3.11.9

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

Image

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (2 by maintainers)

Most upvoted comments

Thanks @unclecode , probably the main confusion is coming from discrepancy between the docs and the code. Cheers

RCA

@BellaBe mentioned the RateLimitConfig is now deprecated in favor of CrawlerRunConfig (mean_delay, max_range, semaphore_count).

Fix suggestions

So we need to update the documentation.

  • docs/md_v2/api/async-webcrawler.md
  • docs/md_v2/api/parameters.md
  • docs/md_v2/core/browser-crawler-config.md

We are on the lookout for talented Open source contributors. Comment below “Interested” and issue will be assigned to you. @BellaBe Would you be in interested in updating the docs ? 😇 - for some open-source karma 😎

It looks like it was omitted in v0.4.3 versions and some values were flattened into CrawlerRunConfig (mean_delay, max_range, semaphore_count), but the docs are still mentioning it