howdoi: Sorry, couldn't find any help with that topic?

Hey, could the Europeans in this thread try out my fork over at #393? I attached a small consent cookie to the request, which seems to have bypassed the Google GDPR Page for me 😃

Duckduckgo works for the first request, then consecutive requests are blocked by the search engine with 403 Client Error: Forbidden

INFO: Fetching answers for query: format date in python
INFO: Searching duckduckgo with URL: https://duckduckgo.com/html?q=site:stackoverflow.com%20format%20date%20in%20python&t=hj&ia=web
INFO: http erro: HTTPError('403 Client Error: Forbidden for url: https://html.duckduckgo.com/html?q=site:stackoverflow.com%20format%20date%20in%20python&t=hj&ia=web')
INFO: result is None
ERROR: Unable to find an answer because the search engine temporarily blocked the request. Please wait a few minutes or select a different search engine.
Traceback (most recent call last):
  File "/howdoi/howdoi/howdoi.py", line 607, in howdoi
    res = _get_answers(args)
  File "/howdoi/howdoi/howdoi.py", line 415, in _get_answers
    question_links = _get_links_with_cache(args['query'])
  File "/howdoi/howdoi/howdoi.py", line 394, in _get_links_with_cache
    links = _get_links(query)
  File "/howdoi/howdoi/howdoi.py", line 281, in _get_links
    raise BlockError('Temporary block by search engine')
howdoi.howdoi.BlockError: Temporary block by search engine

For the folks who are getting blocked, are you in Europe? I read this might be GDPR related.

https://stackoverflow.com/questions/64846902/how-to-get-rid-of-the-google-cookie-pop-up-with-my-selenium-automation https://stackoverflow.com/questions/64927752/google-consent-automatic-accepting

I found these articles that refer to bypassing consent.google.com, but I don’t know how to reliably trigger the page.

For the folks who are getting blocked, are you in Europe? I read this might be GDPR related.

https://stackoverflow.com/questions/64846902/how-to-get-rid-of-the-google-cookie-pop-up-with-my-selenium-automation https://stackoverflow.com/questions/64927752/google-consent-automatic-accepting

I found these articles that refer to bypassing consent.google.com, but I don’t know how to reliably trigger the page.

Yep I think it is related. I base in Europe too.

@kingpitka Aha! So there’s some kind of agreement that Google is requesting. I wonder how we can avoid or accept that.

What happens for bing (and duckduckgo)?

Hi all - thanks for being patient as we try to figure out what is wrong.

I’d like to try our new explainability feature (https://github.com/gleitz/howdoi/pull/381) to see if we can track down the issue.

If you don’t mind, try the following:

git clone https://github.com/gleitz/howdoi.git
git checkout origin/explainability-v2
python -m howdoi format date bash --explain -C

Here’s the output I receive:

» python -m howdoi format date bash --explain -C
INFO: Version: 2.0.14
Cache cleared successfully
INFO: Fetching answers for query: format date bash
INFO: Searching google with URL: https://www.google.com/search?q=site:stackoverflow.com%20format%20date%20bash&hl=en
INFO: stackoverflow.com links found on google: 1
INFO: Answers requested: 1, Starting at position: 1
INFO: Total answers returned: 1
# put current date as yyyy-mm-dd in $date
# -1 -> explicit current date, bash >=4.3 defaults to current time if not provided
# -2 -> start time for shell
printf -v date '%(%Y-%m-%d)T\n' -1

# put current date as yyyy-mm-dd HH:MM:SS in $date
printf -v date '%(%Y-%m-%d %H:%M:%S)T\n' -1

# to print directly remove -v flag, as such:
printf '%(%Y-%m-%d)T\n' -1
# -> current date printed to terminal

Hopefully the output will help us track down this spooky issue! 👻