wallabag: Wrong display in wallabag lesjours.fr

Before submitting the issue, please read: If wallabag can’t parse / extract content for a given link, please first read the documentation about it: http://doc.wallabag.org/en/user/errors_during_fetching.html#how-can-i-help-to-fix-that

We have a lot of requests about fetching config issue. It’ll help us A LOT if you give a try to fix it on your own following the doc. If you failed to fix it yourself, tick the following boxes:

  • I’ve tried myself without success
  • I’ve replaced HOST in the issue title with the host of the URL that can’t be fetched (ie: nytimes.com, 20minutes.fr, bbc.com, etc.)

Content related:

Describe what’s wrong:

Don’t have any content. The title is “ep5-allocution-macron”.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 44 (7 by maintainers)

Commits related to this issue

Most upvoted comments

Hello,

I’m sorry, I’ve just found this ticket. I’m a developer at lesjours.fr. I can’t find Wallabag traffic in the Cloudflare logs.

Do you think it would be possible for Wallabag to use a specific user-agent so it’s easier to identify the traffic and allow it through Cloudflare? Something like Mastodon for exemple:

Mastodon/4.3.2 (http.rb/5.2.0; +https://piaille.fr/)
Mastodon/4.3.2 (http.rb/5.2.0; +https://laserdisc.party/) Bot

Hi @qwertygc @HolgerAusB ,

I finally managed to find some time. 🙂 I slightly changed your version @HolgerAusB to clean the article content. I also changed the User-Agent to avoid using the identity of a Mastodon server (Wallabag-Test is allowed). This is the version I have :

http_header(User-Agent): Wallabag-Test

title: //h1[@class="h2"]
body: //div[@class="article-holder"]

prune: no
tidy: no

# Clean the body
strip: //*[@id="bulletpoints"]
strip: //div[@class="episode-extra-container"]

# Activate graphics:
find_string: src="data
replace_string: foo="data
find_string: data-src="
replace_string: src="

# Wallabag-specific login directives (not supported in FTR)
requires_login: yes

login_uri: https://lesjours.fr/connexion
login_username_field: login-mail
login_password_field: login-password

not_logged_in_xpath: //*[@id="account"]//a[@href="#login"]

test_url: https://lesjours.fr/obsessions/pole-financier/ep12-marcel-campion/

I replaced the content of vendor/j0k3r/graby-site-config/lesjours.fr.txt with theses lines and I tried it on several articles without having any issues. Let me know if you still have issues.

Hi @HolgerAusB This one not_logged_in_xpath: //*[@id="account"]//a[@href="#login"] is safer to check if the user is not connected. It’s the login button in our header so it’s a very stable element of our code.

I’ll try to have a look at your configuration file in the coming days.

I changed our Cloudflare configuration to allow traffic with your Wallabag-Test/5.0 user-agent. 🙂 I think it can be nice to have a default user-agent like Wallabag/%WALLBAG_VERSION% (Guzzle/%GUZZLE_VERSION%), for example: Wallabag/2.6.10 (Guzzle/5.3.4).

I’ll try to open a PR for that.

Actually, I have the same issue with the version on the master branch. The user-agent defined in the configuration file for the website is not passed here: https://github.com/wallabag/wallabag/blob/5b71ede258d95c3533adcac57d51c1829e4ad56e/src/Helper/HttpClientFactory.php#L52

The connexion request is using the Guzzle’s default user-agent: Guzzle/<Guzzle_Version> curl/<curl_version> PHP/<PHP_VERSION>

I’ll try to open a PR in the coming days

Hello @qwertygc @e-adrien, I also have an account if I can help for the debug