pleroma-bot: Failing to process tweets - multiple issues
Hi @robertoszek, Posting from Twitter to Mastodon here. I am using a Twitter dev token with Elevated access.
I have 3 users, two of which are working fine. Whenever I try to gather tweets for the third one, I get the following error:
Error log
ℹ 2023-01-21 14:36:18,416 - pleroma_bot - INFO - config path: /home/mastodon/pleroma-bot/config.yml
ℹ 2023-01-21 14:36:18,416 - pleroma_bot - INFO - tweets temp folder: /home/mastodon/pleroma-bot/tweets
ℹ 2023-01-21 14:36:18,422 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 14:36:18,422 - pleroma_bot - INFO - Processing user: user1 (up and running)
✖ 2023-01-21 14:36:19,315 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:717)
Traceback (most recent call last):
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 577, in main
raise Exception(
Exception: Invalid forceDate format, use "YYYY-mm-dd"
ℹ 2023-01-21 14:36:19,315 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 14:36:19,316 - pleroma_bot - INFO - Processing user: user2 (up and running)
✖ 2023-01-21 14:36:20,066 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:717)
Traceback (most recent call last):
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 577, in main
raise Exception(
Exception: Invalid forceDate format, use "YYYY-mm-dd"
ℹ 2023-01-21 14:36:20,067 - pleroma_bot - INFO - ======================================
ℹ 2023-01-21 14:36:20,067 - pleroma_bot - INFO - Processing user: problematic new user
ℹ 2023-01-21 14:36:21,980 - pleroma_bot - INFO - How far back should we retrieve tweets from the Twitter account?
ℹ 2023-01-21 14:36:21,980 - pleroma_bot - INFO -
Enter a date (YYYY-MM-DD):
[Leave it empty to retrieve *ALL* tweets or enter 'continue'
if you want the bot to execute as normal (checking date of
last post in the Fediverse account)]
2022-10-01
⚠ 2023-01-21 14:36:30,552 - pleroma_bot - WARNING - Raising max_tweets to the maximum allowed value (_utils.py:606)
Gathering tweets... 1207
ℹ 2023-01-21 14:36:47,105 - pleroma_bot - INFO - tweets gathered: 1207
Processing tweets... : 0%| | 0/1207 [00:00<?, ?it/s]
✖ 2023-01-21 14:36:47,549 - pleroma_bot - ERROR - Exception occurred for user, skipping... (cli.py:717)
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 103, in process_tweets
tweet["text"] = _get_rt_text(self, tweet)
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_processing.py", line 280, in _get_rt_text
tweet_ref = self._get_tweets("v2", tweet_ref_id)
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 548, in _get_tweets
tweets_v2 = self._get_tweets_v2(
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 662, in _get_tweets_v2
response = self.twitter_api_request(
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_twitter.py", line 83, in twitter_api_request
logger.info(_(
TypeError: 'list' object is not callable
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/cli.py", line 654, in main
tweets_to_post = process_parallel(
File "/home/mastodon/.local/lib/python3.10/site-packages/pleroma_bot/_utils.py", line 120, in process_parallel
for idx, res in enumerate(
File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
raise value
TypeError: 'list' object is not callable
I can’t seem to be able to locate the reason why this would be failing. One difference is that the third user has include_rts set to true - but it failed as well when I tried again with include_rts: false on this user. Here’s my (partially redacted) config.yml:
Config
# Global Mapping
#
pleroma_base_url: XXX
max_tweets: 40
twitter_token: XXX
delay_post: 1
# User mapping
users:
- twitter_username: XXX
pleroma_username: XXX
pleroma_token: XXX
signature: false
include_rts: false
include_replies: false
include_quotes: true
visibility: "unlisted"
avoid_duplicates: true
media_upload: true
twitter_bio: true
bio_text: "\U0001F916 BEEP BOOP \U0001F916 \nI'm a bot that mirrors\
\ {{ twitter_username }} Twitter's account. \nAny issues please\
\ contact @XXX \n \n "
- twitter_username: XXX
pleroma_username: XXX
pleroma_token: XXX
signature: false
include_rts: false
include_replies: false
include_quotes: true
visibility: "unlisted"
avoid_duplicates: true
media_upload: true
twitter_bio: true
bio_text: "\U0001F916 BEEP BOOP \U0001F916 \nI'm a bot that mirrors\
\ {{ twitter_username }} Twitter's account. \nAny issues please\
\ contact @XXX \n \n "
- twitter_username: XXX
pleroma_username: XXX
pleroma_token: XXX
signature: false
include_rts: true
include_replies: false
include_quotes: true
visibility: "unlisted"
avoid_duplicates: true
media_upload: true
twitter_bio: true
bio_text: "\U0001F916 BEEP BOOP \U0001F916 \nI'm a bot that mirrors\
\ {{ twitter_username }} Twitter's account. \nAny issues please\
\ contact @XXX \n \n "
Any assistance you could provide is appreciated.
Best, Thomas
About this issue
- Original URL
- State: open
- Created a year ago
- Comments: 18 (7 by maintainers)
That would actually be fantastic if it could do that yeah. I’m assuming this was an already known issue, apologies for bothering you with that. I will post the results of 1.2.1rc8 soon.
I see, I think we’re close to cracking the root cause of the bug. Can you test with
1.2.1rc8?:Nevermind my last comment about missing a text field, reading the traceback again it seems like the
idfield inreferenced_tweetsmay be a list instead.Could you try if that’s the issue by running
1.2.1rc5?:pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.2.1rc5