instaloader: unable to fetch HQ version of reshared story/post

it because you fetch too many downloads. try to put some delays or stop downloading for a few hours

What you said is plainly wrong. The “400 Bad Request” response has absolutely nothing to do with “too many requests” or “too many downloads”. In your case, Instagram server would return “Error 429”.

FYI, for debugging purpose, I used a rarely used account to download just 2 stories (a reshared video story and a reshared image story) from 1 target instagram account. Then, I checked the variable self._iphone_struct which should store a dictionary of the json data returned. However, there was no data returned because 400 occurred. Hence, line 1264 of structure.py threw an exception.

[giturfreakon] made it clear that only reshared story/post would throw this error and I can confirm that he is right. Others may have yet to encounter this issue but that doesn’t mean this issue isn’t real. While you yourself thumbing up your own reply would not help, I believe the Instaloader community would appreciate more helpful ideas on how to fix the bug.

it because you fetch too many downloads. try to put some delays or stop downloading for a few hours

Just to clarify. As you said, the key here is to not trigger HTTP 400.

It happens when you trying to retrieve media info for a reposted story (whose mediaid isn’t valid for that API endpoint). And that would cause your session being flagged, or even account suspended.

This is what my PR (#1675) is trying to solve. (Also, I only deal with stories. If anyone can provide a problematic re-shared post, please let me know.)

Using the “wrong UA” won’t cause that; the API would simply return “useragent mismatch” (no 400). And the UA in the code currently, albeit old, won’t even cause that. I never see “useragent mismatch” when using Instaloader myself

(Actually, in my testing, you can even use desktop browser UA to fetch from the iPhone API…)

Once your session/account got flagged, it would constantly throw HTTP 400 on other otherwise valid API endpoints (such as api/v1/users/{}/info/, which we used to retrieve highQ profile pic) regardless your UA.

Hi friends. I was doing tests. Forwarding a request, changing the user agent (with the Firefox development options, in my case).

Error 400, gives it when the request is wrong. For example if you use:

i.instagram.com/api/v1/users/web_profile_info/?username={username}

JSON error:

message:    "useragent mismatch"
status:     "fail"

If we do what I put, changing the (mobile) user agent (complete user agent) and with the right header, of course. The petition is accepted, giving a message 200.

complete user agent (example):

Mozilla/5.0 (iPhone; CPU iPhone OS 15_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Instagram 243.1.0.14.111 (iPhone12,1; iOS 15_5; en_US; en-US; scale=2.00; 828x1792; 382468104)

JSON:

data: -> user: -> ...
status: "ok"

Greetings.

no need to change anything, it was the correct end point for hq images and videos

Hi friends. I was doing tests. Forwarding a request, changing the user agent (with the Firefox development options, in my case).

Error 400, gives it when the request is wrong. For example if you use:

i.instagram.com/api/v1/users/web_profile_info/?username={username}

JSON error:

message:    "useragent mismatch"
status:     "fail"

If we do what I put, changing the (mobile) user agent (complete user agent) and with the right header, of course. The petition is accepted, giving a message 200.

complete user agent (example):

Mozilla/5.0 (iPhone; CPU iPhone OS 15_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Instagram 243.1.0.14.111 (iPhone12,1; iOS 15_5; en_US; en-US; scale=2.00; 828x1792; 382468104)

JSON:

data: -> user: -> ...
status: "ok"

Greetings.

The current included UA works fine. Sometimes you just need to re-create your session/cookie.

have anyone try this old api change?

-scraping-instagram-media-by-adding-at-the-end-of-url-a-1/72582413#72582413

[giturfreakon] made it clear that only reshared story/post would throw this error and I can confirm that he was right. Others may have yet to encounter this issue but that doesn’t mean this issue isn’t real. While you yourself thumbing up your own reply would not help, I believe the Instaloader community would appreciate more helpful ideas on how to fix the bug.

You’re right! ALL my 400’s seem to be reshares. But it seems that instagram blocks the account after enough of those, so the question is: Can these be somehow detected? BEFORE hitting Instagram with a request which clearly could only come from an outside app? I guess the request simply doesn’t work for reshares and therefore it shouldn’t be done at all.

It seems that the following api endpoint no longer works (i.e. api/v1/media/{}/info):

https://github.com/instaloader/instaloader/blob/da0dcb106cd1aba7d44d3dcdd133532f730245ed/instaloader/structures.py#L1175

This line of code appears twice in structure.py. Once inside the class Post and once inside the class Story. Disabling the following lines of code could solve the issue temporarily in my own user case (which deals with video stories). However, I don’t know if it might break other parts of the program for other user cases. My suggestion may not work for everybody.

https://github.com/instaloader/instaloader/blob/da0dcb106cd1aba7d44d3dcdd133532f730245ed/instaloader/structures.py#L1262-L1266

If you wish to disable the same with regards to image stories, disable the following line of codes:

https://github.com/instaloader/instaloader/blob/da0dcb106cd1aba7d44d3dcdd133532f730245ed/instaloader/structures.py#L1233-L1240

To the best of my knowledge, the function that contains the above code aims to find out the best video/image quality to download. It takes the json response returned from the ‘api/v1/media/{}/info/’ endpoint . As I am not familiar with the iphone api endpoints, perhaps someone else could help find out the correct endpoint to use and suggest the appropriate changes to be made.

Alternately, it might also be feasible to replace the iphone api with the instagram web api which offers more stability and consistency.

Thoughts?

yes its downloading normal resolution photos

Thank you for your the reply. How do know which ones that downloaded are normal resolution - is there a way to automatically replace them?

Yeees, I really want to know this, too. And I would also like to disable the fallback to lower resolution versions.

yes its downloading normal resolution photos

Thank you for your the reply. How do know which ones that downloaded are normal resolution - is there a way to automatically replace them?

yes its downloading normal resolution photos

looks like your fork fixed the problem. ty!

you can also use this user agent to make it more legitimate https://developers.whatismybrowser.com/useragents/explore/operating_platform_string/oneplus/8

Precisely from there I obtain it, for the tests.

Greetings.

looks like your fork fixed the problem. ty!

you can also use this user agent to make it more legitimate https://developers.whatismybrowser.com/useragents/explore/operating_platform_string/oneplus/8

@fireattack friend, your code works perfect. Not a single 400 error.

Greetings.

There doesn’t seem to have a way to tell if a post/story is reshared or not in the node data.

{
  "audience": "MediaAudience.DEFAULT",
  "__typename": "GraphStoryVideo",
  "id": "2892247832508454562",
  "dimensions": {
    "height": 1136,
    "width": 640
  },
  "display_resources": [
  ],
  "display_url": "...",
  "media_preview": "...",
  "gating_info": null,
  "fact_check_overall_rating": null,
  "fact_check_information": null,
  "sharing_friction_info": {
    "should_have_sharing_friction": false,
    "bloks_app_url": null
  },
  "media_overlay_info": null,
  "sensitivity_friction_info": null,
  "taken_at_timestamp": 1659002820,
  "expiring_at_timestamp": 1659089220,
  "story_cta_url": null,
  "story_view_count": null,
  "is_video": true,
  "owner": {
    "id": "4009894782",
    "profile_pic_url": "...",
    "username": "caririn.official",
    "followed_by_viewer": true,
    "requested_by_viewer": false
  },
  "tracking_token": "...",
  "has_audio": true,
  "overlay_image_resources": null,
  "video_duration": 12.0,
  "video_resources": [
    {
      "src": "...",
      "config_width": 480,
      "config_height": 852,
      "mime_type": "video/mp4; codecs=\"avc1.42E01E, mp4a.40.2\"",
      "profile": "BASELINE"
    }
  ],
  "tappable_objects": [
    {
      "__typename": "GraphTappableMention",
      "x": 0.5,
      "y": 0.571392141414043,
      "width": 0.8055839082649361,
      "height": 0.8055839082649361,
      "rotation": 0.0,
      "custom_title": null,
      "attribution": null,
      "username": "amusemobile",
      "full_name": "Amuse+(アミュプラ)",
      "is_private": false
    }
  ],
  "story_app_attribution": null,
  "edge_media_to_sponsor_user": {
    "edges": []
  },
  "muting_info": null
}

Otherwise we probably can find a way to only call iPhone API when it’s “original”.

Looks like the error is gone for me - at least for 15 hours.

still got 400 error

it because you fetch too many downloads. try to put some delays or stop downloading for a few hours

What you said is plainly wrong. The “400 Bad Request” response has absolutely nothing to do with “too many requests” or “too many downloads”. In your case, Instagram server would return “Error 429”.

FYI, for debugging purpose, I used a rarely used account to download just 2 stories (a reshared video story and a reshared image story) from 1 target instagram account. Then, I checked the variable self._iphone_struct which should store a dictionary of the json data returned. However, there was no data returned because 400 occurred. Hence, line 1264 of structure.py threw an exception.

[giturfreakon] made it clear that only reshared story/post would throw this error and I can confirm that he was right. Others may have yet to encounter this issue but that doesn’t mean this issue isn’t real. While you yourself thumbing up your own reply would not help, I believe the Instaloader community would appreciate more helpful ideas on how to fix the bug.

https://github.com/instaloader/instaloader/issues/1640#issue-1306778503

Another example that downloading just 1 story would cause Error 400 to occur. This issue is definitely not caused by ‘too many downloads’ .

no need to change anything, it was the correct end point for hq images and videos

During debugging, it returns 400 Bad Request. Perhaps something has changed. Meanwhile, the web api endpoints work fine.

I have experienced the same issue and my account was eventually locked by Instagram. Perhaps there might me some changes to the json response when retrieving stories?

I had to do the reset-password-your-account-is-compromised shinanigan as well. Strangely enough, this only applies to one of my accounts, the other one seems to be doing well. Maybe some kinda trap by Meta?