backstage: 🐛 Bug Report: fetch:plain:file fails with stream is not readable

📜 Description

When I created the following template step it fails with an error:

steps:
    - id: fetch-gitignore
      name: Fetch gitignore
      action: fetch:plain:file
      input:
        url: "https://github.com/AcmeInc/test123/blob/main/.gitignore"
        targetPath: '.gitignore'

👍 Expected behavior

i expect it to fetch the file and write it to the workspace directory at the targetPath

👎 Actual Behavior with Screenshots

It fails with the folling error:

2023-03-24T09:10:24.000Z Beginning step Fetch Repo
2023-03-24T09:10:24.000Z info: Fetching plain content from remote URL
2023-03-24T09:10:25.000Z InternalServerError: stream is not readable
    at readStream (/Users/brianfletcher/git-repos/backstage/node_modules/raw-body/index.js:185:17)
    at executor (/Users/brianfletcher/git-repos/backstage/node_modules/raw-body/index.js:120:5)
    at new Promise (<anonymous>)
    at getRawBody (/Users/brianfletcher/git-repos/backstage/node_modules/raw-body/index.js:119:10)
    at Object.buffer (webpack-internal:///../backend-common/src/reading/ReadUrlResponseFactory.ts:48:73)
    at fetchFile (webpack-internal:///../../plugins/scaffolder-backend/src/scaffolder/actions/builtin/fetch/helpers.ts:67:34)
    at async Object.handler (webpack-internal:///../../plugins/scaffolder-backend/src/scaffolder/actions/builtin/fetch/plainFile.ts:61:13)
    at async NunjucksWorkflowRunner.executeStep (webpack-internal:///../../plugins/scaffolder-backend/src/scaffolder/tasks/NunjucksWorkflowRunner.ts:196:13)
    at async NunjucksWorkflowRunner.execute (webpack-internal:///../../plugins/scaffolder-backend/src/scaffolder/tasks/NunjucksWorkflowRunner.ts:258:17)
    at async TaskWorker.runOneTask (webpack-internal:///../../plugins/scaffolder-backend/src/scaffolder/tasks/TaskWorker.ts:88:33)

👟 Reproduction steps

Create a template with the step shown above and run it.

📃 Provide the context for the Bug.

No response

🖥️ Your Environment

No response

👀 Have you spent some time to check if this bug has been raised before?

I checked and didn’t find similar issue

🏢 Have you read the Code of Conduct?

I have read the Code of Conduct

Are you willing to submit PR?

Yes I am willing to submit a PR!

About this issue

Original URL
State: closed
Created a year ago
Reactions: 6
Comments: 39 (31 by maintainers)

Most upvoted comments

We’re on 1.17.5, and have the same issue.

And can confirm that it works fine by fetching from public repositories, but when fetching from our org, it fails with stream is not readable

alper98 on Oct 5, 2023

@brunosantosnet is it possible that you can setup a minimal reproduction repository for this so we can dig in further? Also what change caused this to start breaking and failing? 🙏

benjdlambert on Nov 17, 2023

I’ve used fetch:plain:file several times without any problems. Now I have this problem

brunosantosnet on Nov 16, 2023

Hey, has there been any progress/updates on this?

athifongoqa on Aug 24, 2023

@benjdlambert something like this https://github.com/backstage/backstage/pull/17632

punkle on May 3, 2023

I wonder would someone in the @backstage/catalog-maintainers group be able to help us here. I believe the issue is being surfaced when accessing the buffer of the URL reader which I would expect would work in this case. Perhaps this task is not using the url reader correctly?

https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/src/scaffolder/actions/builtin/fetch/helpers.ts#L79

punkle on Apr 27, 2023

Hi @gavlyukovskiy, I have the same problem, it seems to be related to private repository files. When I try to get the https://github.com/backstage/backstage/blob/master/mkdocs.yml file everything works fine, but when I try to get a repo from my organization (https://github.com/ORG/REPO/blob/main/mkdocs.yml) I get the same error.

2023-04-19T15:02:29.082Z Beginning step Get repo file
2023-04-19T15:02:29.091Z info: Fetching plain content from remote URL
2023-04-19T15:02:29.256Z InternalServerError: stream is not readable
    at readStream (/app/node_modules/raw-body/index.js:185:17)
    at executor (/app/node_modules/raw-body/index.js:120:5)

fabidick22 on Apr 19, 2023