got: GOT not able to handle Passthrough streams

Current Behaviour Hi, I am trying to upload the file stream to another server. Please check the code below

const fd = new FormData({
    readable: true
});
const stream = getFileStream();
fd.append('file', stream, {
    filename: 'upload.txt',
});
await got.post('http://localhost:3001/upload', {
    body: fd,
});

Hi, I am creating a file stream like the below snippet.

const getFileStream = () => {
    const fileStream = createReadStream(filePath);
    const passThrough = new PassThrough();
    return fileStream.pipe(passThrough);
};

I have to use passthrough for some reasons.

If I do something like the above I get an error ‘ERR_GOT_REQUEST_ERROR’

But if I change getFileStream function like below it works basically I am not using Passthrough

const getFileStream = () => {
    const fileStream = createReadStream(filePath);
    const passThrough = new PassThrough();
    return fileStream;
};

If I do the same in Axios and node fetch it’s working with passthrough.

Expected Behaviour: It should upload the file

Node Version v14.18.2

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 16 (3 by maintainers)

Most upvoted comments

@octet-stream @szmarczak Removing the Content-Length header fixed the issue. Now I am able to use passthrough streams. Below I am pasting the working code so that it will act as a reference for someone facing the same issue

/**
 * App that will upload the stream to the server using got
 */
import express from "express";
import { createReadStream } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { PassThrough, Readable } from "node:stream";
import { got } from "got";
import { fileURLToPath } from "node:url";
import cors from "cors";
import { FormData } from "formdata-node";
import { FormDataEncoder } from "form-data-encoder";

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

const app = express();
const filePath = resolve(join(__dirname, "../assets/upload.txt"));
app.use(cors());

/**
 * Imitate streams coming from other destinations
 */
const getFileStream = () => {
  const fileStream = createReadStream(filePath);
  const passThrough = new PassThrough();
  fileStream.pipe(passThrough);
  return passThrough;
};

app.post("/sendFile", async (req, res) => {
  try {
    const fd = new FormData();
    
    fd.set("file", {
      type: "text/plain",
      name: "upload.txt",
      [Symbol.toStringTag]: "File",
      stream() {
        return getFileStream();
      },
    });

    const encoder = new FormDataEncoder(fd);

    const headers = {
      "Content-Type": encoder.headers["Content-Type"],
    };
    await got.post("http://localhost:3001/upload", {
      body: Readable.from(encoder),
      headers: headers,
    });

    res.send("uploaded");
  } catch (error) {
    res.send("uploaded error");
    console.log(error);
  }
});

app.listen(3000, () => {
  console.log("Example app listening on port 3000!");
});

Thanks a lot for the help @octet-stream @szmarczak

vijhhh2 on Aug 17, 2022