axios: Axios seems to truncate big data responses intermittently
Describe the bug It appears that axios is intermittently returning async / await requests before they’re completely fulfilled, returning a truncated version of it, in case of payloads with big responses. In my case the response is a json string of ~8M, containing 177.000 lines.
To Reproduce Just a normal async / await request using axios
// http-client.ts
import axios from 'axios';
export class HttpClient {
private readonly baseUrl: string;
constructor(baseUrl: string) {
this.baseUrl = baseUrl;
}
public async getBigResponse(): Promise<BigDataStructure> {
const url = `${this.baseUrl}/test`;
const { data } = await axios.get(url);
return data as any;
}
}
// http-repo.ts
import { HttpClient } from './http-client';
export class HttpRepo {
private httpClient: HttpClient;
constructor(httpClient: HttpClient) {
this.httpClient = httpClient;
}
public async getBitData(): Promise<BigDataStructure> {
let bigData;
try {
bigData = await this.httpClient.getBigResponse();
} catch (error) {
console.error(error.toString());
throw error.toString();
}
if (bigData.data.length === undefined) {
console.log(`Big data undefined: ${bigData}`);
throw `bigData undefined`;
try {
JSON.parse(bigData);
} catch (e) {
console.log(`Error parsing bigData into JSON: ${e.toString()}`);
}
} else {
console.log(`big data ok`);
}
return bigData;
}
}
What happens is that sometimes, randomly, the if (bigData.data.length === undefined)
is evaluated true as the (partial) json string is returned, and not the whole object I’m expecting. The following console line, if printed, will show a partial json, not enclosed. That has been verified by using the JSON.parse
function too.
Expected behavior This should never happen, all the requests should be fulfilled and the object is always returned from the client
Environment:
- Axios Version: 0.19.0
- OS: OSX 10.15.1 (Catalina)
- Additional Library Versions [e.g. React 16.7, React Native 0.58.0] (This refers to my local machine, not to the lambda setup)
Additional context We firstly noticed this problem in an AWS Lambda function, but I was able to reproduce it locally by making multiple requests to the APIs using a simple for loop:
it('test im not crazy', async () => {
const httpClient = new HttpClient('https://axios-truncation-issue.free.beeceptor.com');
const httpRepo = new HttpRepo(httpClient);
for (let i = 0; i < 50; i++) {
console.log(`Starting execution of ${i}`);
const menu = await httpRepo.getBitData();
expect(menu).toHaveLength(1);
}
});
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 3
- Comments: 15 (3 by maintainers)
@chinesedfan I just want to point out that this also happens when running on nodejs. So there is no XHR, but axios use http adapter instead…
Just had this issue. Apache was compressing the response (
Content-Encoding: gzip
), and Axios was intermittently not handling that correctly. We would receive 9779 bytes of data, and we eventually figured out that theContent-Length
header was reporting 9779 bytes, but that is the length of the gzipped data. The actual data was 73,104 bytes. Sometimes we’d get 9,779 bytes, sometimes we’d get 73,104 bytes. By telling Axiosheaders: { 'Accept-Encoding': 'gzip' }
we now consistently receive all data correctly.You should try Gaxios which is a library based on Fetch an has an similar API to Axios. Give it a try