firebase-tools: Storage hangs without completing or errors when uploading or deleting files in parallel
[REQUIRED] Environment info
firebase-tools: 11.7.0
Platform: Windows 10 Pro running Ubuntu via WSL2
[REQUIRED] Test case
As you start to increase the number of items attempted to be uploaded to the storage emulator you will increase the chances of that upload failing. It will fail will even relatively small numbers of parallel uploads if attempted enough time (e.g. 2 or 3 files). I believe I also had it fail with a single file but haven’t been able to reproduce as the failure is much less likely.
For example, if you try to upload 5x files via an approach similar to the one below: The first time maybe 3/5 will upload next 2/5 next 4/5 next 5/5
[REQUIRED] Steps to reproduce
`
// Upload Issue
const uploadFiles = () => {
let uploadTasks = [];
for (let i = 0; i < files.value.length; i++) {
let curFile = files.value[i];
let storageRef = fb_ref(storage, "imgs/" + curFile.name);
const uTask = uploadBytesResumable(storageRef, curFile);
uploadTasks.push(uTask);
// Listen for state changes, errors, and completion of the upload.
uploadTasks[i].on(
"state_changed",
(snapshot) => {
// Get task progress, including the number of bytes uploaded and the total number of bytes to be uploaded
console.log("stage changed for ", i, snapshot);
const progress = snapshot.bytesTransferred / snapshot.totalBytes;
//Update Progress
uploadProgress.value[i].percent = progress;
uploadProgress.value[i].color = "green-2";
if (i == 1) {
uploadProgress.value[i].color = "purple-2";
}
const progressPercent = progress * 100;
// console.log("Upload is " + progressPercent + "% done");
switch (snapshot.state) {
case "paused":
console.log("Upload is paused");
break;
case "running":
console.log("Upload is running");
break;
}
},
(error) => {
console.log("failed ============================== ", error);
switch (error.code) {
case "storage/unknown":
// Unknown error occurred, inspect error.serverResponse
break;
}
},
() => {
// Upload completed successfully, now we can get the download URL
//Update Progress
console.log("Trying to get url for foo: ", i);
getDownloadURL(uploadTasks[i].snapshot.ref)
.then((downloadURL) => {
console.log("File Download foo: " + i, downloadURL);
})
.catch((e) => {
console.log("error foo: ", e);
});
}
);
}
Promise.all(uploadTasks).then(() => {
console.log("Files done uploading");
});`
// Delete Issue
try {
let deletePromises = [];
console.log("deleting storage ", imgArray);
imgArray.forEach((img) => {
let imgPath = `${img.path}${img.name}`;
let deleteRef = ref(storage, imgPath);
console.log("path: ", imgPath);
deletePromises.push(deleteObject(deleteRef));
});
await Promise.all(deletePromises);
} catch (e) {
console.log("Error: ", e);
} finally {
Loading.hide();
}
[REQUIRED] Expected behavior
It should be possible to consistently upload or delete multiple files in parallel to storage
[REQUIRED] Actual behavior
The process will randomly hang without showing any errors. It just never completes.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 24 (10 by maintainers)
I’m seeing the same issue. I get lines like this in the console and the storage emulator stops responding.
I’m calling
getDownloadUrl
to fetch multiple images from storage at once. It doesn’t break every time, but it’s often enough that the storage emulator is basically useless.I’m seeing a similar issue only with
getDownloadUrl()
Using the emulators and looping through image references, it hangs on the promise.Doesn’t appear to be a staging or production issue
I experienced the same issue on MacOS with firebase-tools 11.8.0