google-cloud-java: Storage - StorageWriter how to fail the entire write of InputStream

All documentation points to the following being the best practice for uploading a large file

String bucketName = "my_unique_bucket";
 String blobName = "my_blob_name";
 BlobId blobId = BlobId.of(bucketName, blobName);
 InputStream inputStream = new FileInputStream(new File("largefile.zip"));
 BlobInfo blobInfo = BlobInfo.builder(blobId).contentType("application/octet-stream").build();
 try (WriteChannel writer = storage.writer(blobInfo)) {
   try {
       while ((limit = inputStream.read(buffer)) >= 0) {
            writer.write(ByteBuffer.wrap(buffer, 0, limit));
       }
   } catch (Exception ex) {
     // handle exception
   }
 }

The question is how do we fail an upload if an exception is thrown midway?

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 15 (9 by maintainers)

Commits related to this issue

chore(main): release 3.4.0 (#1232) :robot: I have created a release *beep* *boop* --- ## [3.4.0](https://github.com/googleapis/java-asset/compare/v3.3.1...v3.4.0) (2022-06-30) ### Features * Ena... — committed to suztomo/google-cloud-java by release-please[bot] 2 years ago
chore(deps): update dependency com.google.cloud:libraries-bom to v26.1.1 (#1232) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the follo... — committed to googleapis/google-cloud-java by renovate-bot 2 years ago

Most upvoted comments

A bit more information that might be useful: we don’t use resumable uploads. What we would like to guarantee is that a halfwritten file is not commited to GCS. The solution was not to close the writer which led to the question about finalizer or GC closing it automatically.

fondberg on Sep 26, 2016

FYI @stephenplusplus

mziccard on Sep 15, 2016

WriteChannel finalizes the upload when close() is called. To avoid finalizing the upload when an exception is thrown you can use explicit close() instead of try-with-resources. Roughly, something like:

WriteChannel writer = storage.writer(blobInfo);
boolean uploadFailed = false;
try {
  while ((limit = inputStream.read(buffer)) >= 0) {
    writer.write(ByteBuffer.wrap(buffer, 0, limit));
  }
} catch (Exception ex) {
  uploadFailed = true;
  // handle exception
}
if (!uploadFailed) {
  writer.close();
}

We might consider adding an abort() method to WriteChannel to be called inside a catch statement to prevent following close() from finalizing the stream. /cc @aozarov

mziccard on Sep 8, 2016