google-cloud-cpp: [Q] Failed to use GCS client to download file in a basic scenario
Using C++ client to test basic upload/download scenarios with GCS.
TEST(StorageTest, TestGcsSync) {
auto fs = nebula::storage::makeFS("gs", "nebula-com");
auto lfs = nebula::storage::makeFS("local");
auto content = "test";
auto local = lfs->temp(false);
std::ofstream out(local);
out << content;
out.close();
// upload
LOG(INFO) << "upload local file";
fs->sync(local, "cdn/test.txt");
// download
auto local2 = lfs->temp(false);
fs->sync("cdn/test.txt", local2);
std::ifstream in(local2);
std::string v;
std::getline(in, v);
in.close();
EXPECT_EQ(v, content);
}
Basically in the download path - it just makes this function call
google::cloud::Status status = client_->DownloadToFile(bucket_, key, local);
In this test, upload works fine, while download failed with these traces:
...
<< curl(Recv Header): alt-svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
<< curl(Recv Header):
>> curl(Recv Data): size=4
test 74657374
(/Users/shawncao/nebula/build/gcp/src/gcp/google/cloud/storage/internal/curl_handle.cc:151)
2021-02-18T20:31:19.766902000Z [DEBUG] <0x10e7c3dc0> ~CurlHandle == curl(Info): Failure writing output to destination
== curl(Info): stopped the pause stream!
== curl(Info): Connection #0 to host storage.googleapis.com left intact
(/Users/shawncao/nebula/build/gcp/src/gcp/google/cloud/storage/internal/curl_handle.cc:151)
W0218 12:31:19.766983 243023296 GCS.cpp:97] Failed to download: DownloadFileImpl(ReadObjectRangeRequest={bucket_name=nebula-com, object_name=cdn/test.txt, disable-md5-hash=1}, /tmp/nebula.M1aNqa): cannot open download source object - status.message=Permanent error in Read(): EasyPause() - CURL error [43]=A libcurl function was given a bad argument [UNKNOWN]
/Users/shawncao/nebula/src/storage/test/TestStorage.cpp:240: Failure
...
From the trace - looks like content downloaded successfully but failed to write to a temp file passed in as variable “local2”.
Not sure how to dig more, can you help?
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 35 (14 by maintainers)
Just to close the loop. It seems like you are using a super build, which starts here, with no option to pick any particular lib curl:
https://github.com/varchar-io/nebula/blob/745fcbcb3d04b2e95b11e099f8256d182cd2b592/build.mac.sh#L74
That calls our CMake configuration steps here:
https://github.com/varchar-io/nebula/blob/745fcbcb3d04b2e95b11e099f8256d182cd2b592/ext/Gcp_Ext.cmake#L82
Our CMake code to find libcurl is here:
https://github.com/googleapis/google-cloud-cpp/blob/eeeb29f61d31cc407b17dd251c3dcab93c27bd0e/cmake/FindCurlWithTargets.cmake#L26
Most likely
FindCURLfinds the Xcode or the systemlibcurlversion, you need to setup the search path for CMake to find the one in/usr/local/opt/curl.PS: We recently added options so now you only need to list the libraries you want, and avoid the long list of
...=OFF:https://github.com/varchar-io/nebula/blob/745fcbcb3d04b2e95b11e099f8256d182cd2b592/ext/Gcp_Ext.cmake#L60-L74
There do not seem to be further questions, closing for now. Feel free to reopen if needed.
I was about to ask if there are two versions of libcurl in your system.
Which one is real? I mean, if both are available how are we to chose for you? Dependency management is a gigantic headache in C and C++. 😭
Can you tell us how you configured
google-cloud-cpp? What arguments did you pass tocmake? Maybe that gives us some ideas to improve our CMake files to check for things better.You might consider using
vcpkgto install your dependencies, includinggoogle-cloud-cpp. That will give you a consistent set of libraries to work with.Just FYI, I think we can rule out libcurl-7.74 by itself as a source of problems. I built libcurl from source myself, it is possible that the version compiled by homebrew has a lot more features enabled, or it links a weird version of another library.
@shawncao I appreciate that you have already spent quite a while troubleshooting this, but we really will need your help with a way to reproduce this.
I am a bit at a loss here. Can I ask you do do a little more troubleshooting? Well, maybe a lot more. The level of tracing we need is normally disabled at compile-time. Can you change this line:
https://github.com/googleapis/google-cloud-cpp/blob/eeeb29f61d31cc407b17dd251c3dcab93c27bd0e/google/cloud/storage/internal/curl_download_request.cc#L50
to read
GCP_LOG(INFO)instead ofGCP_LOG(TRACE)and run your test again? The log main contain all kinds of confidential information, consider scrubbing it or we can figure out a way for you to send it to my email address (should be guessable, I work at google, my username is the same as my github name).Thanks, I missed the full error message, the error starts with
EasyPause(), fortunately there are only two places where we call this:https://github.com/googleapis/google-cloud-cpp/blob/eeeb29f61d31cc407b17dd251c3dcab93c27bd0e/google/cloud/storage/internal/curl_download_request.cc#L153-L165
and
https://github.com/googleapis/google-cloud-cpp/blob/eeeb29f61d31cc407b17dd251c3dcab93c27bd0e/google/cloud/storage/internal/curl_download_request.cc#L97-L99
I think it is the first one, since it says it was in
Read(). That ugly#ifdefI never liked, it means we are doing something wrong, or at least weird… Did you mention what version of curl you were using? The first time we noticed the problem was with 7.69 or 7.69.1, but maybe it was a problem in earlier versions too and we never tested with (for example) 7.68This is a long shot, but I see in your documentation that you recommend installing libcurl4-gnutls-dev but we test with libcurl4-openssl-dev. What happens if you use libcurl4-openssl-dev instead?
Ack.
The
DownloadToFile()API returns aStatusobject that may have more information, can you log it?