server: Keeps on getting "Invalid private key" when using tritonclient.grpc with SSL

As mentioned in the subject, I keep on getting, Invalid private key when using python tritonclient.grpc with SSL. Below is the error I am getting:

E0116 08:19:15.076809590  416183 ssl_transport_security.cc:821] Invalid private key.
E0116 08:19:15.076844389  416183 ssl_security_connector.cc:115] Handshaker factory creation failed with TSI_INVALID_ARGUMENT.
E0116 08:19:15.076853196  416183 secure_channel_create.cc:87] Failed to create secure subchannel for secure name <AWS HTTPS ALB>'
E0116 08:19:15.076858538  416183 secure_channel_create.cc:47] Failed to create channel args during subchannel creation.

Below is the snippet code on how I set the ssh_credentials, and initialize InferenceServerClient:

creds = grpc.ssl_channel_credentials(
        root_certificates=open("<path/to/crt>", "rb").read(),
        private_key=open("path/to/key>", "rb").read(),
        certificate_chain=open("path/to/ca-bundle", "rb").read(),
        )

triton_grpc_client = tritongrpcclient.InferenceServerClient(
        url=url,
        ssl=True,
        creds=creds,
        verbose=False)

I also tried this way, but still the result is the same:

triton_grpc_client = tritongrpcclient.InferenceServerClient(
        url=url,
        ssl=True,
        root_certificates="path/to/crt",
        private_key="path/to/key",
        certificate_chain="path/to/ca-bundle",
        verbose=False)

I know that the crt, private key, and ca-bundle are valid because the AWS loadbalancer is accessible and valid by using openssl s_client <url:443>. It returns this:

---
SSL handshake has read 5317 bytes and written 498 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    ...
    ...
    ...
    Start Time: 1673856922
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

Below are the details regarding the tritonclient version I’ve installed:

Name: tritonclient
Version: 2.29.0
Summary: Python client library and utilities for communicating with Triton Inference Server
Home-page: https://developer.nvidia.com/nvidia-triton-inference-server
Author: NVIDIA Inc.
Author-email: sw-dl-triton@nvidia.com

Kindly provide guidance on how to resolve this for I am quite stuck for days already. Thanks in advance!

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Comments: 24 (10 by maintainers)

Most upvoted comments

Yes. But it depends upon the ssl configuration whether you’d want to match the common name or not. I am not very sure how gRPC handles it under the hood. They have some documentation here: https://grpc.io/docs/guides/auth/ @jaysonpryde If you are still struggling, then can you provide us with a simple reproducer and our team can look into more details. cc @jbkyang-nvi @matthewkotila

@tanmayv25 I haven’t tested it yet. Been swamped with other tasks. I’ll find sometime later within the day