sedna: Joint Inference Service is failed on the agent

I am deploying Example1:Using Joint Inference Service in Helmet Detection Scenario.

In the last step Create joint inference service, master is okay, but joint-inference-helmet-detection-little can’t work on the edge: you can see there is no pod on the edge(I have already had the big and little images locally and version is v0.4.3) image

kubectl describe ji

  Edge Worker:
    Hard Example Mining:
      Name:  IBT
      Parameters:
        Key:    threshold_img
        Value:  0.9
        Key:    threshold_box
        Value:  0.9
    Model:
      Name:  helmet-detection-inference-little-model
    Template:
      Spec:
        Containers:
          Env:
            Name:             input_shape
            Value:            416,736
            Name:             video_url
            Value:            rtsp://localhost/video
            Name:             all_examples_inference_output
            Value:            /data/output
            Name:             hard_example_cloud_inference_output
            Value:            /data/hard_example_cloud_inference_output
            Name:             hard_example_edge_inference_output
            Value:            /data/hard_example_edge_inference_output
          Image:              kubeedge/sedna-example-joint-inference-helmet-detection-little:v0.4.3
          Image Pull Policy:  IfNotPresent
          Name:               little-model
          Resources:
            Limits:
              Memory:  2Gi
            Requests:
              Cpu:     100m
              Memory:  64M
          Volume Mounts:
            Mount Path:  /data/
            Name:        outputdir
        Node Name:       wspn2
        Volumes:
          Host Path:
            Path:  /joint_inference/output
            Type:  Directory
          Name:    outputdir
Status:
  Active:  1
  Conditions:
    Last Heartbeat Time:   2022-03-23T13:34:59Z
    Last Transition Time:  2022-03-23T13:34:59Z
    Status:                True
    Type:                  Running
    Last Heartbeat Time:   2022-03-23T13:34:59Z
    Last Transition Time:  2022-03-23T13:34:59Z
    Message:               the worker of service failed
    Reason:                workerFailed
    Status:                True
    Type:                  Failed
  Failed:                  1
  Start Time:              2022-03-23T13:34:59Z
Events:                    <none>

on the edge, edgemesh and sedna are okay, but there is no joint-inference-helmet-detection-little image

docker logs k8s_lc_lc-zlrzg_sedna

image

I have already tried to uninstall sdena and restart it again, the problem still exists.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (2 by maintainers)

Most upvoted comments

OK,I solve this problem by check gm logs

I0721 04:09:35.631703       1 worker.go:180] pod helmet-detection-inference-example-cloud-dfb4c is created successfully for sedna.io/v1alpha1, Kind=JointInference default/helmet-detection-inference-example
W0721 04:09:35.654432       1 worker.go:220] failed to create service for JointInference default/helmet-detection-inference-example, err:services "helmet-detection-inference-example-cloud" already exists

check k8s service kubectl get svc

NAME                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
helmet-detection-inference-example-cloud   ClusterIP   10.105.223.115   <none>        5000/TCP   6d20h
kubernetes                                 ClusterIP   10.96.0.1        <none>        443/TCP    51d

delete svc kubectl delete svc helmet-detection-inference-example-cloud and deploy joint-inferenc is ok。 like this

NAMESPACE     NAME                                             READY   STATUS    RESTARTS   AGE     IP                NODE             NOMINATED NODE   READINESS GATES
default       helmet-detection-inference-example-cloud-5bx58   1/1     Running   0          69s     10.244.0.210      cloud.kubeedge   <none>           <none>
default       helmet-detection-inference-example-edge-cffwv    1/1     Running   0          69s     192.168.31.7      edge.cambricon   <none>           <none>