chaos-mesh: AttachNotSupportedException! JVM Chaos Injection failed.

Bug Report

What version of Kubernetes are you using? 1.20.4

What version of Chaos Mesh are you using? v2.2.0

What did you do?

inject JVMChaose:

kind: JVMChaos
apiVersion: chaos-mesh.org/v1alpha1
metadata:
  namespace: pt-shared-services
  name: memory-stress-uc-1
  annotations:
    experiment.chaos-mesh.org/pause: 'false'
spec:
  selector:
    namespaces:
      - pt-shared-services
    labelSelectors:
      app: uc-grpc
  mode: all
  action: stress
  port: 9277
  memType: heap
  name: '--stress-1654584195'
  value: ''
  exception: ''
  latency: 0
  ruleData: ''

What did you expect to see?

inject successfully.

What did you see instead? injection failed.

error log:

2022-06-07T06:57:50.175Z INFO chaos-daemon.daemon-server.background-process-manager.process-builder pb/chaosdaemon.pb.go:3926 build command {“namespacedName”: “pt-shared-services/memory-stress-uc”, “command”: “sh -c bminstall.sh -b -Dorg.jboss.byteman.transform.all -Dorg.jboss.byteman.verbose -Dorg.jboss.byteman.compileToBytecode -p 9277 3500466”} 2022-06-07T06:58:11.526Z ERROR chaos-daemon.daemon-server chaosdaemon/server.go:187 com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file /proc/3500466/root/tmp/.java_pid3500466: target process 3500466 doesn’t respond within 10500ms or HotSpot VM not loaded com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file /proc/3500466/root/tmp/.java_pid3500466: target process 3500466 doesn’t respond within 10500ms or HotSpot VM not loaded at jdk.attach/sun.tools.attach.VirtualMachineImpl.<init>(VirtualMachineImpl.java:100) at jdk.attach/sun.tools.attach.AttachProviderImpl.attachVirtualMachine(AttachProviderImpl.java:58) at jdk.attach/com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:207) at org.chaos_mesh.agent_installer.Install.attach(Install.java:203) at org.chaos_mesh.agent_installer.Install.main(Install.java:47) {“namespacedName”: “pt-shared-services/memory-stress-uc”, “error”: “exit status 1”} github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon.MetadataExtractor.func1 /home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/server.go:187 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1 /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1 /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1 /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 github.com/chaos-mesh/chaos-mesh/pkg/grpc.TimeoutServerInterceptor /home/runner/work/chaos-mesh/chaos-mesh/pkg/grpc/utils.go:205 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1 /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1 /tmp/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34 github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb._ChaosDaemon_InstallJVMRules_Handler /home/runner/work/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb/chaosdaemon.pb.go:3928 google.golang.org/grpc.(*Server).processUnaryRPC /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1297 google.golang.org/grpc.(*Server).handleStream /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:1626 google.golang.org/grpc.(*Server).serveStreams.func1.2 /tmp/go/pkg/mod/google.golang.org/grpc@v1.40.0/server.go:941

other informations:

  1. pid 3500466 is the correct java process id.
  2. -XX:+StartAttachListener has been added .

image

go into chaos deamon pod,we found that the two proc id is not match , this may be the problem?

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 22 (13 by maintainers)

Most upvoted comments

It seems that it is similar to https://github.com/chaos-mesh/chaos-mesh/issues/2588

Can you check the Linux kernel’s version of the K8s’s node? It requires v4.2 at least

openjdk-8

so, the requirement should be added in docs…

Hi, @iguoyr , we are using v2.2.0.

Maybe this issue is not related to #2751 pid200534 and pid12 are the same pid. 200534 is the pid on host machine. 12 is then pid in container.