temurin-build: Cannot create worker GC thread. Out of system resources.
What are you trying to do? trying to start skywalking using https://github.com/apache/skywalking.git i hava edited .env file
ES_VERSION=7.4.2
OAP_IMAGE=apache/skywalking-oap-server:latest
UI_IMAGE=apache/skywalking-ui:latest
Expected behaviour: all container started Observed behaviour: apache/skywalking-oap-server:latest cann’t start Any other comments:
docker error json log
{"log":"[Entrypoint] Apache SkyWalking Docker Image\n","stream":"stdout","time":"2022-07-02T04:16:58.830391698Z"}
{"log":"+ exec java -Xms4096m -Xmx4096m -classpath '/skywalking/ext-libs/*.jar:oap-libs/zstd-jni-1.4.3-1.jar:oap-libs/zookeeper-jute-3.5.7.jar:oap-libs/zookeeper-3.5.7.jar:oap-libs/zipkin-receiver-plugin-9.1.0.jar:oap-libs/zipkin-query-plugin-9.1.0.jar:oap-libs/zipkin-2.23.16.jar:oap-libs/vavr-match-0.10.3.jar:oap-libs/vavr-0.10.3.jar:oap-libs/tool-profile-snapshot-server-mock-9.1.0.jar:oap-libs/tool-profile-snapshot-bootstrap-9.1.0.jar:oap-libs/telemetry-prometheus-9.1.0.jar:oap-libs/telemetry-api-9.1.0.jar:oap-libs/swagger-annotations-1.6.3.jar:oap-libs/storage-tidb-plugin-9.1.0.jar:oap-libs/storage-jdbc-hikaricp-plugin-9.1.0.jar:oap-libs/storage-elasticsearch-plugin-9.1.0.jar:oap-libs/storage-banyandb-plugin-9.1.0.jar:oap-libs/snappy-java-1.1.7.3.jar:oap-libs/snakeyaml-1.28.jar:oap-libs/slf4j-api-1.7.30.jar:oap-libs/skywalking-zabbix-receiver-plugin-9.1.0.jar:oap-libs/skywalking-trace-receiver-plugin-9.1.0.jar:oap-libs/skywalking-sharing-server-plugin-9.1.0.jar:oap-libs/skywalking-profile-receiver-plugin-9.1.0.jar:oap-libs/skywalking-meter-receiver-plugin-9.1.0.jar:oap-libs/skywalking-mesh-receiver-plugin-9.1.0.jar:oap-libs/skywalking-management-receiver-plugin-9.1.0.jar:oap-libs/skywalking-log-recevier-plugin-9.1.0.jar:oap-libs/skywalking-jvm-receiver-plugin-9.1.0.jar:oap-libs/skywalking-event-receiver-plugin-9.1.0.jar:oap-libs/skywalking-ebpf-receiver-plugin-9.1.0.jar:oap-libs/skywalking-clr-receiver-plugin-9.1.0.jar:oap-libs/skywalking-browser-receiver-plugin-9.1.0.jar:oap-libs/simpleclient_httpserver-0.12.0.jar:oap-libs/simpleclient_hotspot-0.6.0.jar:oap-libs/simpleclient_common-0.6.0.jar:oap-libs/simpleclient-0.6.0.jar:oap-libs/server-starter-9.1.0.jar:oap-libs/server-health-checker-9.1.0.jar:oap-libs/server-core-9.1.0.jar:oap-libs/server-alarm-plugin-9.1.0.jar:oap-libs/retrofit-2.5.0.jar:oap-libs/receiver-proto-9.1.0.jar:oap-libs/reactive-streams-1.0.2.jar:oap-libs/query-graphql-plugin-9.1.0.jar:oap-libs/proto-google-common-protos-2.0.1.jar:oap-libs/protobuf-java-util-3.19.4.jar:oap-libs/protobuf-java-3.19.4.jar:oap-libs/prometheus-fetcher-plugin-9.1.0.jar:oap-libs/postgresql-42.3.3.jar:oap-libs/perfmark-api-0.25.0.jar:oap-libs/otel-receiver-plugin-9.1.0.jar:oap-libs/okio-1.17.2.jar:oap-libs/okhttp-3.14.9.jar:oap-libs/oal-rt-9.1.0.jar:oap-libs/oal-grammar-9.1.0.jar:oap-libs/netty-transport-native-unix-common-4.1.77.Final-linux-x86_64.jar:oap-libs/netty-transport-native-unix-common-4.1.77.Final.jar:oap-libs/netty-transport-native-epoll-4.1.77.Final-linux-x86_64.jar:oap-libs/netty-transport-native-epoll-4.1.77.Final.jar:oap-libs/netty-transport-classes-epoll-4.1.77.Final.jar:oap-libs/netty-transport-4.1.77.Final.jar:oap-libs/netty-tcnative-classes-2.0.52.Final.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-windows-x86_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-osx-x86_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-osx-aarch_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-linux-x86_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-linux-aarch_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final.jar:oap-libs/netty-resolver-dns-native-macos-4.1.77.Final-osx-x86_64.jar:oap-libs/netty-resolver-dns-native-macos-4.1.77.Final-osx-aarch_64.jar:oap-libs/netty-resolver-dns-classes-macos-4.1.77.Final.jar:oap-libs/netty-resolver-dns-4.1.77.Final.jar:oap-libs/netty-resolver-4.1.77.Final.jar:oap-libs/netty-handler-proxy-4.1.77.Final.jar:oap-libs/netty-handler-4.1.77.Final.jar:oap-libs/netty-common-4.1.77.Final.jar:oap-libs/netty-codec-socks-4.1.77.Final.jar:oap-libs/netty-codec-http-4.1.77.Final.jar:oap-libs/netty-codec-http2-4.1.77.Final.jar:oap-libs/netty-codec-haproxy-4.1.77.Final.jar:oap-libs/netty-codec-dns-4.1.77.Final.jar:oap-libs/netty-codec-4.1.77.Final.jar:oap-libs/netty-buffer-4.1.77.Final.jar:oap-libs/nacos-common-1.4.2.jar:oap-libs/nacos-client-1.4.2.jar:oap-libs/nacos-api-1.4.2.jar:oap-libs/mvel2-2.4.8.Final.jar:oap-libs/micrometer-core-1.8.2.jar:oap-libs/meter-analyzer-9.1.0.jar:oap-libs/lz4-java-1.6.0.jar:oap-libs/logging-interceptor-4.9.1.jar:oap-libs/log-analyzer-9.1.0.jar:oap-libs/log4j-slf4j-impl-2.17.1.jar:oap-libs/log4j-over-slf4j-1.7.30.jar:oap-libs/log4j-core-2.17.1.jar:oap-libs/log4j-api-2.17.1.jar:oap-libs/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:oap-libs/library-util-9.1.0.jar:oap-libs/library-server-9.1.0.jar:oap-libs/library-module-9.1.0.jar:oap-libs/library-elasticsearch-client-9.1.0.jar:oap-libs/library-datacarrier-queue-9.1.0.jar:oap-libs/library-client-9.1.0.jar:oap-libs/LatencyUtils-2.0.3.jar:oap-libs/kotlinx-coroutines-reactive-1.5.0-native-mt.jar:oap-libs/kotlinx-coroutines-jdk8-1.5.0-native-mt.jar:oap-libs/kotlinx-coroutines-core-jvm-1.5.0-native-mt.jar:oap-libs/kotlinx-coroutines-core-1.5.0-native-mt.jar:oap-libs/kotlin-stdlib-jdk8-1.5.0.jar:oap-libs/kotlin-stdlib-jdk7-1.5.0.jar:oap-libs/kotlin-stdlib-common-1.5.0.jar:oap-libs/kotlin-stdlib-1.5.0.jar:oap-libs/kotlin-reflect-1.5.0.jar:oap-libs/kafka-fetcher-plugin-9.1.0.jar:oap-libs/kafka-clients-2.4.1.jar:oap-libs/jsr305-3.0.2.jar:oap-libs/jose4j-0.7.9.jar:oap-libs/joda-time-2.10.5.jar:oap-libs/jetcd-resolver-0.5.3.jar:oap-libs/jetcd-core-0.5.3.jar:oap-libs/jetcd-common-0.5.3.jar:oap-libs/jcl-over-slf4j-1.7.30.jar:oap-libs/javax.inject-1.jar:oap-libs/javassist-3.25.0-GA.jar:oap-libs/java-dataloader-3.1.0.jar:oap-libs/jackson-module-kotlin-2.13.1.jar:oap-libs/jackson-module-afterburner-2.12.2.jar:oap-libs/jackson-datatype-jdk8-2.9.10.jar:oap-libs/jackson-datatype-guava-2.9.10.jar:oap-libs/jackson-databind-2.13.2.2.jar:oap-libs/jackson-core-2.13.2.jar:oap-libs/jackson-annotations-2.13.2.jar:oap-libs/j2objc-annotations-1.3.jar:oap-libs/httpcore-nio-4.4.13.jar:oap-libs/httpcore-4.4.13.jar:oap-libs/httpclient-4.5.13.jar:oap-libs/httpasyncclient-4.1.3.jar:oap-libs/HikariCP-3.1.0.jar:oap-libs/HdrHistogram-2.1.12.jar:oap-libs/h2-2.1.212.jar:oap-libs/guice-4.1.0.jar:oap-libs/guava-31.1-jre.jar:oap-libs/gson-fire-1.8.5.jar:oap-libs/gson-2.9.0.jar:oap-libs/grpc-stub-1.46.0.jar:oap-libs/grpc-protobuf-lite-1.46.0.jar:oap-libs/grpc-protobuf-1.46.0.jar:oap-libs/grpc-netty-1.46.0.jar:oap-libs/grpc-grpclb-1.46.0.jar:oap-libs/grpc-core-1.46.0.jar:oap-libs/grpc-context-1.46.0.jar:oap-libs/grpc-configuration-sync-9.1.0.jar:oap-libs/grpc-api-1.46.0.jar:oap-libs/groovy-3.0.8.jar:oap-libs/graphql-java-tools-12.0.2.jar:oap-libs/graphql-java-extended-scalars-17.0.jar:oap-libs/graphql-java-17.3.jar:oap-libs/freemarker-2.3.31.jar:oap-libs/flatbuffers-java-1.12.0.jar:oap-libs/failureaccess-1.0.1.jar:oap-libs/failsafe-2.3.4.jar:oap-libs/exporter-9.1.0.jar:oap-libs/event-analyzer-9.1.0.jar:oap-libs/error_prone_annotations-2.11.0.jar:oap-libs/envoy-metrics-receiver-plugin-9.1.0.jar:oap-libs/data-generator-9.1.0.jar:oap-libs/curator-x-discovery-4.3.0.jar:oap-libs/curator-recipes-4.3.0.jar:oap-libs/curator-framework-4.3.0.jar:oap-libs/curator-client-4.3.0.jar:oap-libs/converter-jackson-2.5.0.jar:oap-libs/consul-client-1.4.2.jar:oap-libs/configuration-zookeeper-9.1.0.jar:oap-libs/configuration-nacos-9.1.0.jar:oap-libs/configuration-k8s-configmap-9.1.0.jar:oap-libs/configuration-etcd-9.1.0.jar:oap-libs/configuration-discovery-receiver-plugin-9.1.0.jar:oap-libs/configuration-consul-9.1.0.jar:oap-libs/configuration-apollo-9.1.0.jar:oap-libs/configuration-api-9.1.0.jar:oap-libs/commons-text-1.4.jar:oap-libs/commons-logging-1.2.jar:oap-libs/commons-lang3-3.12.0.jar:oap-libs/commons-io-2.7.jar:oap-libs/commons-compress-1.21.jar:oap-libs/commons-collections4-4.4.jar:oap-libs/commons-codec-1.11.jar:oap-libs/commons-beanutils-1.9.4.jar:oap-libs/cluster-zookeeper-plugin-9.1.0.jar:oap-libs/cluster-standalone-plugin-9.1.0.jar:oap-libs/cluster-nacos-plugin-9.1.0.jar:oap-libs/cluster-kubernetes-plugin-9.1.0.jar:oap-libs/cluster-etcd-plugin-9.1.0.jar:oap-libs/cluster-consul-plugin-9.1.0.jar:oap-libs/client-java-proto-14.0.0.jar:oap-libs/client-java-api-14.0.0.jar:oap-libs/client-java-14.0.0.jar:oap-libs/classmate-1.5.1.jar:oap-libs/checker-qual-3.12.0.jar:oap-libs/brotli4j-1.6.0.jar:oap-libs/bcutil-jdk15on-1.69.jar:oap-libs/bcprov-jdk15on-1.69.jar:oap-libs/bcprov-ext-jdk15on-1.69.jar:oap-libs/bcpkix-jdk15on-1.69.jar:oap-libs/banyandb-java-client-0.1.0.jar:oap-libs/audience-annotations-0.5.0.jar:oap-libs/armeria-protobuf-1.14.1.jar:oap-libs/armeria-graphql-protocol-1.14.1.jar:oap-libs/armeria-graphql-1.14.1.jar:oap-libs/armeria-1.14.1.jar:oap-libs/apollo-core-1.8.0.jar:oap-libs/apollo-client-1.8.0.jar:oap-libs/apm-network-9.1.0.jar:oap-libs/aopalliance-1.0.jar:oap-libs/antlr4-runtime-4.9.2.jar:oap-libs/annotations-13.0.jar:oap-libs/animal-sniffer-annotations-1.19.jar:oap-libs/agent-analyzer-9.1.0.jar:config:' org.apache.skywalking.oap.server.starter.OAPServerStartUp\n","stream":"stderr","time":"2022-07-02T04:16:58.855343376Z"}
{"log":"[0.005s][warning][os,thread] Failed to start thread - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.\n","stream":"stdout","time":"2022-07-02T04:16:58.86878564Z"}
{"log":"#\n","stream":"stdout","time":"2022-07-02T04:16:58.868984268Z"}
{"log":"# There is insufficient memory for the Java Runtime Environment to continue.\n","stream":"stdout","time":"2022-07-02T04:16:58.86900135Z"}
{"log":"# Cannot create worker GC thread. Out of system resources.\n","stream":"stdout","time":"2022-07-02T04:16:58.86900765Z"}
{"log":"# An error report file with more information is saved as:\n","stream":"stdout","time":"2022-07-02T04:16:58.869468693Z"}
{"log":"# /skywalking/hs_err_pid1.log\n","stream":"stdout","time":"2022-07-02T04:16:58.869486777Z"}
jvm error log
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.
# Possible reasons:
# The system is out of physical RAM or swap space
# The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# JVM is running with Unscaled Compressed Oops mode in which the Java heap is
# placed in the first 4GB address space. The Java Heap base address is the
# maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
# to set the Java Heap base and to place the Java Heap above 4GB virtual address.
# This output file may be truncated or incomplete.
#
# Out of Memory Error (workerManager.hpp:70), pid=1, tid=1
#
# JRE version: (11.0.15+10) (build )
# Java VM: OpenJDK 64-Bit Server VM (11.0.15+10, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
# Core dump will be written. Default location: /skywalking/core.1
#
--------------- S U M M A R Y ------------
Command Line: -Xms2048m -Xmx2048m org.apache.skywalking.oap.server.starter.OAPServerStartUp
Host: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz, 16 cores, 98G, Ubuntu 22.04 LTS
Time: Fri Jul 1 05:51:19 2022 UTC elapsed time: 0.005089 seconds (0d 0h 0m 0s)
--------------- T H R E A D ---------------
Current thread (0x0000563405091000): JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007ffc8d296000,0x00007ffc8da94000)]
Stack: [0x00007ffc8d296000,0x00007ffc8da94000], sp=0x00007ffc8da8e3c0, free space=8160k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xf424ca] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x1ca
V [libjvm.so+0xf4338b] VMError::report_and_die(Thread*, char const*, int, unsigned long, VMErrorType, char const*, __va_list_tag*)+0x2b
V [libjvm.so+0x6cf005] report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*, ...)+0xd5
V [libjvm.so+0xf6f58b] AbstractWorkGang::add_workers(unsigned int, bool)+0x14b
V [libjvm.so+0x7aa8c5] G1CollectedHeap::G1CollectedHeap(G1CollectorPolicy*)+0x5b5
V [libjvm.so+0x79c3b9] G1Arguments::create_heap()+0x69
V [libjvm.so+0xefdc11] Universe::initialize_heap()+0x31
V [libjvm.so+0xefdec6] universe_init()+0x46
V [libjvm.so+0x8a4207] init_globals()+0x57
V [libjvm.so+0xed5537] Threads::create_vm(JavaVMInitArgs*, bool*)+0x357
V [libjvm.so+0x960262] JNI_CreateJavaVM+0x52
C [libjli.so+0x49af] JavaMain+0x8f
C [libjli.so+0x9d94] CallJavaMainInNewThread+0x74
C [libjli.so+0x637d] ContinueInNewThread+0x4d
C [libjli.so+0x7a00] JLI_Launch+0x1470
C [java+0xa73]
--------------- P R O C E S S ---------------
Threads class SMR info:
_java_thread_list=0x0000563405054790, length=0, elements={
}
Java Threads: ( => current thread )
Other Threads:
=>0x0000563405091000 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007ffc8d296000,0x00007ffc8da94000)]
Threads with active compile tasks:
VM state:not at safepoint (not fully initialized)
VM Mutex/Monitor currently owned by a thread: None
[error occurred during error reporting (printing compressed oops mode), id 0xb, SIGSEGV (0xb) at pc=0x00007f98acefc543]
GC Heap History (0 events):
No events
Deoptimization events (0 events):
No events
Classes redefined (0 events):
No events
Internal exceptions (0 events):
No events
Events (2 events):
Event: 0.002 Protecting memory [0x00007ffc8d296000,0x00007ffc8d29a000] with protection modes 0
Event: 0.003 Loaded shared library /opt/java/openjdk/lib/libzip.so
Dynamic libraries:
56340325f000-563403260000 r-xp 00000000 08:03 36117702 /opt/java/openjdk/bin/java
563403460000-563403461000 r--p 00001000 08:03 36117702 /opt/java/openjdk/bin/java
563403461000-563403462000 rw-p 00002000 08:03 36117702 /opt/java/openjdk/bin/java
56340504e000-5634050d1000 rw-p 00000000 00:00 0 [heap]
7f989626e000-7f98964dd000 rw-p 00000000 00:00 0
7f98964dd000-7f989674d000 rwxp 00000000 00:00 0
7f989674d000-7f9896c1c000 ---p 00000000 00:00 0
7f9896c1c000-7f9896e8c000 rwxp 00000000 00:00 0
7f9896e8c000-7f989e07c000 ---p 00000000 00:00 0
7f989e07c000-7f989e2ec000 rwxp 00000000 00:00 0
7f989e2ec000-7f98a54dd000 ---p 00000000 00:00 0
7f98a54dd000-7f98ab000000 r--s 00000000 08:03 585853954 /opt/java/openjdk/lib/modules
7f98ab000000-7f98ab007000 r-xp 00000000 08:03 585853952 /opt/java/openjdk/lib/libzip.so
7f98ab007000-7f98ab207000 ---p 00007000 08:03 585853952 /opt/java/openjdk/lib/libzip.so
7f98ab207000-7f98ab208000 r--p 00007000 08:03 585853952 /opt/java/openjdk/lib/libzip.so
7f98ab208000-7f98ab209000 rw-p 00008000 08:03 585853952 /opt/java/openjdk/lib/libzip.so
7f98ab2b2000-7f98ab400000 rw-p 00000000 00:00 0
7f98ab400000-7f98ab41b000 r-xp 00000000 08:03 585853929 /opt/java/openjdk/lib/libjimage.so
7f98ab41b000-7f98ab61a000 ---p 0001b000 08:03 585853929 /opt/java/openjdk/lib/libjimage.so
7f98ab61a000-7f98ab61c000 r--p 0001a000 08:03 585853929 /opt/java/openjdk/lib/libjimage.so
7f98ab61c000-7f98ab61d000 rw-p 0001c000 08:03 585853929 /opt/java/openjdk/lib/libjimage.so
7f98ab6fe000-7f98ab6ff000 ---p 00000000 00:00 0
7f98ab6ff000-7f98ab800000 rw-p 00000000 00:00 0
7f98ab800000-7f98ab828000 r-xp 00000000 08:03 585853923 /opt/java/openjdk/lib/libjava.so
7f98ab828000-7f98aba28000 ---p 00028000 08:03 585853923 /opt/java/openjdk/lib/libjava.so
7f98aba28000-7f98aba29000 r--p 00028000 08:03 585853923 /opt/java/openjdk/lib/libjava.so
7f98aba29000-7f98aba2a000 rw-p 00029000 08:03 585853923 /opt/java/openjdk/lib/libjava.so
7f98aba2a000-7f98aba2b000 rw-p 00000000 00:00 0
7f98aba2e000-7f98aba33000 rw-p 00000000 00:00 0
7f98aba33000-7f98abb17000 ---p 00000000 00:00 0
7f98abb17000-7f98abb1c000 rw-p 00000000 00:00 0
7f98abb1c000-7f98abc00000 ---p 00000000 00:00 0
7f98abc00000-7f98abc0d000 r-xp 00000000 08:03 585853951 /opt/java/openjdk/lib/libverify.so
7f98abc0d000-7f98abe0c000 ---p 0000d000 08:03 585853951 /opt/java/openjdk/lib/libverify.so
7f98abe0c000-7f98abe0e000 r--p 0000c000 08:03 585853951 /opt/java/openjdk/lib/libverify.so
7f98abe0e000-7f98abe0f000 rw-p 0000e000 08:03 585853951 /opt/java/openjdk/lib/libverify.so
7f98abe19000-7f98abf19000 rw-p 00000000 00:00 0
7f98abf19000-7f98abf27000 r--p 00000000 08:03 580295711 /usr/lib/x86_64-linux-gnu/libm.so.6
7f98abf27000-7f98abfa3000 r-xp 0000e000 08:03 580295711 /usr/lib/x86_64-linux-gnu/libm.so.6
7f98abfa3000-7f98abffe000 r--p 0008a000 08:03 580295711 /usr/lib/x86_64-linux-gnu/libm.so.6
7f98abffe000-7f98abfff000 r--p 000e4000 08:03 580295711 /usr/lib/x86_64-linux-gnu/libm.so.6
7f98abfff000-7f98ac000000 rw-p 000e5000 08:03 580295711 /usr/lib/x86_64-linux-gnu/libm.so.6
7f98ac000000-7f98ad270000 r-xp 00000000 08:03 585853960 /opt/java/openjdk/lib/server/libjvm.so
7f98ad270000-7f98ad470000 ---p 01270000 08:03 585853960 /opt/java/openjdk/lib/server/libjvm.so
7f98ad470000-7f98ad535000 r--p 01270000 08:03 585853960 /opt/java/openjdk/lib/server/libjvm.so
7f98ad535000-7f98ad570000 rw-p 01335000 08:03 585853960 /opt/java/openjdk/lib/server/libjvm.so
7f98ad570000-7f98ad5c7000 rw-p 00000000 00:00 0
7f98ad5d8000-7f98ad600000 r--p 00000000 08:03 580151838 /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad600000-7f98ad795000 r-xp 00028000 08:03 580151838 /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad795000-7f98ad7ed000 r--p 001bd000 08:03 580151838 /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad7ed000-7f98ad7f1000 r--p 00214000 08:03 580151838 /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad7f1000-7f98ad7f3000 rw-p 00218000 08:03 580151838 /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad7f3000-7f98ad800000 rw-p 00000000 00:00 0
7f98ad800000-7f98ad810000 r-xp 00000000 08:03 36123812 /opt/java/openjdk/lib/jli/libjli.so
7f98ad810000-7f98ada0f000 ---p 00010000 08:03 36123812 /opt/java/openjdk/lib/jli/libjli.so
7f98ada0f000-7f98ada10000 r--p 0000f000 08:03 36123812 /opt/java/openjdk/lib/jli/libjli.so
7f98ada10000-7f98ada11000 rw-p 00010000 08:03 36123812 /opt/java/openjdk/lib/jli/libjli.so
7f98ada78000-7f98ada7d000 rw-p 00000000 00:00 0
7f98ada7d000-7f98ada87000 ---p 00000000 00:00 0
7f98ada87000-7f98ada8f000 rw-s 00000000 08:03 313133055 /tmp/hsperfdata_root/1
7f98ada8f000-7f98ada90000 r--p 00000000 08:03 580347790 /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada90000-7f98ada91000 r-xp 00001000 08:03 580347790 /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada91000-7f98ada92000 r--p 00002000 08:03 580347790 /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada92000-7f98ada93000 r--p 00002000 08:03 580347790 /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada93000-7f98ada94000 rw-p 00003000 08:03 580347790 /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada94000-7f98ada99000 rw-p 00000000 00:00 0
7f98ada99000-7f98ada9a000 r--p 00000000 08:03 580151854 /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9a000-7f98ada9b000 r-xp 00001000 08:03 580151854 /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9b000-7f98ada9c000 r--p 00002000 08:03 580151854 /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9c000-7f98ada9d000 r--p 00002000 08:03 580151854 /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9d000-7f98ada9e000 rw-p 00003000 08:03 580151854 /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9e000-7f98ada9f000 r--p 00000000 08:03 580345503 /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98ada9f000-7f98adaa0000 r-xp 00001000 08:03 580345503 /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98adaa0000-7f98adaa1000 r--p 00002000 08:03 580345503 /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98adaa1000-7f98adaa2000 r--p 00002000 08:03 580345503 /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98adaa2000-7f98adaa3000 rw-p 00003000 08:03 580345503 /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98adaa3000-7f98adaa5000 r--p 00000000 08:03 580349309 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adaa5000-7f98adab6000 r-xp 00002000 08:03 580349309 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adab6000-7f98adabc000 r--p 00013000 08:03 580349309 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adabc000-7f98adabd000 ---p 00019000 08:03 580349309 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adabd000-7f98adabe000 r--p 00019000 08:03 580349309 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adabe000-7f98adabf000 rw-p 0001a000 08:03 580349309 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adabf000-7f98adac0000 ---p 00000000 00:00 0
7f98adac0000-7f98adac1000 r--p 00000000 00:00 0
7f98adac1000-7f98adac3000 rw-p 00000000 00:00 0
7f98adac3000-7f98adac5000 r--p 00000000 08:03 580151816 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f98adac5000-7f98adaef000 r-xp 00002000 08:03 580151816 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f98adaef000-7f98adafa000 r--p 0002c000 08:03 580151816 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f98adafb000-7f98adafd000 r--p 00037000 08:03 580151816 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f98adafd000-7f98adaff000 rw-p 00039000 08:03 580151816 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7ffc8d296000-7ffc8d29a000 ---p 00000000 00:00 0
7ffc8da73000-7ffc8da96000 rw-p 00000000 00:00 0 [stack]
7ffc8dbfc000-7ffc8dbfe000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
VM Arguments:
jvm_args: -Xms2048m -Xmx2048m
java_command: org.apache.skywalking.oap.server.starter.OAPServerStartUp
java_class_path (initial): /skywalking/ext-libs/*.jar:oap-libs/zstd-jni-1.4.3-1.jar:oap-libs/zookeeper-jute-3.5.7.jar:oap-libs/zookeeper-3.5.7.jar:oap-libs/zipkin-receiver-plugin-9.1.0.jar:oap-libs/zipkin-query-plugin-9.1.0.jar:oap-libs/zipkin-2.23.16.jar:oap-libs/vavr-match-0.10.3.jar:oap-libs/vavr-0.10.3.jar:oap-libs/tool-profile-snapshot-server-mock-9.1.0.jar:oap-libs/tool-profile-snapshot-bootstrap-9.1.0.jar:oap-libs/telemetry-prometheus-9.1.0.jar:oap-libs/telemetry-api-9.1.0.jar:oap-libs/swagger-annotations-1.6.3.jar:oap-libs/storage-tidb-plugin-9.1.0.jar:oap-libs/storage-jdbc-hikaricp-plugin-9.1.0.jar:oap-libs/storage-elasticsearch-plugin-9.1.0.jar:oap-libs/storage-banyandb-plugin-9.1.0.jar:oap-libs/snappy-java-1.1.7.3.jar:oap-libs/snakeyaml-1.28.jar:oap-libs/slf4j-api-1.7.30.jar:oap-libs/skywalking-zabbix-receiver-plugin-9.1.0.jar:oap-libs/skywalking-trace-receiver-plugin-9.1.0.jar:oap-libs/skywalking-sharing-server-plugin-9.1.0.jar:oap-libs/skywalking-profile-receiver-plugin-9.1.0.jar:oap-libs/skywalking-meter-receiver-plugin-9.1.0.jar:oap-libs/skywalking-mesh-receiver-plugin-9.1.0.jar:oap-libs/skywalking-management-receiver-plugin-9.1.0.jar:oap-libs/skywalking-log-recevier-plugin-9.1.0.jar:oap-libs/skywalking-jvm-receiver-plugin-9.1.0.jar:oap-libs/skywalking-event-receiver-plugin-9.1.0.jar:oap-libs/skywalking-ebpf-receiver-plugin-9.1.0.jar:oap-libs/skywalking-clr-receiver-plugin-9.1.0.jar:oap-libs/skywalking-browser-receiver-plugin-9.1.0.jar:oap-libs/simpleclient_httpserver-0.12.0.jar:oap-libs/simpleclient_hotspot-0.6.0.jar:oap-libs/simpleclient_common-0.6.0.jar:oap-libs/simpleclient-0.6.0.jar:oap-libs/server-starter-9.1.0.jar:oap-libs/server-health-checker-9.1.0.jar:oap-libs/server-core-9.1.0.jar:oap-libs/server-alarm-plugin-9.1.0.jar:oap-libs/retrofit-2.5.0.jar:oap-libs/receiver-proto-9.1.0.jar:oap-libs/reactive-streams-1.0.2.jar:oap-libs/query-graphql-plugin-9.1.0.jar:oap-libs/proto-google-common-protos-2.0.1.jar:oap-libs/protobuf-java-util-3.19.4.jar:
Launcher Type: SUN_STANDARD
[Global flags]
intx CICompilerCount = 12 {product} {ergonomic}
uint G1ConcRefinementThreads = 13 {product} {ergonomic}
size_t G1HeapRegionSize = 1048576 {product} {ergonomic}
uintx GCDrainStackTargetSize = 64 {product} {ergonomic}
size_t InitialHeapSize = 2147483648 {product} {command line}
size_t MaxHeapSize = 2147483648 {product} {command line}
size_t MinHeapDeltaBytes = 1048576 {product} {ergonomic}
uintx NonNMethodCodeHeapSize = 7594288 {pd product} {ergonomic}
uintx NonProfiledCodeHeapSize = 122031976 {pd product} {ergonomic}
uintx ProfiledCodeHeapSize = 122031976 {pd product} {ergonomic}
uintx ReservedCodeCacheSize = 251658240 {pd product} {ergonomic}
bool SegmentedCodeCache = true {product} {ergonomic}
bool UseCompressedClassPointers = true {lp64_product} {ergonomic}
bool UseCompressedOops = true {lp64_product} {ergonomic}
bool UseG1GC = true {product} {ergonomic}
Logging:
Log output configuration:
#0: stdout all=warning uptime,level,tags
#1: stderr all=off uptime,level,tags
Environment Variables:
JAVA_HOME=/opt/java/openjdk
PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LANG=en_US.UTF-8
LC_ALL=en_US.UTF-8
Signal Handlers:
SIGSEGV: [libjvm.so+0xf44160], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.so+0xf44160], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.so+0xf44160], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.so+0xc669b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.so+0xc669b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.so+0xf44160], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR2: [libjvm.so+0xc66850], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGINT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGTERM: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGQUIT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
--------------- S Y S T E M ---------------
OS:DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04 LTS"
uname:Linux 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64
OS uptime: 116 days 0:44 hours
libc:glibc 2.35 NPTL 2.35
rlimit (soft/hard): STACK 8192k/infinity , CORE infinity/infinity , NPROC infinity/infinity , NOFILE 1048576/1048576 , AS infinity/infinity , CPU infinity/infinity , DATA infinity/infinity , FSIZE infinity/infinity , MEMLOCK 64k/64k
load average:0.12 0.09 0.06
/proc/meminfo:
MemTotal: 102945540 kB
MemFree: 76121656 kB
MemAvailable: 97823172 kB
Buffers: 2076 kB
Cached: 16185204 kB
SwapCached: 0 kB
Active: 11590136 kB
Inactive: 4824488 kB
Active(anon): 901108 kB
Inactive(anon): 377560 kB
Active(file): 10689028 kB
Inactive(file): 4446928 kB
Unevictable: 940132 kB
Mlocked: 940160 kB
SwapTotal: 33554428 kB
SwapFree: 33554428 kB
Dirty: 36 kB
Writeback: 0 kB
AnonPages: 1176724 kB
Mapped: 309772 kB
Shmem: 880484 kB
Slab: 7748008 kB
SReclaimable: 7242500 kB
SUnreclaim: 505508 kB
KernelStack: 10656 kB
PageTables: 10048 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 85027196 kB
Committed_AS: 3584344 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 394548 kB
VmallocChunk: 34359310332 kB
Percpu: 36352 kB
HardwareCorrupted: 0 kB
AnonHugePages: 675840 kB
CmaTotal: 0 kB
CmaFree: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 151360 kB
DirectMap2M: 5091328 kB
DirectMap1G: 101711872 kB
/sys/kernel/mm/transparent_hugepage/enabled:
[always] madvise never
/sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts parameter):
[always] madvise never
Process Memory:
Virtual Size: 384192K (peak: 384192K)
Resident Set Size: 11884K (peak: 11884K) (anon: 5596K, file: 6288K, shmem: 0K)
Swapped out: 0K
C-Heap outstanding allocations: 419K
/proc/sys/kernel/threads-max (system-wide limit on the number of threads):
804094
/proc/sys/vm/max_map_count (maximum number of memory map areas a process may have):
262144
/proc/sys/kernel/pid_max (system-wide limit on number of process identifiers):
4194303
container (cgroup) information:
container_type: cgroupv1
cpu_cpuset_cpus: 0-15
cpu_memory_nodes: 0
active_processor_count: 16
cpu_quota: -1
cpu_period: 100000
cpu_shares: -1
memory_limit_in_bytes: -1
memory_and_swap_limit_in_bytes: -1
memory_soft_limit_in_bytes: -1
memory_usage_in_bytes: 6201344
memory_max_usage_in_bytes: 6201344
VMWare virtualization detected
Steal ticks since vm start: 0
Steal ticks percentage since vm start: 0.000
CPU:total 16 (initial active 16) (8 cores per cpu, 1 threads per core) family 6 model 85 stepping 4 microcode 0x2006a08, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, 3dnowpref, lzcnt, tsc, tscinvbit, bmi1, bmi2, adx, fma
CPU Model and flags from /proc/cpuinfo:
model name : Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec arat pku ospke md_clear spec_ctrl intel_stibp flush_l1d arch_capabilities
Online cpus:
0-15
Offline cpus:
16-127
BIOS frequency limitation:
<Not Available>
Frequency switch latency (ns):
<Not Available>
Available cpu frequencies:
<Not Available>
Current governor:
<Not Available>
Core performance/turbo boost:
<Not Available>
Memory: 4k page, physical 102945540k(76121656k free), swap 33554428k(33554428k free)
vm_info: OpenJDK 64-Bit Server VM (11.0.15+10) for linux-amd64 JRE (11.0.15+10), built on Apr 19 2022 21:36:55 by "" with gcc 7.5.0
END.
docker compose file
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
version: '3.8'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch-oss:${ES_VERSION}
container_name: elasticsearch
ports:
- "9200:9200"
healthcheck:
test: [ "CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1" ]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
environment:
- discovery.type=single-node
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
oap:
image: ${OAP_IMAGE}
container_name: oap
depends_on:
elasticsearch:
condition: service_healthy
links:
- elasticsearch
ports:
- "11800:11800"
- "12800:12800"
healthcheck:
test: [ "CMD-SHELL", "/skywalking/bin/swctl ch" ]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
environment:
SW_STORAGE: elasticsearch
SW_STORAGE_ES_CLUSTER_NODES: elasticsearch:9200
SW_HEALTH_CHECKER: default
SW_TELEMETRY: prometheus
JAVA_OPTS: "-Xms4096m -Xmx4096m"
ui:
image: ${UI_IMAGE}
container_name: ui
depends_on:
oap:
condition: service_healthy
links:
- oap
ports:
- "8080:8080"
environment:
SW_OAP_ADDRESS: http://oap:12800
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 17 (6 by maintainers)
Ah OK, yes please upgrade to the 20.x engine
I used this base image eclipse-temurin:11-jre-centos7 in my env with docker 19.03 to soloved this problem.The skywalking-oap-server Dockerfile use eclipse-temurin:11-jre default will occured upper probelm.