zeroall edited a comment on issue #2969: OutOfMemoryError by com.google.common.reflect.ClassPath.from
URL: https://github.com/apache/skywalking/issues/2969#issuecomment-506927724
Hi Wu Sheng,
I took a look at Skywalking with the Arthas tool and got some information:
```
$ dashboard
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
1 main main 5 RUNNABLE 98 0:33 false false
17 AsyncAppender-Worker-arthas-cache.result.A system 9 WAITING 0 0:0 false true
15 Attach Listener system 9 RUNNABLE 0 0:0 false true
11 Curator-ConnectionStateManager-0 main 5 TIMED_WAITING 0 0:0 false true
14 Curator-Framework-0 main 5 WAITING 0 0:0 false true
3 Finalizer system 8 WAITING 0 0:0 false true
2 Reference Handler system 10 WAITING 0 0:0 false true
4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true
26 Timer-for-arthas-dashboard-4a94f150-5fea-4 system 10 RUNNABLE 0 0:0 false true
25 as-command-execute-daemon system 10 TIMED_WAITING 0 0:0 false true
19 job-timeout system 9 TIMED_WAITING 0 0:0 false true
13 main-EventThread main 5 WAITING 0 0:0 false true
12 main-SendThread(192.168.1.214:2181) main 5 RUNNABLE 0 0:0 false true
20 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false
Memory used total max usage GC
heap 677M 954M 954M 70.97% gc.ps_scavenge.count 29
ps_eden_space 33M 205M 212M 16.01% gc.ps_scavenge.time(ms) 788
ps_survivor_space 0K 67584K 67584K 0.00% gc.ps_marksweep.count 2
ps_old_gen 647M 683M 683M 94.78% gc.ps_marksweep.time(ms) 554
nonheap 31M 32M -1 97.82%
code_cache 5M 5M 240M 2.46%
metaspace 22M 23M -1 96.89%
Runtime
os.name Linux
os.version 3.10.0-957.21.3.el7.x86_64
java.version 1.8.0_201
java.home /opt/jdk1.8.0_201/jre
systemload.average 0.38
processors 4
uptime 35s
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
1 main main 5 RUNNABLE 97 0:37 false false
26 Timer-for-arthas-dashboard-4a94f150-5fea-4 system 10 RUNNABLE 1 0:0 false true
17 AsyncAppender-Worker-arthas-cache.result.A system 9 WAITING 0 0:0 false true
15 Attach Listener system 9 RUNNABLE 0 0:0 false true
11 Curator-ConnectionStateManager-0 main 5 TIMED_WAITING 0 0:0 false true
14 Curator-Framework-0 main 5 WAITING 0 0:0 false true
3 Finalizer system 8 WAITING 0 0:0 false true
2 Reference Handler system 10 WAITING 0 0:0 false true
4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true
25 as-command-execute-daemon system 10 TIMED_WAITING 0 0:0 false true
19 job-timeout system 9 TIMED_WAITING 0 0:0 false true
13 main-EventThread main 5 WAITING 0 0:0 false true
12 main-SendThread(192.168.1.214:2181) main 5 RUNNABLE 0 0:0 false true
20 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false
Memory used total max usage GC
heap 755M 954M 954M 79.19% gc.ps_scavenge.count 29
ps_eden_space 73M 205M 212M 34.48% gc.ps_scavenge.time(ms) 788
ps_survivor_space 0K 67584K 67584K 0.00% gc.ps_marksweep.count 6
ps_old_gen 682M 683M 683M 99.97% gc.ps_marksweep.time(ms) 1778
nonheap 31M 32M -1 98.05%
code_cache 6M 6M 240M 2.54%
metaspace 22M 23M -1 98.00%
Runtime
os.name Linux
os.version 3.10.0-957.21.3.el7.x86_64
java.version 1.8.0_201
java.home /opt/jdk1.8.0_201/jre
systemload.average 0.43
processors 4
uptime 40s
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
1 main main 5 RUNNABLE 99 0:41 false false
17 AsyncAppender-Worker-arthas-cache.result.A system 9 WAITING 0 0:0 false true
15 Attach Listener system 9 RUNNABLE 0 0:0 false true
11 Curator-ConnectionStateManager-0 main 5 TIMED_WAITING 0 0:0 false true
14 Curator-Framework-0 main 5 WAITING 0 0:0 false true
3 Finalizer system 8 WAITING 0 0:0 false true
2 Reference Handler system 10 WAITING 0 0:0 false true
4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true
26 Timer-for-arthas-dashboard-4a94f150-5fea-4 system 10 RUNNABLE 0 0:0 false true
25 as-command-execute-daemon system 10 TIMED_WAITING 0 0:0 false true
19 job-timeout system 9 TIMED_WAITING 0 0:0 false true
13 main-EventThread main 5 WAITING 0 0:0 false true
12 main-SendThread(192.168.1.214:2181) main 5 RUNNABLE 0 0:0 false true
20 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false
Memory used total max usage GC
heap 853M 954M 954M 89.40% gc.ps_scavenge.count 29
ps_eden_space 170M 205M 212M 80.43% gc.ps_scavenge.time(ms) 788
ps_survivor_space 0K 67584K 67584K 0.00% gc.ps_marksweep.count 12
ps_old_gen 682M 683M 683M 99.97% gc.ps_marksweep.time(ms) 3008
nonheap 31M 33M -1 95.90%
code_cache 5M 6M 240M 2.36%
metaspace 22M 23M -1 97.18%
Runtime
os.name Linux
os.version 3.10.0-957.21.3.el7.x86_64
java.version 1.8.0_201
java.home /opt/jdk1.8.0_201/jre
systemload.average 0.48
processors 4
uptime 45s
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
1 main main 5 RUNNABLE 99 0:43 false false
17 AsyncAppender-Worker-arthas-cache.result.A system 9 WAITING 0 0:0 false true
15 Attach Listener system 9 RUNNABLE 0 0:0 false true
11 Curator-ConnectionStateManager-0 main 5 TIMED_WAITING 0 0:0 false true
14 Curator-Framework-0 main 5 WAITING 0 0:0 false true
3 Finalizer system 8 WAITING 0 0:0 false true
2 Reference Handler system 10 WAITING 0 0:0 false true
4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true
26 Timer-for-arthas-dashboard-4a94f150-5fea-4 system 10 RUNNABLE 0 0:0 false true
25 as-command-execute-daemon system 10 TIMED_WAITING 0 0:0 false true
19 job-timeout system 9 TIMED_WAITING 0 0:0 false true
13 main-EventThread main 5 WAITING 0 0:0 false true
12 main-SendThread(192.168.1.214:2181) main 5 TIMED_WAITING 0 0:0 false true
20 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false
Memory used total max usage GC
heap 881M 954M 954M 92.35% gc.ps_scavenge.count 29
ps_eden_space 198M 205M 212M 93.73% gc.ps_scavenge.time(ms) 788
ps_survivor_space 0K 67584K 67584K 0.00% gc.ps_marksweep.count 25
ps_old_gen 682M 683M 683M 99.97% gc.ps_marksweep.time(ms) 5618
nonheap 31M 33M -1 95.69%
code_cache 5M 6M 240M 2.33%
metaspace 22M 23M -1 97.33%
Runtime
os.name Linux
os.version 3.10.0-957.21.3.el7.x86_64
java.version 1.8.0_201
java.home /opt/jdk1.8.0_201/jre
systemload.average 1.00
processors 4
uptime 50s
jvm
RUNTIME
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MACHINE-NAME 12887@centos0
JVM-START-TIME 2019-06-29 12:56:05
MANAGEMENT-SPEC-VERSION 1.2
SPEC-NAME Java Virtual Machine Specification
SPEC-VENDOR Oracle Corporation
SPEC-VERSION 1.8
VM-NAME Java HotSpot(TM) 64-Bit Server VM
VM-VENDOR Oracle Corporation
VM-VERSION 25.201-b09
INPUT-ARGUMENTS -Xms1G
-Xmx1G
-Doap.logDir=/root/data/skywalking/apache-skywalking-apm-bin/logs
CLASS-PATH /root/data/skywalking/apache-skywalking-apm-bin/oap-libs/zookeeper-3.4.10.jar:/root/data/skywalking/apache-skywalking-ap
m-bin/oap-libs/zipkin-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/zipkin-2.9.1.ja
r:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/telemetry-prometheus-6.1.0.jar:/root/data/skywalking/apache-s
kywalking-apm-bin/oap-libs/telemetry-api-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/t-digest-3.2
.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/swagger-annotations-1.5.12.jar:/root/data/skywalking/apach
e-skywalking-apm-bin/oap-libs/storage-zipkin-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/s
torage-jdbc-hikaricp-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/storage-jaeger-plugin-6.1
.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/storage-elasticsearch-plugin-6.1.0.jar:/root/data/skywal
king/apache-skywalking-apm-bin/oap-libs/snakeyaml-1.18.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/slf4
j-api-1.7.25.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/skywalking-trace-receiver-plugin-6.1.0.jar:/ro
ot/data/skywalking/apache-skywalking-apm-bin/oap-libs/skywalking-sharing-server-plugin-6.1.0.jar:/root/data/skywalking/a
pache-skywalking-apm-bin/oap-libs/skywalking-register-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-
apm-bin/oap-libs/skywalking-mesh-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/skyw
alking-jvm-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/skywalking-clr-receiver-pl
ugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/simpleclient_httpserver-0.6.0.jar:/root/data/sky
walking/apache-skywalking-apm-bin/oap-libs/simpleclient_hotspot-0.6.0.jar:/root/data/skywalking/apache-skywalking-apm-bi
n/oap-libs/simpleclient_common-0.6.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/simpleclient-0.6.0.jar
:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/sharding-jdbc-core-2.0.3.jar:/root/data/skywalking/apache-skyw
alking-apm-bin/oap-libs/server-starter-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/server-core-6.
1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/server-alarm-plugin-6.1.0.jar:/root/data/skywalking/apa
che-skywalking-apm-bin/oap-libs/retrofit-2.3.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/reflectasm-1
.11.7.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/receiver-proto-6.1.0.jar:/root/data/skywalking/apache
-skywalking-apm-bin/oap-libs/reactive-streams-1.0.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/rank-ev
al-client-6.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/query-graphql-plugin-6.1.0.jar:/root/data/s
kywalking/apache-skywalking-apm-bin/oap-libs/proto-google-common-protos-1.0.0.jar:/root/data/skywalking/apache-skywalkin
g-apm-bin/oap-libs/protobuf-java-3.4.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/parent-join-client-6
.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/opencensus-contrib-grpc-metrics-0.12.3.jar:/root/data/
skywalking/apache-skywalking-apm-bin/oap-libs/opencensus-api-0.12.3.jar:/root/data/skywalking/apache-skywalking-apm-bin/
oap-libs/okio-1.13.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/okhttp-ws-2.7.5.jar:/root/data/skywalk
ing/apache-skywalking-apm-bin/oap-libs/okhttp-3.9.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/okhttp-
2.7.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-transport-4.1.27.Final.jar:/root/data/skywalkin
g/apache-skywalking-apm-bin/oap-libs/netty-tcnative-boringssl-static-2.0.7.Final.jar:/root/data/skywalking/apache-skywal
king-apm-bin/oap-libs/netty-resolver-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-han
dler-proxy-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-handler-4.1.27.Final.jar:/roo
t/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-common-4.1.27.Final.jar:/root/data/skywalking/apache-skywalki
ng-apm-bin/oap-libs/netty-codec-socks-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-co
dec-http-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-codec-http2-4.1.27.Final.jar:/r
oot/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-codec-4.1.27.Final.jar:/root/data/skywalking/apache-skywalk
ing-apm-bin/oap-libs/netty-buffer-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-3.10.5
.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/mysql-connector-java-5.1.47.jar:/root/data/skywalkin
g/apache-skywalking-apm-bin/oap-libs/lucene-suggest-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/l
ucene-spatial-extras-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-spatial-7.3.1.jar:/root/d
ata/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-spatial3d-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm
-bin/oap-libs/lucene-sandbox-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-queryparser-7.3.1
.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-queries-7.3.1.jar:/root/data/skywalking/apache-skyw
alking-apm-bin/oap-libs/lucene-misc-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-memory-7.3
.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-join-7.3.1.jar:/root/data/skywalking/apache-skywa
lking-apm-bin/oap-libs/lucene-highlighter-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-grou
ping-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-core-7.3.1.jar:/root/data/skywalking/apac
he-skywalking-apm-bin/oap-libs/lucene-backward-codecs-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs
/lucene-analyzers-common-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/logging-interceptor-2.7.5.ja
r:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/log4j-slf4j-impl-2.9.0.jar:/root/data/skywalking/apache-skywa
lking-apm-bin/oap-libs/log4j-over-slf4j-1.7.25.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/log4j-core-2
.9.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/log4j-api-2.9.0.jar:/root/data/skywalking/apache-skywa
lking-apm-bin/oap-libs/log4j-1.2.16.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/library-util-6.1.0.jar:
/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/library-server-6.1.0.jar:/root/data/skywalking/apache-skywalkin
g-apm-bin/oap-libs/library-module-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/library-client-6.1.
0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/library-buffer-6.1.0.jar:/root/data/skywalking/apache-sky
walking-apm-bin/oap-libs/kotlin-stdlib-1.1.60.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/kotlin-reflec
t-1.1.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jsr305-1.3.9.jar:/root/data/skywalking/apache-skywa
lking-apm-bin/oap-libs/jopt-simple-4.6.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/joda-time-2.9.9.jar:
/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jna-4.5.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/o
ap-libs/jline-0.9.94.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-util-9.4.2.v20170220.jar:/root/d
ata/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-servlet-9.4.2.v20170220.jar:/root/data/skywalking/apache-skywalk
ing-apm-bin/oap-libs/jetty-server-9.4.2.v20170220.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-sec
urity-9.4.2.v20170220.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-io-9.4.2.v20170220.jar:/root/da
ta/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-http-9.4.2.v20170220.jar:/root/data/skywalking/apache-skywalking-
apm-bin/oap-libs/jcl-over-slf4j-1.7.25.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/javax.servlet-api-3.
1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/java-dataloader-2.0.2.jar:/root/data/skywalking/apache-
skywalking-apm-bin/oap-libs/jaeger-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/ja
ckson-module-kotlin-2.8.8.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-mapper-asl-1.9.13.jar:/ro
ot/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-datatype-jdk8-2.9.5.jar:/root/data/skywalking/apache-skywa
lking-apm-bin/oap-libs/jackson-datatype-guava-2.9.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson
-dataformat-yaml-2.8.10.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-dataformat-smile-2.8.10.jar
:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-dataformat-cbor-2.8.10.jar:/root/data/skywalking/apach
e-skywalking-apm-bin/oap-libs/jackson-databind-2.9.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackso
n-core-asl-1.9.13.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-core-2.9.5.jar:/root/data/skywalk
ing/apache-skywalking-apm-bin/oap-libs/jackson-annotations-2.9.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap
-libs/httpcore-nio-4.4.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/httpcore-4.4.5.jar:/root/data/skyw
alking/apache-skywalking-apm-bin/oap-libs/httpclient-4.5.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/
httpasyncclient-4.1.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/hppc-0.7.1.jar:/root/data/skywalking/
apache-skywalking-apm-bin/oap-libs/HikariCP-3.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/HdrHistog
ram-2.1.9.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/guava-20.0.jar:/root/data/skywalking/apache-skywa
lking-apm-bin/oap-libs/gson-2.8.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/grpc-stub-1.15.1.jar:/roo
t/data/skywalking/apache-skywalking-apm-bin/oap-libs/grpc-protobuf-lite-1.15.1.jar:/root/data/skywalking/apache-skywalki
ng-apm-bin/oap-libs/grpc-protobuf-1.15.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/grpc-netty-1.15.1.
jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/grpc-core-1.15.1.jar:/root/data/skywalking/apache-skywalkin
g-apm-bin/oap-libs/grpc-context-1.15.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/groovy-2.4.5-indy.ja
r:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/graphql-java-tools-5.2.3.jar:/root/data/skywalking/apache-sky
walking-apm-bin/oap-libs/graphql-java-8.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/generated-analysi
s-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/exporter-6.1.0.jar:/root/data/skywalking/apache-sky
walking-apm-bin/oap-libs/error_prone_annotations-2.2.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/envo
y-metrics-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/elasticsearch-x-content-6.3
.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/elasticsearch-secure-sm-6.3.2.jar:/root/data/skywalking/
apache-skywalking-apm-bin/oap-libs/elasticsearch-rest-high-level-client-6.3.2.jar:/root/data/skywalking/apache-skywalkin
g-apm-bin/oap-libs/elasticsearch-rest-client-6.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/elastics
earch-core-6.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/elasticsearch-cli-6.3.2.jar:/root/data/sky
walking/apache-skywalking-apm-bin/oap-libs/elasticsearch-6.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-l
ibs/curator-x-discovery-4.0.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/curator-recipes-4.0.1.jar:/ro
ot/data/skywalking/apache-skywalking-apm-bin/oap-libs/curator-framework-4.0.1.jar:/root/data/skywalking/apache-skywalkin
g-apm-bin/oap-libs/curator-client-4.0.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/converter-jackson-2
.3.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/consul-client-1.2.6.jar:/root/data/skywalking/apache-s
kywalking-apm-bin/oap-libs/commons-pool-1.5.4.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/commons-lang3
-3.7.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/commons-io-2.6.jar:/root/data/skywalking/apache-skywal
king-apm-bin/oap-libs/commons-dbcp-1.4.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/commons-codec-1.11.j
ar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/cluster-zookeeper-plugin-6.1.0.jar:/root/data/skywalking/apa
che-skywalking-apm-bin/oap-libs/cluster-standalone-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-
libs/cluster-kubernetes-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/cluster-consul-plugin-
6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/client-java-proto-2.0.0.jar:/root/data/skywalking/apa
che-skywalking-apm-bin/oap-libs/client-java-api-2.0.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/clien
t-java-2.0.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/caffeine-2.6.2.jar:/root/data/skywalking/apach
e-skywalking-apm-bin/oap-libs/bcprov-jdk15on-1.59.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/bcprov-ex
t-jdk15on-1.59.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/bcpkix-jdk15on-1.59.jar:/root/data/skywalkin
g/apache-skywalking-apm-bin/oap-libs/apm-util-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/apm-net
work-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/apm-datacarrier-6.1.0.jar:/root/data/skywalking/
apache-skywalking-apm-bin/oap-libs/antlr4-runtime-4.7.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/ann
otations-13.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/animal-sniffer-annotations-1.17.jar:/root/dat
a/skywalking/apache-skywalking-apm-bin/oap-libs/aggs-matrix-stats-client-6.3.2.jar:/root/data/skywalking/apache-skywalki
ng-apm-bin/config:.:/opt/java/lib/dt.jar:/opt/java/lib/tools.jar
BOOT-CLASS-PATH /opt/jdk1.8.0_201/jre/lib/resources.jar:/opt/jdk1.8.0_201/jre/lib/rt.jar:/opt/jdk1.8.0_201/jre/lib/sunrsasign.jar:/opt/j
dk1.8.0_201/jre/lib/jsse.jar:/opt/jdk1.8.0_201/jre/lib/jce.jar:/opt/jdk1.8.0_201/jre/lib/charsets.jar:/opt/jdk1.8.0_201/
jre/lib/jfr.jar:/opt/jdk1.8.0_201/jre/classes
LIBRARY-PATH /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CLASS-LOADING
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
LOADED-CLASS-COUNT 4478
TOTAL-LOADED-CLASS-COUNT 4478
UNLOADED-CLASS-COUNT 0
IS-VERBOSE false
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
COMPILATION
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
NAME HotSpot 64-Bit Tiered Compilers
TOTAL-COMPILE-TIME 8097(ms)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GARBAGE-COLLECTORS
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
PS Scavenge 29/788(ms)
[count/time]
PS MarkSweep 301/49445(ms)
[count/time]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MEMORY-MANAGERS
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CodeCacheManager Code Cache
Metaspace Manager Metaspace
Compressed Class Space
PS Scavenge PS Eden Space
PS Survivor Space
PS MarkSweep PS Eden Space
PS Survivor Space
PS Old Gen
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MEMORY
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
HEAP-MEMORY-USAGE 1000865792(954.50 MiB)/1073741824(1.00 GiB)/1000865792(954.50 MiB)/930960160(887.83 MiB)
[committed/init/max/used]
NO-HEAP-MEMORY-USAGE 34603008(33.00 MiB)/2555904(2.44 MiB)/-1(-1 B)/33061200(31.53 MiB)
[committed/init/max/used]
PENDING-FINALIZE-COUNT 0
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
OPERATING-SYSTEM
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
OS Linux
ARCH amd64
PROCESSORS-COUNT 4
LOAD-AVERAGE 4.47
VERSION 3.10.0-957.21.3.el7.x86_64
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
THREAD
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
COUNT 18
DAEMON-COUNT 12
PEAK-COUNT 20
STARTED-COUNT 23
DEADLOCK-COUNT 0
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
FILE-DESCRIPTOR
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MAX-FILE-DESCRIPTOR-COUNT 65536
OPEN-FILE-DESCRIPTOR-COUNT 244
Affect(row-cnt:0) cost in 55728 ms.
$
$ Connection closed by foreign host.
Then I looked at the oap-lib directory, which contains the following files:
# root @ centos0 in ~/data/skywalking/apache-skywalking-apm-bin/oap-libs [13:10:10]
$ ls
aggs-matrix-stats-client-6.3.2.jar httpcore-4.4.5.jar mysql-connector-java-5.1.47.jar
animal-sniffer-annotations-1.17.jar httpcore-nio-4.4.5.jar netty-3.10.5.Final.jar
annotations-13.0.jar jackson-annotations-2.9.5.jar netty-buffer-4.1.27.Final.jar
antlr4-runtime-4.7.1.jar jackson-core-2.9.5.jar netty-codec-4.1.27.Final.jar
apm-datacarrier-6.1.0.jar jackson-core-asl-1.9.13.jar netty-codec-http2-4.1.27.Final.jar
apm-network-6.1.0.jar jackson-databind-2.9.5.jar netty-codec-http-4.1.27.Final.jar
apm-util-6.1.0.jar jackson-dataformat-cbor-2.8.10.jar netty-codec-socks-4.1.27.Final.jar
bcpkix-jdk15on-1.59.jar jackson-dataformat-smile-2.8.10.jar netty-common-4.1.27.Final.jar
bcprov-ext-jdk15on-1.59.jar jackson-dataformat-yaml-2.8.10.jar netty-handler-4.1.27.Final.jar
bcprov-jdk15on-1.59.jar jackson-datatype-guava-2.9.5.jar netty-handler-proxy-4.1.27.Final.jar
caffeine-2.6.2.jar jackson-datatype-jdk8-2.9.5.jar netty-resolver-4.1.27.Final.jar
client-java-2.0.0.jar jackson-mapper-asl-1.9.13.jar netty-tcnative-boringssl-static-2.0.7.Final.jar
client-java-api-2.0.0.jar jackson-module-kotlin-2.8.8.jar netty-transport-4.1.27.Final.jar
client-java-proto-2.0.0.jar jaeger-receiver-plugin-6.1.0.jar okhttp-2.7.5.jar
cluster-consul-plugin-6.1.0.jar java-dataloader-2.0.2.jar okhttp-3.9.0.jar
cluster-kubernetes-plugin-6.1.0.jar javax.servlet-api-3.1.0.jar okhttp-ws-2.7.5.jar
cluster-standalone-plugin-6.1.0.jar jcl-over-slf4j-1.7.25.jar okio-1.13.0.jar
cluster-zookeeper-plugin-6.1.0.jar jetty-http-9.4.2.v20170220.jar opencensus-api-0.12.3.jar
commons-codec-1.11.jar jetty-io-9.4.2.v20170220.jar opencensus-contrib-grpc-metrics-0.12.3.jar
commons-dbcp-1.4.jar jetty-security-9.4.2.v20170220.jar parent-join-client-6.3.2.jar
commons-io-2.6.jar jetty-server-9.4.2.v20170220.jar protobuf-java-3.4.0.jar
commons-lang3-3.7.jar jetty-servlet-9.4.2.v20170220.jar proto-google-common-protos-1.0.0.jar
commons-pool-1.5.4.jar jetty-util-9.4.2.v20170220.jar query-graphql-plugin-6.1.0.jar
consul-client-1.2.6.jar jline-0.9.94.jar rank-eval-client-6.3.2.jar
converter-jackson-2.3.0.jar jna-4.5.1.jar reactive-streams-1.0.2.jar
curator-client-4.0.1.jar joda-time-2.9.9.jar receiver-proto-6.1.0.jar
curator-framework-4.0.1.jar jopt-simple-4.6.jar reflectasm-1.11.7.jar
curator-recipes-4.0.1.jar jsr305-1.3.9.jar retrofit-2.3.0.jar
curator-x-discovery-4.0.1.jar kotlin-reflect-1.1.1.jar server-alarm-plugin-6.1.0.jar
elasticsearch-6.3.2.jar kotlin-stdlib-1.1.60.jar server-core-6.1.0.jar
elasticsearch-cli-6.3.2.jar library-buffer-6.1.0.jar server-starter-6.1.0.jar
elasticsearch-core-6.3.2.jar library-client-6.1.0.jar sharding-jdbc-core-2.0.3.jar
elasticsearch-rest-client-6.3.2.jar library-module-6.1.0.jar simpleclient-0.6.0.jar
elasticsearch-rest-high-level-client-6.3.2.jar library-server-6.1.0.jar simpleclient_common-0.6.0.jar
elasticsearch-secure-sm-6.3.2.jar library-util-6.1.0.jar simpleclient_hotspot-0.6.0.jar
elasticsearch-x-content-6.3.2.jar log4j-1.2.16.jar simpleclient_httpserver-0.6.0.jar
envoy-metrics-receiver-plugin-6.1.0.jar log4j-api-2.9.0.jar skywalking-clr-receiver-plugin-6.1.0.jar
error_prone_annotations-2.2.0.jar log4j-core-2.9.0.jar skywalking-jvm-receiver-plugin-6.1.0.jar
exporter-6.1.0.jar log4j-over-slf4j-1.7.25.jar skywalking-mesh-receiver-plugin-6.1.0.jar
generated-analysis-6.1.0.jar log4j-slf4j-impl-2.9.0.jar skywalking-register-receiver-plugin-6.1.0.jar
graphql-java-8.0.jar logging-interceptor-2.7.5.jar skywalking-sharing-server-plugin-6.1.0.jar
graphql-java-tools-5.2.3.jar lucene-analyzers-common-7.3.1.jar skywalking-trace-receiver-plugin-6.1.0.jar
groovy-2.4.5-indy.jar lucene-backward-codecs-7.3.1.jar slf4j-api-1.7.25.jar
grpc-context-1.15.1.jar lucene-core-7.3.1.jar snakeyaml-1.18.jar
grpc-core-1.15.1.jar lucene-grouping-7.3.1.jar storage-elasticsearch-plugin-6.1.0.jar
grpc-netty-1.15.1.jar lucene-highlighter-7.3.1.jar storage-jaeger-plugin-6.1.0.jar
grpc-protobuf-1.15.1.jar lucene-join-7.3.1.jar storage-jdbc-hikaricp-plugin-6.1.0.jar
grpc-protobuf-lite-1.15.1.jar lucene-memory-7.3.1.jar storage-zipkin-plugin-6.1.0.jar
grpc-stub-1.15.1.jar lucene-misc-7.3.1.jar swagger-annotations-1.5.12.jar
gson-2.8.1.jar lucene-queries-7.3.1.jar t-digest-3.2.jar
guava-20.0.jar lucene-queryparser-7.3.1.jar telemetry-api-6.1.0.jar
HdrHistogram-2.1.9.jar lucene-sandbox-7.3.1.jar telemetry-prometheus-6.1.0.jar
HikariCP-3.1.0.jar lucene-spatial3d-7.3.1.jar zipkin-2.9.1.jar
hppc-0.7.1.jar lucene-spatial-7.3.1.jar zipkin-receiver-plugin-6.1.0.jar
httpasyncclient-4.1.2.jar lucene-spatial-extras-7.3.1.jar zookeeper-3.4.10.jar
httpclient-4.5.2.jar lucene-suggest-7.3.1.jar
```
mbaight@foxmail.com
From: mbaight@foxmail.com
Date: 2019-06-29 12:02
To: reply+abtzq63yj4gngmgzzia6r753eqhh3evbnhhbxdmihi
Subject: Re: Re: [apache/skywalking] OutOfMemoryError by com.google.common.reflect.ClassPath.from (#2969)
Hi Wu Sheng:
I ran skywalking in another linux environment, and it worked.
Jdk:
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
But when using the same JDK to run in the faulty machine, it will not work. It should be a problem of the OS system.
I unzipped skywalking to the directory /root/data/skywalking/apache-skywalking-apm-bin/， and changed the following file:
application.yml
log4j2.xml
I put the document in the attachment.
mbaight@foxmail.com
From: 吴晟 Wu Sheng
Date: 2019-06-29 11:33
To: apache/skywalking
CC: zeroall; Author
Subject: Re: [apache/skywalking] OutOfMemoryError by com.google.common.reflect.ClassPath.from (#2969)
This is new to me. We have just tested the features in different
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services