监控平台——SkyWalking部署

一、环境准备

先下载SkyWalking安装包,需要注意的是SkyWalking 版本在10.X以上使用的nacos-client是2.X,如果安装的Nacos版本是1.X就会存在兼容性的问题。由于本人使用的SpringBoot项目是2.7.X版本,安装的Nacos版本只能是1.X版本的,就选择最新的是1.4.8,所以只能选择SkyWalking版本是9.7.0,对应的nacos-client版本是1.4.2。

1、下载安装

wget https://archive.apache.org/dist/skywalking/9.7.0/apache-skywalking-apm-9.7.0.tar.gz
tar -zxvf apache-skywalking-apm-9.7.0.tar.gz
cd apache-skywalking-apm-bin

2、配置集群方式

修改SkyWalking的配置文件 config/application.yml中集群的方式:

cluster:
  selector: ${SW_CLUSTER:nacos}
  nacos:
    serviceName: ${SW_SERVICE_NAME:"SkyWalking_OAP_Cluster"}
    hostPort: ${SW_CLUSTER_NACOS_HOST_PORT:10.60.1.63:8848}
    namespace: ${SW_CLUSTER_NACOS_NAMESPACE:"public"}  # 替换为你的Namespace ID,这里使用默认的命名空间
    username: ${SW_CLUSTER_NACOS_USERNAME:"nacos"}  # nacos用户名
    password: ${SW_CLUSTER_NACOS_PASSWORD:"nacos"}  # nacos登录密码
    # 高级配置(可选)
    clusterName: ${SW_CLUSTER_NACOS_CLUSTER_NAME:"DEFAULT"}
    healthCheckInterval: ${SW_CLUSTER_NACOS_HEALTH_CHECK_INTERVAL:5}

3、配置 Elasticsearch 8 存储

关于ES8存储的配置出现了很多问题,搞了几个小时才成功,主要是 安全证书问题,针对该问题,这里会详细描述遇到的问题和解决方案。

首先第一步是使用如下命令将oap-libs中oap-libs/storage-elasticsearch-plugin-9.7.0.jar复制到plugins文件夹下。

# 进入skywalking安装目录下
cd /home/app/apache-skywalking-apm-bin

#创建plugins文件夹
mkdir plugins

#将storage-elasticsearch-plugin-9.7.0.jar拷贝到plugins文件夹下
cp oap-libs/storage-elasticsearch-plugin-9.7.0.jar plugins/

 由于Elasticsearch 自动生成的自签名CA证书http_ca.crt 是 PEM 格式证书,但 SkyWalking 9.7.0 默认期望 JKS 或 PKCS12 格式的密钥库。如果不转化就会报如下错误信息:

2025-03-30 07:06:12,544 - org.apache.skywalking.oap.server.starter.OAPServerBootstrap - 64 [main] ERROR [] - Invalid keystore format
org.apache.skywalking.oap.server.library.module.ModuleStartException: Invalid keystore format
        at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.StorageModuleElasticsearchProvider.start(StorageModuleElasticsearchProvider.java:281) ~[storage-elasticsearch-plugin-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.library.module.BootstrapFlow.start(BootstrapFlow.java:46) ~[library-module-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.library.module.ModuleManager.init(ModuleManager.java:75) ~[library-module-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.starter.OAPServerBootstrap.start(OAPServerBootstrap.java:52) [server-starter-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.starter.OAPServerStartUp.main(OAPServerStartUp.java:23) [server-starter-9.7.0.jar:9.7.0]
Caused by: java.io.IOException: Invalid keystore format
        at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:688) ~[?:?]
        at sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:221) ~[?:?]
        at java.security.KeyStore.load(KeyStore.java:1473) ~[?:?]

所以需要将 PEM 证书转换为 PKCS12 格式(推荐)

# 进入elasticsearch安装包的证书目录
cd /home/app/elasticsearch-8.17.4/config/certs

# 转换证书(无密码版)
openssl pkcs12 -export -nokeys -in http_ca.crt -out http_ca.p12 -passout pass:

# 设置权限
chmod 644 http_ca.p12

接下来执行下面脚本需要验证证书有效性

# 检查PKCS12文件有效性
keytool -list -v -keystore /home/app/elasticsearch-8.17.4/config/certs/http_ca.p12 -storepass ""

验证结果如下:

此时可以配置SkyWalking的application.yml文件的存储模块内容如下:

storage:
  selector: ${SW_STORAGE:elasticsearch}
  elasticsearch:
    nameSpace: ${SW_NAMESPACE:""}
    clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:10.60.1.63:9200}  # 修改为单节点地址
    protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"https"}
    trustStorePath: ${SW_STORAGE_ES_SSL_JKS_PATH:"/home/app/elasticsearch-8.17.4/config/certs/http_ca.p12"}  # 使用CA证书
    trustStorePass: ${SW_STORAGE_ES_SSL_JKS_PASS:""}  # 如果证书有密码需填写
    user: ${SW_ES_USER:"elastic"}
    password: ${SW_ES_PASSWORD:"HAIyi123*"}
    indexShardsNumber: ${SW_STORAGE_ES_INDEX_SHARDS_NUMBER:1}    # 单节点建议设为1
    indexReplicasNumber: ${SW_STORAGE_ES_INDEX_REPLICAS_NUMBER:0} # 单节点必须设为0
    secretsManagementFile: ${SW_ES_SECRETS_MANAGEMENT_FILE:"/home/app/elasticsearch-8.17.4/config/certs/credentials.json"}  # 可选密钥文件

由于trustStorePass为空,在启动skywalking时会报如下错误信息:

2025-03-30 07:02:56,422 - org.apache.skywalking.oap.server.starter.OAPServerBootstrap - 64 [main] ERROR [] - Cannot invoke "String.toCharArray()" because "this.trustStorePass" is null
org.apache.skywalking.oap.server.library.module.ModuleStartException: Cannot invoke "String.toCharArray()" because "this.trustStorePass" is null
        at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.StorageModuleElasticsearchProvider.start(StorageModuleElasticsearchProvider.java:281) ~[storage-elasticsearch-plugin-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.library.module.BootstrapFlow.start(BootstrapFlow.java:46) ~[library-module-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.library.module.ModuleManager.init(ModuleManager.java:75) ~[library-module-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.starter.OAPServerBootstrap.start(OAPServerBootstrap.java:52) [server-starter-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.starter.OAPServerStartUp.main(OAPServerStartUp.java:23) [server-starter-9.7.0.jar:9.7.0]
Caused by: java.lang.NullPointerException: Cannot invoke "String.toCharArray()" because "this.trustStorePass" is null
        at org.apache.skywalking.library.elasticsearch.ElasticSearchBuilder.build(ElasticSearchBuilder.java:167) ~[library-elasticsearch-client-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.library.client.elasticsearch.ElasticSearchClient.connect(ElasticSearchClient.java:152) ~[library-client-9.7.0.jar:9.7.0]
        at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.StorageModuleElasticsearchProvider.start(StorageModuleElasticsearchProvider.java:268) ~[storage-elasticsearch-plugin-9.7.0.jar:9.7.0]
        ... 4 more

也可以在执行上面的转换证书是进行加密,如下:

keytool -list -v -keystore /home/app/elasticsearch-8.17.4/config/certs/http_ca.p12 -storepass "HAIyi123*"  # 设置证书的密码

然后指定trustStorePass,再次启动skywalking时会报如下错误信息:

Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:200) ~[?:?]
        at java.security.cert.PKIXParameters.<init>(PKIXParameters.java:120) ~[?:?]
        at java.security.cert.PKIXBuilderParameters.<init>(PKIXBuilderParameters.java:104) ~[?:?]
        at sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:98) ~[?:?]
        at sun.security.validator.Validator.getInstance(Validator.java:181) ~[?:?]
        at sun.security.ssl.X509TrustManagerImpl.getValidator(X509TrustManagerImpl.java:309) ~[?:?]
        at sun.security.ssl.X509TrustManagerImpl.checkTrustedInit(X509TrustManagerImpl.java:183) ~[?:?]
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:255) ~[?:?]
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:144) ~[?:?]
        at io.netty.handler.ssl.EnhancingX509ExtendedTrustManager.checkServerTrusted(EnhancingX509ExtendedTrustManager.java:69) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.ssl.ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify(ReferenceCountedOpenSslClientContext.java:235) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.ssl.ReferenceCountedOpenSslContext$AbstractCertificateVerifier.verify(ReferenceCountedOpenSslContext.java:790) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.internal.tcnative.CertificateVerifierTask.runTask(CertificateVerifierTask.java:36) ~[netty-tcnative-classes-2.0.61.Final.jar:2.0.61.Final]
        at io.netty.internal.tcnative.SSLTask.run(SSLTask.java:48) ~[netty-tcnative-classes-2.0.61.Final.jar:2.0.61.Final]
        at io.netty.internal.tcnative.SSLTask.run(SSLTask.java:42) ~[netty-tcnative-classes-2.0.61.Final.jar:2.0.61.Final]
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.runAndResetNeedTask(ReferenceCountedOpenSslEngine.java:1534) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.access$700(ReferenceCountedOpenSslEngine.java:96) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine$TaskDecorator.run(ReferenceCountedOpenSslEngine.java:1509) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1647) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1493) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1345) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1385) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[netty-codec-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[netty-codec-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[netty-codec-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152) ~[netty-handler-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[netty-transport-classes-epoll-4.1.100.Final.jar:4.1.100.Final]

这表明 Java 安全库无法从您的证书文件中提取可信的 CA 证书链。以下是完整的解决方案:

步骤1:验证证书完整性
# 检查证书内容
openssl x509 -in /home/app/elasticsearch-8.17.4/config/certs/http_ca.crt -noout -text

# 检查证书链(应显示完整的CA链)
openssl crl2pkcs7 -nocrl -certfile /home/app/elasticsearch-8.17.4/config/certs/http_ca.crt | openssl pkcs7 -print_certs -noout
步骤2:重建证书链

如果证书链不完整,手动构建完整链:

# 获取Elasticsearch生成的CA证书
cat /home/app/elasticsearch-8.17.4/config/certs/http_ca.crt > full_chain.crt

# 追加系统CA证书(可选)
cat /etc/ssl/certs/ca-certificates.crt >> full_chain.crt

# 转换为PKCS12格式(必须)
openssl pkcs12 -export -nokeys -in full_chain.crt -out full_chain.p12 -passout pass:

# 设置权限
chmod 644 full_chain.p12
chown skywalking:skywalking full_chain.p12
步骤3:将自签名证书加入Java信任库(推荐)
# 1. 进入证书目录
cd /home/app/elasticsearch-8.17.4/config/certs

# 2. 将CA证书导入Java默认信任库
sudo keytool -importcert \
    -alias elasticsearch-ca \
    -file http_ca.crt \
    -keystore $JAVA_HOME/lib/security/cacerts \
    -storepass changeit \
    -noprompt

# 3. 修改SkyWalking配置(不再需要指定trustStore)
storage:
  elasticsearch:
    protocol: "HTTPS"
    # 注释掉trustStore相关配置
    # trustStorePath: ""
    # trustStorePass: ""
    user: "elastic"
    password: "HAIyi123*"
步骤4:验证Java信任库
keytool -list -keystore $JAVA_HOME/lib/security/cacerts -storepass changeit | grep elasticsearch

应显示:

elasticsearch-ca, Mar 30, 2025, trustedCertEntry
测试HTTPS连接
curl --cacert /home/app/elasticsearch-8.17.4/config/certs/http_ca.crt \
     -u elastic:HAIyi123* \
     https://10.60.1.63:9200/_cluster/health

应显示:

[root@localhost certs]# curl --cacert /home/app/elasticsearch-8.17.4/config/certs/http_ca.crt -u elastic:HAIyi123* https://10.60.1.63:9200/_cluster/health
{"cluster_name":"my-es-cluster","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":3,"active_shards":3,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"unassigned_primary_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}[root@localhost certs]# 
[root@localhost certs]# 

二、启动服务

  1. 启动OAP服务:

    cd /home/app/apache-skywalking-apm-bin/bin
    ./oapService.sh
  2. Web UI部署

    cd /home/app/apache-skywalking-apm-bin/bin
    ./webappService.sh

    启动后,直接可以在浏览器上输入http://10.60.1.63:8080/打开SkyWalking的页面:

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值