jest连接AWS版本的es

jest怎么连es?

可参考https://blog.youkuaiyun.com/mingover/article/details/82941506

情况一,需要有key和secret

这边要连 aws 的es,在使用jest去连es的时候，抛了一个错

Caused by: com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain
        at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:117)
        at vc.inreach.aws.request.AWSSigner.getSignedHeaders(AWSSigner.java:104)
        at vc.inreach.aws.request.AWSSigningRequestInterceptor.process(AWSSigningRequestInterceptor.java:32)
        at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:133)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:183)

跟踪后发现是和 AWS_ACCESS_KEY_ID 有关.

https://github.com/awslabs/amazon-kinesis-connectors/issues/42

https://www.youtube.com/watch?v=tgb_MRVylWw

export AWS_ACCESS_KEY_ID=…
export AWS_SECRET_ACCESS_KEY=…

情况二,请求的时候es的路径错了，显示localhost:9200

如果用aws，发现会自己去new一个bean,如JestClientFactory ,如果自定义了es的配置，记得做好逻辑.

增加pom配置

在pom中加上配置 (https://github.com/VanRoy/spring-data-jest

        <dependency>
            <groupId>com.github.vanroy</groupId>
            <artifactId>spring-boot-starter-data-jest</artifactId>
            <version>3.0.0.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>vc.inreach.aws</groupId>
            <artifactId>aws-signing-request-interceptor</artifactId>
            <version>0.0.21</version>
        </dependency>

断点查看

在ElasticsearchRepositoryFactory的L73打断点,如下的栈. 这个 elasticsearchOperations中的client,是 localhost:9200
同时发现，这个elasticsearchOperations并不是我们在config中配置的…

打个断点,相应的栈为:
查看里面的对象elasticsearchOperations中的对象，url是localhost:9200

getTargetRepository:73, ElasticsearchRepositoryFactory (org.springframework.data.elasticsearch.repository.support)
getRepository:298, RepositoryFactorySupport (org.springframework.data.repository.core.support)
lambda$afterPropertiesSet$3:287, RepositoryFactoryBeanSupport (org.springframework.data.repository.core.support)
get:-1, 1227859754 (org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport$$Lambda$418)
getNullable:141, Lazy (org.springframework.data.util)
get:63, Lazy (org.springframework.data.util)
afterPropertiesSet:290, RepositoryFactoryBeanSupport (org.springframework.data.repository.core.support)
afterPropertiesSet:67, ElasticsearchRepositoryFactoryBean (org.springframework.data.elasticsearch.repository.support)
invokeInitMethods:1767, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
initializeBean:1704, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
doCreateBean:581, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
createBean:503, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
lambda$doGetBean$0:317, AbstractBeanFactory (org.springframework.beans.factory.support)
getObject:-1, 696165690 (org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$117)
getSingleton:222, DefaultSingletonBeanRegistry (org.springframework.beans.factory.support)
doGetBean:315, AbstractBeanFactory (org.springframework.beans.factory.support)
getBean:199, AbstractBeanFactory (org.springframework.beans.factory.support)
resolveCandidate:251, DependencyDescriptor (org.springframework.beans.factory.config)
doResolveDependency:1138, DefaultListableBeanFactory (org.springframework.beans.factory.support)
resolveDependency:1065, DefaultListableBeanFactory (org.springframework.beans.factory.support)
inject:584, AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement (org.springframework.beans.factory.annotation)
inject:91, InjectionMetadata (org.springframework.beans.factory.annotation)
postProcessPropertyValues:373, AutowiredAnnotationBeanPostProcessor (org.springframework.beans.factory.annotation)
populateBean:1350, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
doCreateBean:580, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
createBean:503, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
lambda$doGetBean$0:317, AbstractBeanFactory (org.springframework.beans.factory.support)
getObject:-1, 696165690 (org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$117)
getSingleton:222, DefaultSingletonBeanRegistry (org.springframework.beans.factory.support)
doGetBean:315, AbstractBeanFactory (org.springframework.beans.factory.support)
getBean:199, AbstractBeanFactory (org.springframework.beans.factory.support)
resolveCandidate:251, DependencyDescriptor (org.springframework.beans.factory.config)
doResolveDependency:1138, DefaultListableBeanFactory (org.springframework.beans.factory.support)
resolveDependency:1065, DefaultListableBeanFactory (org.springframework.beans.factory.support)
inject:584, AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement (org.springframework.beans.factory.annotation)
inject:91, InjectionMetadata (org.springframework.beans.factory.annotation)
postProcessPropertyValues:373, AutowiredAnnotationBeanPostProcessor (org.springframework.beans.factory.annotation)
populateBean:1350, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
doCreateBean:580, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
createBean:503, AbstractAutowireCapableBeanFactory (org.springframework.beans.factory.support)
lambda$doGetBean$0:317, AbstractBeanFactory (org.springframework.beans.factory.support)
getObject:-1, 696165690 (org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$117)
getSingleton:222, DefaultSingletonBeanRegistry (org.springframework.beans.factory.support)
doGetBean:315, AbstractBeanFactory (org.springframework.beans.factory.support)
getBean:199, AbstractBeanFactory (org.springframework.beans.factory.support)
preInstantiateSingletons:760, DefaultListableBeanFactory (org.springframework.beans.factory.support)
finishBeanFactoryInitialization:869, AbstractApplicationContext (org.springframework.context.support)
refresh:550, AbstractApplicationContext (org.springframework.context.support)
refresh:140, ServletWebServerApplicationContext (org.springframework.boot.web.servlet.context)
refresh:759, SpringApplication (org.springframework.boot)
refreshContext:395, SpringApplication (org.springframework.boot)
run:327, SpringApplication (org.springframework.boot)
run:1255, SpringApplication (org.springframework.boot)
run:1243, SpringApplication (org.springframework.boot)
main:18, busiApplication (com.sasa.degaa.busi)

这个localhost:9200是哪里来的?

在类的 JestClientFactory中，我们发现

public JestClient getObject() {

JestHttpClient client = new JestHttpClient();
if (httpClientConfig == null) {
    log.debug("There is no configuration to create http client. Going to create simple client with default values");
    httpClientConfig = new HttpClientConfig.Builder("http://localhost:9200").build();
}
....

为什么httpClientConfig为null?

进一步跟踪,我在自定义的esconfig中发现这段不走依赖？,那这个jestClientFactory对象哪来的？

@Bean
public JestClient jestClient(@Qualifier("jestClientFactory")JestClientFactory jestClientFactory) {
    return jestClientFactory.getObject();
}

我去找了下哪里引用了JestClientFactory，后来
我在 ElasticsearchJestAWSAutoConfiguration类中找到

@Bean
public JestClientFactory jestClientFactory(AWSCredentialsProvider credentialsProvider) {

    final AWSSigner awsSigner = new AWSSigner(credentialsProvider, getRegion(), AWS_SERVICE, CLOCK);

    final AWSSigningRequestInterceptor requestInterceptor = new AWSSigningRequestInterceptor(awsSigner);
    return new JestClientFactory() {
        @Override
        protected HttpClientBuilder configureHttpClient(HttpClientBuilder builder) {
            builder.addInterceptorLast(requestInterceptor);
            return builder;
        }
        @Override
        protected HttpAsyncClientBuilder configureHttpClient(HttpAsyncClientBuilder builder) {
            builder.addInterceptorLast(requestInterceptor);
            return builder;
        }
    };
}

这里有一个bean名字为jestClientFactory的对象。。。。。。。

所以，是ElasticsearchJestAWSAutoConfiguration自己定义了这个bean.

解决办法就是把bean的名字给改了,如改成projectJestClientFactory

@Bean
public JestClient jestClient(@Qualifier("projectJestClientFactory")JestClientFactory jestClientFactory) {
    return jestClientFactory.getObject();
}

当然，相应的依赖也要改!!!

特别提醒

JestElasticsearchTemplate会有一些不支持操作，即会 throw new UnsupportedOperationException();
执行的时候会抛错， 记得处理。

发现spring-boot-starter-data-jest的3.1.5.RELEASE版本
createIndexBuilder.payload(String.valueOf(settings)); 这一行 payload没实现 (搞不懂为什么)
后来换成3.0.0.RELEASE 是OK的