elasticsearch批量插入数据的时候出现java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection「建议收藏」

elasticsearch批量插入数据的时候出现java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection「建议收藏」问题:elasticsearch每次都批量插入几万数据量,然后就会出现下列问题。看这个问题应该是配置的问题ERROR[https-jsse-nio-443-exec-4]2020-07-0923:31:54(EsMiniDaansouDataInfoWithBLOBsUtil.java:80)java.net.SocketTimeoutException:30,000millisecondstimeoutonconnectionhttp-outgoing-0[ACTIVE]

大家好,又见面了,我是你们的朋友全栈君。如果您正在找激活码,请点击查看最新教程,关注关注公众号 “全栈程序员社区” 获取激活教程,可能之前旧版本教程已经失效.最新Idea2022.1教程亲测有效,一键激活。

Jetbrains全系列IDE使用 1年只要46元 售后保障 童叟无欺

问题:

elasticsearch每次都批量插入几万数据量,然后就会出现下列问题。看这个问题应该是配置的问题

ERROR [https-jsse-nio-443-exec-4] 2020-07-09 23:31:54 (EsMiniDaansouDataInfoWithBLOBsUtil.java:80) java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]
	at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:808)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:248)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1611)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1581)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1551)
	at org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:531)
	at cn.dxszxw.elasticsearch.esUtils.EsMiniDaansouDataInfoWithBLOBsUtil.increaseEsData(EsMiniDaansouDataInfoWithBLOBsUtil.java:76)
	at cn.dxszxw.controller.ElasticsearchController.createMiniDaansouDataInfoWithBLOBsAllIndexByRange(ElasticsearchController.java:146)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at com.alibaba.druid.support.http.WebStatFilter.doFilter(WebStatFilter.java:123)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387)
	at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92)
	at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39)
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
	... 1 more

 

解决办法(我仅处理elasticsearch7.8.0版本的,可以看到下面的方法出现的是RestHighLevelClient高级客户端的):

创建下面列举的配置类即可,然后相应修改下自己的需要的时间

import org.apache.http.HttpHost;
import org.apache.http.client.config.RequestConfig;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ElasticSearchClientConfig {

	@Bean
	@Qualifier("highLevelClient")
	public RestHighLevelClient restHighLevelClient() {
		// RestHighLevelClient highLevelClient = new RestHighLevelClient(
		// RestClient.builder(new HttpHost("127.0.0.1", 9200, "http")));

		RestHighLevelClient highLevelClient = new RestHighLevelClient(
				RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))
						.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
							// 该方法接收一个RequestConfig.Builder对象,对该对象进行修改后然后返回。
							@Override
							public RequestConfig.Builder customizeRequestConfig(
									RequestConfig.Builder requestConfigBuilder) {
								return requestConfigBuilder.setConnectTimeout(5000 * 1000) // 连接超时(默认为1秒)
										.setSocketTimeout(6000 * 1000);// 套接字超时(默认为30秒)//更改客户端的超时限制默认30秒现在改为100*1000分钟
							}
						}));// 调整最大重试超时时间(默认为30秒).setMaxRetryTimeoutMillis(60000);

		return highLevelClient;
	}

}

RestClientBuilder支持同时提供一个RequestConfigCallback和一个HttpClientConfigCallback,你可以定制 the Apache Async Http Client 公开的配置。这两个回调函数可以修改某些特定的行为,而不会覆盖RestClient初始化的所有其他默认配置。 

使用elasticsearch查询百亿级数据必然会影响查询的速度但是es与客户端的的默认响应时间只有30秒,查询大量的数据时30秒的时间一般都不够用。

版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 举报,一经查实,本站将立刻删除。

发布者:全栈程序员-用户IM,转载请注明出处:https://javaforall.cn/179193.html原文链接:https://javaforall.cn

【正版授权,激活自己账号】: Jetbrains全家桶Ide使用,1年售后保障,每天仅需1毛

【官方授权 正版激活】: 官方授权 正版激活 支持Jetbrains家族下所有IDE 使用个人JB账号...

(0)


相关推荐

  • 未将对象引用设置到对象的实例–可能出现的问题总结

    未将对象引用设置到对象的实例–可能出现的问题总结

    2021年11月15日
  • fastjson教程 List转JSONArray以及JSONArray转List[通俗易懂]

    fastjson教程 List转JSONArray以及JSONArray转List[通俗易懂]1.fastjsonList转JSONArrayList<T>list=newArrayList<T>();JSONArrayarray=JSONArray.parseArray(JSON.toJSONString(list));2.fastjsonJSONArray转ListJSONArrayarray=newJSONArray();List<T>list=JSONObject.parseArray(array.toJSONSt..

  • Python学习笔记一(基础知识)

    Python学习笔记一(基础知识)1.Python 的特点: 1.1. 语法简介,实现同样的功能,代码行数少; 1.2. 作为解释型语言,性能与C/C++有差距,甚至比不上java,发布代码必须发布源码,而C等编译类语言,只需要把编译后的机器码发布即可; 1.3. 可实现网站或游戏后台,网络爬虫工具等,YouTube、豆瓣、Google、Yahoo等都在使用Python; 1.4. 丰富的基础代码库,覆盖了网络、文件、GUI、数据库、文本等大量内容; 1.5. Python有两个版本,一个是2.x版

  • Java分布式应用:Java分布式通讯方式

    Java分布式应用:Java分布式通讯方式第一部分Java分布式通讯的方式1.基于消息方式实现系统间通讯a.数据传输部分:TCP/IP、UDP/IPTCP(端口号为23)与UDP(端口号为53)的区别1、TCP面向连接(如打电话要先拨号建立连接);UDP是无连接的,即发送数据之前不需要建立连接2、TCP提供可靠的服务。也就是说,通过TCP连接传送的数据,无差错,不丢失,不重复,且按序到达;UDP尽最大努力交付,即不保证…

  • tomcat配置虚拟主机

    tomcat配置虚拟主机

    2021年11月16日
  • 电子信息系统机房设计规范 GB50174-2017

    电子信息系统机房设计规范 GB50174-2017一、物理安全1.1物理安全主要包括:(1)机房环境安全(2)通信线路安全(3)设备安全(4)电源安全1.1.1机房的安全等级分为三个基本类别:A类:对计算机机房的安全有严格的要求,有完

发表回复

您的电子邮箱地址不会被公开。

关注全栈程序员社区公众号