How to completely release connection when using HttpClient?
0
I use HttpClient (https://hc.apache.org/httpcomponents-client-4.5.x/index.html) to make many http calls back to back and in parallel. After running for a while, it gets this exception:
java.net.BindException: Address already in use: connect
I tried to closing everything I can see, but I must still miss something because it still has that error.
How to properly release connections to avoid this connection leak problem?
Here's the test case to reproduce the problem, running in Java8 on Windows:
public void test(String url) throws Exception {
List<Thread> threads = new ArrayList<Thread>();
for(int t=0; t<40; t++) {
int tt = t;
threads.add(new Thread(() -> {
for(int i=0; i<Integer.MAX_VALUE; i++) {
URI metadataUri;
try {
metadataUri = new URI(url);
} catch (Exception e1) {
e1.printStackTrace();
continue;
}
HttpPost httpRequest = new HttpPost(metadataUri);
httpRequest.setEntity(new ByteArrayEntity( "abc".getBytes()));
//httpRequest.addHeader("Connection", "close");
CloseableHttpClient httpclient = HttpClients.custom().build();
try {
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest);
metadataResponse2.close();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
httpRequest.completed();
httpRequest.releaseConnection();
} catch (Exception e) {
e.printStackTrace();
}
try {
httpclient.close();
} catch (Exception e) {
e.printStackTrace();
}
}
System.out.println("thread " + tt + " round " + i);
}
}));
}
for(Thread thread : threads) {
thread.start();
}
}
java apache-httpclient-4.x connection-leaks
asked Nov 13 '18 at 16:35
DavidDavid
5915
add a comment |
0
I use HttpClient (https://hc.apache.org/httpcomponents-client-4.5.x/index.html) to make many http calls back to back and in parallel. After running for a while, it gets this exception:
java.net.BindException: Address already in use: connect
I tried to closing everything I can see, but I must still miss something because it still has that error.
How to properly release connections to avoid this connection leak problem?
Here's the test case to reproduce the problem, running in Java8 on Windows:
public void test(String url) throws Exception {
List<Thread> threads = new ArrayList<Thread>();
for(int t=0; t<40; t++) {
int tt = t;
threads.add(new Thread(() -> {
for(int i=0; i<Integer.MAX_VALUE; i++) {
URI metadataUri;
try {
metadataUri = new URI(url);
} catch (Exception e1) {
e1.printStackTrace();
continue;
}
HttpPost httpRequest = new HttpPost(metadataUri);
httpRequest.setEntity(new ByteArrayEntity( "abc".getBytes()));
//httpRequest.addHeader("Connection", "close");
CloseableHttpClient httpclient = HttpClients.custom().build();
try {
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest);
metadataResponse2.close();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
httpRequest.completed();
httpRequest.releaseConnection();
} catch (Exception e) {
e.printStackTrace();
}
try {
httpclient.close();
} catch (Exception e) {
e.printStackTrace();
}
}
System.out.println("thread " + tt + " round " + i);
}
}));
}
for(Thread thread : threads) {
thread.start();
}
}
java apache-httpclient-4.x connection-leaks
asked Nov 13 '18 at 16:35
DavidDavid
5915
that exception just means that something is using that port, not necessarily that you're using it....
– Roddy of the Frozen Peas
Nov 13 '18 at 16:38
But this is client port, not server port right? My understand is client port is dynamically allocated some where in TCP/IP layer and should be closed when the connection is close. But apparently, HttpClient lib is not closing connections in the above test code, so after running fine for about 12,000 http requests, it starts throwing that exception.
– David
Nov 13 '18 at 16:48
add a comment |
0
0
0
I use HttpClient (https://hc.apache.org/httpcomponents-client-4.5.x/index.html) to make many http calls back to back and in parallel. After running for a while, it gets this exception:
java.net.BindException: Address already in use: connect
I tried to closing everything I can see, but I must still miss something because it still has that error.
How to properly release connections to avoid this connection leak problem?
Here's the test case to reproduce the problem, running in Java8 on Windows:
public void test(String url) throws Exception {
List<Thread> threads = new ArrayList<Thread>();
for(int t=0; t<40; t++) {
int tt = t;
threads.add(new Thread(() -> {
for(int i=0; i<Integer.MAX_VALUE; i++) {
URI metadataUri;
try {
metadataUri = new URI(url);
} catch (Exception e1) {
e1.printStackTrace();
continue;
}
HttpPost httpRequest = new HttpPost(metadataUri);
httpRequest.setEntity(new ByteArrayEntity( "abc".getBytes()));
//httpRequest.addHeader("Connection", "close");
CloseableHttpClient httpclient = HttpClients.custom().build();
try {
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest);
metadataResponse2.close();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
httpRequest.completed();
httpRequest.releaseConnection();
} catch (Exception e) {
e.printStackTrace();
}
try {
httpclient.close();
} catch (Exception e) {
e.printStackTrace();
}
}
System.out.println("thread " + tt + " round " + i);
}
}));
}
for(Thread thread : threads) {
thread.start();
}
}
java apache-httpclient-4.x connection-leaks
asked Nov 13 '18 at 16:35
DavidDavid
5915
I use HttpClient (https://hc.apache.org/httpcomponents-client-4.5.x/index.html) to make many http calls back to back and in parallel. After running for a while, it gets this exception:
java.net.BindException: Address already in use: connect
I tried to closing everything I can see, but I must still miss something because it still has that error.
How to properly release connections to avoid this connection leak problem?
Here's the test case to reproduce the problem, running in Java8 on Windows:
public void test(String url) throws Exception {
List<Thread> threads = new ArrayList<Thread>();
for(int t=0; t<40; t++) {
int tt = t;
threads.add(new Thread(() -> {
for(int i=0; i<Integer.MAX_VALUE; i++) {
URI metadataUri;
try {
metadataUri = new URI(url);
} catch (Exception e1) {
e1.printStackTrace();
continue;
}
HttpPost httpRequest = new HttpPost(metadataUri);
httpRequest.setEntity(new ByteArrayEntity( "abc".getBytes()));
//httpRequest.addHeader("Connection", "close");
CloseableHttpClient httpclient = HttpClients.custom().build();
try {
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest);
metadataResponse2.close();
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
httpRequest.completed();
httpRequest.releaseConnection();
} catch (Exception e) {
e.printStackTrace();
}
try {
httpclient.close();
} catch (Exception e) {
e.printStackTrace();
}
}
System.out.println("thread " + tt + " round " + i);
}
}));
}
for(Thread thread : threads) {
thread.start();
}
}
java apache-httpclient-4.x connection-leaks
java apache-httpclient-4.x connection-leaks
asked Nov 13 '18 at 16:35
DavidDavid
5915
asked Nov 13 '18 at 16:35
DavidDavid
5915
asked Nov 13 '18 at 16:35
DavidDavid
5915
asked Nov 13 '18 at 16:35
DavidDavid
5915
asked Nov 13 '18 at 16:35
DavidDavid
5915
5915
that exception just means that something is using that port, not necessarily that you're using it....
– Roddy of the Frozen Peas
Nov 13 '18 at 16:38
But this is client port, not server port right? My understand is client port is dynamically allocated some where in TCP/IP layer and should be closed when the connection is close. But apparently, HttpClient lib is not closing connections in the above test code, so after running fine for about 12,000 http requests, it starts throwing that exception.
– David
Nov 13 '18 at 16:48
add a comment |
that exception just means that something is using that port, not necessarily that you're using it....
– Roddy of the Frozen Peas
Nov 13 '18 at 16:38
But this is client port, not server port right? My understand is client port is dynamically allocated some where in TCP/IP layer and should be closed when the connection is close. But apparently, HttpClient lib is not closing connections in the above test code, so after running fine for about 12,000 http requests, it starts throwing that exception.
– David
Nov 13 '18 at 16:48
that exception just means that something is using that port, not necessarily that you're using it....
– Roddy of the Frozen Peas
Nov 13 '18 at 16:38
that exception just means that something is using that port, not necessarily that you're using it....
– Roddy of the Frozen Peas
Nov 13 '18 at 16:38
But this is client port, not server port right? My understand is client port is dynamically allocated some where in TCP/IP layer and should be closed when the connection is close. But apparently, HttpClient lib is not closing connections in the above test code, so after running fine for about 12,000 http requests, it starts throwing that exception.
– David
Nov 13 '18 at 16:48
But this is client port, not server port right? My understand is client port is dynamically allocated some where in TCP/IP layer and should be closed when the connection is close. But apparently, HttpClient lib is not closing connections in the above test code, so after running fine for about 12,000 http requests, it starts throwing that exception.
– David
Nov 13 '18 at 16:48
add a comment |
1 Answer
1
active
oldest
votes
0
Normally, a try with resource (see reference) should handle this:
try (CloseableHttpClient httpclient = HttpClients.custom().build();
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest)) {
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
httpRequest.completed();
httpRequest.releaseConnection();
}
answered Nov 13 '18 at 16:39
maio290maio290
2,029414
Thanks maio290! I tried that too, but still it doesn't help.
– David
Nov 13 '18 at 16:41
@David - how many rounds do you need to get this error? I added the metadataResponse2 to the try as well and don't have any error after ~ 1000 rounds.
– maio290
Nov 13 '18 at 16:43
I just did that too, and consistently got the error after about 400 rounds on each of those 40 threads.
– David
Nov 13 '18 at 17:02
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53285571%2fhow-to-completely-release-connection-when-using-httpclient%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
Normally, a try with resource (see reference) should handle this:
try (CloseableHttpClient httpclient = HttpClients.custom().build();
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest)) {
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
httpRequest.completed();
httpRequest.releaseConnection();
}
answered Nov 13 '18 at 16:39
maio290maio290
2,029414
Thanks maio290! I tried that too, but still it doesn't help.
– David
Nov 13 '18 at 16:41
@David - how many rounds do you need to get this error? I added the metadataResponse2 to the try as well and don't have any error after ~ 1000 rounds.
– maio290
Nov 13 '18 at 16:43
I just did that too, and consistently got the error after about 400 rounds on each of those 40 threads.
– David
Nov 13 '18 at 17:02
add a comment |
0
Normally, a try with resource (see reference) should handle this:
try (CloseableHttpClient httpclient = HttpClients.custom().build();
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest)) {
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
httpRequest.completed();
httpRequest.releaseConnection();
}
answered Nov 13 '18 at 16:39
maio290maio290
2,029414
Thanks maio290! I tried that too, but still it doesn't help.
– David
Nov 13 '18 at 16:41
@David - how many rounds do you need to get this error? I added the metadataResponse2 to the try as well and don't have any error after ~ 1000 rounds.
– maio290
Nov 13 '18 at 16:43
I just did that too, and consistently got the error after about 400 rounds on each of those 40 threads.
– David
Nov 13 '18 at 17:02
add a comment |
0
0
0
Normally, a try with resource (see reference) should handle this:
try (CloseableHttpClient httpclient = HttpClients.custom().build();
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest)) {
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
httpRequest.completed();
httpRequest.releaseConnection();
}
answered Nov 13 '18 at 16:39
maio290maio290
2,029414
Normally, a try with resource (see reference) should handle this:
try (CloseableHttpClient httpclient = HttpClients.custom().build();
CloseableHttpResponse metadataResponse2 = httpclient.execute(httpRequest)) {
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
httpRequest.completed();
httpRequest.releaseConnection();
}
answered Nov 13 '18 at 16:39
maio290maio290
2,029414
edited Nov 13 '18 at 16:46
answered Nov 13 '18 at 16:39
maio290maio290
2,029414
answered Nov 13 '18 at 16:39
maio290maio290
2,029414
answered Nov 13 '18 at 16:39
maio290maio290
2,029414
2,029414
Thanks maio290! I tried that too, but still it doesn't help.
– David
Nov 13 '18 at 16:41
@David - how many rounds do you need to get this error? I added the metadataResponse2 to the try as well and don't have any error after ~ 1000 rounds.
– maio290
Nov 13 '18 at 16:43
I just did that too, and consistently got the error after about 400 rounds on each of those 40 threads.
– David
Nov 13 '18 at 17:02
add a comment |
Thanks maio290! I tried that too, but still it doesn't help.
– David
Nov 13 '18 at 16:41
@David - how many rounds do you need to get this error? I added the metadataResponse2 to the try as well and don't have any error after ~ 1000 rounds.
– maio290
Nov 13 '18 at 16:43
I just did that too, and consistently got the error after about 400 rounds on each of those 40 threads.
– David
Nov 13 '18 at 17:02
Thanks maio290! I tried that too, but still it doesn't help.
– David
Nov 13 '18 at 16:41
Thanks maio290! I tried that too, but still it doesn't help.
– David
Nov 13 '18 at 16:41
@David - how many rounds do you need to get this error? I added the metadataResponse2 to the try as well and don't have any error after ~ 1000 rounds.
– maio290
Nov 13 '18 at 16:43
@David - how many rounds do you need to get this error? I added the metadataResponse2 to the try as well and don't have any error after ~ 1000 rounds.
– maio290
Nov 13 '18 at 16:43
I just did that too, and consistently got the error after about 400 rounds on each of those 40 threads.
– David
Nov 13 '18 at 17:02
I just did that too, and consistently got the error after about 400 rounds on each of those 40 threads.
– David
Nov 13 '18 at 17:02
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53285571%2fhow-to-completely-release-connection-when-using-httpclient%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
that exception just means that something is using that port, not necessarily that you're using it....
– Roddy of the Frozen Peas
Nov 13 '18 at 16:38
But this is client port, not server port right? My understand is client port is dynamically allocated some where in TCP/IP layer and should be closed when the connection is close. But apparently, HttpClient lib is not closing connections in the above test code, so after running fine for about 12,000 http requests, it starts throwing that exception.
– David
Nov 13 '18 at 16:48