Elasticsearch Java REST Client: see if number of eligible hits exceeded SearchSourceBuilder size
I have an Elasticsearch request:
SearchSourceBuilder source = new SearchSourceBuilder()
.size(limit)
.from(offset)
.query(queryBuilder)
SearchRequest searchRequest = new SearchRequest(indexName)
.types(type)
.source(source);
SearchResponse searchResponse = client.search(searchRequest);
I need to optionally return a pagination token based on whether or not the number of results returned was limited by the limit parameter (in other words, are there more results to return). Is there a way to see if the number of eligible search hits that matched the query exceeded limit, other than either doing another query, or by setting the size to limit + 1 and removing the last hit in the response?
java
elasticsearch limit asked Nov 12 at 18:41
pterry26
4441517
add a comment |
I have an Elasticsearch request:
SearchSourceBuilder source = new SearchSourceBuilder()
.size(limit)
.from(offset)
.query(queryBuilder)
SearchRequest searchRequest = new SearchRequest(indexName)
.types(type)
.source(source);
SearchResponse searchResponse = client.search(searchRequest);
I need to optionally return a pagination token based on whether or not the number of results returned was limited by the limit parameter (in other words, are there more results to return). Is there a way to see if the number of eligible search hits that matched the query exceeded limit, other than either doing another query, or by setting the size to limit + 1 and removing the last hit in the response?
java
elasticsearch limit asked Nov 12 at 18:41
pterry26
4441517
add a comment |
I have an Elasticsearch request:
SearchSourceBuilder source = new SearchSourceBuilder()
.size(limit)
.from(offset)
.query(queryBuilder)
SearchRequest searchRequest = new SearchRequest(indexName)
.types(type)
.source(source);
SearchResponse searchResponse = client.search(searchRequest);
I need to optionally return a pagination token based on whether or not the number of results returned was limited by the limit parameter (in other words, are there more results to return). Is there a way to see if the number of eligible search hits that matched the query exceeded limit, other than either doing another query, or by setting the size to limit + 1 and removing the last hit in the response?
java
elasticsearch limit asked Nov 12 at 18:41
pterry26
4441517
I have an Elasticsearch request:
SearchSourceBuilder source = new SearchSourceBuilder()
.size(limit)
.from(offset)
.query(queryBuilder)
SearchRequest searchRequest = new SearchRequest(indexName)
.types(type)
.source(source);
SearchResponse searchResponse = client.search(searchRequest);
I need to optionally return a pagination token based on whether or not the number of results returned was limited by the limit parameter (in other words, are there more results to return). Is there a way to see if the number of eligible search hits that matched the query exceeded limit, other than either doing another query, or by setting the size to limit + 1 and removing the last hit in the response?
java
elasticsearch limit java
elasticsearch limit asked Nov 12 at 18:41
pterry26
4441517
asked Nov 12 at 18:41
pterry26
4441517
asked Nov 12 at 18:41
pterry26
4441517
asked Nov 12 at 18:41
pterry26
4441517
asked Nov 12 at 18:41
pterry26
4441517
4441517
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
Elasticsearch returns the total number of documents matching your query in hits.totalHits property in the response. If it is bigger than offset+limit then you have next page.
answered Nov 23 at 18:56
martin-g
12k1825
I did end up using the totalHits property; thanks!
– pterry26
Nov 26 at 22:50
add a comment |
I think that what you need is first just a count search, with this I mean do the query you want and set the size to 0 this make the result to just return the total hits, where you can get the size of each page.
int pageSize = 10;
long totalCount = searchResponse.getHits().getTotalHits();
PageCount pageCount = new PageCount();
if (totalCount <= 0)
return result;
long pages = (totalCount < pageSize) ? 1l : ((totalCount % pageSize) == 0) ? (totalCount / pageSize) : ((totalCount / pageSize) + 1);
if (totalCount <= 10000) {
result.setTotal(totalCount);
result.setPages(pages);
} else {
result.setTotal(10000);
result.setPages((10000 / pageSize));
}
Here I use the 10000 just as a limit so that Elasticsearch to limit the amount of memory it consumes. Once you have this information you can return the number of pages for the query and make requests for a given page.
Once you have this, you could setup the search request like:
long size = ((pageCount.getTotal() < pageSize) ? pageCount.getTotal() : pageSize);
if (pageCount.getTotal() > pageSize && (page + 1) >= pageCount.getPages()) {
size = ((pageCount.getTotal() % pageSize) != 0 ? pageCount.getTotal() % pageSize : pageSize);
}
searchRequest.setSize((int) size).setFrom((int) (page * pageSize))
This is just sample code, I hope this get you enough.
Cheers
answered Nov 23 at 19:52
nmorenor
1309
Thanks, but it looks like I can get the totalHits property from the original query. It turns out this is equal to the total number of possible results matching the query, and is unaffected by the size limit.
– pterry26
Nov 26 at 22:52
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53268231%2felasticsearch-java-rest-client-see-if-number-of-eligible-hits-exceeded-searchso%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Elasticsearch returns the total number of documents matching your query in hits.totalHits property in the response. If it is bigger than offset+limit then you have next page.
answered Nov 23 at 18:56
martin-g
12k1825
I did end up using the totalHits property; thanks!
– pterry26
Nov 26 at 22:50
add a comment |
Elasticsearch returns the total number of documents matching your query in hits.totalHits property in the response. If it is bigger than offset+limit then you have next page.
answered Nov 23 at 18:56
martin-g
12k1825
I did end up using the totalHits property; thanks!
– pterry26
Nov 26 at 22:50
add a comment |
Elasticsearch returns the total number of documents matching your query in hits.totalHits property in the response. If it is bigger than offset+limit then you have next page.
answered Nov 23 at 18:56
martin-g
12k1825
Elasticsearch returns the total number of documents matching your query in hits.totalHits property in the response. If it is bigger than offset+limit then you have next page.
answered Nov 23 at 18:56
martin-g
12k1825
answered Nov 23 at 18:56
martin-g
12k1825
answered Nov 23 at 18:56
martin-g
12k1825
answered Nov 23 at 18:56
martin-g
12k1825
12k1825
I did end up using the totalHits property; thanks!
– pterry26
Nov 26 at 22:50
add a comment |
I did end up using the totalHits property; thanks!
– pterry26
Nov 26 at 22:50
I did end up using the totalHits property; thanks!
– pterry26
Nov 26 at 22:50
I did end up using the totalHits property; thanks!
– pterry26
Nov 26 at 22:50
add a comment |
I think that what you need is first just a count search, with this I mean do the query you want and set the size to 0 this make the result to just return the total hits, where you can get the size of each page.
int pageSize = 10;
long totalCount = searchResponse.getHits().getTotalHits();
PageCount pageCount = new PageCount();
if (totalCount <= 0)
return result;
long pages = (totalCount < pageSize) ? 1l : ((totalCount % pageSize) == 0) ? (totalCount / pageSize) : ((totalCount / pageSize) + 1);
if (totalCount <= 10000) {
result.setTotal(totalCount);
result.setPages(pages);
} else {
result.setTotal(10000);
result.setPages((10000 / pageSize));
}
Here I use the 10000 just as a limit so that Elasticsearch to limit the amount of memory it consumes. Once you have this information you can return the number of pages for the query and make requests for a given page.
Once you have this, you could setup the search request like:
long size = ((pageCount.getTotal() < pageSize) ? pageCount.getTotal() : pageSize);
if (pageCount.getTotal() > pageSize && (page + 1) >= pageCount.getPages()) {
size = ((pageCount.getTotal() % pageSize) != 0 ? pageCount.getTotal() % pageSize : pageSize);
}
searchRequest.setSize((int) size).setFrom((int) (page * pageSize))
This is just sample code, I hope this get you enough.
Cheers
answered Nov 23 at 19:52
nmorenor
1309
Thanks, but it looks like I can get the totalHits property from the original query. It turns out this is equal to the total number of possible results matching the query, and is unaffected by the size limit.
– pterry26
Nov 26 at 22:52
add a comment |
I think that what you need is first just a count search, with this I mean do the query you want and set the size to 0 this make the result to just return the total hits, where you can get the size of each page.
int pageSize = 10;
long totalCount = searchResponse.getHits().getTotalHits();
PageCount pageCount = new PageCount();
if (totalCount <= 0)
return result;
long pages = (totalCount < pageSize) ? 1l : ((totalCount % pageSize) == 0) ? (totalCount / pageSize) : ((totalCount / pageSize) + 1);
if (totalCount <= 10000) {
result.setTotal(totalCount);
result.setPages(pages);
} else {
result.setTotal(10000);
result.setPages((10000 / pageSize));
}
Here I use the 10000 just as a limit so that Elasticsearch to limit the amount of memory it consumes. Once you have this information you can return the number of pages for the query and make requests for a given page.
Once you have this, you could setup the search request like:
long size = ((pageCount.getTotal() < pageSize) ? pageCount.getTotal() : pageSize);
if (pageCount.getTotal() > pageSize && (page + 1) >= pageCount.getPages()) {
size = ((pageCount.getTotal() % pageSize) != 0 ? pageCount.getTotal() % pageSize : pageSize);
}
searchRequest.setSize((int) size).setFrom((int) (page * pageSize))
This is just sample code, I hope this get you enough.
Cheers
answered Nov 23 at 19:52
nmorenor
1309
Thanks, but it looks like I can get the totalHits property from the original query. It turns out this is equal to the total number of possible results matching the query, and is unaffected by the size limit.
– pterry26
Nov 26 at 22:52
add a comment |
I think that what you need is first just a count search, with this I mean do the query you want and set the size to 0 this make the result to just return the total hits, where you can get the size of each page.
int pageSize = 10;
long totalCount = searchResponse.getHits().getTotalHits();
PageCount pageCount = new PageCount();
if (totalCount <= 0)
return result;
long pages = (totalCount < pageSize) ? 1l : ((totalCount % pageSize) == 0) ? (totalCount / pageSize) : ((totalCount / pageSize) + 1);
if (totalCount <= 10000) {
result.setTotal(totalCount);
result.setPages(pages);
} else {
result.setTotal(10000);
result.setPages((10000 / pageSize));
}
Here I use the 10000 just as a limit so that Elasticsearch to limit the amount of memory it consumes. Once you have this information you can return the number of pages for the query and make requests for a given page.
Once you have this, you could setup the search request like:
long size = ((pageCount.getTotal() < pageSize) ? pageCount.getTotal() : pageSize);
if (pageCount.getTotal() > pageSize && (page + 1) >= pageCount.getPages()) {
size = ((pageCount.getTotal() % pageSize) != 0 ? pageCount.getTotal() % pageSize : pageSize);
}
searchRequest.setSize((int) size).setFrom((int) (page * pageSize))
This is just sample code, I hope this get you enough.
Cheers
answered Nov 23 at 19:52
nmorenor
1309
I think that what you need is first just a count search, with this I mean do the query you want and set the size to 0 this make the result to just return the total hits, where you can get the size of each page.
int pageSize = 10;
long totalCount = searchResponse.getHits().getTotalHits();
PageCount pageCount = new PageCount();
if (totalCount <= 0)
return result;
long pages = (totalCount < pageSize) ? 1l : ((totalCount % pageSize) == 0) ? (totalCount / pageSize) : ((totalCount / pageSize) + 1);
if (totalCount <= 10000) {
result.setTotal(totalCount);
result.setPages(pages);
} else {
result.setTotal(10000);
result.setPages((10000 / pageSize));
}
Here I use the 10000 just as a limit so that Elasticsearch to limit the amount of memory it consumes. Once you have this information you can return the number of pages for the query and make requests for a given page.
Once you have this, you could setup the search request like:
long size = ((pageCount.getTotal() < pageSize) ? pageCount.getTotal() : pageSize);
if (pageCount.getTotal() > pageSize && (page + 1) >= pageCount.getPages()) {
size = ((pageCount.getTotal() % pageSize) != 0 ? pageCount.getTotal() % pageSize : pageSize);
}
searchRequest.setSize((int) size).setFrom((int) (page * pageSize))
This is just sample code, I hope this get you enough.
Cheers
answered Nov 23 at 19:52
nmorenor
1309
answered Nov 23 at 19:52
nmorenor
1309
answered Nov 23 at 19:52
nmorenor
1309
answered Nov 23 at 19:52
nmorenor
1309
1309
Thanks, but it looks like I can get the totalHits property from the original query. It turns out this is equal to the total number of possible results matching the query, and is unaffected by the size limit.
– pterry26
Nov 26 at 22:52
add a comment |
Thanks, but it looks like I can get the totalHits property from the original query. It turns out this is equal to the total number of possible results matching the query, and is unaffected by the size limit.
– pterry26
Nov 26 at 22:52
Thanks, but it looks like I can get the totalHits property from the original query. It turns out this is equal to the total number of possible results matching the query, and is unaffected by the size limit.
– pterry26
Nov 26 at 22:52
Thanks, but it looks like I can get the totalHits property from the original query. It turns out this is equal to the total number of possible results matching the query, and is unaffected by the size limit.
– pterry26
Nov 26 at 22:52
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53268231%2felasticsearch-java-rest-client-see-if-number-of-eligible-hits-exceeded-searchso%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
