elasticsearch 分页可以任意跳转到某一页当超过10000条怎么解决给出具体java代码示例
当超过10000条时,可以使用scroll API进行分页查询,具体的Java代码示例如下:
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.Client;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortOrder;
import java.util.ArrayList;
import java.util.List;
public class ElasticsearchUtil {
private static final int PAGE_SIZE = 1000;
/**
* 普通分页查询
*
* @param client
* @param index
* @param type
* @param queryBuilder
* @param pageNum
* @return
*/
public static List<Object> searchPage(Client client, String index, String type, QueryBuilder queryBuilder, int pageNum) {
int from = (pageNum - 1) * PAGE_SIZE;
SearchResponse searchResponse = client.prepareSearch(index)
.setTypes(type)
.setQuery(queryBuilder)
.setFrom(from)
.setSize(PAGE_SIZE)
.addSort("_id", SortOrder.ASC)
.execute()
.actionGet();
SearchHits hits = searchResponse.getHits();
List<Object> list = new ArrayList<>();
for (SearchHit hit : hits) {
Object obj = hit.getSourceAsString();
list.add(obj);
}
return list;
}
/**
* 使用scroll API进行分页查询
*
* @param client
* @param index
* @param type
* @param queryBuilder
* @param pageNum
* @return
*/
public static List<Object> searchScroll(Client client, String index, String type, QueryBuilder queryBuilder, int pageNum) {
String scrollId = null;
List<Object> list = new ArrayList<>();
try {
SearchResponse searchResponse = client.prepareSearch(index)
.setTypes(type)
.setQuery(queryBuilder)
.setSize(PAGE_SIZE)
.addSort("_id", SortOrder.ASC)
.setScroll("1m")
.execute()
.actionGet();
scrollId = searchResponse.getScrollId();
while (true) {
SearchHits hits = searchResponse.getHits();
for (SearchHit hit : hits) {
Object obj = hit.getSourceAsString();
list.add(obj);
}
searchResponse = client.prepareSearchScroll(scrollId)
.setScroll("1m")
.execute()
.actionGet();
if (searchResponse.getHits().getHits().length == 0) {
break;
}
}
} finally {
if (scrollId != null) {
client.prepareClearScroll().addScrollId(scrollId).execute().actionGet();
}
}
int from = (pageNum - 1) * PAGE_SIZE;
int to = from + PAGE_SIZE;
if (to > list.size()) {
to = list.size();
}
return list.subList(from, to);
}
}
其中,普通分页查询使用了Elasticsearch的from和size参数进行分页查询,但是当数据量很大时,这种方式会非常慢。因此,我们使用了scroll API进行分页查询,它可以通过游标的方式进行查询,性能更高。具体使用方法是,首先使用prepareSearch方法进行查询,设置scroll参数,获取第一批数据和scrollId,然后使用prepareSearchScroll方法进行下一批数据的查询,直到没有数据为止。最后将查询结果进行分页返回即可
原文地址: http://www.cveoy.top/t/topic/htPb 著作权归作者所有。请勿转载和采集!