当先锋百科网

首页 1 2 3 4 5 6 7

springboot快速整合elasticsearch+elasticsearch简介

 

1、ElasticSearch简介

我们的应用经常需要添加检索功能，开源的ElasticSearch是目前全文搜索引擎的首选。他可以快速存储、搜索和分析海量数据。Spring Boot通过整合Spring Data Elasticsearch为我们提供了非常便捷的检索功能支持；

ElasticSearch是一个分布式搜索服务，提供Restful API,底层基于Lucene，采用多shard（分片）的方式保证数据安全，并且提供自动resharding的功能，github等大型的站点也是采用了ElasticSearch作为其搜索服务。

2、ElasticSearch镜像的安装与启动

docker search elasticsearch
docker pull elasticsearch:7.11.1
docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -e "discovery.type=single-node" -d -p 9200:9200 -p 9300:9300 --name es002 bc3d45eba361

因为ElasticSearch占用内存为2g，我的虚拟机不支持，设立了默认内存256MB和最大内存256MB

如果不加 -e "discovery.type=single-node" 会自动停止ElasticSearch容器。-d 后台运行 -p 虚拟机和容器进行端口映射 -p 多节点分布式端口映射。

如果还是自动停止容器：请按照以下步骤操作：

通过docker ps -a可以发现elasticsearch自动退出

通过docker logs -f 容器id查看日志，可以看到提示空间不足

使用命令find / -name jvm.options找到jvm.options文件位置

通过vim 文件位置命令进入该文件

找到xms参数位置，将分配空间修改小一些，减小值到系统可用的值，如

-Xms512m
-Xmx512m

有可能报max_map_count [65530] is too low之类的错误，需要重置max_map_count大小

如sysctl -w vm.max_map_count=262144

查看max_map_count

sysctl -a|grep vm.max_map_count

然后重新启动容器

ps.

容器启动失败后，需要进入docker ps -a并记下容器id

然后通过docker rm 容器id删除当前容器，才可重新启动

pps.

如果是通过阿里云/腾讯云等云服务器的docker部署，千万不要忘记开放端口号！！！！！！！

此时，在物理机浏览器访问虚拟机端口：http://192.168.44.128:9200/,返回json，证明启动成功

{
  "name" : "ee56c20ae504",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "lO4zw-yiQxC6MMv9pDE9FQ",
  "version" : {
    "number" : "7.11.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "ff17057114c2199c9c1bbecc727003a907c0db7a",
    "build_date" : "2021-02-15T13:44:09.394032Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

 通过官方文档操作elasticsearch

https://www.elastic.co/guide/cn/elasticsearch/guide/current/_indexing_employee_documents.html

PUT /megacorp/employee/1
{
    "first_name" : "John",
    "last_name" :  "Smith",
    "age" :        25,
    "about" :      "I love to go rock climbing",
    "interests": [ "sports", "music" ]
}
PUT /megacorp/employee/2
{
    "first_name" :  "Jane",
    "last_name" :   "Smith",
    "age" :         32,
    "about" :       "I like to collect rock albums",
    "interests":  [ "music" ]
}

PUT /megacorp/employee/3
{
    "first_name" :  "Douglas",
    "last_name" :   "Fir",
    "age" :         35,
    "about":        "I like to build cabinets",
    "interests":  [ "forestry" ]
}
GET /megacorp/employee/1
GET /megacorp/employee/_search
GET /megacorp/employee/_search?q=last_name:Smith


GET /megacorp/employee/_search
{
    "query" : {
        "match" : {
            "last_name" : "Smith"
        }
    }
}


GET /megacorp/employee/_search
{
    "query" : {
        "bool": {
            "must": {
                "match" : {
                    "last_name" : "smith" 
                }
            },
            "filter": {
                "range" : {
                    "age" : { "gt" : 30 } 
                }
            }
        }
    }
}
//全文搜索
GET /megacorp/employee/_search
{
    "query" : {
        "match" : {
            "about" : "rock climbing"
        }
    }
}


//短语搜索
GET /megacorp/employee/_search
{
    "query" : {
        "match_phrase" : {
            "about" : "rock climbing"
        }
    }
}
//高亮所搜
GET /megacorp/employee/_search
{
    "query" : {
        "match_phrase" : {
            "about" : "rock climbing"
        }
    },
    "highlight": {
        "fields" : {
            "about" : {}
        }
    }
}

ElasticSearch的数据结构是

 

 

 

 

 

 

springboot整合ElasticSearch和REST搜索

以前是jest，现在是rest ，autoconfig包里边：

springBoot 2.3.0版本及以后版本不支持es查询工具jestClient自动注入

 

 

 

springboot默认有两种方式来和ElasticSearch交互

 1、jest默认不生效，因为没有引入pom依赖

 2、springdata ElasticSearch 默认生效，但是springdata ElasticSearch版本得和ElasticSearch容器版本匹配

     client节点信息，clusterNodes clusterName

     ElasticsearchTemplate操作es

     编写一个ElasticSearchRepository的子接口来操作es

 

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.4.4</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.pshdhx.ElasticSearch</groupId>
    <artifactId>demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>demo</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

package com.pshdhx.elasticsearch.demo.config;

import io.searchbox.client.JestClient;
import io.searchbox.client.JestClientFactory;
import io.searchbox.client.config.HttpClientConfig;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.RestClientFactoryBean;

/**
 * @Authtor pshdhx
 * @Date 2021/3/3121:58
 * @Version 1.0
 */
@Configuration
public class EsConfig {
    /**
     *  重新注入RestHighLevelClient
     * @return
     */
    @Bean
    public RestHighLevelClient restHighLevelClient(){
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost("http://192.168.44.128:9200", 9200, "http")
                )
        );
        return client;
    }

    @Bean
    public JestClient getJestCline() {
        JestClientFactory factory = new JestClientFactory();
        factory.setHttpClientConfig(new HttpClientConfig
                .Builder("http://192.168.44.128:9200/")
                .multiThreaded(true)
                .build());
        return factory.getObject();
    }
//
//    @Bean
//    public RestClient geRestCline() {
//        RestClientFactoryBean factoryBean = new RestClientFactoryBean();
//        factoryBean.setHosts("http://192.168.44.128:9200");
//        RestHighLevelClient object = factoryBean.getObject();
//
//        JestClientFactory factory = new JestClientFactory();
//        factory.setHttpClientConfig(new HttpClientConfig
//                .Builder("http://192.168.44.128:9200")
//                .multiThreaded(true)
//                .build());
//        return factory.getObject();
//    }
}

 

package com.pshdhx.elasticsearch.demo;

import com.pshdhx.elasticsearch.demo.bean.Article;
import io.searchbox.client.JestClient;
import io.searchbox.core.Index;
import io.searchbox.core.Search;
import io.searchbox.core.SearchResult;
import org.elasticsearch.client.AsyncSearchClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.elasticsearch.RestClientBuilderCustomizer;
import org.springframework.boot.test.autoconfigure.web.client.RestClientTest;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

import java.io.IOException;

//@Service
@SpringBootTest
class DemoApplicationTests {


//    RestTemplate restTemplate;
//
//    JestClient01 jestClient01;
//
//
//    @Autowired
//    RestClientTest restClientTest;
//
//    RestClientBuilderCustomizer restClientBuilderCustomizer;
//
//
//
//
//
//    @Autowired
//    RestHighLevelClient restHighLevelClient;

    @Autowired
    JestClient jestClient;

    @Test
    void contextLoads() {
        Article article = new Article(111,"psd","消息","news hello");
        Index index = new Index.Builder(article).index("pshdhx").type("news").build();
        try {
//            JestClient jestCline = jestClient01.getJestCline();
//            jestCline.execute(index);
//            restHighLevelClient.search
//            AsyncSearchClient asyncSearchClient = restHighLevelClient.asyncSearch();
//            String forObject = restTemplate.getForObject("http://192.168.44.128:9200?first_name=John", String.class);
//            System.out.println(forObject);
            jestClient.execute(index);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Test
    void Search(){
        String json = "{\n" +
                "    \"query\" : {\n" +
                "        \"match\" : {\n" +
                "            \"last_name\" : \"Smith\"\n" +
                "        }\n" +
                "    }\n" +
                "}\n";
        Search search = new Search.Builder(json).addIndex("pshdhx").addType("news").build();
        try {
            SearchResult result = jestClient.execute(search);
            System.out.println(result);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

结果：

http://192.168.44.128:9200/_search

{
    "took": 307,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "megacorp",
                "_type": "employee",
                "_id": "1",
                "_score": 1.0,
                "_source": {
                    "first_name": "John",
                    "last_name": "Smith",
                    "age": 25,
                    "about": "I love to go rock climbing",
                    "interests": [
                        "sports",
                        "music"
                    ]
                }
            },
            {
                "_index": "pshdhx",
                "_type": "news",
                "_id": "111",
                "_score": 1.0,
                "_source": {
                    "id": 111,
                    "author": "psd",
                    "title": "消息",
                    "content": "news hello"
                }
            }
        ]
    }
}

ElasticSearch

 

配置类：

package com.pshdhx.elasticsearch.demo.config;

import io.searchbox.client.JestClient;
import io.searchbox.client.JestClientFactory;
import io.searchbox.client.config.HttpClientConfig;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.ClientConfiguration;
import org.springframework.data.elasticsearch.client.RestClientFactoryBean;
import org.springframework.data.elasticsearch.client.RestClients;

/**
 * @Authtor pshdhx
 * @Date 2021/3/3121:58
 * @Version 1.0
 */
@Configuration
public class EsConfig {
    /**
     * 重新注入RestHighLevelClient
     *
     * @return
//     */
//    @Bean
//    public RestHighLevelClient restHighLevelClient() {
//        RestHighLevelClient client = new RestHighLevelClient(
//                RestClient.builder(
//                        new HttpHost("http://192.168.44.128:9200", 9200, "http")
//                )
//        );
//        return client;
//    }

    @Bean
    public JestClient getJestCline() {
        JestClientFactory factory = new JestClientFactory();
        factory.setHttpClientConfig(new HttpClientConfig
                .Builder("http://192.168.44.128:9200/")
                .multiThreaded(true)
                .build());
        return factory.getObject();
    }
//
//    @Bean
//    public RestClient geRestCline() {
//        RestClientFactoryBean factoryBean = new RestClientFactoryBean();
//        factoryBean.setHosts("http://192.168.44.128:9200");
//        RestHighLevelClient object = factoryBean.getObject();
//
//        JestClientFactory factory = new JestClientFactory();
//        factory.setHttpClientConfig(new HttpClientConfig
//                .Builder("http://192.168.44.128:9200")
//                .multiThreaded(true)
//                .build());
//        return factory.getObject();
//    }

    //    @Bean
//    RestHighLevelClient elasticsearchClient() {
//        ClientConfiguration configuration = ClientConfiguration.builder()
//                .connectedTo("http://192.168.44.128:9200/")
//                //.withConnectTimeout(Duration.ofSeconds(5))
//                //.withSocketTimeout(Duration.ofSeconds(3))
//                //.useSsl()
//                //.withDefaultHeaders(defaultHeaders)
//                //.withBasicAuth(username, password)
//                // ... other options
//
//                .build();
//        RestHighLevelClient client = RestClients.create(configuration).rest();
//        return client;
//    }
    @Bean
    public RestHighLevelClient restHighLevelClient() {
        ClientConfiguration configuration = ClientConfiguration.builder(
        )
                .connectedTo("192.168.44.128:9200")
                //.withConnectTimeout(Duration.ofSeconds(5))
                //.withSocketTimeout(Duration.ofSeconds(3))
                //.useSsl()
                //.withDefaultHeaders(defaultHeaders)
                //.withBasicAuth(username, password)
                // ... other options
                .build();
        RestHighLevelClient client = RestClients.create(configuration).rest();
        return client;
    }

}

实体类：

package com.pshdhx.elasticsearch.demo.bean;

import org.springframework.data.elasticsearch.annotations.Document;

/**
 * @Authtor pshdhx
 * @Date 2021/4/118:38
 * @Version 1.0
 */
@Document(indexName = "pshdhx",indexStoreType = "book")
public class Book {
    private Integer id;
    private String bookName;
    private String author;

    public Integer getId() {
        return id;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public String getBookName() {
        return bookName;
    }

    public void setBookName(String bookName) {
        this.bookName = bookName;
    }

    public String getAuthor() {
        return author;
    }

    public void setAuthor(String author) {
        this.author = author;
    }
}

 实现类：

package com.pshdhx.elasticsearch.demo.repository;

import com.pshdhx.elasticsearch.demo.bean.Book;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

import java.util.List;

/**
 * @Authtor pshdhx
 * @Date 2021/4/118:37
 * @Version 1.0
 */
public interface BookRepository extends ElasticsearchRepository<Book,Integer> {
    public List<Book> findByBookNameLike(String bookName);
}

测试类：

package com.pshdhx.elasticsearch.demo;

import com.pshdhx.elasticsearch.demo.bean.Article;
import com.pshdhx.elasticsearch.demo.bean.Book;
import com.pshdhx.elasticsearch.demo.repository.BookRepository;
import io.searchbox.client.JestClient;
import io.searchbox.core.Index;
import io.searchbox.core.Search;
import io.searchbox.core.SearchResult;
import org.elasticsearch.client.AsyncSearchClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.elasticsearch.RestClientBuilderCustomizer;
import org.springframework.boot.test.autoconfigure.web.client.RestClientTest;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

import java.io.IOException;

//@Service
@SpringBootTest
class DemoApplicationTests {


//    RestTemplate restTemplate;
//
//    JestClient01 jestClient01;
//
//
//    @Autowired
//    RestClientTest restClientTest;
//
//    RestClientBuilderCustomizer restClientBuilderCustomizer;
//
//
//
//
//
//    @Autowired
//    RestHighLevelClient restHighLevelClient;

    @Autowired
    JestClient jestClient;

    @Test
    void contextLoads() {
        Article article = new Article(111,"psd","消息","news hello");
        Index index = new Index.Builder(article).index("pshdhx").type("news").build();
        try {
//            JestClient jestCline = jestClient01.getJestCline();
//            jestCline.execute(index);
//            restHighLevelClient.search
//            AsyncSearchClient asyncSearchClient = restHighLevelClient.asyncSearch();
//            String forObject = restTemplate.getForObject("http://192.168.44.128:9200?first_name=John", String.class);
//            System.out.println(forObject);
            jestClient.execute(index);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Test
    void Search(){
        String json = "{\n" +
                "    \"query\" : {\n" +
                "        \"match\" : {\n" +
                "            \"last_name\" : \"Smith\"\n" +
                "        }\n" +
                "    }\n" +
                "}\n";
        Search search = new Search.Builder(json).addIndex("pshdhx").addType("news").build();
        try {
            SearchResult result = jestClient.execute(search);
            System.out.println(result);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    @Autowired
    BookRepository repository;
    @Test
    void test02(){
        Book b = new Book();
        b.setAuthor("pshdhx");
        b.setId(1);
        b.setBookName("book1");
        repository.save(b);
    }

    @Test
    void Test3(){
        for(Book book:repository.findByBookNameLike("bo")){
            System.out.println(book);
        }
    }

}

结果：