侧边栏壁纸
博主头像
成培培的博客

分享技术，记录生活

  • 累计撰写 56 篇文章
  • 累计创建 7 个标签
  • 累计收到 6 条评论

目 录CONTENT

文章目录
Java

Embedding模型与向量数据库-图文搜索

成培培
成培培
2025-03-19 / 0 评论 / 0 点赞 / 15 阅读 / 0 字

上篇博文介绍了Embedding模型与向量数据库及相关的RAG技术，这里介绍另一个应用场景，基于Embedding模型与向量数据库实现一个图文搜索功能，可以根据用户输入的文字搜索其相关的图片。

多模向量模型

既然要搜索图片，当然需要将图片内容进行向量化，这里就使用了阿里百炼平台提供的通用多模态向量模型multimodal-embedding-v1
https://www.chengpei.top/upload/multimodal-embedding-v1.png
向量数据库还是选用milvus本地部署

遇到的问题

既然使用阿里百炼提供的模型，当然集成框架自然会使用spring ai alibaba，配置好向量模型相关的参数直接注入框架提供的DashScopeEmbeddingModel，调用call()传入图片url或Base64编码即可向量化才对，甚至结合MilvusVectorStore去写入图片搜索图片时，它会自动调用本地的EmbeddingModel对象向量化，类似上篇博文中的知识库导入、RAG搜索一样，用不了几行代码即可实现。但是实际操作遇到一个奇怪的报错：
{"code":"InvalidParameter","message":"url error, please check url！","request_id":"267f626a-9e80-9ca4-8c14-34d2ad2bf934"}

org.springframework.ai.retry.NonTransientAiException: 400 - {"code":"InvalidParameter","message":"url error, please check url！","request_id":"267f626a-9e80-9ca4-8c14-34d2ad2bf934"}
	at org.springframework.ai.autoconfigure.retry.SpringAiRetryAutoConfiguration$2.handleError(SpringAiRetryAutoConfiguration.java:100) ~[spring-ai-spring-boot-autoconfigure-1.0.0-M5.jar:1.0.0-M5]
	at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.client.StatusHandler.lambda$fromErrorHandler$1(StatusHandler.java:71) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.client.StatusHandler.handle(StatusHandler.java:146) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.applyStatusHandlers(DefaultRestClient.java:711) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:200) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.readBody(DefaultRestClient.java:698) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntityInternal(DefaultRestClient.java:668) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntity(DefaultRestClient.java:657) ~[spring-web-6.1.16.jar:6.1.16]
	at com.alibaba.cloud.ai.dashscope.api.DashScopeApi.embeddings(DashScopeApi.java:285) ~[spring-ai-alibaba-core-1.0.0-M5.1.jar:1.0.0-M5.1]
	at com.alibaba.cloud.ai.dashscope.embedding.DashScopeEmbeddingModel.lambda$call$1(DashScopeEmbeddingModel.java:136) ~[spring-ai-alibaba-core-1.0.0-M5.1.jar:1.0.0-M5.1]
	at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:357) ~[spring-retry-2.0.11.jar:na]
	at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:230) ~[spring-retry-2.0.11.jar:na]
	at com.alibaba.cloud.ai.dashscope.embedding.DashScopeEmbeddingModel.lambda$call$3(DashScopeEmbeddingModel.java:133) ~[spring-ai-alibaba-core-1.0.0-M5.1.jar:1.0.0-M5.1]
	at io.micrometer.observation.Observation.observe(Observation.java:565) ~[micrometer-observation-1.13.10.jar:1.13.10]
	at com.alibaba.cloud.ai.dashscope.embedding.DashScopeEmbeddingModel.call(DashScopeEmbeddingModel.java:132) ~[spring-ai-alibaba-core-1.0.0-M5.1.jar:1.0.0-M5.1]
	at top.chengpei.ai.picturesearch.controller.PictureSearchController.upload(PictureSearchController.java:38) ~[classes/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:926) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:831) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) ~[tomcat-embed-core-10.1.34.jar:6.0]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.1.16.jar:6.1.16]
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.34.jar:6.0]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.16.jar:6.1.16]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.16.jar:6.1.16]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.1.16.jar:6.1.16]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.16.jar:6.1.16]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:397) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:905) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-embed-core-10.1.34.jar:10.1.34]
	at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]

这里的InvalidParameter肯定是阿里云返回的，在错误信息列表里确实有这个错误码https://help.aliyun.com/zh/model-studio/developer-reference/error-code?spm=0.0.0.i3
但是并没有错误信息为url error, please check url！的详细说明，这就很奇怪，各种尝试没有解决，最后在github的spring-ai-alibaba项目里找到有人提问类似的报错，得到回复是当前框架并不支持multimodal-embedding-v1模型。

解决方案

既然调用multimodal-embedding-v1模型不能使用spring ai alibaba框架，那么查看官方提供的API调用示例，提供了Python调用与HTTP接口调用两种方式，这里只好使用手写http调用解析的过程去向量化数据了，当然也无法使用MilvusVectorStore直接写入或者查询数据了，但是可以使用更加底层的MilvusServiceClient，手动将向量化后的数据写入向量数据库，再手动向量化用户的输入本文，传入向量化的结果到库中查询。

代码演示

具体代码如下，两个接口功能分别是初始化数据以及根据用户输入本文搜索图片

@GetMapping(value = "/init", produces = "application/json")
public String init() {
    List<String> picUrlList = new ArrayList<>();
    picUrlList.add("https://home.chengpei.top:8443/root/picture/aircraft1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/aircraft2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/bot1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/bot2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/car1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/car2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/car3.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/cat1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/cat2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/cat3.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/desk1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/desk2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/dog1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/dog2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/dog3.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/horse1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/horse2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/house1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/house2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/phone1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/phone2.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/phone3.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/tree1.jpg");
    picUrlList.add("https://home.chengpei.top:8443/root/picture/tree2.jpg");

    List<String> docIdArray = new ArrayList<>();
    List<String> contentArray = new ArrayList<>();
    List<JSONObject> metadataArray = new ArrayList<>();
    List<List<Float>> embeddingArray = new ArrayList<>();

    for (String picUrl : picUrlList) {
        docIdArray.add(UUID.randomUUID().toString());
        contentArray.add(picUrl);
        metadataArray.add(new JSONObject());
        embeddingArray.add(EmbeddingUtils.embedding(picUrl, "image"));
    }
    List<InsertParam.Field> fields = new ArrayList<>();
    fields.add(new InsertParam.Field(MilvusVectorStore.DOC_ID_FIELD_NAME, docIdArray));
    fields.add(new InsertParam.Field(MilvusVectorStore.CONTENT_FIELD_NAME, contentArray));
    fields.add(new InsertParam.Field(MilvusVectorStore.METADATA_FIELD_NAME, metadataArray));
    fields.add(new InsertParam.Field(MilvusVectorStore.EMBEDDING_FIELD_NAME, embeddingArray));

    InsertParam insertParam = InsertParam.newBuilder()
            .withDatabaseName(MilvusVectorStore.DEFAULT_DATABASE_NAME)
            .withCollectionName("picture_search")
            .withFields(fields)
            .build();
    this.milvusServiceClient.insert(insertParam);
    return "success";
}

@GetMapping(value = "/search", produces = "application/json")
public List<JSONObject> search(String message) {
    List<String> outFieldNames = new ArrayList<>();
    outFieldNames.add(MilvusVectorStore.DOC_ID_FIELD_NAME);
    outFieldNames.add(MilvusVectorStore.CONTENT_FIELD_NAME);
    var searchParamBuilder = SearchParam.newBuilder()
            .withDatabaseName(MilvusVectorStore.DEFAULT_DATABASE_NAME)
            .withCollectionName("picture_search")
            .withTopK(5)
            .withOutFields(outFieldNames)
            .withVectors(List.of(EmbeddingUtils.embedding(message, "text")))
            .withVectorFieldName(MilvusVectorStore.EMBEDDING_FIELD_NAME);
    R<SearchResults> searchResults = milvusServiceClient.search(searchParamBuilder.build());
    SearchResults resultsData = searchResults.getData();
    System.out.println(resultsData.getResults());
    List<JSONObject> result = new ArrayList<>();
    SearchResultData results = resultsData.getResults();
    for (int i = 0; i < results.getTopK(); i++) {
        JSONObject jsonObject = new JSONObject();
        jsonObject.put("docId", results.getFieldsData(0).getScalars().getStringData().getData(i));
        jsonObject.put("content", results.getFieldsData(1).getScalars().getStringData().getData(i));
        jsonObject.put("scores", results.getScores(i));
        result.add(jsonObject);
    }
    return result;
}

图片准备了24张不同类似的图片，有猫、狗、汽车、飞机、树等，向量数据库创建了一个collection存储图片的向量数据以及URL
https://www.chengpei.top/upload/attu.png
这里需要注意的是向量字段使用维度为1024，跟上一篇博文里的是不一样的，因为根据阿里百炼里multimodal-embedding-v1模型的描述，这个模型的向量维度就是1024，所以创建这个collection时维度一定要一致。
调用接口搜索结果如下：
https://www.chengpei.top/upload/search_result.png
这里因为TopK为5，所以搜索出来的是5条数据，并且前三条的scores较高。
具体代码放到了github：
https://github.com/chengpei/spring-ai-demo
模块picture-search-demo为图片搜索相关代码

0
经验分享
版权归属： 成培培
本文链接： https://blog.chengpei.top/archives/embedding-model-picture-search
许可协议： 本文使用《署名-非商业性使用-相同方式共享 4.0 国际 (CC BY-NC-SA 4.0)》协议授权

评论区