Skip to main content

Documentation Index

Fetch the complete documentation index at: https://nvd-54.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

概述

本概述涵盖基于文本的向量嵌入模型。LangChain 目前不支持多模态向量嵌入。
向量嵌入模型将原始文本(如句子、段落或推文)转换为一个固定长度的数字向量,以捕获其语义含义。这些向量使机器能够基于含义而非精确词汇来比较和搜索文本。 在实践中,这意味着具有相似含义的文本在向量空间中被放置在相近的位置。例如,向量嵌入不仅可以匹配 “machine learning” 这一短语,还可以找到讨论相关概念的文档,即使使用了不同的措辞。

工作原理

  1. Vectorization — The model encodes each input string as a high-dimensional vector.
  2. Similarity scoring — Vectors are compared using mathematical metrics to measure how closely related the underlying texts are.

相似度指标

通常使用以下几种指标来比较向量嵌入:
  • Cosine similarity — measures the angle between two vectors.
  • Euclidean distance — measures the straight-line distance between points.
  • Dot product — measures how much one vector projects onto another.

接口

LangChain 通过 Embeddings 接口为文本向量嵌入模型(如 OpenAI、Cohere、Hugging Face)提供标准接口。 提供两个主要方法:
  • embedDocuments(documents: string[]) → number[][]: Embeds a list of documents.
  • embedQuery(text: string) → number[]: Embeds a single query.
该接口允许查询和文档使用不同的策略进行嵌入,尽管大多数提供商在实践中以相同方式处理它们。

安装和使用

安装依赖:
npm i @langchain/openai
添加环境变量:
OPENAI_API_KEY=your-api-key
实例化模型:
import { OpenAIEmbeddings } from "@langchain/openai";

const embeddings = new OpenAIEmbeddings({
  model: "text-embedding-3-large"
});
安装依赖
npm i @langchain/openai
添加环境变量:
AZURE_OPENAI_API_INSTANCE_NAME=<YOUR_INSTANCE_NAME>
AZURE_OPENAI_API_KEY=<YOUR_KEY>
AZURE_OPENAI_API_VERSION="2024-02-01"
实例化模型:
import { AzureOpenAIEmbeddings } from "@langchain/openai";

const embeddings = new AzureOpenAIEmbeddings({
  azureOpenAIApiEmbeddingsDeploymentName: "text-embedding-ada-002"
});
安装依赖:
npm i @langchain/aws
添加环境变量:
BEDROCK_AWS_REGION=your-region
实例化模型:
import { BedrockEmbeddings } from "@langchain/aws";

const embeddings = new BedrockEmbeddings({
  model: "amazon.titan-embed-text-v1"
});
安装依赖:
npm i @langchain/google-genai
添加环境变量:
GOOGLE_API_KEY=your-api-key
实例化模型:
import { GoogleGenerativeAIEmbeddings } from "@langchain/google-genai";

const embeddings = new GoogleGenerativeAIEmbeddings({
  model: "text-embedding-004"
});
安装依赖:
npm i @langchain/google-vertexai
添加环境变量:
GOOGLE_APPLICATION_CREDENTIALS=credentials.json
实例化模型:
import { VertexAIEmbeddings } from "@langchain/google-vertexai";

const embeddings = new VertexAIEmbeddings({
  model: "gemini-embedding-001"
});
安装依赖:
npm i @langchain/mistralai
添加环境变量:
MISTRAL_API_KEY=your-api-key
实例化模型:
import { MistralAIEmbeddings } from "@langchain/mistralai";

const embeddings = new MistralAIEmbeddings({
  model: "mistral-embed"
});
安装依赖:
npm i @langchain/cohere
添加环境变量:
COHERE_API_KEY=your-api-key
实例化模型:
import { CohereEmbeddings } from "@langchain/cohere";

const embeddings = new CohereEmbeddings({
  model: "embed-english-v3.0"
});
安装依赖:
npm i @langchain/ollama
实例化模型:
import { OllamaEmbeddings } from "@langchain/ollama";

const embeddings = new OllamaEmbeddings({
  model: "llama2",
  baseUrl: "http://localhost:11434", // Default value
});

缓存

向量嵌入可以被存储或临时缓存,以避免重新计算。 可以使用 CacheBackedEmbeddings 来缓存向量嵌入。此包装器将向量嵌入存储在键值存储中,文本被哈希处理,哈希值用作缓存中的键。 The main supported way to initialize a CacheBackedEmbeddings is fromBytesStore. It takes the following parameters:
  • underlyingEmbeddings: The embedder to use for embedding.
  • documentEmbeddingStore: Any BaseStore for caching document embeddings.
  • options.namespace: (optional, defaults to "") The namespace to use for the document cache. Helps avoid collisions (e.g., set it to the embedding model name).
import { CacheBackedEmbeddings } from "@langchain/classic/embeddings/cache_backed";
import { InMemoryStore } from "@langchain/core/stores";

const underlyingEmbeddings = new OpenAIEmbeddings();

const inMemoryStore = new InMemoryStore();

const cacheBackedEmbeddings = CacheBackedEmbeddings.fromBytesStore(
  underlyingEmbeddings,
  inMemoryStore,
  {
    namespace: underlyingEmbeddings.model,
  }
);

// Example: caching a query embedding
const tic = Date.now();
const queryEmbedding = cacheBackedEmbeddings.embedQuery("Hello, world!");
console.log(`First call took: ${Date.now() - tic}ms`);

// Example: caching a document embedding
const tic = Date.now();
const documentEmbedding = cacheBackedEmbeddings.embedDocuments(["Hello, world!"]);
console.log(`Cached creation time: ${Date.now() - tic}ms`);
在生产环境中,你通常会使用更稳健的持久化存储,如数据库或云存储。请参阅存储集成了解可选方案。

所有集成

Alibaba Tongyi

Azure OpenAI

Baidu Qianfan

Amazon Bedrock

ByteDance Doubao

Cloudflare Workers AI

Cohere

DeepInfra

Fireworks

Google Generative AI

Google Vertex AI

Gradient AI

HuggingFace Inference

IBM watsonx.ai

Jina

Llama CPP

Minimax

MistralAI

Mixedbread AI

Nomic

Ollama

Oracle AI Database

OpenAI

Pinecone

Prem AI

Tencent Hunyuan

TensorFlow

TogetherAI

HuggingFace Transformers

Voyage AI

ZhipuAI