> ## Documentation Index
> Fetch the complete documentation index at: https://nvd-54.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Apify dataset 集成

> 使用 LangChain Python 集成 Apify dataset 文档加载器。

> [Apify Dataset](https://docs.apify.com/platform/storage/dataset) is a scalable append-only storage with sequential access built for storing structured web scraping results, such as a list of products or Google SERPs, and then export them to various formats like JSON, CSV, or Excel. Datasets are mainly used to save results of [Apify Actors](https://apify.com/store)—serverless cloud programs for various web scraping, crawling, and data extraction use cases.

本 notebook 展示how to load Apify datasets to LangChain.

### 集成详情

| Class                                                            | Package                                                        | Serializable | [JS support](https://js.langchain.com/docs/integrations/document_loaders/web_loaders/apify_dataset) |                                            Version                                            |
| :--------------------------------------------------------------- | :------------------------------------------------------------- | :----------: | :-------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------: |
| [`ApifyDatasetLoader`](https://github.com/apify/langchain-apify) | [`langchain-apify`](https://pypi.org/project/langchain-apify/) |       ❌      |                                                  ✅                                                  | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-apify?style=flat-square\&label=%20) |

### 加载器特性

|     Source    | Document Lazy Loading | Native Async Support |
| :-----------: | :-------------------: | :------------------: |
| Apify Dataset |           ❌           |           ❌          |

## 前提条件

You need to have an existing dataset on the Apify platform. This example shows how to load a dataset produced by the [Website Content Crawler](https://apify.com/apify/website-content-crawler).

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
pip install -qU langchain langchain-apify langchain-openai
```

First, import `ApifyDatasetLoader` into your source code:

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain_apify import ApifyDatasetLoader
from langchain_core.documents import Document
```

Find your [Apify API token](https://console.apify.com/account/integrations) and [OpenAI API key](https://platform.openai.com/account/api-keys) and initialize these into environment variable:

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
import os

os.environ["APIFY_TOKEN"] = "your-apify-token"
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
```

## Pricing

Apify Actors can be priced in different ways, depending on the Actor you run.
Many Actors support [Pay-Per-Event (PPE) pricing](https://docs.apify.com/platform/actors/publishing/monetize/pay-per-event), where you pay for explicit events defined by the Actor author (for example, per dataset item). This can be a good fit for agent workloads where you want clear, per-operation costs.

## Map dataset items to documents

Next, define a function that maps Apify dataset record fields to LangChain [`Document`](https://reference.langchain.com/python/langchain-core/documents/base/Document) format.

For example, if your dataset items are structured like this:

```json theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
{
    "url": "https://apify.com",
    "text": "Apify is the best web scraping and automation platform."
}
```

The mapping function in the code below will convert them to LangChain [`Document`](https://reference.langchain.com/python/langchain-core/documents/base/Document) format, so that you can use them further with any LLM model (e.g. for question answering).

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
loader = ApifyDatasetLoader(
    dataset_id="your-dataset-id",
    dataset_mapping_function=lambda dataset_item: Document(
        page_content=dataset_item["text"], metadata={"source": dataset_item["url"]}
    ),
)
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
data = loader.load()
```

## An example with question answering

In this example, we use data from a dataset to answer a question.

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
from langchain.indexes import VectorstoreIndexCreator
from langchain_apify import ApifyWrapper
from langchain_core.documents import Document
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_openai import ChatOpenAI
from langchain_openai.embeddings import OpenAIEmbeddings
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
loader = ApifyDatasetLoader(
    dataset_id="your-dataset-id",
    dataset_mapping_function=lambda item: Document(
        page_content=item["text"] or "", metadata={"source": item["url"]}
    ),
)
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
index = VectorstoreIndexCreator(
    vectorstore_cls=InMemoryVectorStore, embedding=OpenAIEmbeddings()
).from_loaders([loader])
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
llm = ChatOpenAI(model="gpt-5-mini")
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
query = "What is Apify?"
result = index.query_with_sources(query, llm=llm)
```

```python theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
print(result["answer"])
print(result["sources"])
```

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
 Apify is a platform for developing, running, and sharing serverless cloud programs. It enables users to create web scraping and automation tools and publish them on the Apify platform.

https://docs.apify.com/platform/actors, https://docs.apify.com/platform/actors/running/actors-in-store, https://docs.apify.com/platform/security, https://docs.apify.com/platform/actors/examples
```

***

## Using the Apify MCP server

Unsure which Actor to use or what parameters it requires? The [Apify MCP (Model Context Protocol) server](https://mcp.apify.com) can help you discover available Actors, explore their input schemas, and understand parameter requirements.

When connecting to the Apify MCP server over HTTP, include your Apify token in the request headers:

```text theme={"theme":{"light":"catppuccin-latte","dark":"catppuccin-mocha"}}
Authorization: Bearer <APIFY_TOKEN>
```

更多信息请参阅 the [LangChain MCP documentation](/oss/python/langchain/mcp).

***

<div className="source-links">
  <Callout icon="terminal-2">
    [通过 MCP 将这些文档连接到](/use-these-docs) Claude、VSCode 等工具以获取实时答案。
  </Callout>

  <Callout icon="edit">
    [在 GitHub 上编辑此页面](https://github.com/langchain-ai/docs/edit/main/src/oss/python/integrations/document_loaders/apify_dataset.mdx) 或 [提交 issue](https://github.com/langchain-ai/docs/issues/new/choose)。
  </Callout>
</div>