类class agentscope.rag

ChunkerBase

class ChunkerBase(ABC)

源码Source agentscope/rag/_chunker/_base.py:26–91

切分器基类，把私有文档转化为可检索的知识。位于 agentscope.rag 包，继承自 ABC。核心成员包括 chunk。

Subclasses implement a specific chunking strategy (by character count, by token count, by semantic boundary, etc.). The chunker is configured once at construction time and reused across many chunk() calls within the same knowledge base.

Subclasses must set :attr:chunker_type and may override the :class:Parameters inner class to declare their tunable parameters.

Subclasses must guarantee:

No cross-Section merging: every output :class:Chunk is derived from exactly one input :class:Section.
DataBlock pass-through: a Section whose content is a :class:~agentscope.message.DataBlock becomes a single Chunk with the same content; multimodal data is never sliced.
No empty-text chunks: a Section whose text content is empty or whitespace-only produces no Chunk, so blank pages never reach the embedding step.
Continuous indexing: chunk_index runs from 0 to total_chunks - 1 across the entire output list, even when the input contains many Sections.
Consistent total_chunks: every output Chunk carries the same total_chunks value (the length of the output list).
Metadata inheritance: each output Chunk's source and metadata are copied from its parent Section.

嵌套类摘要Nested Class Summary

类Class	类型Kind	说明Description
`Parameters`	类class	参数，把私有文档转化为可检索的知识。 The tunable parameters of the chunker.

构造器摘要Constructor Summary

构造器Constructor	说明Description
`def __init__(parameters: 'ChunkerBase.Parameters \| None' = None) -> None`	初始化 `ChunkerBase` 实例。 Initialize the chunker.

属性摘要Attribute Summary

属性Attribute	类型Type	说明Description	值Value
`chunker_type`	str	切分器类型字段。	—

方法摘要Method Summary

方法Method	说明Description
`async def chunk(sections: list[Section]) -> list[Chunk]`asyncabstractabstractmethod	设置分块。 Split a list of Sections into Chunks.

构造器详情Constructor Detail

ChunkerBase

def __init__(parameters: 'ChunkerBase.Parameters | None' = None) -> None

初始化 ChunkerBase 实例。

Initialize the chunker.

参数Parameters

名称Name	类型Type	说明Description
`parameters`默认default `None`	'ChunkerBase.Parameters \| None'	参数（类型 `'ChunkerBase.Parameters \| None'`）。The chunker parameters. Defaults to `self.Parameters()` when not provided.

源码Source agentscope/rag/_chunker/_base.py:62–75

属性详情Attribute Detail

chunker_type

str

切分器类型字段。声明为 str。

The chunker type field.

源码Source agentscope/rag/_chunker/_base.py:56

方法详情Method Detail

chunk

async def chunk(sections: list[Section]) -> list[Chunk]

asyncabstractabstractmethod

设置分块。

Split a list of Sections into Chunks.

参数Parameters

名称Name	类型Type	说明Description
`sections`	list[Section]	sections（类型 `list[Section]`）。The Sections produced by a :class:`ParserBase`, in document order.

返回Returns list[Chunk]

返回分块列表。`list[Chunk]`:

源码Source agentscope/rag/_chunker/_base.py:78–91

嵌套类详情Nested Class Detail

Parameters

class Parameters

参数，把私有文档转化为可检索的知识。位于 agentscope.rag 包。

The tunable parameters of the chunker. Declared in agentscope.rag.