TextChunker
public final class TextChunker源码Source
io/agentscope/core/rag/reader/TextChunker.java:28–195文本切分器，把私有文档转化为可检索的知识。位于 io.agentscope.core.rag.reader 包。核心成员包括 chunkText。
Utility class for chunking text into smaller pieces.
This class provides methods to split text according to different strategies, with support for overlap between chunks to preserve context.
方法摘要Method Summary
| 方法Method | 说明Description |
|---|---|
public static List<String> chunkText(String text, int chunkSize, SplitStrategy strategy, int overlapSize)static | 执行分块文本相关操作。 Chunks text according to the specified strategy. |
方法详情Method Detail
chunkText
public static List<String> chunkText(String text, int chunkSize, SplitStrategy strategy, int overlapSize)static
执行分块文本相关操作。
Chunks text according to the specified strategy.
参数Parameters
| 名称Name | 类型Type | 说明Description |
|---|---|---|
text | String | 文本到分块。the text to chunk |
chunkSize | int | 分块大小（类型 `int`）。the target size for each chunk (interpreted based on strategy) |
strategy | SplitStrategy | strategy（类型 `SplitStrategy`）。the splitting strategy |
overlapSize | int | overlap 大小（类型 `int`）。the number of characters/tokens to overlap between chunks |
List<String>返回列出。a list of text chunks
源码Source
io/agentscope/core/rag/reader/TextChunker.java:46–71