ExternalApiReader

Generic external API document parsing reader that supports integration with any third-party document parsing service.

This reader is designed to be highly abstract and configurable, adapting to different APIs through functional interfaces:

• Supports custom HTTP request construction (headers, body, method, etc.)

• Supports custom response parsing (extracting markdown from JSON/XML/other formats)

• Supports polling for asynchronous task completion

• Supports file upload or URL passing

Example usage - Integrating with MinerU Tianshu:

`ExternalApiReader reader = ExternalApiReader.builder()
.requestBuilder((filePath, client) -> {
// Build task submission request
String json = String.format(
"{\"file_path\":\"%s\",\"engine\":\"pipeline\"}",
filePath
);
return new Request.Builder()
.url("http://localhost:8000/api/v1/tasks/submit")
.header("Content-Type", "application/json")
.post(RequestBody.create(json, MediaType.get("application/json")))
.build();
})
.responseParser((response, client) -> {
// Extract task_id from response, then poll for results
String responseBody = response.body().string();
// Parse JSON and poll...
return markdown;
})
.chunkSize(512)
.splitStrategy(SplitStrategy.PARAGRAPH)
.overlapSize(50)
.supportedFormats("pdf", "docx", "pptx", "xlsx", "txt") // Specify supported formats
.build();

List<Document> docs = reader.read(ReaderInput.fromString("path/to/file.pdf")).block();
`

Example usage - Integrating with other APIs:

`ExternalApiReader reader = ExternalApiReader.builder()
.requestBuilder((filePath, client) -> {
// Custom request building logic
byte[] fileBytes = Files.readAllBytes(Paths.get(filePath));
RequestBody body = RequestBody.create(fileBytes, MediaType.get("application/octet-stream"));
return new Request.Builder()
.url("https://api.example.com/parse")
.header("Authorization", "Bearer YOUR_TOKEN")
.post(body)
.build();
})
.responseParser((response, client) -> {
// Custom response parsing logic
return response.body().string();
})
.build();
`