docs / operations / knowledge-flow-backend

Sizing the PDF Ingestion Worker (Docling)

The medium and rich ingestion profiles run every PDF through Docling (OCR + layout) on a Temporal worker. Get the thread count wrong relative to worker concurrency and CPU cores, and ingestion doesn't error — it just goes silent for minutes at a time. This page is the operational reference for avoiding and diagnosing that.

Sizing principle & formula

A Temporal worker runs up to scheduler.temporal.ingestion_max_concurrent_activities PDF extractions at the same time, each in its own activity thread. Each of those extractions independently spins up its own Docling accelerator pool, sized by processing.profiles.<profile>.pdf.docling_num_threads (OMP/accelerator threads — Pydantic field, default 4, only meaningful for the medium and rich profiles, since those are the only ones with extractor: docling).

Nothing ties these two numbers together automatically. Total OMP threads in flight at any moment is their product, and that product has to fit inside the CPU you actually have — not the CPU you think you have.

The formula: 
docling_num_threads ≈ available_cores ÷ ingestion_max_concurrent_activities
  • Native (make run-worker): available_cores = nproc — the host's logical core count.
  • Kubernetes: available_cores = the worker pod's resources.limits.cpu — never the host node's core count. A pod capped at 2 vCPU on a 64-core node still only gets 2 vCPU worth of scheduler time; sizing off the node's nproc reproduces the exact same thrash inside the cgroup.

A pod limited to 2 vCPU with ingestion_max_concurrent_activities: 3 must not exceed docling_num_threads: 1 — going higher hits the same symptom as the native case: the pod pins at 100% of its cgroup CPU limit and extractions appear to stall, even though nothing has actually deadlocked.

Source: knowledge_flow_backend/common/structures.py (docling_num_threads field, DoclingPdfExtractor.__init__ wiring) · config/configuration.yaml, config/configuration_prod.yaml (inline sizing comment on both medium and rich profiles) · deploy/charts/fred/values.yaml (sizing comment next to ingestion_max_concurrent_activities — no value is imposed there, since it depends on the pod's real CPU limit)

Threads × concurrency abacus

Zones below are relative to available_cores (host nproc natively, pod CPU limit in Kubernetes) — read ratio, not the absolute thread count, when translating to your own hardware or pod sizing.

Total OMP threads (num_threads × concurrent activities)Ratio vs. available coresZone
≤ available cores≤ 1.0×safe
up to ~1.5× available cores1.0×–1.5×caution — some scheduler contention, still makes forward progress
> 1.5× available cores>1.5×danger — severe OMP thread thrashing, looks like a hang

Two measured points on an 8-logical-core dev machine (Intel Core Ultra 7 258V, 4 P-cores @4.7-4.8GHz + 4 E-cores @3.7GHz, no hyperthreading), ingestion_max_concurrent_activities: 3 in both cases:

ConfigTotal OMP threadsRatio vs. 8 coresZoneObserved behavior
docling_num_threads=8 × 3 concurrent243.0×dangerCPU 253%, 131 system threads, logs silent >2 min, one document took 67s
docling_num_threads=2 × 3 concurrent60.75×safeCPU ~400%, continuous cadence, 15–45s/doc, no stalls
These two numbers are one concrete data point, not a universal benchmark — Docling's per-document time depends on page count, image density, and table complexity. What transfers to other hardware is the ratio (safe at ≤1.0×, danger above 1.5×), not the raw seconds-per-document figures.

Diagnostic checklist

If ingestion looks stuck, check per-core CPU before concluding it's a deadlock. A thrashing worker and a genuinely hung one look identical from the ingestion UI — both show "Processing" with no progress — but they need opposite fixes.

  1. Check per-core CPU, not just aggregate load. Run top (process-level %CPU) and mpstat -P ALL 1 (per-core breakdown) on the worker host/pod. Aggregate CPU pinned near ingestion_max_concurrent_activities × 100% with every core individually saturated is oversubscription, not a hang — proceed to step 2, not to restarting the worker.
  2. Count OS threads on the worker process. ps -o nlwp <pid> or cat /proc/<pid>/status | grep Threads. A count far above ingestion_max_concurrent_activities (dozens to 100+) points at OMP over-threading, not stuck activities.
  3. Confirm the ratio. Multiply the configured docling_num_threads by ingestion_max_concurrent_activities and compare against available_cores (see the formula above) — in Kubernetes, read the pod's actual resources.limits.cpu, don't assume the node's. Above the 1.5× danger threshold, resize docling_num_threads down before investigating anything else.
  4. Rule out the extractor silently falling back. Docling initialization failures (missing model artifacts, bad path_base_model) fail fast — in under a millisecond — and fall back to markitdown silently: no OCR, no layout, but no visible error either, and no CPU thrash symptom. If ingestion completes too fast for a scanned/layout-heavy PDF and quality looks degraded, check the worker's startup logs for a Docling init failure before assuming the CPU-sizing issue above.
  5. Only after 1–4: consider a genuine stall (network-dependent model download hanging, a Temporal activity stuck waiting on I/O). If per-core CPU is idle while ingestion is silent, that's the real signal for an actual hang — worth checking second, not first, since CPU thrash is the far more common cause in practice.

What we actually hit

Three distinct issues surfaced in one live debug session on medium/rich-profile PDF ingestion — worth knowing about even if you never reproduce them verbatim, since two of the three don't fail loudly.

1. Docker model path used in native execution

config/configuration_worker.yaml had processing.path_base_model: "/app" (commented "Use for docker"), while configuration.yaml and configuration_prod.yaml use ".". Running the worker natively (make run-worker) resolved Docling's artifacts_path to /app/models — a path that doesn't exist outside a container — so pipeline initialization failed in ~1ms for every document, falling back to markitdown silently (OCR and layout extraction lost, no visible error). Fixed by aligning path_base_model to "." in configuration_worker.yaml.

2. Shared work directory across parallel extractions

application_context.get_input_processor_instance() caches one PdfMarkdownProcessor singleton per class for the worker's whole lifetime. The processor used to create a single self.folder temp directory in __init__, shared by every document processed concurrently across Temporal activity threads. A thread that finished ran self._remove_all_files(self.folder) in its finally block and deleted images another thread was still reading — FileNotFoundError: img0.png, silent fallback to markitdown. Fixed: every call to convert_file_to_markdown now creates its own tempfile.mkdtemp(), passed explicitly to the extractor and cleaned up only for that call.

3. CPU oversubscription from a hardcoded OMP thread count

DoclingPdfExtractor hardcoded AcceleratorOptions(num_threads=8), independent of ingestion_max_concurrent_activities or the cores actually available. See the measured numbers above for what that produced on an 8-core machine at concurrency 3. Fixed by adding processing.profiles.<profile>.pdf.docling_num_threads (default 4), threaded through DoclingPdfExtractor.__init__(num_threads=...).

Bugs 1 and 2 are silent by design of the fallback path — markitdown succeeding means no exception surfaces, so ingestion "works" with degraded output. If OCR/layout quality looks wrong on medium/rich-profile PDFs, check worker startup logs for a Docling init failure before assuming a model or content problem.

Files: knowledge_flow_backend/common/structures.py · core/processors/input/pdf_markdown_processor/{docling_processor,pdf_markdown_processor}.py · config/configuration{,_prod,_worker}.yaml · deploy/charts/fred/values.yaml