Build a document search index
Create term postings, file summaries, and an index completion manifest.
Use Document search index to turn a scoped set of stored files into structured search data. The prepared process extracts document segments, builds a word index, and retains the index rows for a search experience or another consumer.
Supply the source
| Parameter | What to enter |
|---|---|
workflow_name | A nonempty name that identifies the indexed collection. |
source_connection_id | The saved store connection containing the documents. |
source_catalog_slug | Its connector type; default platform_store. |
prefix | Optional file-key prefix limiting the indexed collection. |
Start with a small collection containing words you can verify. A blank prefix can admit a much broader source than a single folder or uploaded file.
Build and verify
- Select the template and configure the source.
- Create the workflow and inspect
parse → index → write_results. - Validate the saved definition.
- Start a Manual run and wait for completion.
- Inspect the index's posting, file-summary, and manifest records.
- Connect an appropriate search interface to the resulting workflow source.
The extraction job emits document_segment records. It sets ignore_input_records: true, so its configured source and prefix govern the collection rather than arbitrary trigger records. The index job consumes those segments and avoids separately publishing its intermediate output. The destination retains records stamped as search_index.
Read the index correctly
The payload distinguishes search_posting, search_file, and search_index_meta rows. Postings associate terms with files, positions, and snippets. File summaries describe indexed files. The completion manifest identifies the completed index context.
This is a word-search index, not a vector embedding collection or a promise of semantic retrieval. Use vector-search steps when that different retrieval model is required.
A workflow source containing index data is not automatically a deployed search app. Bind a compatible app or reader explicitly and verify its first query with a known term.
Repeated runs have their own run-based output identities. Ensure consumers select the intended index/run rather than mixing unrelated historical postings. If expected terms are absent, inspect extraction output before changing the indexing stage.
See Build text search index, document extraction, and workflow data sources.
Prepared job settings
The starter is enabled and uses a Manual trigger. These settings are supplied by the recipe; review them in the editor before adapting the process. ${…} values below refer to the corresponding template parameter or run variable.
| Setting | Prepared value |
|---|---|
parse.config.source.kind | "connection" |
parse.config.source.connection_id | "${source_connection_id}" |
parse.config.source.catalog_slug | "${source_catalog_slug}" |
parse.config.prefix | "${prefix}" |
parse.config.record_type | "document_segment" |
parse.config.ignore_input_records | true |
index.config.record_type | "search_index" |
index.config.segment_record_type | "document_segment" |
index.config.persist_outputs | false |
write_results.config.record_type | "search_index" |
write_results.config.idempotency_key_template | "${run_id}:${step_id}:${record_seq}" |