Branching and parallel execution
Split data processing into branches and control conditional steps.
A workflow can branch when several jobs depend on the same upstream job. This lets you analyze the same input in different ways, separate destinations, or join results later.
Build a branch
- Add a source and verify its records.
- Connect two processing jobs to that source.
- Configure each branch independently.
- Add a merge or multi-input step when results need to come together.
- Inspect the combined output with sample records that make the two branches distinguishable.
Independent branches can execute concurrently within the workspace's execution limits. Parallelism is an opportunity, not a promise that every step starts at the same instant. A branch that depends on a slow parent must wait for that parent.
Gate a step with a condition
A step condition tests a JSONPath against incoming records. Supported comparisons include equals, not equals, contains, membership, and existence. Any runs the step when at least one record matches; All requires every record to match.
A false condition skips the entire step and produces an empty output. It does not automatically skip all descendants. A downstream source or handler that can act on empty input may still run. When you want to remove individual nonmatching records, use Filter instead.
Empty input has precise consequences: Any does not match an empty set; All does. Test this case before using a condition to guard a destination.
Distinguish parallel branches from record mapping
Map fields reshapes each record; it is not a parallel loop that creates one child workflow per row. A subworkflow invokes a separate workflow and returns its terminal outputs. Choose the operation that matches the intended process.
An aborting branch failure can leave independent branches already running. Some completed external writes may therefore remain even when the overall run fails. Design destinations for safe repetition and inspect each step before retrying. See reliability.