Parallelism with FORK_JOIN

The basic workflow fetched two cities one after another. When steps are independent, running them in sequence wastes time. FORK_JOIN runs several task chains in parallel and rejoins them before the workflow continues.

The shape

A FORK_JOIN task holds a _forkTasks array. Each element is itself an array of tasks, one independent branch. All branches run concurrently; the workflow waits for every branch to finish before moving to the next task.

{    _type: "FORK_JOIN",    _name: "parallel_weather_fetch",    _sequenceno: 1,    _inputParams: {},          // required even when empty    _forkTasks: [        // Branch 1: New York        [            {                 _type: "SCRIPT_EXECUTION",                _name: "fetch_newyork",                _sequenceno: 1,                _inputParams: { _userType: "weather_workflow_demo_static",                    _scriptName: "generateNewYorkWeatherData" } },            {                 _type: "SCRIPT_EXECUTION",                 _name: "store_newyork",                 _sequenceno: 2,                _inputParams: { _userType: "weather_workflow_demo_static",                    _scriptName: "parseAndStoreWeatherFromWorkflow",                    weatherData: "${fetch_newyork._output.scriptOutput}" } }        ],        // Branch 2: San Francisco  (same shape)        [ /* fetch_sanfrancisco -> store_sanfrancisco */ ],        // Branch 3: London        [ /* fetch_london -> store_london */ ]    ]}

Then a following task aggregates the three branches' results:

{    _type: "SCRIPT_EXECUTION",    _name: "aggregate_results",    _sequenceno: 2,    _inputParams: {        _userType: "weather_workflow_demo_static",        _scriptName: "aggregateWeatherResults",        newyork:      "${store_newyork._output.scriptOutput}",        sanfrancisco: "${store_sanfrancisco._output.scriptOutput}",        london:       "${store_london._output.scriptOutput}"    }}

Two Rules Worth Memorising

These are easy to get wrong and both are enforced by validation:

1. _inputParams is required on the FORK_JOIN task, even if it is {}.
2. Every task inside a branch needs its own _sequenceno, numbered within that branch (each branch starts again at 1). That's why each branch for New York, San Francisco, and London each have a task with _sequenceno: 1.

Addressing Branch Outputs

This is the subtle part. After the join, the sub-tasks are addressable by name at the top level. You reference ${store_newyork._output.scriptOutput}, not ${parallel_weather_fetch.store_newyork...}. The parent FORK_JOIN task's own _output is an empty {}; it does not re-expose its children under sub-keys.

So the aggregator binds directly to the branch task names:

newyork:      "${store_newyork._output.scriptOutput}",sanfrancisco: "${store_sanfrancisco._output.scriptOutput}",london:       "${store_london._output.scriptOutput}"

The aggregateWeatherResults script then simply collects them:

async function aggregateWeatherResults(input, libraries, ctx) {    const p = getParams(input);    const allResults = [p.newyork, p.sanfrancisco, p.london].filter(Boolean);    return {        totalCities: allResults.length,        totalReadings: allResults.reduce((sum, r) => sum + (r.readingsCount || 0), 0),        cities: allResults.map(r => r.city),        details: allResults    };}

Run the Workflow

Continue using Code

1. Run runWeatherWorkflow with input copying and pasting this value into the input field:

   { "workflowUserType": "weather_workflow_forkjoin_static" }.

2. Monitor with getLatestWorkflowRunByUserType for the same _userType.

In the completed run you will see the three branches (New York, San Francisco, London) each with their fetch and store tasks, followed by aggregate_results reporting all three cities. All three branches ran at the same time, so the run is bounded by the slowest branch rather than the sum of them.

Continue using IDE Extension

Right click the Parallel_Weather_Fetch_ForkJoin workflow in the Workflow Service panel and select 'Run Workflow'.

After the workflow starts right click it again and select 'Show Workflow Runs'. Refresh the runs until the Workflow shows as COMPLETE.

You can examine the workflows _output to see the results of the workflow run for the three cities.

When to reach for FORK_JOIN

Use it when several units of work are independent and you need all of them before continuing: fetching the same resource from several hubs, calling several external services at once, or processing several partitions of a dataset. The next lesson on the Splitter–Aggregator pattern composes FORK_JOIN with a fan-out script to process a dynamic list.

---