A fetch function executes an HTTP request as a step in a workflow. Any data returned to a fetch function is merged into the original notify
data provided on workflow trigger and made available to all subsequent steps in the workflow.
With the fetch function, you can acquire additional data for your channel step templates that may not be immediately available when you first trigger a workflow. A common case is combining a fetch function with a batch function to retrieve notify data for a group of activities after a batch window has closed. You can also use the fetch function to trigger side effects in your systems as Knock processes your workflow.
As with channel steps, you use the Knock template editor to configure the shape of your request. For each fetch step, you can edit the following attributes:
- Request method - You can select one of
- URL - A valid HTTP URL.
- Headers - Any headers Knock should include in the request. You manage these via a key-value editor, with the key being the header name and the value being the header value.
- Query parameters - Any query parameters to encode into the URL. You also manage these via a key-value editor.
- Request body - When building a
PUTrequest, you can build a request body to include in the request. Knock will always encode the request body as JSON.
Aside from the request method selector, each of the above fields is a Liquid-compatible input. This means you can use Liquid variables and control flow to inject variable data, access Knock-controlled workflow state attributes (e.g.,
recipient), and dynamically shape the request per workflow run.
See the Knock template editor reference for a detailed guide on working with Liquid templates in Knock.
When executing the request for a fetch function, Knock expects the following from your service:
- The response to the request is one of:
201 Created, or
204 No Content.
- If the request response contains data, it's encoded as JSON and can be decoded into a map/dictionary/hash.
- The response to the request takes no longer than 15 seconds for Knock to receive.
When the response sent to Knock for a fetch function request contains JSON data, Knock will merge the decoded result into the
data you originally passed to the workflow trigger call. Knock uses a shallow-merge strategy here where:
- Data from the request overwrites the original workflow run data.
- Top-level attributes are merged, and nested attributes are completely overwritten.
The merged data result from a fetch function step then becomes the global notify data for all subsequent steps in the workflow run.
The example below illustrates how this could look in practice.
Knock will automatically retry request execution for a fetch function following certain types of errors. The first retry will be delayed by 30 seconds, and the second by 60 seconds. These retryable errors are:
- Server errors - Any
5xxlevel HTTP error code.
- Request timeouts - This is any fetch function request from Knock that does not receive a completed response within the 15 second limit.
All other errors or unexpected responses are immediately fatal. These include:
- Any other HTTP response code.
- Some issue with the structure of the request, such as an invalid URL.
- Any issue JSON-encoding a request body.
- Response data that cannot be JSON-decoded as expected.
After two failed retries for a retryable error or any non-retryable error, Knock will mark the fetch function step as a failure and halt your workflow run.
As you develop, you can execute test runs of your fetch step from right within the template editor. This should look and feel similar to executing test runs of your workflows, but here Knock will execute just your fetch step, ignoring any other steps that may exist before or after.
To run a fetch step test:
- Click the button that sits to the right of the URL field in the template editor. This should open the Knock test runner modal.
- Specify the appropriate trigger parameters (actor, recipient, notify data, and tenant) for the test run. NOTE: If your fetch step expects data from a preceding batch step or fetch step, you'll need to explicitly include it here in the "Data" field. Since Knock will test this step in isolation, it cannot know what preceding data may be present when the full workflow runs.
- Click the "Run test" button in the modal. The modal will close and the test console should display a loading state as Knock executes the test.
- When the test run has completed, Knock will load the result into the test console for your review. You can then use use the "Request" and "Response" buttons to toggle between the two views in the test console. The "Response" section will show any data returned by the request that would be made accessible to subsequent steps in your workflow.
When running fetch step tests, Knock will not retry a failed request on any error. For the retryable errors outlined above, Knock will indicate in the test console result that they would be retried during a full workflow run.
You can use the workflow run logs to debug your fetch function steps. For each fetch function, you can expect to see in the logs:
- The request URL (with encoded query parameters), headers, and body as sent by Knock.
- The duration of the request (in milliseconds).
- The response headers and body data.
In the workflow run overview, you'll also see any data that Knock successfully received from your fetch function steps and merged into your workflow run state.
If the request encounters an error, you can also expect to see details about the error in the logs. And finally, if the fetch function retries the request on a retryable error, you can expect to see details enumerated for each request attempt.
See the guide on debugging workflows for more details about workflow debugging and run logs.