Powering human-in-the-loop flows with the Knock agent toolkit
The Knock agent toolkit allows you to power rich human-in-the-loop interactions.
For example, you can use the agent toolkit to create a workflow that lets a person to approve or reject work that the agent has done, powered by Knock's cross-channel messaging workflows.
The agent toolkit exposes a set of helper methods that make it trivial for you to wrap your tool calls in asynchronous human approvals or inputs.
Knock takes care of the heavy lifting for you, making it easy to build rich human-in-the-loop flows with little effort.
Use cases
Using Knock, it's possible to power human-in-the-loop interactions that:
- Send cross-channel notifications to email, SMS, in-app feeds, and workplace chat applications like Slack and MS Teams.
- Power advanced cross-channel escalation flows to bubble up messages between channels or across team members.
- Solicit structured input such as approve/reject buttons, or unstructured feedback in the form of text or prompt modifications.
- Handle long-running async interactions that "interrupt" and "resume" your agent workflows.
- Give full visibility into the messages sent from your AI agent workflows and track engagement of those messages across channels.
How Knock powers human-in-the-loop flows
Knock uses workflows to power human-in-the-loop messaging across channels. Here's a high-level overview of how this works:
- You setup a workflow in the Knock dashboard that describes the message to send and the channels to send it to.
- Your agent calls to Knock to trigger the workflow with a set of recipients and any additional context for the request.
- Your user receives the message on the channel, which includes the context for the request and a set of actions to take.
- The user then interacts with the message and the result is sent back to Knock.
- Knock then triggers an outbound webhook back to your agent application with the result of the interaction.
- Your agent application can then use the incoming interaction event from Knock to resume execution, or respond to the interaction.
Helper methods
The Knock agent toolkit exposes a set of helper methods that you can use to power human-in-the-loop flows for common cases, as well as low-level primitives should you need to implement a custom flow.
toolkit.requiresHumanInput
The requiresHumanInput helper method allows you to wrap a tool in requiring human input to proceed. When a tool is wrapped with this method, when the LLM tries to invoke the call it will instead delegate the tool call to Knock, passing the tool execution context to the Knock workflow.
Calling requireHumanInput will return a new tool that you pass in place of the original tool in your model. When the LLM invokes the new tool, it will instead delegate the tool call to Knock, passing the tool execution context to the Knock workflow.
toolkit.handleMessageInteracted
The handleMessageInteracted helper method allows you to handle an incoming message.interacted event from Knock and determine if there's an interaction with the message that should be handled by your agent workflow.
toolkit.resumeToolExecution
The resumeToolExecution helper method allows you to resume tool execution after a human has interacted with the message and you have received an affirmative response to proceed.
You pass the resumeToolExecution method the DeferredToolCallInteractionResult that you've received from Knock via the handleMessageInteracted method. Here's an example of how you might use this method:
When using the resumeToolExecution method, you must have first registered any tools that require human input using the requiresHumanInput method. This ensures that the toolkit has access to the necessary information to resume the tool execution.
In the AI SDK, the resumeToolExecution method will return a tool status object where you can access the result under the result property. The original toolCallId is also included.
Example: In-app approval flow
In this guide we show you how to setup an approval flow that sends an in-app notification to a notification feed requesting approval when the agent wishes to invoke a particular tool.
Setting up a workflow to power approvals
To start, you'll want to setup a Knock workflow with a single, in-app feed channel step that sends an approval message to an in-app feed. You can find a workflow template for this in the workflow templates repository.
Rendering the feed
Next, you'll need to render the in-app feed in your application. You can find a guide on how to do this in the building in-app feed guide. By default, your in-app feed will handle the interaction on the "Approve" and "Reject" buttons and send the message.interacted event to Knock.
Setting up an outbound webhook in Knock
A human-in-the-loop interaction is inherently an asynchronous action, and therefore requires that we might need to handle the approval at some later point. Knock uses outbound webhooks to forward your events to your application, helping to implement this asynchronous behavior.
To set up an outbound webhook, you'll need to create a new webhook under Developers > Webhooks in the Knock dashboard. Add a URL for your service and select the message.interacted event. You can read more about the available event types in our guide.
Wrapping a tool as requiring human input
The agent toolkit exposes a requiresHumanInput helper that you can use to wrap a tool call as one that requires human input to proceed.
Now, when your agent calls your classificationTool, it will instead call your wrappedClassificationTool and delegate the tool call to Knock, bringing one or more humans into the agent's loop.
Handling the response and resuming tool execution
Finally, you will need to implement a handler for the message.interacted webhook event that fires when someone interacts with the "approve" or "reject" buttons in the notification feed.
You'll use this method to handle the incoming interaction event and to resume your agent's execution. Depending on which agent framework you're using, you'll need to implement this handler differently. We've left this example fairly abstract as a result.