Using Tools from External Apps in LLM-Powered Applications

This guide outlines the steps to use tools from external apps in LLM-powered applications.

Overview

Most LLMs support tool use via one mechanism or another.

• In OpenAI models, it's called Function calling or tools.
• In Google models, it's called Function calling.
• In Anthropic models, it's called Tool use.

There are also emerging LLM-agnostic ways to use tools, such as Model Context Protocol.

All these mechanisms work in a similar way and are supported natively by our platform.

Each tool has the following:

• A name
• An optional description
• Input schema, described as JSONSchema, or in our case a Data Schema
• Output schema, also described as a JSONSchema
• An API endpoint to call the tool

You can use different building blocks to implement tools: API Operations, Data Collections, Data Sources, Flows, etc.

In this guide, we'll use the most common way of implementing: Actions.

Actions are the best way to implement tools for a few reasons:

• The structure of an action is very similar to the structure of a tool.
• You can use other building blocks inside actions.
• You can implement fully custom tools using your own code through Actions.

Adding Tools

To add a tool that can be used in your LLM-powered application, simply create an action.

You will need to configure the following:

• Name will be used by the LLM and end-users to determine if it's the right tool to use.
• Description will provide additional context to the LLM and end-users.
• Input schema will be used both by the LLM to suggest an input for the tool and by end-users to confirm and correct the input.
• Output schema will let you determine how to display and/or process the output of the tool.
• Implementation determines what the tool does.

Auto-generating tools

If your application needs a lot of tools, you can auto-generate them from other building blocks such as API Operations or Data Collections.

For this, you can use the following logic described in pseudocode:

# List integrations you want to generate tools for
integrations = ['integration1', 'integration2']

for integration in integrations:
   # Get building blocks from the integration that you want to generate tools for
   apiOperations = integrationApp.get('/integrations/' + integration + '/operations')

   # Filter the building blocks if you don't want to generate tools for all of them

   for apiOperation in apiOperations:
      # Create a tool from the building block
      # See action types documentation to get details on how to create one.
      action = {
         'name': apiOperation.name,
         'description': apiOperation.description,
         'inputSchema': apiOperation.inputSchema,
         'outputSchema': apiOperation.outputSchema,
         'type': 'api-operation',
         'config': {
            'operationKey': apiOperation.key,
         }
      }

This approach lets you save a lot of time on creating tools manually while giving you full control over the resulting list of tools.

You can augment this approach by manually creating missing tools or editing them to fit your needs better.

Using tools

To use tools in your application, you need to get a list of available tools and send them to your LLM via a corresponding SDK or API.

To get a list of tools/actions, use our Actions API.

Filtering down tools

In many cases, you will have way more tools available than can fit into the LLM context window comfortably.
In this case, we recommend filtering down the list of tools to only include the ones that are most relevant to the current user or the current task.

The most common way to do this is to generate embeddings for tools, put them into your vector database, and select the top X relevant tools based on the user's query.

Running tools

To run a tool, simply execute the corresponding action:

await integrationApp
  .connection('hubspot')
  .action('{ACTION_KEY}')
  .run('{INPUT}')