Creating a Knowledge Base based on Research Agent outputs

A Knowledge Base (KB) is a collection of source data that powers an Insight Booster (IB) project.

A Research Agent-based Knowledge Base lets you gather information from online sources using predefined research goals. By clicking “Create”, you can launch a Research Agent that searches the web, collecting and summarizing relevant content. Unlike news or social media, you don’t need to write a specific search query. Instead, simply describe what you’re looking for, and the Research Agent will find and summarize the information for you.

In this article, you’ll learn how to:

To create a Knowledge Base, navigate to the Knowledge Base icon in the blue ribbon at the top of the page. Alternatively, you can open an existing Insight Booster project and create a Knowledge Base directly within that project. More details can be found in the article titled “Creating a Knowledge Base”.

Click “Select input data”.

Clearly describe the research task for the Research Agent. You can include details such as which sources to use, what geographies to focus on, the search language, and your preferred tone or length for the results. Below is an example task description.

In the example above, the terms $area and $country are placeholders. Instead of running a separate Research Agent for each combination of area and country, you can loop through multiple values by using these variables. This allows you to automate your research across various topics and regions efficiently.

Right below the research task description, you’ll find the “Add missing placeholders” button. Use it to define the specific values for each variable, such as the list of areas of interest or countries you want to include in the research.

You can add placeholder values one at a time by typing and pressing Enter, or you can paste a list of newline-separated values and press Enter to add them all at once.

After adding placeholders, make the following configurations:

Number 1 - Placeholder iteration policy: When your task description includes multiple placeholders, choose how to iterate through their values. Select ‘All combinations’ to generate every possible combination of the placeholder values. If each placeholder contains the same number of values and you prefer to pair them in order (i.e., first with first, second with second), select ‘Aligned values’.

Number 2 - Sources: By default, the Research Agent scans both Google News and Google Web. You can choose to focus exclusively on either news or web sources if preferred.

Number 3 - Enable extensive search: Enabling this option allows the Research Agent to analyze a broader set of search engine results. This is especially useful for tasks that require comprehensive information gathering rather than targeting specific answers.

Number 4 - Email notification upon completion: Enable this option to receive a notification when the task is complete. The notification includes a summary of key statistics, such as the number of websites visited, reports analyzed, and other relevant details.

Number 5 - Add the input placeholders as output columns: Enabling this option allows you to get a column for each of the placeholders.

Number 6 - Exclude empty results: If the agent can't find relevant information for a specific task iteration, that output will be left empty. Enable this option to automatically exclude empty results from the final output dataset.

Number 7 - Output column name: Enter the names of the columns you’d like to include in your output dataset. The pre-filled list shows suggested columns, but you can add or remove columns based on your needs.

Number 8 - Output description: Use this field to provide descriptions for each column in your output dataset.

Number 9 - Add columns: Use this to define additional columns to the output.

Number 10 - File name: Name the file that will store the agent run results. File names can only contain alphanumeric characters and hyphens.

Click “Preview” to preview your research task before starting the agent.

In the preview, you’ll see the number of tasks the run will execute, a sample prompt with one combination of placeholder values, and the defined output schema. If anything looks wrong, you can click “Back” and make changes. If everything looks correct, click “Start” to begin the run.

Now you will schedule the Knowledge Base run. Click ”Continue” to proceed to workflow settings.

You can create a Knowledge Base for one-time use, schedule it for automatic updates, or trigger it through an API call for dynamic data integration.

This is the default option, allowing you to run the workflow once. The results will automatically be converted into a Knowledge Base.

Click “Continue” to configure further settings.

You can either create a new Knowledge Base or update an existing one. If you're creating a new Knowledge Base, you'll be prompted to select the relevant fields from the dropdown menu.

Enter a descriptive name for your Knowledge Base, which will appear in your list of Knowledge Bases.

To add metadata, click the plus sign under the Metadata section. In the example below, the country and area values used as placeholders have been added as metadata.

Next, provide a clear and descriptive name for your workflow. You can also add an optional description to give more context. Once all parameters are set, click “Create” to proceed.

If you are updating an existing Knowledge Base, select “Update” as the action type and choose the Knowledge Base name from the list.

Once you have filled out these details, click “Create” to finalize the setup.

Once your Knowledge Base has been created, you will find it under “My Knowledge Bases” accessible through the blue ribbon at the top. Successfully created Knowledge Bases will have a green check mark next to their name.

You can update Knowledge Bases at scheduled intervals. Please note that this option is only available for updating existing Knowledge Bases; it cannot be used to create new ones.

At this stage, you can select your preferred frequency for updating. The settings shown below will create a workflow that runs the RA and updates the Knowledge Base on the 1st of each month at 6 am. You may also select from different frequencies from the “Repeat every (...)” dropdown.

Select the right columns for your Knowledge Base using the dropdown menus, then click “Continue” to proceed.

Your RA will run on the first of each month at 6 am and update the KB automatically.