Creating a Knowledge Base based on social media

A Knowledge Base (KB) is a collection of source data that powers an Insight Booster (IB) project.

A social media-based Knowledge Base lets you gather posts from platforms like X (former Twitter), Facebook, Instagram, and YouTube. By clicking “Create” you can build a KB that draws data from one or more of these channels.

In this article, you’ll learn how to create a Knowledge Base from social media content and how to set up the right search space to capture relevant data.

To create a Knowledge Base, navigate to the Knowledge Base icon in the blue ribbon at the top of the page. Alternatively, you can open an existing Insight Booster project and create a Knowledge Base directly within that project. More details can be found in the article titled "Creating a Knowledge Base”.

Next, click “Select input data” and choose the sources you wish to include in your Knowledge Base. You can select one, multiple, or all sources from the list provided.

In this section, you define the search space through keywords and other restrictions.

Number 1 - Import keywords: Use this button to upload keywords from an Excel spreadsheet. This method allows you to build detailed search queries using Boolean operators within a structured Excel format.

Number 2 - All of these words/phrases: Add keywords here that must be in every post. You can copy and paste them from an Excel column, with each row becoming a separate keyword, or enter them one by one. In Boolean terms, these terms are connected with "AND".

Number 3 - Any of these words/phrases: At least one of these keywords must appear in each post. As demonstrated below, here we are downloading posts containing any of the five terms. You can copy and paste them from an Excel column, with each row as a separate keyword, or add them individually. In Boolean terms, these are connected with "OR." If you fill both keyword sections, the query combines the “All of these words/phrases” and “Any of these words/phrases” keywords.

Number 4 - None of these words/phrases: Keywords here will be excluded from your search, even if they contain the search terms. You can add keywords that may bring irrelevant content. For instance, to avoid posts about a celebrity with the same name as the company you’re analyzing, add the celebrity’s full name here.

Number 5 - Language: You can specify the language of the posts you want to scan to align with the terms used in your search query. Choosing the appropriate language can help avoid ambiguities related to false friends -words that have different meanings in different languages. For example, the word "gift" means "present" in English but translates to "poison" or “wedded” in Swedish. Properly setting the language ensures more accurate and relevant search results.

Number 6 - Country: You can choose the countries from which to download data using the provided list. However, please note that some posts may not have country information. When you make your selections, the analysis will only include posts that contain country information and are sourced from those specific countries. While there may be additional posts from those countries, they might not have country data encoded. These posts will be missed from your analysis.

For this reason, we recommend refraining from adding any countries when comprehensive information is critical. This advice is specifically relevant to social media posts; however, it does not apply to news articles, as all news articles include a source country value.

Number 7 - Posted after: Select the earliest date to be included in your analysis. Please note that social media posts are only available for the past month, so access to posts before that time is unavailable.

Number 8 - Posted before: Select the latest date to be included in your analysis.

Number 9 - Sort by: Sort the collected posts by overall score or publication date. This is particularly useful for sampling data.

Number 10 - Post order: This setting controls the order of posts when you sample data. By default, posts are sorted from the most recent to the oldest. If you choose to sample data, the system will select the latest posts based on this order. If you aren’t using the sampling feature, you can keep this setting as it is, as it won’t impact the contents of your Knowledge Base.

Once you are set, click “Find posts”.

After clicking “Find posts”, you'll see the number of matches for your query, which helps you gauge the available data volume for your analysis. If the total number of posts exceeds your desired volume, you can refine your selection.

Once you’ve entered the preferred file name, click “Import”.

Click ”Continue” to proceed to workflow settings.

If there’s something specific you’d like to focus on within the data, you can enter it here. The workflow will then extract parts of the posts that relate to your area of interest. You can also choose a language for the summaries, regardless of the original data's language.

You can create a Knowledge Base for one-time use, schedule it for automatic updates, or trigger it through an API call for dynamic data integration.

This is the default option, allowing you to run the workflow once. The results will automatically be converted into a Knowledge Base. Please note that to schedule a workflow, you must first create it through an immediate single run.

Click “Continue” to configure further settings.

You can either create a new Knowledge Base or update an existing one. If you are creating a new one, provide a clear name for your Knowledge Base, and it will appear under this name in the Knowledge Base list.

Next, enter a name for the workflow, which will be displayed under “My Workflows”. You may also add a description for the workflow if desired.

If you are updating an existing Knowledge Base, select “Update” as the action type and choose the Knowledge Base name from the list below.

Once you have filled out these details, click “Create” to finalize the setup.

Once your Knowledge Base has been created, you will find it under “My Knowledge Bases” accessible through the blue ribbon at the top. Successfully created Knowledge Bases will have a green check mark next to their name.

You can also update Knowledge Bases at scheduled intervals. Please note that this functionality can only be applied to existing Knowledge Bases, as outlined in the previous section, “Immediate single run”.

At this stage, you can select your preferred frequency for updating. The settings shown below will create a workflow that updates the Knowledge Base every Monday at 8 AM with data from the previous week.

Click “Create” to proceed.

Select the Knowledge Base you want to automate from the dropdown menu.

If you want the results in the Insight Booster project to update automatically, enable the setting indicated by the arrow to regenerate the results.

Provide a name and optionally a description for the workflow, and click “Create”.

Select the “Trigger through API call” option, then click the plus sign to configure the variables you wish to modify for each API run.

You can choose a parameter from the dropdown menu to change the search query (i.e., the downloaded data) and assign a name to the variable. For example, if you want to modify the country for each run, simply select “country” from this menu. When you initiate a new run through the API, you will be prompted to configure a value for the country variable.

You can also define settings through operations. Click on “Add operation”, select the relevant operation, and configure the associated variables. These operations represent the steps your data undergo to transform into a Knowledge Base.

For instance, if you select the node that displays the “Sample” operation and click “Select”, you’ll be able to configure parameters related to the sample, in addition to the country.

Here, you can choose the sample size and set it for the upcoming run. Once you have selected your variables, click “Continue”.

Choose the Knowledge Base where you want to trigger runs through API, enter a workflow name, and click “Create”.

Once your Knowledge Base has been created, you’ll see it under “My Knowledge Bases” accessed through the blue ribbon on top. You will also receive an email once the Knowledge Base has been created. Successfully created KBs have the green tick mark next to their name.