The article explains how to set up a simple flow for an Agent Copilot workspace. It does not include the basic setup of the connection between Cognigy and your contact center. Information on how to do that can be found here:
Getting Started with Agent Copilot
This article also assumes the user has the following preexisting abilities:
- Basic Javascript knowledge
- Basic HTML knowledge
- Adaptive Cards knowledge
The Copilot Workspace
At a high level Cognigy Copilot workspaces are webpages separated into customizable "tiles". Each tile represents either a miniature HTML based webpage or Microsoft Adaptive Card.
Even though the Copilot Workspace is one large webpage each tile is its own independent process, at least from a flow design and technical perspective. In other words, an individual process must be built for each tile in your workspace.
But before we think about the functions of the individual tiles, We need to consider how they fit into the design of the webpage. This is referred to in Cognigy as the grid.
Setting Tiles in the Grid
The grid is what is used to determine the exact size and placement of individual tiles and is the first thing which needs to be set in your flow. Do this by clicking the "+" in the Cognigy flow and adding the "Copilot: Set Grid" node.
This node is used to define what the grid should look like and is preset with common tile types in predetermined placements and sizes.
this can be changed in the Copilot Grid Configuration field above the preview. But what do the individual elements mean?
Grid
The "grid" element determines the amount of columns and rows in the workspace as well as the gap between each tile on all sides. The whole workspace and each tile individually changes its size dynamically depending on the size of the page they are integrated in. The lowest value which can be entered into any of these variables is "1".
The number of columns and rows given here doesn’t determine the actual size of the individual tiles, but rather how granular you wish to set the size of them.
For example if you set both your columns and rows value to "1" your grid would only have enough space for one large tile.
If you want to two tiles split in the middle horizontally you can set your columns to "1" and your rows to "2".
We will go into more details when we discuss the actual tile settings.
Because we normally only need to set this one time in the flow, the best-practice is to add this behind a once node.
The more columns and rows you set up the more granular you can set your tiles. Exactly how to set your tiles is explained in the next section.
Tile Positioning and Size
After determining the amount of rows and columns in your grid, we now need to set the individual tiles.
The first thing you will notice is that The name of the object also affects the name of the tile in the grid.
This is important as this name is not only used to identify the tiles but also has a functional purpose within the flow (which will be explained below), so please make sure to name the tiles in a clear and concise way.
X and Y Anchor Points
The "x" and "y" values in the object determine anchor point of the tile within the grid. This is always the upper left of the tile.
For example, if we have a 5x5 grid (Grid settings columns = 5 & rows = 5] and set the x and y values to both to 1 the upper left of the tile would be in the upper left of the grid.
But, if the x and y values are both set to 2, the anchor point would be on the one down and right in the grid.
Rows and Columns
The row and column variables determine the size of the tiles relative to the entire grid.
Let's take an example of the 5x5 grid and an x & y of 1 from above. If we set both the column and row values to 3 it would take up a 3x3 space in our 5x5 grid.
As we mentioned before this is all relative to the size of our grid. So as a point of comparison let's assume we have a grid and tiles with the following properties:
- Grid = 6x6
- X anchor = 1
- Y anchor = 1
- Column size = 3
- Row size = 6
It would look like this:
However, in the actual co-pilot workspace and the UI the agent interacts with there is no difference between that and a grid and tiles with the following property:
- Grid = 2x1
- X anchor = 1
- Y anchor = 1
- Column size = 1
- Row size = 1
This is because these setting only determine the relative size and relative position of tiles. The actual size of the tiles and grid is determined by the size of the window for the embedded workspace in the contact center. In other words,The actual size is dynamic, and design testing in the workspace is absolutely necessary.
Flow Structure
Before going into detail about the tile contents we should define a few rules about the flow structure in the Copilot workspace.
In general just like with any other flow each user input is sent to the Copilot flow (in the case of VoiceGateway even the agent audio can be transcribed) so when building the flow we need to assume that every user input needs to be processed. Even if the input isn't relevant for the copilot we need to make sure we have processes in place to ensure it doesn't make unnecessary changes to your workspace.
High level all copilot workspaces should have the following structure:
In addition we should assume that the flow will go from Start to End with every user input. For this reason Question Nodes, Wait for Input Nodes and other such "Stopper" Nodes should not be used in the flow if possible. Of course there will be exceptions to this, but as a general rule it should not be necessary.
Setting Tile Content
After we have established the size and position of the tiles we now want to set the content of the tiles. In the co-pilot workspace we have both predefined tiles and customizable tiles.
Nodes for Copilot tiles must be added after the Copilot: Set Grid node.
Remember how we mentioned that the names of the tiles in the grid are also functional? This was referring to how we determine the content of each tile. The name established in the grid must match with the name in the Tile ID field of your tile.
Depending on the type of tile and what you wish to do with it, this tile either needs to be triggered only once (On First Time) or multiple times based on logic within the flow.
Let's take a closer look at the Node/Tiles available. Cognigy offers both predefined tiles as well as customizable ones.
Predefined Tiles
As the predefined tiles are thoroughly described in our Copilot Docs page we will only go into detail which has not been explicitly mentioned in those pages.
Copilot: Knowledge Tile
The knowledge tile leverages LLMs and our Knowledge AI to help the agent find answers within the environment. Detailed information on how to use our Knowledge Tile can be found in our docs.
Copilot: Sentiment Tile
The Sentiment Tile uses LLMs to detect the users mood in the conversation and give the agent live feedback. This is only based on the transcribed text and not the actual tone of voice. Detailed Information on how to use our Sentiment Tile can be found in our docs.
Copilot: Transcript Tile
The Transcript Tile displays the callers last input as transcribed by the STT system. It does not show the whole transcript live. If you wish to do this we can leverage the Get Transcript Node and an HTML tile to create a custom solution.
Copilot: Identify Tile
The Copilot Identity Tile displays user information in the workspace.
The fields are normal CognigyScript which means it can be filled with data from a Cognigy Contact Profile or from external customer management systems via API:
If you decide to use the Cognigy Contact Profile for data please keep in mind that in voice use-cases a new sessionId and userId are created for the Copilot endpoint.
This is usually the userId and sessionId of the original starting session prepended with "assist-". So if the original call had the following userId
491234567
the copilot user userId would be
assist-491234567
In this case it would be very easy to use the Merge Profile node to keep the data between the sessions consistent.
Copilot: Next Action Tile
The Next Action Tile either leverages your flow logic to give predefined text based on the user's input or can also use Knowledge AI to generate a suggested answer.
Custom Tiles
Alongside tiles with predefined design we also have much more customizable tiles. These are tiles where the flow builder has a more granular ability to determine the content, look and function of the tile.
These include the Copilot: HTML Tile, Copilot: Send Data, Copilot: iFrame Tile, and Copilot: Adaptive Card Tile nodes.
Let's go through them one by one and explain how they work starting with the easiest to work with.
Copilot: Adaptive Card Tile
The easiest way to create your own forms in a copilot workspace is using Adaptive Cards. However, they offer many advantages and disadvantages.
Pros
- Very short development lead time
- Ease of creation via the Microsoft Adaptive Card Designer
Cons
- Design and colors are not customizable
- Field sizes not dynamic
- Changes to the tile can only be done by completely reloading it with a separate node (Send Data node will not work)
Defining the Layout and Functionality of the Adaptive Card
In general most Adaptive Card functionality will work in the tile as described in the Microsoft documentation. So let's take a simple example and show how we can leverage the Copilot and Cognigy functionality with them.
Let's assume we want to create a very simple form which looks like this:
But what if we want to prefill is with data, for example, from the user's contact profile? In this case we can simply use CognigyScript to define the value variable in the Adaptive Card.
{
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"type": "AdaptiveCard",
"version": "1.5",
"body": [
{
"type": "TextBlock",
"text": "User Information",
"wrap": true
},
{
"type": "Input.Text",
"id": "name",
"placeholder": "Enter your name here...",
"value": "{{profile.firstname}} {{profile.lastname}}"
}
],
"actions": [
{
"type": "Action.Submit",
"title": "Submit"
}
]
}
And in your workspace it will be prefilled:
Keep in mind you will only be able to send variables in the sting format to the adaptive card so anything more complicated in regards to format will probably need to be preformatted via a code node in the flow.
Now that we have a form, how do we send the data to the flow?
Sending Data from Adaptive Card to the Flow
If we have a Submit Action properly setup in our adaptive card, the data send when we submit will be sent back to the flow as input data with the following JSON data object. This will then be in the following object:
input.data._cognigy._agentAssist.payload.ID_OF_INPUT_FIELD
If we have the flow set up properly it should be very easy to 'catch' this data and process it properly within the Cognigy flow.
Changing Content of Adaptive Card
If we are using an Adaptive Card for informational purposes (or if we want to make any changes to the adaptive card in general) we will need to reload the whole Adaptive Card in the flow.
So let's say, for example, we want to replace the above Adaptive card form with a generic message after submitting the data.
{
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"type": "AdaptiveCard",
"version": "1.5",
"body": [
{
"type": "TextBlock",
"text": "Information has been submitted",
"wrap": true,
"size": "Large",
"weight": "Bolder",
"horizontalAlignment": "Center",
"color": "Good"
}
]
}
In this case we can create a flow which looks like this, wherein the form is defined in the 'On First Time' branch and the update with the message is in the 'data catching' part of the If Node in 'Afterwards':
Keep in mind that this will completely reload the tile because we are replacing the old Adaptive Card, with a new one, which will be apparent in the workspace UI.
Copilot: HTML Tile
In addition to Adaptive Cards we can also add tiles with custom HTML to the copilot workspace. This also has both advantages and disadvantages.
Pros
- Highly customizable via CSS etc.
- Don't need to be reloaded to update data
- More general functionality via Javascript integration directly in the node
Cons
- Requires more technical and design knowledge to build, in particular HTML, CSS and Javascript
Defining the Layout and Functionality of the HTML Tile
Let's take the same use case we had above: a simple form with one field.
In HTML we can probably build it like this:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
</head>
<body>
<h1>User Information</h1>
<label for="name">Name:</label><br />
<input type="text" id="name" name="name" value="Max Mustermann" /><br />
<button
type="button"
onclick="SDK.postback({ name: document.getElementById('name').value });"
>
Submit
</button>
</body>
</html>
Which would like this when it is first initiated.
This isn't very pretty but it gets the job done.
But there are some parts of the code which are exclusive to Cognigy, so let's break that down.
Sending Data from the HTML Tile to Flow
This part defines the body of the HTML. The most important part is the SDK.postback function which, as is also described in our documentation, sends data back to the flow. In this case we are sending data from the field which we have defined as "name" when the button is clicked.
Just as with the adaptive card the data is sent in the following object:
input.data._cognigy._agentAssist.payload.ID_OF_INPUT_FIELD
This means we can also use the same pattern in the flow to send data back to the flow and catch it in the exact same way.
Of course it is also possible to add this as a Javascript function:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
</head>
<body>
<script>
function sendName() {
const nameText = document.getElementById("name").value;
SDK.postback({ name: nameText });
}
</script>
<h1>User Information</h1>
<label for="name">Name:</label><br />
<input type="text" id="name" name="name" /><br />
<button type="button" onclick="sendName()">Submit</button>
</body>
</html>
Sending Data from the Flow to the HTML Tile
If we look at the example above though, we are hard-coding the prefilled value. So what do we do if we want to prefill it with dynamic data from the flow?
The simplest way to do this is simply with CognigyScript:
And this would work in a very simple use case with very simple data, however if the tile needs to be more dynamic and requires more complicated data it is very limiting.
In this case we would need to add an event listener to the HTML as described here. This works in coordination with the JSON Data field of the Copilot: HTML Tile Node.
To keep it simple, by adding the data into the JSON Data field we are telling the HTML Tile what data from the flows can be used by the HTML tile at the time it is triggered in the flow. In this case our HTML code also needs to be adapted to be able to process this information.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
</head>
<body>
<script>
// Optional: observe messages coming back from Copilot host
window.addEventListener("message", (event) => {
if (event.data && event.data.firstname && event.data.lastname) {
const name = document.getElementById("name")
name.value = event.data.firstname + ' ' + event.data.lastname
}
});
</script>
<h1>User Information</h1>
<label for="name">Name:</label><br>
<input type="text" id="name" name="name"><br>
<button
type="button"
onclick="SDK.postback({ name: document.getElementById('name').value });"
>
Submit
</button>
</body>
</html>
So in this case we are "listening" if the values 'firstname' and 'lastname' exist in the JSON Data field and if they do we are 'pushing' them into the standard value of the input field.
his method has a major advantage. If we need to update the data in the HTML tile in any way we simply need to use the Send Data node to update the data and we do not need to reload it like the Adaptive Card. We will explain how that works below.
Changing the Look of the HTML Tile with CSS
Now that we have an HTML Tile with basic functionality, we might want to adapt it to match the look and feel of the company.
Because this is a HTML code it is possible to add CSS, Google fonts, and other styling in order to give the tile a more professional look.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Figtree:ital,wght@0,300..900;1,300..900&family=Mulish:ital,wght@0,200..1000;1,200..1000&display=swap"
rel="stylesheet"
/>
<style>
h1 {
font-family: "Mulish", sans-serif;
font-weight: 500;
font-size: 18px;
line-height: 100%;
}
label {
font-family: "Mulish", sans-serif;
font-weight: 500;
font-size: 16px;
line-height: 100%;
}
input {
width: 50%;
font-family: "Mulish", sans-serif;
font-weight: 400;
font-size: 20px;
line-height: 140%;
letter-spacing: 0;
color: #000;
resize: none;
}
button {
font-family: "Mulish", sans-serif;
border: none;
outline: none;
font-weight: 500;
font-size: 16px;
line-height: 100%;
text-transform: uppercase;
color: #ffffff;
background: #003781;
width: 300px;
height: 39px;
}
button:hover {
background: #0055a6;
}
</style>
</head>
<body>
<script>
// Optional: observe messages coming back from Copilot host
window.addEventListener("message", (event) => {
if (event.data && event.data.firstname && event.data.lastname) {
const name = document.getElementById("name");
name.value = event.data.firstname + " " + event.data.lastname;
}
});
</script>
<h1>User Information</h1>
<label for="name">Name:</label><br />
<input type="text" id="name" name="name" /><br />
<button
type="button"
onclick="SDK.postback({ name: document.getElementById('name').value });"
>
Submit
</button>
</body>
</html>
Copilot: Send Data
As mentioned above it is possible to update data in the HTML Tile without reloading it. This can be done using the Copilot: Send Data node.
We do this by adding the node in a pattern based on the one above.
In our Afterwards branch we add the Send Data node wherever we want to use it. Two things are important when it comes to this:
- The Title ID must match the Title ID of the original HTML Tile Node
- The JSON Data must also match the basic structure of the data as defined in the original HTML Tile Node or be data which can be processed by the event listener.
Copilot: IFrame Tile
The Copilot: IFrame Tile node is used to integrate external websites into the Cognigy Copilot Workspace. This means the tile development will need to take place completely outside of the Copilot environment.
This is good for integrating external systems which are already in place in a company's organization.
It can even use the JSON Data field and Send Data node to update data within the IFrame just like the HTML Tile Node. However there are some considerations which need to be addressed when using this.
- Make sure the format of the embedded webpage fits within the Copilot workspace. You will more than likely have less space in your Copilot environments than a normal webpage.
- If you want to receive data you will need to use the same code and patterns as in the HTML Tile Node, otherwise this functionality is not available.
- Sending data from the IFrame to the Cognigy flow requires a development specific to the IFrame.
Sending data from the IFrame to the Cognigy flow
Unlike the Copilot: HTML Tile Node, the IFrame node cannot send data directly to the Cognigy flow via the SDK.postback() function simply because this is a function directly integrated into the parent window of the copilot workspace. The IFrame cannot access this for security reasons (in order to prevent injection attacks from third parties). Instead we need to send data to the main copilot workspace and handle the postback in a separate HTML Node. In other words, in order to integrate an IFrame at least two tiles will be required.
From a high level perspective, the IFrame is the child property/window in the the copilot workspace, which is the parent property. Because of this we need to send data to the parent page and "catch" it in a separate tile in order to process it.
If we look at our example from above, the first step would be to replace the SDK postback function with the following pattern:
<button
type="button"
onclick="window.parent.postMessage({ name: document.getElementById('name').value }, '*')"
>
Submit
</button>
To break it down, the SDK.postback is replaced with an HTML window.postMessage function. The '*' denotes the page we are sending the data to. If we want to add a layer of security we can replace this with the base URL of the Copilot workspace, for example 'https://ai-copilot-trial.cognigy.ai' for the Cognigy Trial environment. This would prevent embedding into any other environments:
The above sends the value of the field to the copilot workspace/webpage, however this does not yet send it to the flow. For this we will need a Copilot: HTML Tile. This tile must be within the grid but does not need to be visible.
This tile only requires the following code as part if the HTML:
<script>
window.parent.addEventListener("message", function (event) {
console.log("Content of in parent message: " + JSON.stringify(event.data));
if (event.data.name) {
SDK.postback(event.data.name);
}
});
</script>
This code is similar to the "normal" event listener of the HTML node but rather than catching data within the node, it is catching data from within the whole workspace. In this case we are specifically "listening" for the event which sends the name and in that case uses the SDK.postback function in a separate node to send it to the flow.
To summarize the individual parts interact with each other such as seen in the image below.
High level structure: Parent and Child window interaction
Our tip is to add this script to an already existing node in your workspace. Alternatively it can be "hidden" behind another tile by giving it no information and the same coordinates as another tile, however this can lead to some design problems.
General Tips and Tricks
Now that we can build a Copilot environment here are some tips and tricks which might help you in your design.
- If you want to add a logo to your environment consider giving it its own tile, this will help with the general look of the workspace
- If you can use HTML Tile Nodes over Adaptive Cards. You will have more flexibility in your design and function
- Keep the size of the Copilot workspace in mind when designing
- Each tile is its own individual website, and should be designed accordingly
Comments
0 comments