Cognigy.AI Lab
You've entered the Cognigy.AI Lab. This content and example code is inspirational.
This article will explain how to deal with files uploaded with in an app created for Microsoft Teams via methods other than using the paperclip 
below the text box in the chat, for example, copy and pasting the image directly into the text box or taking a picture via the Microsoft Teams app with a cell phone camera (so called "inline images").
Unfortunately unlike with the normal process Teams does not create a download link for images uploaded this way, for this reason a workaround needs to be created in order to process the image.
Knowledge of extension creation would also be extremely useful.
Extension
In order to retrieve the file from Teams we will need to create a token which will be used in order to retrieve the image later in the process. In this case we have created an extension which does just that and which has been attached as a download to this article.
Because this is a workaround you will not find this extension in the Extension Marketplace or on the Extensions Overview page.
This extension creates the Microsoft app credentials using the Microsoft Bot Framework Connector NPM package provided by Microsoft.
The extension requires the same credentials used to connect your endpoint to the Microsoft Azure Bot Channel. You will need to enter these regardless of whether you have entered the information in your endpoint.
Add the extension to the flow at any point before you want to process the uploaded image. It will create a token which can be saved to the context and used later.
If you would like to make adjustments to the Get Token Node of the extension or create your own the code for that node is as follows:
import { createNodeDescriptor, INodeFunctionBaseParams } from "@cognigy/extension-tools";
import { MicrosoftAppCredentials } from "botframework-connector";
export interface IgetTokenParams extends INodeFunctionBaseParams {
config: {
apiConnection: {
appId: string;
appPassword: string;
};
storeLocation: string;
inputKey: string;
contextKey: string;
};
}
export const getTokenNode = createNodeDescriptor({
type: "getToken",
defaultLabel: "Get Token",
fields: [
{
key: "apiConnection",
label: "Your Bot Authentication Credentials",
type: "connection",
description: "Bot App ID and Password",
params: {
connectionType: 'microsoftBotCredentials',
required: true
}
},
{
key: "storeLocation",
type: "select",
label: "Where to Store the Result",
params: {
options: [
{
label: "Input",
value: "input"
},
{
label: "Context",
value: "context"
}
],
required: true
},
defaultValue: "input"
},
{
key: "inputKey",
type: "cognigyText",
label: "Input Key to Store Result",
defaultValue: "microsoftToken",
condition: {
key: "storeLocation",
value: "input"
}
},
{
key: "contextKey",
type: "cognigyText",
label: "Context Key to store Result",
defaultValue: "microsoftToken",
condition: {
key: "storeLocation",
value: "context"
}
}
],
sections: [
{
key: "storageOption",
label: "Storage Option",
defaultCollapsed: true,
fields: [
"storeLocation",
"inputKey",
"contextKey"
]
}
],
form: [
{ type: "field", key: "apiConnection"},
{ type: "section", key: "storageOption"}
],
appearance: {
color: "#ed1d24"
},
function: async ({ cognigy, config }: IgetTokenParams) => {
const { api } = cognigy;
const { apiConnection, storeLocation, inputKey, contextKey } = config;
const { appId, appPassword } = apiConnection;
let credentials = await new MicrosoftAppCredentials(appId, appPassword).getToken();
try {
if (storeLocation === 'context') {
api.addToContext(contextKey, credentials, 'simple');
} else {
// @ts-ignore
api.addToInput(inputKey, credentials);
}
} catch (error) {
if (storeLocation === 'context') {
api.addToContext(contextKey, { error: error.message }, 'simple');
} else {
// @ts-ignore
api.addToInput(inputKey, { error: error.message });
}
}
}
});
Retrieving the image
When uploading an image with a method other than the "paper-clip" the file URL can be found in the following input object:
input.data.request.attachments[0].contentUrl
You will notice that unlike the other method this image is not a download URL.
We can however use this information to create an HTTP get request, for example via UiPath, Microsoft Power Automate or any other external program in order to download the image for further processing.
As a URL you would need to use the content URL received from Teams and the token we created would then be used as a bearer token for the HTTP request.
Comments
0 comments