Odata transcript lacks important information (fallback messages, postback values)

I've been using the Odata API with Cognigy's custom extension Get Conversation node to retrieve a transcript. The idea is that an HTML table representation of this transcript can be sent to an arbitrary e-mail address. The mechanism works fine in general, but the transcript data is quite lacking, regardless of whether JSON or HTML is selected in the Get Conversation node.

The HTML transcript lacks complex message formats like Text with Quick Replies or Galleries, and no fallback text is included instead (the messages are simply omitted). If I format the JSON transcript manually to HTML I can include fallback text (a lot of work!), but then, the postback value of when a user clicks one of the quick replies is not in the JSON object... Meaning this information also doesn't show up in the transcript.

Is there any way to circumvent this issue, or get a more descriptive transcript? Else I would consider this a bug.

Didn't find what you were looking for?

New post
Was this post helpful?
0 out of 0 found this helpful

Comments

4 comments

  • Hi Thomas,

    You are right. In the 'Get Conversation Node' not the whole conversation transcript is available.
    Also with the OData analytics endpoint (https://docs.cognigy.com/ai/tools/analytics/odata-analytics-endpoint/) or the OpenAPI viewer (https://api-trial.cognigy.ai/openapi) the entire transcript is not available.

    However, in Cognigy Insights (https://docs.cognigy.com/insights/cognigy-insights/), in the transcript explorer the entire script is available, but you cant use it in a flow, or send it in an Email.

    This is not a bug, but would be a good feature request to get the complete conversation exported in some way

     

    I will submit a feature request for this.

    Reg,
    Edwin

    0
  • Hi Edwin,

    After a bit more messing around I found out that the transcript can actually be obtained via the OData analytics endpoint. The chatHistory variable has all the required information, though some work is needed to parse it and the user responses (e.g. formatting Quick Replies Nodes). So I was able to generate a transcript from this data by calling it in an HTTP Request Node.

    Still, I agree that obtaining a transcript could be a nice feature :) The Extension transcript is a bit limited.

    Regards,

    Thomas

    0
  • Hi Thomas,

    I am glad to hear that you figured out a way to do it.
    How did you clean up the data you got from the from the buttons in the OData endpoint? Care to share?

     

    Reg,

    Edwin

     

     

    0
  • Hi Edwin,

    Sure, I can share the HTTP Request Node configuration and the JS code I wrote in my Code Node. Given time I would wrap this functionality in an Extension and make it more robust, but we're in an MVP phase so time is a bit short :)

     

    Here are the HTTP Request Node settings. Be aware that anyone with Read access on this Flow can retrieve the Odata API key - hence the need for an Extension to encrypt these credentials in production.

     

    The relevant IDs are retrieved from the input. I haven't yet tested this for all endpoints, though.

    I hope the comments explain how the code works. I ripped the HTML table styling from the Get Conversation node styling.

    /**
     * Author: Thomas Maaiveld (thomas.maaiveld@cgi.com)
     */

    /**
     * Sort messages by their timestamp string in ascending order.
     * @param   {Array(object)} messagesArray 
     * @returns {Array(object)} 
     */
    function sortTranscript(messagesArray) {
        return messagesArray.sort((a,b) => a.timestamp > b.timestamp ? 1 : -1)
    }


    /**
    * Removes messages that were marked for omission Cognigy. Such messages include a variable in the Data field ("includeInTranscript: false").
     * @param   {Array(object)} messagesArray 
     * @returns {Array(object)} 
     */
    function scrubTranscript(messagesArray) {
        return messagesArray.filter(el => !(JSON.parse(el.inputData)?.includeInTranscript === false))
    }


    /**
     * Takes an item from an Odata transcript and extracts the source, timestamp and message. 
     * @param   {object} messageObject An object from an array in a Cognigy Odata conversation transcript
     * @returns {object}               Object with source, timestamp, and message text
     */
    function parseMessageObject(messageObject, truncate=true) {

        let inputData = JSON.parse(messageObject.inputData)
        let source = messageObject.source
        let timestamp = formatTimestamp(messageObject.timestamp)

      // if the input text is not null, return it. Else, return the fallback text.
        let message = ""

        if(!(messageObject.inputText === null)) {
            message = messageObject.inputText
        } else {
            message = getFallbackText(inputData)
        }

        return {"source": source, "timestamp": timestamp, "text": message}
    }


    /**
     * 
     * @param   {string} odataTimestamp Source timestamp string
     * @returns {string}                Formatted timestamp (hh:mm:ss)
     */
     function formatTimestamp(odataTimestamp) {
        return odataTimestamp.slice(11,19)
    }


    /**
     * Obtains the (fallback) text in a Cognigy message object.
     * @param   {object} jsonObject Cognigy message object
     * @returns {string}            (Fallback) text of message
     */
    function getFallbackText(jsonObject) {

        var flattenedJSON = Object.assign(
            {}, 
            ...function _flatten(o) { 
                return [].concat(...Object.keys(o)
                .map(k => 
                    typeof o[k] === 'object' ?
                    _flatten(o[k]) : 
                    ({[k]: o[k]})
                )
            );
            }(jsonObject)
        )
        
        if(Object.keys(flattenedJSON).includes('fallbackText')) {
            return flattenedJSON['fallbackText']
        } else if(Object.keys(flattenedJSON).includes('text')){
            return flattenedJSON['text']
        } else {
            return "<message text missing>"
        }
    }


    /**
     * Legibly truncates messages longer than maxChars characters.
     * @param   {string} message  Message to truncate
     * @param   {int}    maxChars Characters after which to truncate
     * @returns                   Truncated string followed by "..." (if longer than maxChars)
     */
    function truncateTLDR(message, maxChars = 140) {
        return `${message.slice(0,maxChars)}${message.length > maxChars? '...':''}`
    }


    /**
     * Tabulate an array of parsed messages and return the table as HTML code
     * @param {Array(object)} parsedMessages 
     * @returns               A formatted table HTML string
     */
    function buildTableFromTranscript(parsedMessages) {
        return formatTableHeader(parsedMessages.map(formatTableRow).join(""))
    }


    function formatTableHeader(tableContents) {
        return `<table style=\"font-family: arial; border-collapse: collapse; outline: thin solid;\">${tableContents}</table>`
    }


    function formatTableRow(rowContents) {

        var bg = ``
        if(rowContents.source=="bot") {bg = ` background: #DDDDFF;`}

        return `<tr class=\"${rowContents.source}\" style=\"padding: 8;${bg}\">${
            formatTableCells(rowContents)
        }</tr>`
    }


    function formatTableCells(rowContents) {
        return `<td class=\"${rowContents.source}-time\" style=\"padding: 8; font-weight: bold;\">${rowContents.timestamp}</td>` +
               `<td class=\"${rowContents.source}-source\" style=\"padding: 8;\">${rowContents.source}</td>` + 
               `<td class=\"${rowContents.source}-text\" style=\"padding: 8; font-weight: bold;\">${rowContents.text}</td>` 
    }


    // -----

    // Retrieve the transcript
    const transcriptArray = ci.odataTranscript?.result?.value

    // sort the messages in time and remove any messages marked for deletion
    var sortedTranscript = scrubTranscript(sortTranscript(transcriptArray))

    // extract the message (inputText if it is present, fallback if it is not)
    var parsedMessages = sortedTranscript.map(parseMessageObject)

    // Format the parsed messages as an HTML table to embed in the email
    var htmlTable = buildTableFromTranscript(parsedMessages)

    api.addToInput("transcriptHTMLTable", htmlTable)
     
    0

Please sign in to leave a comment.