SAP Conversational AI is an NLP powered bot builder platform by SAP to improve user experience by a conversational approach.
In this blog post I would like to share some of the tips and configuration guides I have learned and implemented recently which helped me to improve the user experience. There are two types of bots offering by SAP Conversational AI out of the box: FAQ bot and Action bot. FAQ bot is designed to be much simpler than the Action bot and to serve only FAQ cases. However, there are still a lot of configuration capabilities which can improve the overall user experience.
Below, I will list several tips and configuration guides which I have tried recently collected from SAP colleagues and my personal experience. Please feel free to comment on them and share your experience in the comment section below.
I have also created an open sample FAQ bot to make all the configuration mentioned below available. You can access this bot using your SAP ID by this link.
💡Note: Currently, the link is providing access only to the build configuration and the data and bot test environment is only accessible for the collaborators. If you want to have a collaborator access, please let me know.
In this blog post it has been assumed that you have a basic understanding and hands-on experience of SAP Conversational AI. If that is not the case, please first visit the following learning materials:
The last update of this blog post has been done based on the last version of SAP Conversational AI until October 2021. If you are using a later version, some tips and screenshots might look different and need changes and adjustments.
In addition, the following configuration and guides are provided as an example to show SAP Conversational AI capabilities and are not provided by SAP as official best practices.
In case you have several distinct resources for answering the questions, and generating question and answers takes a massive effort, there are few options to facilitate it. One of them is to use the known keywords to generate few simple questions around them.
For this you can create a new pair for each of your resources. As the questions you can use a list of specific keywords which are related to the resource. Then as the answer you can provide a link to your resource. The link can be provided using Markdown. Here for more details about markdown in SAP Conversational AI.
SAP Conversational AI FAQ bot by default has the capability to provide the three best match answers whenever the highest confidence score is below a certain value (90 percent by default). If your bot’s knowledge domain is not tightly bounded usually users faced situations that none of the providing options are answering their question. In this case, it will make the discussion continuing if you add another option like “None of Them”.
By this additional option you can provide more helping answer if the user chose that option. It can be also beneficial during bot monitoring and we can easily track when the bot couldn’t provide appropriate options.
Step 1: Add a new option to the three questions available
Step 2: Add your hashtag (functional word) to your question answer pairs to make sure it will not interfere with other skills
1. Go to Train tab
2. Upload a new sample file name it “functional-words”
💡Note: It can be any other name, or you can add this pair to the other files you already have
3. Add your functional word as question and any fallback message you want as an answer
💡Note: Skills like small-talk and customer-satisfaction-reply have some predefined triggering words. If you are not using them, you can disable them to reduce the possibility of their false activation.
💡Note: If you want to avoid showing satisfaction questions after functional words, you can add conditions to your faq skill to exclude them when confidence score is high. Then have another action group to propose the answer if the user query is a functional word. You can find an example of it in the sample FAQ bot provided for this blog post.
Always showing the best 3 options is not looking good. Sometimes the confidence score for the second or the third option is very low and providing them to the user will cause unsatisfaction.
In SAP Conversational AI we can configure our faq skill in a way to check the confidence score for all the options and only show the options which are more relevant. For that we can only create one Action Group with sophisticated customized message or just easily create three different Action Group for when we want to answer with three options, two options, or even one option. In this post I am going to describe the second approach.
Step 1: Add conditions to the current option Action Group
Step 2: Add another Action Group for two options
Step 3: Add another Action Group for one option
💡Note: Confidence scores configured here are based on the default scores. You may need to slightly adjust them based on your business case. You can find your best fit by monitoring your user log and see if any adjustment is needed.
Sometimes the confidence score for the first option is higher than 90% but the second or third option also have a very high confidence score. If your questions have a lot of similarities and overlap it might happen more frequently. In this case you can create a new Action Group to provide other options as an additional message if the confidence score is high enough for the other options.
1. Go to the Build tab
2. Click on faq skill
3. Add a new Action Group
4. Set the conditions and message as shown below to propose one additional option
💡Note: The second if-statement is to exclude the condition that we will propose two additional options
5. Add another new Action Group
6. Set the conditions and message as shown below to propose two additional options
💡Note: As the faq answers are sorted by confidence score, we do not need to check the confidence score of the other option.
In addition to configuring welcome message and messages in small-talk Skill, it would improve the user experience if your bot is also capable of answering some general questions. Some examples to be added as question answer pairs are shown below:
💡Note: Too many small-talk capabilities also may cause negative consequences. The faq skill might offer those questions during disambiguation as well. Might not appear appropriate, if a potential disambiguation option would be “Tell me a joke”.
There are some points in the chat flow that your bot cannot provide the desired answer like when the confidence score is lower than 5%, when the user has been clicked on “None of the above” option, or when the user had a negative feedback caught in customer-satisfaction-reply skill.
In this situation, one of the possible additional services could be providing a search result based on the user initial query. We can do this in SAP Conversational AI by calling APIs and formatting the responses. Below you can see an example of using SAP Search API to provide search result from SAP open access information when the confidence score is lower than 5%.
As SAP Search API requires a JWT token for search query, we first need to consume an API to get the token and save it in the memory and then call another API to post our query and get the search result. At the end we just need to format the response body and show the user few search results (e.g., the first five titles and links).
💡Note: To describe it in more details please read my second blog post, Call SAP Search APIs in SAP Conversational AI to Enrich your Fallbacks.
SAP Conversational AI FAQ bot is providing a customer-satisfaction-prompt skill out of the box. Here we just want to make it looks a bit simpler and more user friendly. Therefore, instead of asking a question about user satisfaction and provide buttons as yes and no, we are going to just show like and dislike button after providing each answer.
In the following sample there is also a heart button which can be mapped to a special answer from your bot, and you may track it using the SAP Conversational AI Monitor tab to get energized with the users’ feedback.
Step 1: Create custom message in customer-satisfaction-prompt skill
1. Go to the Build tab
2. Click on customer-satisfaction-prompt skill
3. Click on Choose a Message Type
4. Select Custom
5. Add the configuration shown below
6. Delete the default message
💡Notes:
1. In some user interfaces like JAM there would need to include an empty title in the custom message to show the buttons. Test if your user interface is working without it, delete the “title” property from the custom message to make it looks better.
2. To include the heart button, you need to define a special function word or hashtag like #SPECIAL_BIGHEART and add it to your functional words (described in section 2) with a specific response message to answer your user’s feedback.
There are some cases that you may need to create a Negative List in your bot to make a more meaningful user interaction. For instance:
In these cases, it makes sense to add a few pairs of questions and an answer indicating that your bot is not covering those topics yet or point the user to another help desk.
Prepare a regression test, either manually or automatically to see if your new changes are not affecting your previous skills and setup. For instance, you can have a list of questions for each condition e.g., high confidence answer, three options, two options, and fallback to see if the provided answer is still what you have expected.
💡Note: There is a new Test feature in the Enterprise edition of the SAP Conversational AI which is helping to create your test easily. This feature is available for both action and FAQ bots. More info here.
If in the answers, you want to point to a resource like a pdf, it is always better to provide a direct link to the section or page (or minute of a video) if it is possible. Users usually get lost when they are directed from a bot to a big resource. A direct link to the desired part of the resource can save user’s time and increase user satisfaction.
For instance, in pdf documents usually there is an option to make a direct link to each specific page by adding #page=<pagenumber> at the end of the URL. You can also generate your pdf files in a way to consider the section as destinations (possible in Microsoft Office) and create a direct link to the destination by adding #[destination name] at the end of the URL. More info about pdf deep link here.
SAP Conversational AI FAQ bot is generally the best SAP offering when you want to implement an FAQ bot. It is simple and ready to use after uploading a list of pair of question and answers. However, in some cases there are other bot architectures to be recommended.
For instance, if the knowledge boundary that your bot is going to represent is tightly limited, using an Action bot can result same or better performance which is also providing wider configuration and skill capabilities. In this case you just need to define intends as your questions which can trigger a skill to answer the question. This can help you to have more control on the chat-flow.
Moreover, if you are thinking of having two different bots in one bot or benefit from the both world of FAQ bot and Action bot, you can call SAP Conversational AI bots inside SAP Conversational AI bots in a secure way using API service and OAuth-based Authentication. For more information, look at this SAP blog post written by harinder.singh.batra.
There are still some challenges we are facing using SAP Conversational AI FAQ bot. Please feel free to add your comments on them and discuss on possible solutions or any similar experience you had.
This was my first SAP blog post, and it would have never happened if SAP colleagues didn’t have such an openness and helping culture. To get to this point, there were several SAP colleagues who openly shared their experience and knowledge with me and helped me to realize SAP Conversational AI capabilities. Therefore, I would like to thank gerald.reinhard, SuccessFactors Product Advisory and Partner Success team, albert.neumueller, patrick.heinze, abishek.suvarna, paultormey, Leigha Aherne, giacomo.gasperini, david.duncan, and several other colleagues I may have missed.
Here you can find more SAP Conversational AI related resources and communication channels to ask your questions:
Follow Mohammadhadi Shadmehr on LinkedIn
In this blog post I would like to share some of the tips and configuration guides I have learned and implemented recently which helped me to improve the user experience. There are two types of bots offering by SAP Conversational AI out of the box: FAQ bot and Action bot. FAQ bot is designed to be much simpler than the Action bot and to serve only FAQ cases. However, there are still a lot of configuration capabilities which can improve the overall user experience.
Figure 1 The two types of bots that SAP Conversational AI is currently offering. I am referring Action bot to the left one and FAQ bot to the right one. This is a screenshot from the current version of SAP Conversational AI.
Below, I will list several tips and configuration guides which I have tried recently collected from SAP colleagues and my personal experience. Please feel free to comment on them and share your experience in the comment section below.
Open Access Sample FAQ Bot
I have also created an open sample FAQ bot to make all the configuration mentioned below available. You can access this bot using your SAP ID by this link.
💡Note: Currently, the link is providing access only to the build configuration and the data and bot test environment is only accessible for the collaborators. If you want to have a collaborator access, please let me know.
Perquisitions
In this blog post it has been assumed that you have a basic understanding and hands-on experience of SAP Conversational AI. If that is not the case, please first visit the following learning materials:
- Blog Post: Build Your First FAQ Chatbot with SAP Conversational AI
- Tutorial: Build Your First Chatbot with SAP Conversational AI
- OpenSAP Course: How to Build Chatbots with SAP Conversational AI
Disclaimer
The last update of this blog post has been done based on the last version of SAP Conversational AI until October 2021. If you are using a later version, some tips and screenshots might look different and need changes and adjustments.
In addition, the following configuration and guides are provided as an example to show SAP Conversational AI capabilities and are not provided by SAP as official best practices.
1- Use Keywords to Generate Initial Questions
In case you have several distinct resources for answering the questions, and generating question and answers takes a massive effort, there are few options to facilitate it. One of them is to use the known keywords to generate few simple questions around them.
For this you can create a new pair for each of your resources. As the questions you can use a list of specific keywords which are related to the resource. Then as the answer you can provide a link to your resource. The link can be provided using Markdown. Here for more details about markdown in SAP Conversational AI.
Figure 2 Example usage of keywords for an Implementation Design Principles (IDP) document. It is better to set the resource title as the stared question.
2- A Fourth Option for Disambiguation Questions (None of them)
SAP Conversational AI FAQ bot by default has the capability to provide the three best match answers whenever the highest confidence score is below a certain value (90 percent by default). If your bot’s knowledge domain is not tightly bounded usually users faced situations that none of the providing options are answering their question. In this case, it will make the discussion continuing if you add another option like “None of Them”.
By this additional option you can provide more helping answer if the user chose that option. It can be also beneficial during bot monitoring and we can easily track when the bot couldn’t provide appropriate options.
How to?
Step 1: Add a new option to the three questions available
- Go to the Build tab
- Click on faq skill
- Go to the second Action Group (the Action Group which providing options as answer)
- Click on edit
- Click on Add Quick Reply
- Add a name for your option
- Add a functional word as value
- Functional word could be a hashtag or any text which you know is not like any business word you have or anything that will trigger other skills
- Save it
Figure 3 A fourth option has been added here as None of Them option
Step 2: Add your hashtag (functional word) to your question answer pairs to make sure it will not interfere with other skills
1. Go to Train tab
2. Upload a new sample file name it “functional-words”
💡Note: It can be any other name, or you can add this pair to the other files you already have
3. Add your functional word as question and any fallback message you want as an answer
Figure 4 A sample message for None of Them situation.
💡Note: Skills like small-talk and customer-satisfaction-reply have some predefined triggering words. If you are not using them, you can disable them to reduce the possibility of their false activation.
💡Note: If you want to avoid showing satisfaction questions after functional words, you can add conditions to your faq skill to exclude them when confidence score is high. Then have another action group to propose the answer if the user query is a functional word. You can find an example of it in the sample FAQ bot provided for this blog post.
3- Only Show the Options Which Are Related
Always showing the best 3 options is not looking good. Sometimes the confidence score for the second or the third option is very low and providing them to the user will cause unsatisfaction.
In SAP Conversational AI we can configure our faq skill in a way to check the confidence score for all the options and only show the options which are more relevant. For that we can only create one Action Group with sophisticated customized message or just easily create three different Action Group for when we want to answer with three options, two options, or even one option. In this post I am going to describe the second approach.
How to?
Step 1: Add conditions to the current option Action Group
- Go to the Build tab
- Click on faq skill
- Add the additional conditions as presented in the screenshot below
Figure 5 Configuration for offering 3 options
Step 2: Add another Action Group for two options
- Configure as shown below
Figure 6 Configuration for offering 2 options
Step 3: Add another Action Group for one option
- Configure as shown below
Figure 7 Configuration for offering only one option
💡Note: Confidence scores configured here are based on the default scores. You may need to slightly adjust them based on your business case. You can find your best fit by monitoring your user log and see if any adjustment is needed.
4- Show More Options if the Confidence Score of the Next Options is High Enough
Sometimes the confidence score for the first option is higher than 90% but the second or third option also have a very high confidence score. If your questions have a lot of similarities and overlap it might happen more frequently. In this case you can create a new Action Group to provide other options as an additional message if the confidence score is high enough for the other options.
How to?
1. Go to the Build tab
2. Click on faq skill
3. Add a new Action Group
4. Set the conditions and message as shown below to propose one additional option
Figure 8 Proposing additional option when the confidence score is high enough
💡Note: The second if-statement is to exclude the condition that we will propose two additional options
5. Add another new Action Group
6. Set the conditions and message as shown below to propose two additional options
Figure 9 Proposing two additional options when confidence score is high
💡Note: As the faq answers are sorted by confidence score, we do not need to check the confidence score of the other option.
5- Add Few General Questions to Make the Bot More Fun
In addition to configuring welcome message and messages in small-talk Skill, it would improve the user experience if your bot is also capable of answering some general questions. Some examples to be added as question answer pairs are shown below:
Figure 10 Examples for general question answer pairs.
💡Note: Too many small-talk capabilities also may cause negative consequences. The faq skill might offer those questions during disambiguation as well. Might not appear appropriate, if a potential disambiguation option would be “Tell me a joke”.
6- Call Search APIs to Enrich your Fallbacks
There are some points in the chat flow that your bot cannot provide the desired answer like when the confidence score is lower than 5%, when the user has been clicked on “None of the above” option, or when the user had a negative feedback caught in customer-satisfaction-reply skill.
In this situation, one of the possible additional services could be providing a search result based on the user initial query. We can do this in SAP Conversational AI by calling APIs and formatting the responses. Below you can see an example of using SAP Search API to provide search result from SAP open access information when the confidence score is lower than 5%.
Figure 11 Providing search result when confidence score is low, showing in the SAP Conversational AI chat preview.
As SAP Search API requires a JWT token for search query, we first need to consume an API to get the token and save it in the memory and then call another API to post our query and get the search result. At the end we just need to format the response body and show the user few search results (e.g., the first five titles and links).
💡Note: To describe it in more details please read my second blog post, Call SAP Search APIs in SAP Conversational AI to Enrich your Fallbacks.
7- Like, Dislike Button as a Feedback for All Provided Answers
SAP Conversational AI FAQ bot is providing a customer-satisfaction-prompt skill out of the box. Here we just want to make it looks a bit simpler and more user friendly. Therefore, instead of asking a question about user satisfaction and provide buttons as yes and no, we are going to just show like and dislike button after providing each answer.
In the following sample there is also a heart button which can be mapped to a special answer from your bot, and you may track it using the SAP Conversational AI Monitor tab to get energized with the users’ feedback.
Figure 18 New feedback style showing in the SAP Conversational AI chat preview.
How to?
Step 1: Create custom message in customer-satisfaction-prompt skill
1. Go to the Build tab
2. Click on customer-satisfaction-prompt skill
3. Click on Choose a Message Type
4. Select Custom
5. Add the configuration shown below
Figure 19 A sample custom message for customer satisfaction question
6. Delete the default message
💡Notes:
1. In some user interfaces like JAM there would need to include an empty title in the custom message to show the buttons. Test if your user interface is working without it, delete the “title” property from the custom message to make it looks better.
2. To include the heart button, you need to define a special function word or hashtag like #SPECIAL_BIGHEART and add it to your functional words (described in section 2) with a specific response message to answer your user’s feedback.
Figure 20 Add a new pair to your functional words to reply when user is sending heart to you.
8- Negative List
There are some cases that you may need to create a Negative List in your bot to make a more meaningful user interaction. For instance:
- In case you know there will be few topics that will raise soon but your bot is not covering the related information
- In case your bot is getting frequent unrelated questions
In these cases, it makes sense to add a few pairs of questions and an answer indicating that your bot is not covering those topics yet or point the user to another help desk.
Figure 21 An example of negative list from.
9- A Regression Test
Prepare a regression test, either manually or automatically to see if your new changes are not affecting your previous skills and setup. For instance, you can have a list of questions for each condition e.g., high confidence answer, three options, two options, and fallback to see if the provided answer is still what you have expected.
💡Note: There is a new Test feature in the Enterprise edition of the SAP Conversational AI which is helping to create your test easily. This feature is available for both action and FAQ bots. More info here.
10- Deep Link to the Resources
If in the answers, you want to point to a resource like a pdf, it is always better to provide a direct link to the section or page (or minute of a video) if it is possible. Users usually get lost when they are directed from a bot to a big resource. A direct link to the desired part of the resource can save user’s time and increase user satisfaction.
For instance, in pdf documents usually there is an option to make a direct link to each specific page by adding #page=<pagenumber> at the end of the URL. You can also generate your pdf files in a way to consider the section as destinations (possible in Microsoft Office) and create a direct link to the destination by adding #[destination name] at the end of the URL. More info about pdf deep link here.
Other Architectures
SAP Conversational AI FAQ bot is generally the best SAP offering when you want to implement an FAQ bot. It is simple and ready to use after uploading a list of pair of question and answers. However, in some cases there are other bot architectures to be recommended.
For instance, if the knowledge boundary that your bot is going to represent is tightly limited, using an Action bot can result same or better performance which is also providing wider configuration and skill capabilities. In this case you just need to define intends as your questions which can trigger a skill to answer the question. This can help you to have more control on the chat-flow.
Moreover, if you are thinking of having two different bots in one bot or benefit from the both world of FAQ bot and Action bot, you can call SAP Conversational AI bots inside SAP Conversational AI bots in a secure way using API service and OAuth-based Authentication. For more information, look at this SAP blog post written by harinder.singh.batra.
Some Remaining Challenges
There are still some challenges we are facing using SAP Conversational AI FAQ bot. Please feel free to add your comments on them and discuss on possible solutions or any similar experience you had.
- Facilitate FAQ generation out of documents:
- Generating FAQs quickly and reduce the work to create those as good as possible.
- Update FAQs based on resource updates:
- If you are creating an FAQ bot to answer questions based on several resources, you might have this challenge to keep your bot updated with lower maintenance cost while the resources are updating.
Acknowledgments
This was my first SAP blog post, and it would have never happened if SAP colleagues didn’t have such an openness and helping culture. To get to this point, there were several SAP colleagues who openly shared their experience and knowledge with me and helped me to realize SAP Conversational AI capabilities. Therefore, I would like to thank gerald.reinhard, SuccessFactors Product Advisory and Partner Success team, albert.neumueller, patrick.heinze, abishek.suvarna, paultormey, Leigha Aherne, giacomo.gasperini, david.duncan, and several other colleagues I may have missed.
More Info
Here you can find more SAP Conversational AI related resources and communication channels to ask your questions:
- Another OpenSAP course: Next-Level Chatbot Building with SAP Conversational AI
- FAQ bot video tutorial: How to Build an FAQ Chatbot with SAP Conversational AI
- SAP Community topic page for SAP Conversational AI
- SAP Conversational AI Q&A tag link
- Help Portal
Follow Mohammadhadi Shadmehr on LinkedIn
- SAP Managed Tags:
3 Comments
