Home

Awesome

Notes on Rasa 3.x/ 2.x / 1.x

  1. The main branch of this repo is compatible with Rasa Open Source version 3.x
  2. The bot for Rasa 1.x can be found in the rasa-1 branch.

Financial Services Example Bot

This is an example chatbot demonstrating how to build AI assistants for financial services and banking. This starter pack can be used as a base for your own development or as a reference guide for implementing common banking-industry features with Rasa. It includes pre-built intents, actions, and stories for handling conversation flows like checking spending history and transferring money to another account.

<!-- START doctoc generated TOC please keep comment here to allow auto update --> <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

Table of Contents

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

Install dependencies

Run:

pip install -r requirements.txt

To install development dependencies:

pip install -r requirements-dev.txt
pre-commit install
python -m spacy download en_core_web_md en
python -m spacy link en_core_web_md en

With pre-commit installed, the black and doctoc hooks will run on every git commit. If any changes are made by the hooks, you will need to re-add changed files and re-commit your changes.

Run the bot

Use rasa train to train a model.

Then, to run, first set up your action server in one terminal window, listening on port 5056:

rasa run actions --port 5056

Note that port 5056 is used for the action server, to avoid a conflict when you also run the helpdesk bot as described below in the handoff section.

In another window, run the duckling server (for entity extraction):

docker run -p 8000:8000 rasa/duckling

Then to talk to the bot, run:

rasa shell --debug

Note that --debug mode will produce a lot of output meant to help you understand how the bot is working under the hood. To simply talk to the bot, you can remove this flag.

You can also try out your bot locally using Rasa X by running

rasa x

Refer to our guided workflow in the Wiki page for how to get started with Rasa X in local mode.

Overview of the files

data/nlu/nlu.yml - contains NLU training data

data/nlu/rules.yml - contains rules training data

data/stories/stories*.yml - contains stories training data

actions.py - contains custom action/api code

domain.yml - the domain file, including bot response templates

config.yml - training configurations for the NLU pipeline and policy ensemble

tests/ - end-to-end tests

Things you can ask the bot

The bot currently has five skills. You can ask it to:

  1. Transfer money to another person
  2. Check your earning or spending history (with a specific vendor or overall)
  3. Answer a question about transfer charges
  4. Pay a credit card bill
  5. Tell you your account balance

It also has a limited ability to switch skills mid-transaction and then return to the transaction at hand.

For the purposes of illustration, the bot recognises the following fictional credit card accounts:

It recognises the following payment amounts (besides actual currency amounts):

It recognises the following vendors (for spending history):

You can change any of these by modifying actions.py and the corresponding NLU data.

If configured, the bot can also hand off to another bot in response to the user asking for handoff. More details on handoff below.

Handoff

This bot includes a simple skill for handing off the conversation to another bot or a human. This demo relies on this fork of chatroom to work, however you could implement similar behaviour in another channel and then use that instead. See the chatroom README for more details on channel-side configuration.

Using the default set up, the handoff skill enables this kind of conversation with two bots:

<img src="./handoff.gif" width="200">

Try it out

The simplest way to use the handoff feature is to do the following:

  1. Clone chatroom and Helpdesk-Assistant alongside this repo
  2. In the chatroom repo, install the dependencies:
yarn install
  1. In the chatroom repo, build and serve chatroom:
yarn build
yarn serve
  1. In the Helpdesk-Assistant repo, install the dependencies and train a model (see the Helpdesk-Assistant README)
  2. In the Helpdesk-Assistant repo, run the rasa server and action server at the default ports (shown here for clarity) In one terminal window: 
    rasa run --enable-api --cors "*" --port 5005 --debug
    In another terminal window: 
    rasa run actions --port 5055 --debug
  3. In the Financial-Demo repo (i.e. this repo), run the rasa server and action server at the non-default ports shown below In one terminal window: 
    rasa run --enable-api --cors "*" --port 5006 --debug
    In another terminal window: 
    rasa run actions --port 5056 --debug
  4. Open chatroom_handoff.html in a browser to see handoff in action

How it works

Using chatroom, the general approach is as follows:

  1. User asks original bot for a handoff.
  2. The original bot handles the request and eventually sends a message with the following custom json payload: 
        {
        "handoff_host": "<url of handoff host endpoint>",
        "title": "<title for bot/channel handed off to>"
        }
    This message is not displayed in the Chatroom window.
  3. Chatroom switches the host to the specified handoff_host
  4. The original bot no longer receives any messages.
  5. The handoff host receives the message /handoff{"from_host":"<original bot url">}
  6. The handoff host should be configured to respond to this message with something like, "Hi, I'm <so and so>, how can I help you??"
  7. The handoff host can send a message in the same format as specified above to hand back to the original bot. In this case the same pattern repeats, but with the roles reversed. It could also hand off to yet another bot/human.

Bot-side configuration

The "try it out" section doesn't require any further configuration; this section is for those who want to change or further understand the set up.

For this demo, the user can ask for a human, but they'll be offered a bot (or bots) instead, so that the conversation looks like this:

For handoff to work, you need at least one "handoff_host". You can specify any number of handoff hosts in the file actions/handoff_config.yml.

handoff_hosts:
    helpdesk_assistant:
      title: "Helpdesk Assistant"
      url: "http://localhost:5005"
    ## you can add more handoff hosts to this list e.g.
    # moodbot:
    #   title: "MoodBot"
    #   url: "http://localhost:5007"

Handoff hosts can be other locally running rasa bots, or anything that serves responses in the format that chatroom accepts. If a handoff host is not a rasa bot, you will of course want to update the response text to tell the user who/what they are being handed off to.

The Helpdesk-Assistant bot has been set up to handle handoff in exactly the same way as Helpdesk-Assistant, so the simplest way to see handoff in action is to clone Financial-Demo alongside this repo.

If you list other locally running bots as handoff hosts, make sure the ports on which the various rasa servers & action servers are running do not conflict with each other.

Testing the bot

You can test the bot on the test conversations by:

This will run end-to-end testing on the conversations in tests/test_stories.yml.

All tests must pass.

Rasa X Deployment

To deploy financial-demo, it is highly recommended to make use of the one line deploy script for Rasa X.

As part of the deployment, you'll need to set up git integration to pull in your data and configurations, and build or pull an action server image.

Action Server Image

You will need to have docker installed in order to build the action server image. If you haven't made any changes to the action code, you can also use the public image on Dockerhub instead of building it yourself.

To build & tag the image, run:

export ACTION_SERVER_DOCKERPATH=<dockerID>/<name-of-image>:<tag-of-image>
make docker-build

Run the action server container:

make docker-run

Perform a smoke test on the health endpoint:

make docker-test

Once you have confirmed that the container is working, push the container image to a registry:

# login to a container registry with your credentials
docker login  

# check the registry logged into
docker system info | grep Registry

# push the action server image
make docker-push

CI/CD

Summary

As explained in the Setting up CI/CD section of the Rasa documentation, Continous Integration (CI) is the practice of merging in code changes frequently and automatically testing changes as they are committed. Continuous Deployment (CD) means automatically deploying integrated changes to a staging or production environment. Together, they allow you to make more frequent improvements to your assistant and efficiently test and deploy those changes.

A CI/CD pipeline is used to test, build and deploy the financial-demo bot to AWS EKS.

The pipeline uses GitHub Actions, defined in .github/workflows/cicd.yml. It includes these jobs:

params

params_summary

action_server

rasa_model

aws_eks_create_test_cluster

deploy_to_test_cluster

deploy_to_prod_cluster

GitHub Secrets

Secrets can be added to your GitHub repository by going to Settings > Secrets and selecting New repository secret.

When entering values, be sure to omit quotes.

AWS IAM User API Keys:

To configure the aws cli in Github Actions, create IAM User API Keys as described below, and add them as GitHub secrets to the repository:

Rasa Enterprise License:

To define a pull secret in Github Actions for the private GCR repo, you'll need to retrieve the private values from your Rasa Enterprise license file (docs) and add them as GitHub secrets to the repository:

An alternative approach to GCR repo authentication would be with the gcloud credential helper.

Helm chart Credentials

To use safe_credentials in Github Actions through values.yml (docs), add following GitHub Secrets to the repo, replacing each <safe credential> with a different alphanumeric string and choosing a <username> for the initial user.

(Please use safe credentials to avoid data breaches)

AWS Preparation

The CI/CD pipeline of financial-demo uses AWS for all the storage & compute resources.

After cloning or forking the financial-demo GitHub repository you must set up the following items before the pipeline can run.

IAM User API Keys

The CI/CD pipeline uses the aws cli.

The aws cli needs a set of IAM User API keys for authentication & authorization:

In your AWS Console, go to the IAM dashboard to create a new set of API keys:

SSH Key Pair

To SSH into the EC2 worker nodes of the EKS cluster, you'll need an SSH Key Pair

Local AWS CLI

Before the CI/CD pipeline can run, you will use the AWS CLI locally to create some resources and need to install & configure the CLI locally.

Install AWS CLI v2

See the installation instructions.

Configure your AWS CLI

# AWS CLI version 2.1.26 or later
aws --version

# Configure AWS CLI
aws configure
AWS Access Key ID [None]: -----          # See above: IAM User API Keys
AWS Secret Access Key [None]: -------    # See above: IAM User API Keys
Default region name [None]: us-west-2    # The CI/CD pipeline uses us-west-2 
Default output format [None]: 

# Check your configuration
aws configure list [--profile profile-name]

# verify it works
aws s3 ls

ECR repository & S3 bucket

The CI pipeline creates two artifacts:

Run these commands to create the storage for these artifacts. These commands run the AWS CLI and create an ECR repository and an S3 bucket:

# create ECR repository with name `financial-demo`
make aws-ecr-create-repository

# create S3 bucket with name `rasa-financial-demo`
make aws-s3-create-bucket

EKS production cluster

If the production cluster is already set up, you can skip this section.

This section describes the initial deployment of the financial-demo bot on an EKS production cluster.

This initial deployment is done manually. After that, the deployment is maintained & upgraded automatically by the CI/CD pipeline.

Preparation

Before you can create or interact with an EKS cluster, you must install eksctl, kubectl, helm, jp, and define some environment variables.

Install eksctl

See the installation instructions

If you use Ubuntu, you can issue the command:

make install-eksctl

Install kubectl

See the installation instructions

If you use Ubuntu, you can issue the command:

make install-kubectl

Install helm

See the installation instructions

If you use Ubuntu, you can issue the command:

make install-helm

Install jp

See the installation instructions

If you use Ubuntu, you can issue the command:

make install-jp

Set environment variables

There are many ways to set the required environment variables in your local environment.

One way is to create a file ./secret/envs_export, with this content:

# source this file to set the environment variables the same as the GitHub secrets.
export GCR_AUTH_JSON_PRIVATE_KEY_ID=...
export GCR_AUTH_JSON_PRIVATE_KEY='-----BEGIN PRIVATE KEY-...-END PRIVATE KEY-----\n'
export GCR_AUTH_JSON_CLIENT_EMAIL='...'
export GCR_AUTH_JSON_CLIENT_ID=...
export GLOBAL_POSTGRESQL_POSTGRESQLPASSWORD=...
export GLOBAL_REDIS_PASSWORD=...
export RABBITMQ_RABBITMQ_PASSWORD=...
export RASAX_DEBUG_MODE=false
export RASAX_INITIALUSER_USERNAME=admin
export RASAX_INITIALUSER_PASSWORD=...
export RASAX_JWTSECRET=...
export RASAX_PASSWORDSALT=...
export RASAX_TOKEN=...
export RASA_TOKEN=...

Then, create the environment variables with the command:

source ./secret/envs_export

Create the EKS cluster

Create the EKS production cluster, with name financial-demo-production:

make aws-eks-cluster-create AWS_EKS_CLUSTER_NAME=financial-demo-production

Some other useful 'Makefile functions' to interact with an EKS cluster:

make aws-eks-cluster-list-all

# in commands below, default name of cluster = financial-demo-<current branch name>
#
make aws-eks-cluster-describe AWS_EKS_CLUSTER_NAME=<name of cluster>
make aws-eks-cluster-describe-stacks AWS_EKS_CLUSTER_NAME=<name of cluster>
make aws-eks-cluster-delete AWS_EKS_CLUSTER_NAME=<name of cluster>

Configure kubeconfig

Add the EKS cluster information to ~/.kube/config, and set the current-context to that cluster:

make aws-eks-cluster-update-kubeconfig AWS_EKS_CLUSTER_NAME=financial-demo-production

# make sure kubectl is now looking at the correct EKS cluster
make kubectl-config-current-context

Install/Upgrade Rasa Enterprise

Versions

Select compatible versions

Get compatible versions for rasa-x, rasa and rasa-sdk from compatibility matrix

For the rasa-x-helm chart, the latest version should work. When installing a very old version, you might need to use an older helm chart version, selected with:

#.............NAME   URL
helm repo add rasa-x https://rasahq.github.io/rasa-x-helm
helm repo update

# .................NAME  /CHARTNAME
$ helm search repo rasa-x/rasa-x --versions
NAME         	CHART VERSION	APP VERSION	DESCRIPTION                                 
rasa-x/rasa-x	2.0.0        	0.41.1     	Rasa X Helm chart for Kubernetes / Openshift
rasa-x/rasa-x	1.16.0       	0.40.1     	Rasa X Helm chart for Kubernetes / Openshift
..
rasa-x/rasa-x	1.7.0        	0.33.0     	Rasa X Helm chart for Kubernetes / Openshift
rasa-x/rasa-x	1.6.13       	0.30.1     	Rasa X Helm chart for Kubernetes / Openshift
# NOTE: APP VERSION = Rasa X version

rasa & rasa-sdk

# file: requirements.txt
rasa[spacy]==2.5.2
rasa-sdk==2.5.0

# file: Dockerfile
FROM rasa/rasa-sdk:2.5.0

rasa-x & rasa-x-helm

# file: Makefile

RASAX_TAG := 0.39.3
RASAX_HELM_CHART_VERSION := 2.0.0

Build & push action server docker image

Build, run, test & push the action docker server image, with name:

# check out the `main` branch.
# -> the branch name is used as the image tag
git checkout main

# build the image
make docker-build

# test it
make docker-run
make docker-test
make docker-stop

# Login & push the docker image to the AWS ECR repository
make aws-ecr-docker-login
make docker-push

Install/Upgrade Rasa Enterprise

Install/Upgrade Rasa Enterprise, using the action server docker image that was uploaded to the ECR repository:

# make sure kubectl is looking at the correct EKS cluster
make aws-eks-cluster-update-kubeconfig AWS_EKS_CLUSTER_NAME=financial-demo-production
# check it
make kubectl-config-current-context  AWS_EKS_CLUSTER_NAME=financial-demo-production

# check out the `main` branch.
# -> The action server docker image with tag = current branch name will be installed
git checkout main

# if you want to re-install rasa enterprise completely from scratch:
make rasa-enterprise-uninstall
make rasa-enterprise-delete-pvc-all
make aws-eks-namespace-delete

# create namespace `my-namespace`
make aws-eks-namespace-create

# create/refresh gcr-pull-secret, for Rasa Enterprise image
make pull-secret-gcr-create

# create/refresh ecr-pull-secret, for action server image
make pull-secret-ecr-create

# Install/Upgrade Rasa Enterprise with the action server
make rasa-enterprise-install

# Check Rasa Enterprise Health
make rasa-enterprise-check-health

# To troubleshoot, highly recommended to use OCTANT, see Appendix C

Train, test & upload model to S3

# check out the `main` branch.
git checkout main

# Train the model: `models/<current branch>.tar.gz`
make rasa-train

# In another window, start duckling server
docker run -p 8000:8000 rasa/duckling

# Run the end-to-end tests
make rasa-test

# Upload `models/<current branch>.tar.gz` to S3
# Note: This does not mean the model is deployed to Rasa Enterprise,
#       which is done in the next step.
make aws-s3-upload-rasa-model

Deploy, Tag & Smoketest the trained model

# Configure ~/.kube/config and set current_context
make aws-eks-cluster-update-kubeconfig  AWS_EKS_CLUSTER_NAME=financial-demo-production

# Deploy rasa model
make aws-s3-download-rasa-model
make rasa-enterprise-model-delete
make rasa-enterprise-model-upload
make rasa-enterprise-model-tag

# Wait about 1 minute, so rasa-production can download, upack & load the model

# Smoketest
make rasa-enterprise-smoketest

DNS

Optionally, if you want to access Rasa Enterprise at your own (sub)-domain name, define a DNS record of type CNAME with your domain service provider:

It might take some time for things to propagate, but you can verify it with commands like nslookup & dig:

########################################################################
# use `nslookup -type`
nslookup -type=CNAME aws-financial-demo.my-domain.com
aws-financial-demo.my-domain.com canonical name = ---.us-west-2.elb.amazonaws.com.

########################################################################
# use `dig`
dig CNAME aws-financial-demo.my-domain.com
...
;; ANSWER SECTION:
aws-financial-demo.my-domain.com. ---- IN CNAME ---.us-west-2.elb.amazonaws.com.

Once propagated, you can access Rasa Enterprise at http://aws-financial-demo.my-domain.com

Trouble Shooting

If something goes wrong with the CI/CD pipeline, you should:

The following steps will most likely reveal what is going wrong:

Cleanup of AWS resources

The CI/CD pipeline creates unique artifacts and infrastructure for each branch. This is all cleaned up once a branch is deleted, by a special workflow .github/workflows/cleanup.yml.

You can also do a manual cleanup, either from the AWS CLI, or from the AWS Console.

From the command line:

From the AWS console:

Appendix A: The AWS EKS cluster

We use eksctl to create the clusters. It creates many AWS resources with CloudFormation templates.

Some of these AWS resources are:

The cluster context is also added to the ~/.kube/config file.

The VPC created looks something like this:

The EKS Control Plane interacts with the the EKS Data Plane (the nodes), like this:

img

Appendix B: OCTANT

Octant is a useful open sourced tool for visualizing workloads inside the cluster and troubleshooting issues when they arise.

Install Octant

Installation instructions

Install on Ubuntu

cd ~
mkdir octant
cd octant
wget https://github.com/vmware-tanzu/octant/releases/download/v0.20.0/octant_0.20.0_Linux-64bit.deb
sudo dpkg -i octant_0.20.0_Linux-64bit.deb

Run Octant

# #################################################### #
# Configure kubectl to look at the correct EKS cluster #
# #################################################### #

# -> for a feature branch test cluster
git checkout my-feature-branch

# -> for the production cluster
git branch production # create temporary, local `production` branch
git checkout production

# Configure kubeconfig for the correct EKS cluster
make aws-eks-cluster-update-kubeconfig

# make sure kubectl is now looking at the correct EKS cluster
make kubectl-config-current-context

# #################################################### #
# Run Octant on default port 7777 and open the browser #
# #################################################### #
OCTANT_LISTENER_ADDR=0.0.0.0:7777 octant &

# Within octant, select the namespace `my-namespace`

Appendix D: AWS EKS references

There are two commands to create AWS EKS clusters; eksctl & aws eks.

The following references are useful for learning about AWS EKS, and it is highly recommended to manually build test-clusters with both the eksctl & aws eks commands to help demystify the AWS resources that are being generated: