* * *

Chatbot

A chatbot is a software system, which can interact or “chat” with a human user in natural language (such as English, or whatever other language the chatbot has been built on).

Chatbots can help inform a user, or help them with fulfilling a task. 

Different approaches to making a Chatbot

  • Rule Based

In a rule-based approach, a bot answers questions based on some rules on which it is trained on. The rules defined can be very simple to very complex. One of such language is AIML (Artificial Intelligence Markup Language). Also, writing rules for different scenarios is very time taking and it is impossible to write rules for every possible scenario. The bots can handle simple queries but fail to manage complex queries. A simple rule based chatbot can be constructed using basic NLP and making sense of query by looking in words used in it and keeping in mind nature of sentence and person in which it is.

  • Sequence to Sequence Models

Sequence To Sequence model is one of the strongest Deep NLP techniques available, used commonly in Translation, Question-Answering systems and Chatbots. It consists of two RNNs (Recurrent Neural Network): An Encoder and a Decoder. The encoder takes a sequence (sentence) as input and processes one symbol(word) at each timestep. Its objective is to convert a sequence of symbols into a fixed size feature vector that encodes only the important information in the sequence while losing the unnecessary information. You can visualize data flow in the encoder along the time axis, as the flow of local information from one end of the sequence to another.

There are quite a few challenges in using this model.

  1. The model cannot handle variable length sequences.

  2. Vocabulary size: - The decoder has to run softmax over a large vocabulary of say 20,000 words, for each word in the output. That is going to slow down the training process, even if your hardware is capable of handling it.

  3. The biggest problem is that it requires a large amount of data to train it on. And for specific problems like the one give “Coffee Shop Sales Executive”, its very difficult to get that amount to data to train our model. But once done it gives very good results.

  • Dynamic Memory Networks

A Dynamic Memory Network (DMN) is a neural network architecture optimized for question-answering (QA) problems. Given a training set of input sequences (knowledge) and questions, it can form episodic memories, and use them to generate relevant answers. Whilst classic Encoder-Decoder (Seq2Seq) models can solve QA problems, their performance is limited by the small size of their ‘memory’ - this is what’s encoded by their hidden states and weights, and reflects the information that’s passed between encoder and decoder. This limitation becomes especially apparent when dealing with very long sequences of data, as we might find in sources like books or videos, where the salient facts might occur a long time apart in very different contexts.

The problem here also lies with large data required to train model, but it is most efficient model till date.

Backend Architecture

Databases of all the customers need to be maintained on server storing their order history, which will be used to predict the order of customer when it arrives. For each customer, mobile number can be taken as unique attribute and used for indexing. So User profile will store:

  • Mobile Number(primary key)

  • Name

  • Sex

  • Age

  • Preference of coffee

The prediction can be done in two ways:

1. Statistical

Maintaining the count of different types of coffees purchased with their counts, updating on every order. The prediction will be simply coffee with max count.

2. ML Based

An ML based system can be maintained to predict the next order of customer based on its historical data. It will be more accurate and can account more parameters than simply count.

Keeping things less complicated we can store following parameters for each order (for multiple orders, store each order as separate entry):

  • Timestamp (just of indexing)

  • Time of Day

  • Day of week

  • Weather (Simply cold, hot, warm, rainy)

  • Store ID(purchased from which store)

  • Coffee Name (to be trained for and predicted)

A similar system can be used to predict coffee for a new customer based on age, sex, weather, day time.

API:

GET /ChatBot(sentence): call be made from user interface when user enters a query. Returns Response to sentence.

The following API calls will be used to send and receive data from database. These will be made from inside the chatbot script when encountered with particular type of response

from user.

  • GET /prefered(mobile_number)

  • Return preferred coffee

  • POST /Create_user(user_data): makes a new user entry.

  • POST /order place(mobile_number, coffee): update user purchase data

  • GET /menu: Send json of menu. May be required at initialization of bot. Return menu

 

Frontend

The front can be a simple UI chat panel. It will take a query form user and send to server to get a response and display the response on same. The bot can be made to chat on FB on page so that customers can place orders or on coffee house’s own site. I didn’t have a https server so I was not able to make Facebook App, so I have a created a basic form type UI for conversation and testing.

Approach

Best Possible Approach:

The best approach for making any sort of chatbot as already mentioned earlier will be use Sequence to Sequence network or Dynamic Memory Network and train them on long conversation so that they can learn to talk. The only problem is data gathering. This can be solved with Data Generation Algorithms, which in turn again is a problem in its own.

Rule Based Approach:

If not provided with sufficient data to apply Deep NLP we can use Traditional NLP to create a rule based chatbot. As name suggests Rule based bot simply tries to map the query of user with one the conditions and then respond with an already defined response for that particular condition.

The Rules can vary from very simple like:

If: name of coffee present then,

place order for that coffee

To little complex like:

If: name of coffee present then,

look for whether it’s a question, query or order and then respond accordingly

The Basic Components will be following:

1. Classification Module

Tanning a full chat using ML/DL is not possible because of data restrictions but a small model can be trained to classify “Question”, “Negative”, “Positive”, “Request”, “Acceptance”, and “Rejection”. So this is initially specified what customer is trying to convey. Then for each category query can be treated differently.

2. Presence of Coffee name in query

  • If yes then,

                Separate all coffee with their respective counts

  • If cost related words are present then,

                Separate all coffee and tell prices separately

3. Presence of menu or menu related words in query

4. Presence of cost or cost related words. Each of modules from 2 to 4, can then decide response based on classification done by module 1. Also person of sentence can be used:

I  saying to computer and it’s about the user

You  asking computer to respond

Considering all these basic conversation can be achieved.

Past State:

The response also depends upon last question or answer. Like consider

Computer: Anything else sir,

Response: No thanks or Yes

The response of user will be basically yes or no, or their variants, so next output will be effected by this yes or no like,

For Yes  What sir

For No  Thank you

Response can be classified with help of Module 1, but a state full system needs to be made. Apart from menu or price the user may ask about details of a particular coffee, may ask sales person to suggest something or ask for specialty and in end may ask for bill.

To make bot personalize for user:

  • We can fetch user name, and use it in conversation

  • Fetch gender and use sir or ma’am

 

Approach Followed

I have tried to cover most of the points to a certain extent, but haven’t made the classification module so instead I have used TextBlob’s inbuilt sentiment analysis and tried to classify other classes of sentences by searching for most common words belonging to that particular class. My approach is stateless so It can’t be used to place two different orders at same time thought any number of coffees can be ordered in one go. This also forces to mention coffee name each time, this can be solved by maintaining a history of last suggestion/order made by bot and use that in case a Number is followed by it. The different decisions points are mentioned in code, to make it easily understandable. Also logs of conversations are also being stored, so that it can be used later for improving system. 

Personalizing Bot for User

The bot used randomly chosen phases to reply in general cases, the responses vary depending the query is first person or second and so response is set accordingly. If we know gender of user then use of sir/ma’am will make it more soft talker. Prediction and recommendation (mentioned above) will make bot more personalized for users.