Remember when I asked Ox for its favourite Adele song and it said ‘Hello’? Well, now I’ve added hundreds of lines of code and it can still do it!

>>> ox.tell(“What’s your favourite Adele song?”)
Hello

This time, it’s not just printing ‘Hello’ in response to every input. Instead, it takes apart the question and (to some extent) works out what it means. It searches for all the songs that it knows are by Adele, compares how much it likes them, and says the name of the one it likes most.

a simpler task

In order to explain this, let’s start with the simpler question, “What’s your favourite song?” When Ox reads this question, it searches its vocabulary and finds these two relevant chunks:

The first chunk has a gap in it designed to receive the name of a category, like ‘song’, so it slots the two chunks together to build a sequence that matches what you asked it.

the context

Ox keeps track of the conversation using a list of items called the context. This contains information about what’s salient to the people in the conversation. For example, if you were to talk about a song, and then later say ‘it’, Ox needs to know that ‘it’ refers to the song. Ox can’t do that yet, but that’s the kind of thing the context will be used for.

Now, back to the blue “What’s your favourite” chunk from before. This chunk exists as a piece of code in Ox’s knowledge base. We’ve seen that it contains a sequence of words and a gap for categories; it also includes some instructions for how it should be understood. Like all questions, it puts these instructions in a box marked ‘expectation’ and adds them to the context. This is a way for Ox to remember, when it’s deciding what to say next, that the person talking to it expects an answer to their question.

So here’s what we’ve got so far:

the conversation strategy

Now, just because you understand the question, does that mean you’re automatically going to answer it? Well, yes, for now. But maybe in the future we’ll want Ox to be more sophisticated, so I’ve given it a ConversationStrategy component which at the moment just looks in the context for the first expectation it can find and tells Ox, ‘Deal with this next’.

relations

The next step is to carry out the instructions in the blue box – that is, to work out what Ox’s favourite song is. Ox has two kinds of thing in its knowledge base: there’s a pile of concepts (these are the ‘chunks’ we’ve been talking about already), and there’s a pile of relations that link these concepts together. Here are some examples of relations:

Each relation has a type and a list of arguments. The first one says that ‘Hello’ is a song (I could also use the ‘is a’ relation type in the future to teach Ox that a song is a work of art and that a work of art is a… whatever a work of art is). The second relation says that Ox likes ‘Hello’, and gives a numerical score to show how much. Ox also likes ‘Someone Like You’, but the score argument in that relation is only 900, so it likes ‘Hello’ more.

finding Ox’s favourite

Now we have all the pieces we need for Ox to answer the ‘favourite song’ question: it collects together a list of songs by looking for ‘is a’ relations that link things to the Song category, then it looks for ‘likes’ relations that link Ox itself to those songs. It compares the numeric scores in those ‘likes’ relations to establish that Hello is the one it likes the most, and finally it prints out the sequence of text associated with that concept, which of course is just ‘Hello’.

compound nouns

Now let’s go back to the slightly more complicated question, “What’s your favourite Adele song?” Again, Ox searches its vocabulary and finds the same two relevant chunks:

And following the patterns above, I can extend its vocabulary to include Adele, perhaps with a category ‘singer’ and an ‘is a’ relationship to connect them:

But Ox still isn’t prepared to interpret the question. The blue chunk needs a category to fill its slot, so Ox needs to understand that ‘Adele song’ is a category. And of course, it needs to know that it isn’t the same category as ‘song’ – what if its favourite Adele song is ‘Hello’ but its favourite song overall is ‘Shape of You’ by Ed Sheeran? One solution is to create a new concept for ‘Adele song’ that behaves much like ‘song’, but since language allows infinite combinations, that approach is going to get unwieldy. Also it misses an opportunity for Ox to infer the meaning of a new phrase from components that it already knows.

‘Adele song’ is a compound noun, like ‘water feature’ or ‘kitchen table’. In each of these cases, the parts of the compound noun have different relations to each other. We have a song by Adele, a feature using water, a table in the kitchen. I think when we recognise the compound noun construction, that only tells us that the first noun has some relationship with the second – we then have to use our world knowledge to work out what the relationship is. But in each case, we know we have a kind of song, a kind of feature, a kind of table. So to capture these rules, let’s add them to Ox’s vocabulary as a new item:

Like the “What’s your favourite” chunk from earlier, there are slots here to be filled with other items from the vocabulary. Unlike that other chunk, though, the compound noun concept has no words of its own. It’s just a framework from which we can hang the instructions for working out what it means.

So now Ox once again has enough vocabulary to piece together what you’ve asked it:

It works out the answer in the same way as before, except that this time there’s an extra step where it follows the instructions attached to the purple box. The output of the purple box is a list of songs. To make this list, Ox goes through the things it knows about and checks:

1. 1. Does the thing have an ‘is a’ relation with Song?
   2. Does the thing appear in any kind of relationship where one of the other arguments is a concept associated with the text ‘Adele’?

So now we need to teach Ox one more relation and we’re there:

Why bother with all this?

Now Ox has just enough equipment to answer those two questions. It can’t talk about anything else, but the exercise has given us a sketch of the way conversation works. The next step is to be able to tell Ox things as well as asking things – if Ox can remember the things it’s told, then it can start to learn. So far I’ve had to hard-code everything Ox does, which is a lot of work. But I hope that the more it learns, the more it will be able to do that work for itself.

Remember that you can always see the code for Ox at the github repository: https://github.com/TommyHerbert/ox

A knowledge base is a collection of information in a form that allows computers to make inferences. Ox will need a knowledge base of common sense and linguistic information. On the common sense side, lots of work has been done since I was last active in natural language processing. On the linguistic side, some aspects are well served by existing work: for example, there are lots of resources if you want to find all the synonyms of a noun. But there are other tasks that I haven’t found research about yet, like how to relate a compound noun like ‘Adele song’ to what you know about songs.

Here are some existing knowledge bases:

Yago: https://gate.d5.mpi-inf.mpg.de/webyagospotlx/WebInterface
ConceptNet: http://conceptnet.io/
WebChild: https://gate.d5.mpi-inf.mpg.de/webchild/
Knowledge Graph: https://developers.google.com/knowledge-graph/
NELL: http://rtw.ml.cmu.edu/rtw/
Reverb: http://openie.allenai.org/

Facebook’s new ParlAI platform looks like it might be aimed at doing something along the lines of Ox. It’s a Python library with concepts like ‘learner’ and ‘teacher’. I should definitely explore a bit more. Not sure if other people are working with it – no one seems to have cited it in its first five months.

>>> import ox
>>> ox.tell('Hello, Ox.')
Hello.
>>> ox.tell("What's your favourite Adele track?")
Hello.
>>> ox.tell("What's your favourite hunting call?")
Hello.

Right now, that’s all it can say.

https://github.com/TommyHerbert/ox

Not dated, but the latest reference is 2012:

Click to access amine.pdf

Proposal for a natural-language question answering system. Narrow domain, hand-built interpretation rules, no plan to allow the user to teach things to the machine.

I think Ox will be an online chatbot with an ontology on the back end. It should be able to rewrite its own linguistic rules in response to user input. Here are two points of reference:

owllang

OntBot

“You call him the dumb ox, but in his teaching he will one day produce such a bellowing that it will be heard throughout the world.”

Project Ox will create a chatbot that passes the Turing Test. Right now, it consists of this post.