Overview<\/h1>\n
All three of LabLabLab\u2019s games use a combination of\u00a0<\/span>ChatScript\u00a0<\/a><\/span>for the natural language component and a Unity made interface exported as a web player package. The frontend and backend communicate through the websocket protocol, which operate over TCP sockets.<\/span><\/p>\n Fig. 1, \u00a0Simplified diagram of the back\/front components<\/span><\/p>\n Before going further, one should definitely look-up introductory documentation both to ChatScript and Unity, so as to understand how they work from a high level: see this\u00a0Unity video<\/a><\/span>\u00a0for a quick introduction on how it handle 2D games; the ChatScript\u00a0Input to Output Overview<\/a><\/span>\u00a0should explain the basis, although to make to most of this document, I would recommend reading through the\u00a0Basic User Manual<\/a><\/span>.<\/p>\n Because most of \u00a0the content is based on the LabLabLab\u2019s games, the reader should also play a bit with\u00a0our games<\/a><\/span>\u00a0to understand their gameplay mechanics and how it relate to the technology we\u2019re using.<\/p>\n At the core of the project resides ChatScript, a software created and maintained almost solely by Bruce Wilcox since 2010. It is a complete chatbot engine, used to create complex and rich \u201cconversational robots\u201d, with whom the user can have believable natural language (english) conversations. It implement everything that one might need to do so: natural language parsing, extensive ontologies, conversation volley & rejoinder system, wildcard matching, etc.<\/p>\n While ChatScript is now well into version 6, the versions used by LabLabLab in production are 5.21 (A Tough Sell)\u00a0<\/span>and\u00a0<\/span>5.52 (SimHamlet & SimProphet).<\/p>\n ChatScript (CS) works by using a\u00a0control script<\/a><\/span>\u00a0to direct user inputs towards different topics, where they will be responded to (or not). The matching rules\/responses pairs are implemented using a mix of english, operators, modifiers and functions. Topics are thematically arranged, user-created collections of rules\/response pairs and keywords.<\/p>\n The control script is a topic as well, but one that is used as a dispatcher for other topics, which makes it a bit less readable. For most usage, the default control script should be sufficient and behave more or less like this:<\/p>\n When a response is found, the execution stops, the rule\/response pair is marked as used, and the response is sent to the user.<\/p>\n After determining which topic is the most relevant for a given input, CS parses this input to find a rule\/response pair. For a concise explanation of the matching patterns available in CS, one should refer to the\u00a0Pattern Redux<\/a><\/span>\u00a0documentation available from the\u00a0Github repository<\/a><\/span>. Of course, natural language forces us to assume that most of the time the input will use improper grammar or be incomplete, and CS makes provisions for this.<\/p>\n Let\u2019s look at how this come together in\u00a0SimHamlet<\/a><\/span>, first by looking at the file structure inside the\u00a0\/RAWDATA\/GRAVEDIGGER<\/span>\u00a0folder:<\/p>\n . Fig. 3, Directory structure of SimHamlet<\/span><\/p>\n First off, all CS files use the\u00a0.top<\/span>\u00a0extension. One can also see that we use prefixes to identify special files. In this case,\u00a0a_concepts.top<\/span>\u00a0is not a topic, but rather a place where we gathered all of our custom concepts. Since concepts can be compared to constant variables, using a single file to aggregate all of them curbs the problem of having to hunt for them in dozen of files when we want to change something. It is also important to note that having an\u00a0a_<\/span>\u00a0prefix insures that our concept file is compiled first and made available to files loaded after.<\/p>\n Our regular topics are simply named with the topic they cover, in this case mostly character names. If they would reference or depend on each other in specific ways, one would want to prefix them them with a number to dictate when they are individually compiled.<\/p>\n Finally, we have prefixed our system topics with\u00a0z_<\/span>, which means that they will be compiled after all other files. These files contain most of our script logic and concatenate our controls, making them more easily maintainable. Our control script obviously qualify as system topic (z_simplecontrole.top<\/span>), just as our \u201ccatch all\u201d topic do (z_catchall.top<\/span>).<\/p>\n Topics are paramount in ChatScript. From a structure standpoint, they should be viewed as classes in OOP: they are used to make the code manageable & reusable, while splitting it up in logical, thematic files (.top). This underline the importance of a structured outline, since the interactions will need to be divided and grouped together; the clearer the program outline, the easier it will be to break it into parts.<\/p>\n What these topics consist of depend solely on what kind of narrative arcs we want to create. For\u00a0SimProphet<\/a><\/span>\u00a0we used broad themes such as death, politics and sex to group our matching patterns. For\u00a0SimHamlet<\/a><\/span>\u00a0the topics were centered around the characters in the play: Ophelia, Laertes, Polonious and so on. Thorough, requirements from the narratives dictated the composition of topics, once again denoting the importance of mapping out conversational paths well before implementing them. Topics are built around keywords. Keeping in mind the control script outlined above (Fig. 2), consider this arrangement:<\/p>\n Fig. 4, Topics name and keyword<\/span><\/p>\n Assuming \u00a0that the current topic is\u00a0~intro<\/span>\u00a0and that the previous topic is\u00a0~car\u00a0<\/span>:<\/p>\n This further illustrate why catch all topics should always contain mostly generic rule\/responses & gambits, alongside permanent responses (see the\u00a0<\/a><\/span>^keep\u00a0<\/a><\/span>and\u00a0<\/a><\/span>^repeat<\/a><\/span>\u00a0functions<\/a><\/span>). This insures that all inputs are appropriately responded to, even when we find no match at all.<\/p>\n A different type of topics are system topics, which are used to collect rules that apply to the execution of our script. Marking these topics as system topic will make sure that they are prioritized, never \u201cforgotten\u201d and that their content will not be checked for spelling mistakes. The control script, for instance, is a system topic. By separating our logic code from our regular response topics, we insure that we can maintain our bot more easily and completely oversee the flow of execution through the control script.<\/p>\n System topics are very important to enforce structure and sanitize or process user input before responding to it. For example, a good candidate for a system topic would be a piece of script that parse the input for a certain word, replace it, and then direct the input the another topic for an actual response. For a list of topic flags (modifiers that can be added to your topics), see page 6 of the\u00a0Advanced User Manual<\/a><\/span>.<\/p>\n One way to view rules and response pairs is as\u00a0if<\/span>\u00a0statement\u00a0with words being concurrent boolean conditions: if the word is matched, we get a\u00a0true\u00a0<\/span>and move to the next part of the operation. If all operations add up to\u00a0true<\/span>, the bot will offer the response. This behavior appears naturally when we start adding literal boolean condition in our match patterns, so the bot respond only if certain conditions that might have nothing to do with the input are met.<\/p>\n If you have not already, have a look at the\u00a0Input to Output Overview<\/a><\/span>\u00a0to get a grasp of how content is matched and responded to in CS.<\/p>\n The simplest form of a rule \/ response pair could look like this:<\/p>\n While a slightly more complex example could look like this:<\/p>\n A bit clarification of the syntax if you are not familiar with it:<\/p>\n It is important to\u00a0remember that by default, CS will erase rule\/response pairs as they are used to avoid letting the bot repeat itself on similar or identical inputs; each rule matches an input will \u00a0be deleted for the duration of the conversation. This can be avoided on certain rule\/response by using the\u00a0^keep\u00a0<\/a><\/span>and\u00a0<\/a><\/span>^repeat\u00a0<\/a><\/span>function<\/a><\/span>.<\/p>\n This is also where execution flow becomes important: the files are parsed by CS from top to bottom, so what is at the top will always be tested out before what comes after.<\/p>\n Concepts are clusters of related and synonymous terms that we want considered as a group identified by a single term. Members of the concept could be argotic words, misspelled words, variations or even other concepts. CS will register a match on the concept if any of the terms are found in the input. In our CS files, this looks like this:<\/p>\n Concepts are the green\u00a0~word<\/span>. You can see that we are using the\u00a0IGNORESPELLING<\/span>\u00a0modifier, which will mark the terms in parenthesis as canonical and not validate their spelling against the dictionary when compiling the bot. For a list of system concepts (CS 6.3)\u00a0see this document<\/a><\/span>\u00a0and for more on concept modifiers, see page 6 (\u201cAdvanced Concept\u201d) of the\u00a0Advanced Manual<\/a><\/span>.<\/p>\n It is also important to realize that since concepts are parsed as arrays and can contain an almost infinite number of words and concepts themselves, they are most of the time interchangeable with regular, single words. Let\u2019s have a look at an example from SimProphet, first with the definition of a\u00a0~motanciens\u00a0<\/span>concept:<\/p>\n Fig. 6, Concepts in SimProphet<\/span><\/p>\n Which we might use like this:<\/p>\n Fig. 7, Using concepts in rules<\/span><\/p>\n On line\u00a0288<\/span>, the script will look at the system variable\u00a0%length\u00a0<\/span>and only go forward if it evaluate to more or equal to 3. Then it will simply look for any words in the\u00a0~motanciens<\/span>\u00a0concepts that is present in the input, and will output\u00a0\u201cYou\u2019re using it wrong\u201d<\/span>\u00a0if it finds one.<\/p>\n On line\u00a0289<\/span>, the script will look for any word that is present in\u00a0~motanciens<\/span>\u00a0and, if there is one, will evaluate to true AND captures the word (remember the what the underscore does). It then uses it in the response by using\u00a0the form\u00a0\u2018_0<\/span>. In practice this means that the conversation could go like this:<\/p>\n Player: \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0my god is super ulmash In other words, any matched word from the concept can be captured and reused as is, without reference to the concept it is part of.<\/p>\n There is four types of variable in CS:<\/p>\n As we\u2019ve seen, variables are vital to the progression of our game. They also make the responses more engaging and personal. This is quite obvious in\u00a0SimProphet<\/a><\/span>\u00a0as a lot the the content is generated by the user: god name, follower names, holidays, sacred places, etc. This create supplemental challenges, as some user input needs to be registered exactly as they are typed, even when they are not real words, while other need to be conjugated or made plural:<\/p>\n Fig. 8, Variables in SimProphet<\/span><\/p>\n I\u2019ve only included parts of the routine. Line\u00a068<\/span>\u00a0is the first line that is served to the user: the rule name\u00a0GODNAME\u00a0<\/span>and the lack of any conditional in the rule hint to the fact that this rule is invoked with the\u00a0^gambit<\/span>\u00a0function.<\/p>\n To make sure that the matched content will not be corrected, we change the parsing system by modifying the\u00a0$token<\/span>\u00a0system variable on line\u00a071,<\/span>\u00a0omitting\u00a0DO_SPELLCHECK<\/span>\u00a0for the duration of this rule (remember the linear execution), adding it back at the end on line\u00a0173<\/span>. Since this line is matched every time and used only to reset the\u00a0$token\u00a0<\/span>system variable, we move the respond to the next gambit available in the current topic with the\u00a0^gambit\u00a0<\/span>rule.<\/p>\n Now line\u00a0131<\/span>,\u00a0134<\/span>\u00a0&\u00a0141<\/span>. They are variation of the same need: we want to break apart each component of the god name, so we must capture them individually. First with evaluate the length, then capture exactly one word (_*1<\/span>) on line\u00a0131<\/span>. With the construct\u00a0\u2018_0<\/span>, we make sure to use the literal rather than canonical form. The splitting allows us to capitalize each word, then join them into our permanent\u00a0$godname<\/span>\u00a0variable (line\u00a0132<\/span>,\u00a0137<\/span>\u00a0&\u00a0145<\/span>).<\/p>\n After adding the content to\u00a0$godname<\/span>, we have three slightly different behaviors. In the first example (line\u00a0132<\/span>) we offer no response, so execution will move down to the next match in line (in this case, line\u00a0173<\/span>). In the second example (line\u00a0138<\/span>), assuming a\u00a0$godname\u00a0<\/span>variable of\u00a0\u201cBeetlejuice\u201d<\/span>, the character will respond\u00a0\u201cBeetlejuice? Never hard of it\u201d,\u00a0<\/span>then CS will affix the next gambit available in this topic. Finally, the last match will simply responds with the response in the\u00a0~god_identity<\/span>\u00a0topic named\u00a0INTRIGINGNAME\u00a0<\/span>(line\u00a0146<\/span>).<\/p>\n To illustrate other possible usage and how they affect the games\u00a0made at LabLabLab, let\u2019s look at the\u00a0z_pronouns.top\u00a0<\/span>file in\u00a0A Tough Sell<\/a><\/span>:<\/p>\n Fig. 9, Pronouns resolution in SimHamlet<\/span><\/p>\n None of these rules output a response, instead they parse the input for proper nouns and assign them to variables that will be used elsewhere. Since the control script detects that no response has been found, it keeps searching just as it normally would.<\/p>\n The\u00a0^refine<\/span>\u00a0function act as a filter that passes the query along further: it first looks for a match in\u00a0~noun_proper_singular<\/span>, and if it finds one move the exact same input to the rejoinders (lines\u00a09<\/span>\u00a0&\u00a010<\/span>) where they are further processed. This is not foolproof however, as the name & pronouns could be inferred or they could not be present in ontologies such as\u00a0~noun_proper_singular\u00a0<\/span>or\u00a0~noun_plural<\/span>. For example, someone playing A Tough Sell might have this hypothetical exchange:<\/p>\n Player: \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0This apple looks good. Trying to match the last sentence, which would be parsed as \u201cit be very red\u201d, might not trigger anything in the apple topic where we want to be looking: nowhere in the query does the word \u201capple\u201d appears. Our mechanism from above is more efficient combined with topic specific pronoun marking.<\/p>\n What we want to be doing essentially this: for all input in the current topic, replace all pronouns by what we think they point to. One way to do this is to simply parse and replace the terms:<\/p>\n Fig. 10, Pronoun marking<\/span><\/p>\n As you can tell by the line numbers, this is at the top of the file: all inputs that are made while in this topic, regardless of their terms, will go through these six lines. If any of the words highlighted in yellow are found, they will be replaced by\u00a0apple<\/span>. Adding the\u00a0^repeat<\/span>\u00a0and\u00a0^keep\u00a0<\/span>function insures that CS does not discard these lines after they have been matched once. Since there is no response given, the control script will automatically keep searching for a response.<\/p>\n This brings us to a specific need in the LabLabLab projects: in narrative driven games a lot more than in regular chatbots, it is central to keep track of the subject.<\/p>\n While chatbots need to follow certain narrative arcs to keep up with the conversation, keeping track of progress is a lot more important in games, as they move towards defined goals. How do you bring this behavior out in a stateless system and that is designed to juggle with multiple topic at once? Luckily for us CS has an extensive variable system. Let\u2019s break down the first part of a rule in\u00a0SimProphet<\/a><\/span>\u00a0that uses lots of variable, the rules that register the name of the followers of your god:<\/p>\n Fig. 11, Variable in SimProphet<\/span><\/p>\n This is a gambit named\u00a0NOMCROYANTS<\/span>\u00a0(line\u00a03<\/span>, note the\u00a0t:<\/span>), which means that the bot will offer this response if:<\/p>\n Again, let\u2019s assume that\u00a0$godname<\/span>\u00a0has been set to\u00a0\u201cBeetlejuice\u201d<\/span>\u00a0and walk through this rule:<\/p>\n Since variables can be either words or numerical values, we can use them to track and change gameplay properties and interface values, like it is done on line\u00a013<\/span>. As previously mentioned, this is very important in a game as an end state has to be reached. By using variables and changing them on certain inputs or responses, we collect data that we can feed back to the interface (as ways to keep the score) or the the chatbot itself (as a way to trigger responses).<\/p>\n We\u2019ve seen with variables how to capture, assemble and proceed after matching terms that we want to reuse. But what about when we want to modify them according to context? We need to rely on the\u00a0^pos<\/a><\/span>![\"\"](\"http:\/\/www.lablablab.net\/papers\/images\/image03.png\")
<\/p>\nChatScript<\/h1>\n
Fundamentals<\/h5>\n
\n
![\"\"](\"http:\/\/www.lablablab.net\/papers\/images\/image02.png\" "\\"\\"")
\nFig. 2, Execution of a basic CS control script<\/span><\/p>\nFile structure<\/h5>\n
\n\u251c\u2500\u2500 a_concepts.top
\n\u251c\u2500\u2500 claudius.top
\n\u251c\u2500\u2500 gertrude.top
\n\u251c\u2500\u2500 hamlet_jr.top
\n\u251c\u2500\u2500 hamlet_sr.top
\n\u251c\u2500\u2500 introduction.top
\n\u251c\u2500\u2500 laertes.top
\n\u251c\u2500\u2500 ophelia.top
\n\u251c\u2500\u2500 polonius.top
\n\u251c\u2500\u2500 rosencrantz_guildenstern.top
\n\u251c\u2500\u2500 z_catchall.top
\n\u251c\u2500\u2500 z_culprit.top
\n\u251c\u2500\u2500 z_pronouns.top
\n\u2514\u2500\u2500 z_simplecontrol.top<\/p>\nTopics<\/h5>\n
![\"\"](\"http:\/\/www.lablablab.net\/papers\/images\/image05.png\" "\\"\\"")
<\/p>\n\n
System topics<\/h5>\n
Rule and response pairs<\/h5>\n
u: (how be you) Fine thank you.<\/pre>\n
s: PETAPET(!not _~verb _[dog <\/span>cat]) Wait, you can _0 a _1?!<\/span><\/pre>\n\n
Concepts<\/h5>\n
![\"Selection_004.png\"](\"http:\/\/www.lablablab.net\/papers\/images\/image10.png\" "\\"\\"")
\nFig. 5, Concepts in SimHamlet<\/span><\/p>\n![\"Selection_007.png\"](\"http:\/\/www.lablablab.net\/papers\/images\/image07.png\" "\\"\\"")
<\/p>\n![\"Selection_006.png\"](\"http:\/\/www.lablablab.net\/papers\/images\/image06.png\" "\\"\\"")
<\/p>\n
\nSimProhet:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Sorry to be that guy, but you\u2019re misusing ulmash.<\/p>\nVariables<\/h5>\n
\n
![\"Selection_007.png\"](\"http:\/\/www.lablablab.net\/papers\/images\/image09.png\" "\\"\\"")
<\/p>\n![\"Selection_008.png\"](\"http:\/\/www.lablablab.net\/papers\/images\/image11.png\" "\\"\\"")
<\/p>\n![\"Selection_009.png\"](\"http:\/\/www.lablablab.net\/papers\/images\/image04.png\" "\\"\\"")
<\/p>\nSpecial Needs<\/h5>\n
![\"\"](\"http:\/\/www.lablablab.net\/papers\/images\/image01.png\" "\\"\\"")
<\/p>\n
\nSnow White:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Yes it does! I got it from the store.
\nPlayer: \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0It\u2019s very red.<\/p>\n![\"Selection_004.png\"](\"http:\/\/www.lablablab.net\/papers\/images\/image13.png\" "\\"\\"")
<\/p>\nProgression<\/h5>\n
![\"Selection_004.png\"](\"http:\/\/www.lablablab.net\/papers\/images\/image08.png\" "\\"\\"")
<\/p>\n\n
\n
Part Of Speech (POS)<\/h5>\n