Improvised Theatre Alongside Artificial Intelligences

New paper to be presented at 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment Conference.

This is a cross-post from, where you can find more information on past and future shows, videos, and pictures. Also, find HumanMachine on facebook and Twitter.

A new publication on “Improvised Theatre Alongside Artificial Intelligences” will be presented at the 13th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE’17) in Snowbird, Utah.

This study presents the first report of Artificial Improvisation, or improvisational theatre performed live, on-stage, alongside an artificial intelligence-based improvisational performer. The Artificial Improvisor is a form of artificial conversational agent, or chatbot, focused on open domain dialogue and collaborative narrative generation.

Mathewson (as Albert) and Mirowski (as mYleZ), alongside A.L.Ex. the improvisational theatre robot and an enthusiastic audience member at the Edinburgh Fringe Festival 2017 in Edinburgh, Scotland.

Using state-of-the-art machine learning techniques spanning from natural language processing and speech recognition to reinforcement and deep learning, these chatbots have become more lifelike and harder to discern from humans. Natural human conversations are seldom limited in scope and jump from topic to topic, they are laced with metaphor and subtext and face-to-face communication is supplemented with non-verbal cues.

Live improvised performance takes natural conversation one step further with multiple actors performing in front of an audience. In improvisation, the topic of the conversation is often given by the audience several times during the performance. These suggestions inspire actors to perform novel, unique, and engaging scenes. During each scene, actors must make rapid-fire decisions to collaboratively generate coherent narratives.

This new work introduces Pyggy and A.L.Ex. (Artificial Language Experiment), the first two Artificial Improvisors, each with a unique composition and embodiment. The paper highlights research and development, successes and failures along the way, celebrates collaborations enabling progress, and presents discussions for future work in the space of artificial improvisation.

Read the full paper

Building a Sextbot: botrotica

Building a sextbot for a smutnight art fest cabaret. Exploring the obscenity of interacting with artificial dialog systems.

In this post, I describe the development, deployment, and conclusions of a recent art-project-cum-coding-experiment called botrotica.

botrotica is a sextbot. It is a chat bot that is based on the sext messages that we send to each other. This sextbot is an exploration in sexting developed for futuresmut, a thematic cabaret at Nextfest 2017, an emerging artist festival in Edmonton, Alberta.


The internet enables us to explore our identities and develop social and communication skills. Technology has changed the way that we interact with each other. How do we share ourselves online when only one other is watching? Who do we share ourselves with and what exactly are we sharing? These hypothetical questions remind me of a favorite quote from a song I love:

The television went from being a babysitter to a mistress
Technology made it easy for us to stay in touch while keeping a distance
’til we just stayed distant and never touched
Now all we do is text too much
— Sage Francis, The Best of Times

Thoughts, ideas, pictures, and videos can all now be shared privately with passion partners. It has been found that rather than increasing intimacy in these types of relationships, sexting may act as a buffer for physical intimacy and may be associated with risky behaviour in teens and young adults [1, 2]. botrotica explores questions of sexual communication, instigation, privacy, and censorship.


botrotica is situated behind a phone number. Participants only need a mobile phone for interaction. This interactive cyber-installation piece turned-on for the duration of the futuresmut cabaret. If users texted the number they engaged the bot in a back and forth conversation which ranged from purely deterministic, preset dialog, to generative dialog pieces from a model trained on huge swaths of online erotica, to hooks into machine learning systems to classify objects, faces, details, emotions, and not-safe-for-work flags in the images (this was particularly engaging for those that engaged in sexy dialogs).

An early doodle of the physical installation… modeling the “lonely home”.

It was modeled to respond like a late night sexter. It would talk slightly coy and dirty, ask for (and send) photos (of sexy robots of course). The game was that if the user sent an image to the bot, then it would classify the objects in an image, thereby ‘objectifying’ the user.

Example classification, rating, and leaderboard based on my ‘classic’ gym swole sweaty selfie.

It would also classify potentially NSFW content, thereby rating the image, and it would assign scores to the images completely arbitrarily. Images that scored higher would be then completely randomly ranked on a fictitious leaderboard. For example, in response to a racy MMS botrotica responded:

73.42% explicit nudity. 37.27% graphic male nudity. I see a beautiful face. I am 70.0% sure. You look CONFUSED and a little bit CALM. I would guess you are somewhere between 35 and 52. Looks like a 2 dressed up as a 8… I see people, person, human, tattoo…

There was a pre-recorded message for those who call in to the number which prompted the human to leave a message and send an SMS for a faster response.

Advert cards littered around the physical space, with helloworld and the phone number on the card.

For physical advertising, I had little business cards printing with the phone number on them and the words… helloworld (an homage to the traditional computer program used to introduce novices to a new language or toolset). Behind the card on a table sat a single flower and a tv/vcr combo playing old adverts for phone sex lines with this phone number dubbed over top.

Realized physical installation at futuresmut.

During the night images from this video were interlaced briefly between scenes projected on the walls of the cabaret. For reference, this is what some of the images looked like:

Images from the advert video.


botrotica is a service that runs on a Twilio phone number. It is python code which runs a server on an Amazon EC2 server in a docker container with hooks to Amazon Rekognition image analysis. Additionally, there is a trained tensorflow model running for seeded generation, similar to the models used in my previous (and current) work in building an artificial improvisor. Of course, these models are trained with a (very) different dataset than the improvisational dialog agents.


The results were incredible with 847 messages (731 SMS + 136 MMS) over the course of a single evening (5.5 hours). Interactions varied from innocent to full on sexy-text-messaging exchanging images and dirty talk, back and forth between human and agent. There were 64 minutes of recorded voice messages from 144 voice calls to the number.

From human texts the system parsed and responded from a wide variety of ways. Some of these more engaging than others… similar to human-to-human communication.


botrotica served as an interesting window into the world sext messaging. It illuminated how popular simple chat interactions can be if prepared, produced, developed, and deployed appropriately and in interesting ways.

This project gave me many ideas for future projects. For example, building future interactions of photo scavenger hunts, collaborative photo classification games, live streaming logs and interactions to rooms full of interlocutors, collective collaborations, and formats for large group communication at events and parties.

What we share online changes as new technology lands. How and what I share on facebook is different from how I share on twitter, or this blog, or snapchat, or Youtube, etc. Early on in my creative career I wrote 15 correspondences entitled: “A series of letters to which I do not expect a response”. These letters were written to inanimate objects and famous individuals. In a way, writing these letters gave me a channel through which I could communicate with an embodiment of a concept.

I believe there is value in building these channels. It is not obscene (ala Joseph Weizenbaum), in fact, I think it is beautiful, to think that in the future we may communicate our lusts, thoughts, fears, desires, emotions, judgements, interests, hopes, dreams and memories to artificial agents with full knowledge of their non-sentience. In a more distant future one step beyond one could imagine these systems communicating back.


This project would not have been possible without the help from a small handful of incredible and helpful folks. Thanks go to Sam for making the event happen, curating, and inviting this piece into the event. Sarah for physical installations, discussion,  and deployment of physical interactions, idea storming. Lana for help on writing and aggregating texts and source material. Paul and Julian for testing. And every human who dove deep into experimenting and getting weird with the system on the evening of the event.


  1. Benotsch, Eric G., et al. “Sexting, substance use, and sexual risk behavior in young adults.” Journal of Adolescent Health 52.3 (2013): 307-313.
  2. Lenhart, Amanda. “Teens and sexting.” Pew Internet & American Life Project1 (2009): 1-26.

30 songs, 30 years.

A simple birthday request: tell me what song lies at the intersection of you and I.

As you may know, June 5 is the date of my birth… As you may not know, for the last several years I have attempted to curate very specific gifts from friends and loved ones all over the planet.

This year, my 30th, I would like you to submit a song, your own or one from popular culture. I am hoping to collect 30 songs, so that I may create a double-sided birthday mixtape. Feel free to submit the Artist-Title, the lyrics, or a YouTube link. It can be a song that you wrote, that you performed, or just one that you love.

These can be songs that I introduced you to, or that you introduced me to. They can be musical, or funny, or sad, or capture something in the connection that we share. These can be songs that are long or short, maybe we listened to them together one time, or it was that song that you can never remember the name of but makes you think of me.

Imagine that we are two points floating around in space; if you were to travel from one point to the other… what song would play in our minds the middle of your journey? What song lies at the intersection of you and I?

Feel free to comment on this post, send it to me on facebook, or email me privately to submit.

Some may ask: “Kory, can I submit more than one song?”,
“Thank you for asking, yes of course, but no more than three. And double points if you can tell a rich narrative with three songs.”

As a reminder looking back on past years…

Leadership in Improvisation

Making great teams work better together through creative collaboration.

Recently I was invited to give a talk on Leadership in Improvisation for the Peter Lougheed Leadership College at the University of Alberta breakfast event about Making Connections.

It was a delightful opportunity to share my story, and the audience was engaged, motivated and taught me some things in the process. It is my hope that by sharing the story here, the story can continue.

Note: It was critical to discuss the preparation of the talk with a sample of individuals who would be in the audience the week before, this allowed me to understand who would be in the room, what their expectations would be like, and what has worked well, and not as well, for that group in the past.

My name is Kory Mathewson; I am an improviser with Rapid Fire Theatre, and I am also a Ph.D. student in Computing Science studying artificial intelligence and machine learning at the University of Alberta.

If you want to read Part 1 of the story of me making an Artificial Improvisor, check it out here:

Note: it was a failure. A not a total failure, because I learned a lot.

I had four short and clear goals for the morning:

  • Teach the history of improvisation and
    its growing relevance in leadership.
  • Share my stories of success and failure.
  • Distil everything I have learned over 12 years of improvisation to 6 gemstones takeaways.
  • Explore some improvisational exercises with the audience.

The same question my mom asked when she found my brother (9) and I (6) driving in a convertible we didn’t own. Let’s start with a brief history of my improvisation that should help you understand me and how we got to this point?

February 2005 at the Rapid Fire Theatre Nosebowl High School Improv Tournament. A bright-eyed memory of myself performing impeccable mime in cargo shorts, a popped collar pink polo shirt and rolled up tennis socks, with three of my best friends supporting from the bench. We lost that tournament, by a mile but we gained memories and fashion sense.

Flash forward to this month, on stage at Rapid Fire Theatre. Performing alongside the incredibly talented cast and the Mayor of Edmonton His Worship Don Iveson.

Who here knows what improvisation is? Has anyone done improvisation? Has anyone seen a comedic improvisation show? Or performed jazz? Has anyone built an ingenious solution to a problem facing scarce resources and limited time?

Improvisation, much like my life, is art and science. It sits at the intersection between the rules and pure creativity.

By the end of this talk, I promised, you will all be familiar with improvisation and the ways that the principles of improvisation can help guide your leadership and collaboration. First, it is not about being funny, it is about failing together?

I earned my stripes improvising in long 50-hour marathons. These grueling time tests stretch physical and mental abilities and provide an incomparable safe playground of practice.

Practice makes perfect; but why? Because it elucidates your patterns, obstacles and growth goals, then focuses your efforts on improvements.

The more you can adapt, response, and extemporize the more effective you will be. Veterans of the form respond proactively to sudden changes and road blocks.

The first lesson I learned in improvisation is that “the best improv performers can improvise with anyone.” They make everyone else shine; this is not necessarily true about painters, musicians, mathematicians, or lawyers, but it is right about improvisors, and more importantly: leaders.

Once I was in, I was hooked. So I did what any newly committed student of an art form does, I dug deep into the history of improvisation.

The river ran deep. Documented improvisation starts in Rome in the 4th century BC with the Atellan Farce and mask work. Flash forward thousands of years and we learn about Commedia dell-Art(e) in the early 1600’s exploring character archetypes.

Contemporaries in the field have written, studied, tested, tried and failed at shaping the art form. Each with their form of proselytizing:

  • Stanislavski: The greatest wisdom is to realize one’s lack of it.
  • Spolin 1963: Through spontaneity, we are reformed into ourselves.
  • Boal 1973: Theatre is a form of knowledge; it should and can also be a means of transforming society. Theatre can help us build our future, rather than just waiting for it.
  • Johnstone 1987: If you have a good idea, open your mouth and say something else.

And of course, no discussion of improvisation would be complete without mentioned the tortured genius of the 20th century, Del Close. A coach and mentor of many of today’s most popular comedians including Tina Fey, Amy Poehler.

With many words of wisdom, one that resonates with me as a leader is:

“Don’t bring a cathedral into a scene, bring a brick and let’s build together.” – Del Close 

Bringing us to the first gemstone takeaway.

  • Listen
  • Actually, listen.
  • Most people listen just enough to be able to respond.
  • Be willing to change.
  • Listen like this is the last thing they ever say.
  • Listen to the other is going to change your mind.

“Listen. Listen to one another like you know you are scholars. Artists. Scientists. Athletes. Musicians. Like you know you will be the ones to shape this world.” – Sarah Kay

  • Positivity.
  • Accept, agree, say “Yes”.
  • Accept and offer, say “Yes, and…” – it improves your relationships with others AND with yourself.
  • It is also a base principle of meditation. Accept your currently emotional state. Yes I am feeling like this. If this is true, what else is true.

A series of yeses takes us somewhere. All it takes is one no to stop the momentum. But what do we get when we can say: “Yes, and…”

But wait, there’s more…Practice strengthening your affirmation:

  • Practice saying “Yes, and..”
  • Use it to refocus, redirect,
    collect thoughts.
  • Improvisational leaders accept quickly and look for opportunities immediately.
  • Make your agreement prominent.

Much like building a cathedral, we can try collaborating in real time.

You can try as well, with a partner try telling a story by writing it down on a piece of paper one-line-at-a-time, for an additional challenge try writing the story one-word-at-a-time.

Then read the stories back, and see what it is like to directly collaborate on a creative piece. Remember: there are no wrong answers, no mistakes, and no judgment.

You are walking alone in a wooded forest, you have been without food and water for a few days and you are cold, scared and hungry. You come to a clearning and in the middle of the open area there is a saber-tooth tiger… you are afraid. You feel fear. Scientists think this stems from the amygdala, two almond-shaped bundles of nuclei in the temporal lobes of the brain. What is your emotional response? 

Most would say fight or flight, there is also new research on the freeze mechanism, and of course, the fourth and least favorable option in real-life, fail. You have to do something very critical in this moment, which leads us to the next gemstone.

  • Make choices.
  • In improv we often call them offers.
  • Make offers instead of asking questions.
  • Make your choices specific, unique and novel for bonus points.

Often times when we are forced with a decision we encounter a psychological phenomenon called: analysis paralysis.

Remember that you do not need to be 100% right 100% of the time. In fact, you need to be 100% right only about 10% of the time, the other times you just need to make a decision.

Your choices should be made in an attempt to make others look good. Shelve the ego and embrace the collective elevation and amplification. Endow others with power, status, and agency to create a team that functions better than individuals working independently.

I have tried on 1000 masks and understand the world slightly differently from each perspective.

Alex Williams of the New York Times has a very nice piece on friendships as we progress through life. One of the more salient points to me is that the three elements required for making close friends.

These are the exact characteristics of my interactions with improvisers all over the world. From Liverpool to Austin, to Slovenia, to New York, and back through North America… I have found my community around the world. We build relationships over space and time because we can quickly adapt, work together, collaborate openly and communicate effectively.

My research is in Reinforcement Learning. Training artificially intelligent systems to act in certain ways given rewards as feedback. This is the same way that dolphins are trained to do mind-bending stunts.

You can play this with a partner as well. Attempt to encourage your ‘dolphin’ to accomplish an unstated goal in the environment using only rewards as feedback.

This simple exercise elucidates the importance of clear communication, systems of collaboration, and shared goals.

In life,we are each living our own story (or stories), but we are major and minor characters in many other stories. We are the character that will give the right piece of advice at the wrong time or the hidden romance that ends up falling out of love and stop writing letters right before a chapter ends.

If this is the case, we should aim to make our story interesting. Aim to make offers over questions, decisions over ultimatums, and bring something specific, unique, and novel to each and every interaction.

Stories are about patterns. One of the easiest ways to understand patterns is through images. First, an action is taken establishing normal, then, with a second similar action, the pattern is established thereby creating a solid platform. Finally, the pattern is broken.

We can share the storytelling by trading back and forth on who is setting the patterns and who is breaking them. Improvisation encourages this ebb and flow, the constant back and forth of transferring energy between leading and following.

These are the characteristics of an improvisational leader who can think, speak, and act freely on their feet:

  • Listen actively
  • Amplify with positivity
  • Confidence to make authentic choices
  • Bring out the best in others
  • Tell great stories

Finally, is the failure. The most under discussed reality of the modern leader.

Here is a challenge, for the next conversation you have with a mentor or a peer that you have yet to connect on a deeper level with, ask them:

What has been your biggest failure?

Then, focus on understanding the learning that came from that failure. 

So, does anyone remember the first ‘best’ piece of advice that I ever received?

The best improvisers, make everyone else shine. I wanted to put this to the test. So I did an improv show with the audience member with the least stage experience.

Spoiler alert: He was magnificent.

So I thought, can I do it with an artificial intelligence? I would fuse my love for improvisation and machine learning. I would call it something sexy like “artificial improvisation, ” and there could be a hot Hamlet sequel skull in the bionic arm of a cyborg robot from the future.


And then I would do it a whole bunch, and tell everyone I knew the story and find other people around the world that were similarly passionate.

And then make an art collective in the space, and book a swath of shows in 2017. Learning and growing along the way, building businesses and research that impact millions of people.

That is how I embraced one of my biggest failures as a performer and scientist.

Questions for reflections: What do all the gemstones add up to? What is the big key takeway?

Reinforcement Learning for Profit

Is RL being used in revenue generating systems today?

Recently, one of my facebook friends, and alumni of the University of Alberta (with a PhD in Computing Science), Cosmin Paduraru posed a question:

Where is Reinforcement Learning used in revenue generating systems today?

I have been thinking about this lots over the last month as I attended two international conferences on Artificial Intelligence and Machine Learning (ICML and IJCAI) in NYC, USA. It is important to explore future prospects both inside and outside academia — In case you need a catch up, I am currently at the University of Alberta working on a PhD in Computing Science with a focus on Reinforcement Learning and Artificial Intelligence.

With the success of modern AI systems — out of the winter and into the spring — many companies have invested and continue to invested heavily into modern AI systems, backed by teams of leading researchers in the field (e.g. Facebook, Google, Microsoft, IBM, Twitter, etc.).

With that said, maybe Cosmin is right, Reinforcement Learning (Sutton and Barto 1998, and this killer-intro by the fantastically talented Andrej Karpathy) is seemingly publicly underrepresented in currently deployed systems making money in the real world, or is it?

Adapted from Sutton and Barto 1998 and WALL-E
Adapted from Sutton and Barto 1998 and WALL-E

Luckily I was at the International Joint Conference on Artificial Intelligence where I was attending a panel discussion on The Business of AI, the panel was composed of all speakers from the industry day. A desirable venue to solicit a wide variety of opinions from thought leaders in the field.

So I posed the question to them, their responses went as follows:

Peter Norvig (Director of Research at Google): “well… AlphaGo made a million bucks and then gave it away”… a recent tweet from Demis Hassabis (Google DeepMind) confirms:

Peter Stone (Founder and President, Cogitai. Professor UoT (Austin)) gave lots of great examples of recent applications:

He said,“We are on the cusp of moving from the academic lab to the industry for RL, adaptation, and lifelong learning…We are at the cusp, and that is the main motivation from Cogitai”

He also referenced work by Thomas G. Dietterich on invasive species management, wildfire suppression, by Joelle Pineau on applying RL in healthcare, and by Andrew Ng and Drew Bagnall on helicopter control. All of these could be as a practical demonstrations of specific, developing industrial applications.

Hiroaki Kitano (President & CEO SONY Computer Science Laboratories) said that this is a current research area for Sony and to expect profitability using these and advancing RL algorithms in 2-5 years. Almost 10 years after Sony’s last robotic venture, the Aibo, Sony CEO Kazuo Hirai has just recently (late June 2016) said “the robots we are developing can have emotional bonds with customers, giving them joy and becoming the objects of love”.

Guruduth Banavar (Chief Science Officer, Cognitive Computing, IBM Research) predicted that this is going to happen, sooner rather than later, and his prediction was that it will happen in the domain of conversational systems, dialog systems, and understanding the larger context of conversations. He also mentioned that the illustrious Gerald Tesauro (the man behind TD-Gammon) is working on these problems. Interesting that he did not mention Watson

Some interesting answers from industry leaders. But I was surprised that no one mentioned: recommender systems (like those on Amazon, Netflix, Yelp, and nicely formalized as an RL problem in 2005 by Shani et al.), are these systems all collaborative filtering? Surely not.

No one mentioned that Google Reinforcement Learning Architecture (here is a quick summary), which I can only imagine could be behind some of the personal recommendations and rankings that Google does behind-the-scenes on Search, YouTube, and maybe … Maps?

No one mentioned contextual bandits, sometimes called associative RL (as discussed by Li et al. 2010 for news recommendation), for serving ads and news stories. These systems are surely deployed on large-scale news sites by the publishers to maximize click-through-ratios and create a personalized experience. Microsoft recently announced Multiworld Testing Decision Service, for making context based decisions… I guess there were no Microsoft representatives on the panel to toot this horn (thanks for the catch Pardis)

With so much potentially out there, why was there no mention of these use cases for reinforcement learning? Where else could RL be hiding in the money-making wild? RL seems like an ideal candidate for systems of personalization on large-scale, sequential decision-making problems… so what am I missing?

What’s your Doodle?

It is always surprising to see what people had ready to roll once they had a pen and a blank page.

Back in May 2006 (10 years ago, thanks @Meags for actually dating a doodle), I carried a notebook around with me and when I would have some down time with others I would ask them to share a doodle of theirs (something that they draw when they are on the phone, or in class, or a meeting) in this notebook.

I collected so many of these doodles, from many different individuals. It is always surprising to see what people had ready to roll once they had a pen and a blank page. It seems like everyone has a small doodle … so I ask….what’s your doodle?

Collective Improvisation: A review and thoughts…

Nerding out on improv theory with a bunch of notes on Collective Improvisation: From Theatre to Film by Sonja Vilc.

May 2016, I am invited to Llubjana, Slovenia to teach and perform improvisation. Hosted by the three heads of the Cerberus of Slovenian improvisation Juš, Vid and Peter at IGLU, I had an incredible time seeing what life was really like in Llubjana.

Screen Shot 2016-05-13 at 10.38.12 AM

The improvisational scene is alive and well in Llubjana, with monthly guests from around the world and some lively improv shows in Slovenian and English, it is truly a community worth venturing to for the adventurous improvisor. As well, word on the street is there is a Eurovision inspired Improvision show that draws hundreds and hundreds every year.

Upon departing, Juš was kind enough to give me a book as a gift. By a fellow Slovenian improvisor named Sonja Vilč, the book is Collective Improvisation: From Theatre to Film.

The book is composed of three chapters: 1) The Roots and Teachings of Modern Improvisation, 2) Using the Principles of Improvisation in Film Creation, 3) Looking forward: The Promises of Improvisation.

While the books main focus is the use of improvisational techniques in film creation, I was most interested in the background research on the roots of modern improvisational techniques and sections on the philosophies of improvisation.

I enjoy what I would consider to be the European explorations of improvisational philosophies, and this book is full of great discourse and thoughts on the romantic ideals of what happens when you start making things up.

The Roots of Modern Improvisational Teaching

Vilč explores the history of modern improvisational teaching, clearly laying out a trajectory of learning and evolution of theory dating back to the late 1800’s and early 1900’s. I enjoy how she clearly delineates some lists of ideas and concepts:

Four Johnstone-ian Basic Principles of Improvisation (pg 27):

  1. Accept and embrace your first impulse;
  2. Accept offers made by your other actors;
  3. Advance the action by truthful reaction to what is happening; and
  4. Advance the narrative by connecting previously established elements.


Three Del Close-ian (along with Elaine May and Theodore Flicker) Basic Principles of Improvisation (pg 67):

  1. Do not deny what is said or done onstage.
  2. Take the active as opposed to the passive choice.
  3. The actor’s business is to justify.


She delves into Del’s discussions on where laughter comes from (pg 73): “Laughter is a response to a gestalt formation where two previously incompatible or dissimilar ideas suddenly form into a new piece of understanding. The energy release during that reaction comes out in laughter”.  Which is all to say, that “the truth is funny”.

Vilč discusses the translation of the above principles and other thoughts and writings by improvisational innovators (Johnstone, Viola Spolin, Compass Players, Close, David Shepard, etc., pg. 87):

  1. The idea of individual genius is replaced by the collective group mind.
  2. The idea of a script is replaced by an emergent narrative, formed through the multitude of artistic, and audience, interactions.
  3. The idea of a final product as the goal is replaced by a focus on the risks and failures of an ongoing process.


I enjoy one thought she presents as a derivative of a Randy Dixon-ian idea (pg 91): “Less structure can often mean less freedom”, quite an odd concept, but she continues, “it is the limitations that work as catalysts for the creative processes and not the other way around”. As Dixon said: “[The] forms are a skeleton structure that allow the improvisor to focus…”

Vilč explores the ideas of the audience as the indirect-director, or the ‘indirector” as I call it. They are reacting to what is happening on stage, in real-time, and the actors can modify their actions accordingly. But, the responses may be inconsistent across audience members, and thus the actors, as an ensemble, must synthesize and translate the conscious and subconscious indirections from the collective.

Johnstonian improvisation focuses on the idea of “Yes, and”-ing any offer. But, there is an extension of this that connects to the theory of mind of the improvisors on stage. That is, that “in order to play … one first has to know what the other improvisors want, one has to be able to inspire others”. This note really resonates with me, as my inspiration these days has been stemming from the idea of Finding your Fun in the Fun of Others. Improvisors should be able to quickly embrace multiple impulses (as offers and acceptances), and start to guide their offers to this specific situation (the actor across from them, the audience, the show, the moment) accordingly. I am not sure if this something I truly do on stage, but I like to think that being able to play with different individuals, starts with being able to model the actor that you are improvising with, and updating that model online during a scene/show/performance.

Finally, there is a full chapter which explores some novel philosophies behind improvisation quite eloquently:

  1. “…to enable truly innovative art forms that will be able to address the questions and issues we are interested in … in a way that resonates with contemporary audiences, there needs to be a place and enough time for experimentation… Experimentation means that the artists need to have the freedom to take risks and … to fail (pg 156).
  2. Collective improvisation should be inclusive (pg 157).
  3. “The other actor is the most important focus of my own work and it is only through supporting the other actor that I will be able to shine and succeed myself” (pg 159, incl. Gregor Moder)
  4. Taking risks is a precondition for anything to happen at all, and any mistakes do not exist. Out of the thousands of choices one could make the first and often accidental or associative choice is the best and it will always enable the next thing to happen. Quality is not a matter of pre-meditated action but of complete commitment to what one is doing here and now (adapted from pg 163).
  5. The audience is freed from a role of passive spectator by being included in the show (cf. Ranciere The Emancipated Spectator, pg 164-5).
  6. One can only do things for oneself by doing them for and through the other (cf. Jean-Luc Nancy Inoperative Community, pg 167). Decision making happens because each individual is empowered to make decisions, but this is only possible if all the actors are tuned into the decisions of the others (pg 169).
  7. The more actors there are onstage, the less each individual needs to initiate and the more each individual needs to be actively listening (pg 171).

The thoughts and ideas presented in the book Collective Improvisation are inspiring and well-researched. I admire the preparation and writing of this book, as it formulates a great background for those of us that enjoy the philosophical understanding of improvisation as well as the historical exploration. As Julian Faid says, we are all looking forward to looking back.

Anything in this post totally counter to what you believe? Let me know below! As you can probably imagine, I really dig nerding out on improv theory.




Will you write me something for my birthday?

For my 29th birthday I want you to share with me a short essay of length 290 – 2900 words on “Finding Fun in Others”.

IMG_2091 - New

This year, 2016, I have been very inspired by the ideas of Finding the Fun in Others. In May, I was invited to teach on this principle in workshops in London, England (with C3Something) and Llubjana, Slovenia (with IGLU). What a true delight it was to get to share some of the ideas I have been floating around in my head with some of the best improvisors in the world.

So, that being said, it is only fitting that I will make a request for my 29th birthday. As a reminder, it is June 5th, the 5th of June, JUNE 5, the fifth of June, the 156th day of the year in the Gregorian Calendar, Arbour Day in New Zealand and the Feast of Núr in the Bahá’í calendar (if Bahá’í Naw-Rúz falls on March 21).

The request is as follows, for my 29th birthday I want you to share with me:

A short essay of length 290 – 2900 words on the topic of Finding Fun in Others. 

You can feel free to make it about improvisational theatre, about academia, about art or science, or life in general. You can make it specific or general, postulate wildly or sincerely. Tell me a horrific tale of a fun-free vacuum, or spin a heart-warming story of a land filled with fun-buns and jelly-bellys. What is fun, what are others, what does it mean to find? Feel free to semantically define it for yourself…

It is my hope that I can collect 29 of these short pieces and accumulate them in a collective creation collection. Much like I have done over the last four years. Another small piece in my larger, yearly, creative endeavour to collect, embrace, and feature the creativity around me.

As a note, feel free to add any necessary attachments, be it audio, visual, photos, supplementary writings, tree diagrams, pins on a map, that may help to clarify and expand your essays. 

To deliver the present to me, you can feel free to email it to me, you can  email me for my address and then hand write it and mail it to me (as you know, I love hand written letters). You can hand deliver it to me (if you are in Edmonton, or Toronto (I will be there May 30 – June 5 (which is my birthday as a reminder))) [ How about that, triple-nested parentheses ].

Over the past four years I have asked for some exciting, collective creation style gifts for my birthday!

25th year: 25 Stories on 25 Years of Age
26th year: 26 Love Letters
27th year: 27 Birthday Pictures
28th year: 28 Handwritten Notes

Building an Artificial Improvisor

In June 2015, I started a journey to develop an Artificial Intelligence that I could perform improvisational theatre with. April 8, 2016, my dream becomes a reality. This post documents the progress…


In June 2015, I started a journey to develop an Artificial Intelligence that I could perform improvisational theatre with. April 8, 2016, my dream becomes a reality.


As you may or may not know, I am an improvisor, and a Ph.D. student in Computer Science, studying artificial intelligence. Early on in my improv career a very, very good improvisor told me:

A good improvisor looks great on stage. The best improvisors make everyone else look good.

This was advice that I have held closely through my artistic journeys. Lee White (twitter, wikipedia), a talented Canadian performer (and my improv Uncle) told me about a show where he selected from the audience the individual with the least stage / public speaking experience and then did a show with them. I was up for the challenge and pitched the show to then Rapid Fire Theatre’s Artistic Director Amy Shostak. She fostered the idea (as all great improvisors do) and I performed the show multiple times to great success.


If I could do improvisation with anyone in the audience, could I improvise with anything? (this reminds my of another mentor and early guide Jacob Banigan (twitter), and his work in solo improvisation).

Some more cool background research that I found and followed up on:

The Journey to AI (artificial improvisation)

Improvaganza 2015

It was at Improvaganza 2015 that the idea of building an artificial intelligence came to be. Sitting at the pub with friend-from-across-the-pond Adam Meggido, who just recently won an Olivier (like a TONY for Brits) for Showstopper on the West End of London.

Adam told me to embrace my scientific side, he challenged me to bring the best in artificial improvisational technology to the stage. Most importantly, he told me in no uncertain terms:

The first time will be the worst time, but the science is getting better every year — every month even. So, just start!

Adam was a great initial sounding board, he helped me wrap my ideas around the basic foundations of why this kind of project would happen.

How and why science and art intersect?

Bringing AI to the stage is fascinating, but for me it was due to three main points:

  1. Science broadens our artistic understanding;
  2. Art broadens our scientific curiosity; and
  3. Both can be used to explore and understand humanity.

Deep, I know, but it is in these points that I found my main inspiration.

Science broadens Art

The worlds of art and science are intrinsically linked. We, as artists, grow experimentally with every performance. Similarly, we as scientists, are imbued with creativity in hypothesizing. The combination of technology and storytelling is not the novel convention. Differently stories are told through different means. The medium is the message (as Marshall McLuhan says).  The way that we tell the story, symbolically connects to the messages being portrayed in the contained narrative.

We are connecting the minds of the audience with the performers, sharing their awareness of the performance live and in real-time with the performance as it is happening. By allowing the transparency between the story-teller and the audience, we can collaborate the share in the experience. Our art will push our technology, but our technology should push our art as well. As audiences evolve they want to be challenged, and we should not be afraid to experiment with the newest and most powerful tools to capture that excitement.

Art Broadens Science

Creatives are paving a road of technological experimentation. By fusing the worlds of natural and artificial we can start to see the links between the sentient and the synthetic. We are using a means that is appreciated, understandable and approachable in performance theatre, to share knowledge of advancing technology.

Artificial Intelligence is a tool, this is a specific implementation of that tool. It is the medium with which we can tell these stories.  Emotional Artificial Intelligence (that is, one that can portray empathy or sympathy or compassion) is somewhat abstract and unappreciated in the scientific literature. By creating these links we are able to somehow add dimension to the discussion.

Using Both to Understand Humanity

What is it to be human? What are basic human values? Is morality relative? These are all extremely difficult questions to think about, let alone to start to answer. Perhaps, the combination of the human and the machine, raw and live in front of an audience which shaping the experience, can help to understand how to approach answers.

Can a robot perform a Shakespearian monologue? Can a robot act? Can a synthetic voice perform all the parts in a play? Can the robot be the actor and the director and the audience? Where does the human NEED to come into a theatre piece, if at all? Theatre helps us to understand the human condition, but perhaps it could help us to understand the human as well.

Performance Ideas Pt. 1

The ‘Adam’ discussions (the creationist metaphor is not lost on me) also lead to two interesting ideas for the performance:

  • A potential branching cue structure where I can cue the booth to play the next sound cues, which could be the voice of the AI. Rather than the deterministically selected cue, it could choose randomly from a set. This selection could also be biased from choices it has (or I have) made in the past.
  • The idea of the AI bring a voice that convinces of how real it is by physically embodying an audience member. Perhaps the voice helps select someone from the audience and they then are the physical embodiment of the voice.

January 2016

It was deep in the cold Edmonton winter when the Reinforcement Learning and Artificial Intelligence lab at the University of Alberta was visited by Dr. Michael Littman. A renowned AI scientist, who made time to sit down and chat with me about research ideas and … creative projects. While at I was at first hesitant to share some of my more out-there ideas, Dr. Littman embraced the ideas wholeheartedly. He imbues the sense of “Yes, and”, a notable quality for an academic.

Dr. Littman was thrilled with the idea of the show, and connected me to a fiend of his named Tom Sgouros. While I didn’t realize it at the time, this was one of the critical connections between the art and the science.

Tom Sgouros and JUDY


In Spring 2005 (yes, that is the correct date, I know it seems like a long time ago for what I am about to tell you, so see the date on the headline of this article), Tom performed alongside a robot that he built named Judy. Tom explored the question:


Needless to say, this was utterly inspiring. Tom told me about his history of performance, with so many inspiring shows (and creative uses of VCRs, fake tapes, and cue buttons intelligently hidden around the stage) .

Performance Ideas Pt. 2

I told Tom about the show I was imagining:

  • The show is somewhere in between Her, Pygmalion and tomorrow’s tomorrow. It exists in a world where humans and robots can have relationships. It is, in that sense, some what futurism. It provides some commentary on the judgement of others on the choices we make. I had the idea of using the robot as an introduction to the surrogate; can the robot convince an audience member to be the vessel, and at that point, it could be that the human is the intelligent being?
  • This could show the tradeoff between the human and the machine. Plus then I would get to have a lovely, improvised scene, with someone from the audience, as they are ‘acting’, as if the AI gave them full control.
  • Perhaps there is a place for audience interaction, or maybe the audience has some driving power over the intelligence, how can this feedback help to drive the show.
  • Perhaps there is a natural language processing piece to incorporate, or a dynamic voice synthesis piece (as those are still very developmental).

The Hard Part

With my background research done.  It was time to start with the implementation

My name is  Pyggy

I wanted a robot that I could improvise with, but I was less focused on the hardware and more interested in the dialog engine. With improvisation listening is key, so I needed an intelligence that could hear me and speak. I started building Pyggy.

The name Pyggy comes from the name Pygmalion, and the fact that Pyggy is build in Python (see what I did there), it also stems from this haunting video:

Seemingly alone, Pygmalion sought to create for himself a perfect, pure, unsullied companion. He used his particular skills to this end: he created a statue bride.

What I was setting out to do was build a perfect improvising companion, and there are eternal dangers of seeking idealist ideas.

Pygmalion is also the name of a George Bernad Shaw play about an academic who makes a bet that he can train Cockney flower girl Eliza Doolittle to pass for a duchess. How fitting. Ps. if you haven’t yet, turn on My Fair Lady while you read the rest of this post. You won’t miss much, just the credits and some exposition.

The Technology

It was less the fancy robotics that I wanted to focus on, in improv you have to play so many characters, so fleshing out a body isn’t the top priority. So, having made the decision to stick with the software implementation I started  programming.

Pyggy is free and open-source if you want to try to get it running on your own system.

Behind Pyggy there are several modules:

  • Pandorabots Chatbot
  • Chatterbot learning Chatbot
  • Speech recognition
  • Speech synthesis

Speech Recognition and Synthesis

Speech recognition is done by the python module of the same name. Pyggy uses the newly released Google Speech API (which was majorly updated during development, see Adam, technology won’t stop progressing, so it was right to jump in before it was ‘ready’).

Synthesis is done by the onboard speech text-to-speech system for the mac, but I did download some better sounding voices (for my ears, the best ones are Tom and Samantha).

Pandorabots Chatbot

The Pandorabots chatbot playground is a great place to get your feet wet starting to play with chatbots. They have some great AIML libraries which serve as a nice structured foundation for their bots. As well, their API from makes it super easy to start your chatbot up and deploy it live.

But their training is quite difficult as it all relies on the mostly deterministic AIML. It made for some interesting conversations, but I couldn’t really train it deeply on conversational knowledge, and it worked best only when I was asking the ‘right’ questions.

Finally, it is not free and not open source.

Chatterbot Chatbot

The Chatterbot Chatbot from  Gunther Cox is free and open-source. Which is great because while it is a strong code base to start a project on, I found that I wanted to train it on huge datasets and it was quite slow.

I was able to fork the repository and modify things nice and quickly so my training could happen lightning quick. Maybe it will even make it into the main codebase!

I was also able to deploy it quickly on a working training interface (which is currently not running, but as I wanted to freeze the training), so that I could enlist the help of my fellow improvisors to help seed and train the model.

Screen Shot 2016-04-08 at 3.23.10 PM

The chatbot runs on a database (MongoDB) which is trained on a huge corpus (Cornell Movie-Dialogs Corpus) of conversations.

This corpus is gigantic (300k utterances from over 10k characters), due to the training speed limitations of chatterbot I had to settle for training on a random subset of 20000 conversations, and it still took an hour to train. For the next iteration, I want to use the full corpus.

It works by fuzzy matching my speech recognized input string with some strings in its dictionary and then producing the most likely response to that input.

Gluing it Together

I combined the chatbots, to get the best of both worlds. The chatbot-like feel of the Pandora bot, and the deep conversational knowledge and interest of the chatterbot, and I got Pyggy.


As there will be an audience for the performance, and it will be taking place in a well-equipped theatre, I though that it would be best to find an interesting visualization for Pyggy.

A great recommendation at the 11th hour led me to build a nice face with talking animation using the Magic Music Visualizer and Soundflower to port the audio through.

Performance Ideas Pt. 3 – You’re On

I have been so incredibly supported on my academic and creative artistic sides, it has been a great learning journey over the last year or so.

So, April 8 is the performance. I am performing alongside Pyggy in Rapid Fire Theatre’s Bonfire Festival. Thanks to the current Artistic Director Matt Schuurman for giving me the chance to do it, and pushing me when things were faltering.

I plan to wear a headset microphone, and project Pyggy on the screen, and have a scene with Pyggy. It sounds weird to say, but it is going to happen.

I will pray to the non-denominational demo gods that things don’t crash, I will break legs with the other actors on stage. It feels like my worlds of science and improvisation are colliding right in front of my eyes.

Or, your eyes, if you are going to be there… This time.

But like Adam says, the first time will be the worst time, because technology is only going to progress, Pyggy is only going to get better at conversing, and I will only get better at training Pyggy.


Thank You

As I reflect on Pyggy, I am overwhelmed with the passion and enthusiasm of friends, colleagues, and collaborators. It was common for me to bring it up and for friends to quickly offer help and ideas on creation, so thank you to: Paul, Sarah, Matt, Joel, Brendan, Paul, Joe, Nikki, Leif, Julian, Paul, Lana, Stu… it is moments like this that helped me through the ‘this is never going to work’ phase:

Screen Shot 2016-04-08 at 3.38.26 PM




Visiting Washington State University

Travelling to Washington State University to guest lecture and speak on the Autonomous Anatomy we are building at the University of Alberta. Sometimes you have to travel to new places to see what is really happening here at home.

Recently, I was invited to perform TEDxRFT at the Seattle Festival of Improvised Theatre. It was a delightful opportunity to do our show for a new audience in a new city, to travel to Washington State and to visit some old friends from Portland and abroad!


It was also an opportunity for me to take a shot in the dark and send an email to a professor at Washington State University whose work I admire very much. Dr. Matt Taylor works in the domain of reinforcement learning in the Intelligent Robot Learning Laboratory, or IRL at WSU. I am familiar with his work through my research on current topics in the field of teaching robots (see a recent paper of our thought framework).


The IRL is an amazing place with research in the areas of Reinforcement Learning, Transfer Learning in heterogenous domains, Multi-agent Exploration and Optimization, Autonomous robots, and Deep Knowledge Transfer… read more about their current projects.

Matt enthusiastically invited me to come visit the lab at WSU, to give a talk as part of the Smart Environments Research Center Distinguished Speaker Series and to guest lecture in his Introduction to Robotics class (I gave a lecture titled Don’t Shoot the Robot, inspired in large part by the dog training work of Karen Pryor).

There are many things happening in and around the lab that I very much enjoy:

  • Interest in project teams (3D printed prosthetic hands controlled by EEG and RoboSub);
  • Hardware Hackathon to bring together small, diverse teams for a short hack on a new project;
  • Great science and engineering on remote biophysical monitoring, and assisted catheter insertions;
  • Lots of access to robotic development hardware (UAVs, drones, Turtlebots); and
  • Great science on interactive reinforcement learning (like this amazing work).

Lots of this work happens in the Frank Innovation Zone, or Fiz, which is an amazing Community Studio with Wood, Desktop and Metal Fabrication, Electrical Testing and Fabrication and lab space to experiment with the new robots and algorithms. Students are trained on the tools and machines and they are further supported by staff who can purchase raw materials and advise on projects.


It is fun to  connect with amazing scientists and engineers who are physically far away, but, when it comes to the research we are doing, it feels like they are right next door.

I can’t stress enough how important it is to get out of the place you are working in, and to travel to gain exposure to the work happening elsewhere in parallel. It also helps give you perspective on the work you are doing and how it fits into the global knowledge landscape.

Special thanks to the amazing students of Matt’s who made my stay so easy and enjoyable: Gabe, Bei, James, Chris and everyone else who shared with me their research ideas, goals and progress.

IMG_1664 2