AI professionals like me are part software engineer and part anthropologist. At our best, we’re much more than just numbers geeks who write clean code. We strive to understand human behavior because we are building interfaces that sit between people and the digital world.
But there’s another important reason we study anthropology: As we try to create more intelligent and relatable AI, we often model some of the very characteristics that make us human.
That includes how we learn.
Malcolm Gladwell famously coined the “10,000-Hour Rule” in his book Outliers, a concept that he borrowed from Anders Ericsson’s research on the development of expertise. The simplified idea is that humans need at least 10,000 hours of practice at a skill to become an expert.
There are, naturally, different types of practice (more on that later). But this idea applies to artificial intelligence, too. As we recently wrote, AI is actually pretty dumb without human intelligence — and that intelligence comes through training.
Here’s one personal experience to help illustrate that.
Out of the box, there’s nothing smart about AI
I recently set up a chatbot using Amazon Lex. Lex is one of Amazon’s newest web services. At the core of this service is Amazon’s natural language processing engine used in Alexa. So, Amazon is essentially packaging up years of AI research and development into a toolkit for engineers to efficiently create their own chatbots.
So, I quickly experimented with a “Pizza Ordering” bot that Amazon provides as kind of a template to get started. And out of the box, here’s how my first interaction with the bot went:
Phew. That was exhausting. Now imagine if we wanted this bot to do something that was actually hard, like provide technical assistance for a software-as-a-service like Salesforce or offer health advice for WebMD. Clearly, the out-of-the-box bot has a long way to go.
It needs a lot more practice — deliberate practice.
Deliberate practice and AI
Not all training is the same. In his research, Ericsson states that to become an expert at something (anything), a human will need to engage in “deliberate practice.”
Deliberate practice involves three elements:
Focused time spent practicing
Setting and pursuing a goal
Having an expert teacher provide critical feedback
Ericsson says that at some point, usually after about 10,000 hours of deliberate practice, you get to be very, very good at the skill you’ve practiced.
In his decades of research, Ericsson found that more deliberate practice invariably results in a higher skill level. The violinist who has deliberately practiced for 7,000 hours is always better than the one who has put in 3,500 hours.
If you do not get expert feedback, the benefit of practice time is significantly muted. For example, if you play the same 10 violin concertos over and over again for 10,000 hours, while you may develop an amazing stamina for repetition, you will not improve your skill.
The same formula holds true for AI. Even with all of Amazon’s intelligencesmarts built into the engine, the out-of-the-box PizzaOrdering bot is no smarter than an untrained teenager working his first job behind the counter at Pizza Hut. Actually, the OOB AI is not even close to as smart as this teenager.
The reason: The bot has had zero hours of deliberate practice at taking pizza orders. It simply doesn’t know what to do in 99.9% of pizza ordering situations.
So, how do we make our bot ‘intelligent?’
We train it. We start by feeding it all kinds of conceivable scenarios and utterances. We teach it about crusts and toppings, and how to handle irate customers, and how to interpret common idioms. For example, we teach it that when a customer says, “This pizza is THE BOMB,” it is not cause to auto-dial the police. Eventually, our bot will be able to take an order reasonably well.
But like the violinist-in-training, not all types of training are the same for our pizza bot. If we feed it with 10,000 hours worth of one teenager’s interactions, it’ll only ever get as smart as that teenager. If, on the other hand, we feed it with data culled from the interactions of many experienced pizza-order-takers — who cumulatively have handled even the most challenging customer interactions with finesse and grace — then its skill level will begin to reflect that of its trainers. In other words, the teacher matters.
However, the teacher’s role is not only to demonstrate excellence. The teacher’s role is also to provide feedback. It’s to tell the violin student that a note was slightly flat. Or, that by moving the bow hand in a certain way, the sound will evoke sadness.
We need to take the same approach with our bot. We need our expert pizza-order-takers to tell the bot when it misinterprets a question and to suggest a more appropriate answer. We need our experts to continually help the bot refine its responses as it continues to make mistakes — which it will inherently do. Eventually, with a large enough data set and continual training by experts, our pizza ordering bot will become skilled. It will have accumulated the AI equivalent of 10,000 hours of deliberate practice.
How does this tie into Directly’s work?
In action, our AI-powered customer support service can be smart, really smart. But it’s not because my team wrote brilliant machine learning algorithms (though we are very proud of what we do). The reason is that our CX automation platform deploys expert super-users on the products and services our clients sell. And they train the AI.
The best machine learning system in the world won’t know anything about a company’s very particular customer service issues out-of-the-box. There may be rich history of old customer support “tickets” that can give the AI some data to start. But even this data is not enough — just as a library of concerto videos and tutorials wouldn’t be enough for our student violinist. It’s a good start, but the student needs more than great examples. She needs feedback (and a lot of it).
It’s also important to recognize that most company’s customer service tickets are not as simple as taking pizza orders. They contain compound questions, branching conversations, and require a healthy degree of empathy. So, how do you train AI to handle these complexities and to have empathy?
Some companies have automated support feedback loops that will ask users who received support help to provide limited feedback. It usually comes in the form of: “Was this a good answer?” Most users do not click these buttons. And even when they do, the feedback signal is one dimensional.
Other companies perform AI training by hiring a team of humans to manually label and clean the dataset. They might comb through historical tickets, breaking them apart into logically simple issues. The companies may also run bots through thousands of support case simulations, telling the bot when it has done a good or bad job. You can hire a company like Crowdflower to do this kind of work. They’ll put hundreds of humans to work on these tasks that only humans can do well.
Where Directly shines is by hiring multiple experts to do the training (instead of an average clickworker). It’s like having all of the world’s top violin teachers providing feedback to the student violinist after every note is played.
Having one expert validate the AI response equates to hundreds of hours of real-world practice. By having multiple experts validate each answer and suggest edits means the AI response benefits from thousands of hours of combined expert experience. And the AI gets smart, really smart over time.
So, while it might be a stretch to say a Directly-powered AI response is like a master performance from a concert violinist, it wouldn’t be that much of a stretch. With confidence, we can say that our expert-powered AI benefits from thousands of hours of deliberate practice.
Maybe even 10,000.
Want to see AI training in action?
If you’d like to see AI training in action, contact us today and set up a demo of Directly’s CX automation platform.
