AI: what's the difference between NLP and NLU?
Although they differ by just one small letter, the concepts of NLP (natural language processing) and NLU (natural language understanding) are different but complementary in terms of text recognition and word comprehension.
It's easy to see why natural language understanding is such an important issue for companies wishing to use intelligent robots to communicate with their customers.
The concept of NLP, or Automatic Language Processing in French, emerged in the 60s. Its ambition was to enable machines to record and process information formalized in natural language.
NLU - NLP the difference
In order to function and interact with us smoothly, machines need to learn through natural language processing (NLP).
NLP - Natural Language Processing
Visit NLP or "Natural Language Processing", is a set of text recognition solutions that enable us to understand words and phrases formulated by users.
Objective: to analyze and understand a need expressed naturally by a human being, and to be able to respond to it.
NLP encompasses all technologies that take raw text as input and output an expected result (Natural Language Understanding, synthesis or translation, for example). In concrete terms, NLP makes it possible to understand what a human being is saying, to process the data contained in the message and to provide a natural language response.
The NLU - a sub-component of the NLP
Natural Language Understanding (NLU) consists in analyzing a written or spoken text in natural language and understanding its intentions. It is therefore a sub-component of NLP.
NLP literally interprets what the customer says or writes, while NLU identifies intentions and deeper meaning.
An NLU is an algorithm that is trained to categorize information "inputs" according to "semantic data classes". The model finalized using neural networks is thus able to determine whether an element X belongs to class Y or to another class Z, etc.
"I'm trying to reach Orange, do you know what the number is?" The NLU will understand that the person wants to reach the Orange Group's customer service department.
"In which department is the town of Orange?" The NLU will understand the difference in intent with the previous sentence. We're talking about the city here, not the telecom group.
The NLU is also capable of recognizing entities, i.e. words and expressions which, if recognized in the user's request (Input), can condition the progress of the conversation.
In our orange example, the input is: In + city + Orange
Intention and entity
Understanding natural language therefore relies on two key pieces of information: intention and entity.
Intent enables the user's message to be understood, and is characterized by a typical syntactic structure found in the majority of inputs corresponding to the same objective.
The entity is a piece of information present in the user's request, relevant to understanding his objective, characterized by typical short words and expressions found in a large number of inputs corresponding to the same objective.
- Automatic dialog summary
- Automatic translation
The Callbot, fed with artificial intelligence, will have an advanced understanding of natural language thanks to the NLU. If this isn't precise enough, human intervention is possible on an ad hoc basis, using a low-code conversational agent creation platform, for example.
NLU - NLP and speech recognition
When dealing with voice interaction, it is necessary to define a system for real-time transcription of voice interaction.
This transcription phase takes place just before the analysis and comprehension stages.
Transcription uses algorithms known as Automatic Speech Recognition (ASR), which generate a written version of the conversation in real time.
To put it simply, you can think ofASR as speech recognition software that enables a person to make a voice request.
Speech recognition is not a new subject: historically, the first speech recognition goal was to accurately recognize 10 digits that were transmitted using a hard-wired device (Davis et al., 1952). From 1960 onwards, numerical methods were introduced, effectively improving the recognition of isolated elements of language (such as when you are asked to say 1, 2 or 3 on the telephone, for example). However, it would take much longer to tackle "continuous" speech, which would remain highly complex for a long time to come (Haton et al., 2006).
Like learning to read - first the alphabet, then sounds, and finally words - the transcription of vocal language has evolved with time and technology.
From the simplest to the most complex :
- Recognition of isolated words,
- Recognition of chained words: only one person speaks
- Coarticulation: how one phoneme* can influence another "it's too [P]unfair", for example.
- Taking into account disruptive elements such as continuous speech or the variability of interlocutors, made possible by advances in machine learning.
