Machine translation is a fast developing field where researchers make constant breakthroughs. Despite the huge progress achieved over the last years, machines are still not capable of delivering high quality translations.
Language is ambiguous due to the contextual elements humans that embed into it. Words are important vehicles of communication, but they are often void without context. This is where the main challenge lies in machine translation: computers aren’t very good at identifying and interpreting context. This is the main reason why human translators are better than machine translation algorithms.
Human translators can very well use context to identify the correct meaning conveyed by words, whereas machines are still using a limited word-based approach. This is where artificial intelligence enters the scene. By integrating AI into machine translation, researchers have significantly improved the accuracy of machine translation.
Convolutional neural networks in machine translation
Facebook’s Artificial Intelligence Research (FAIR) team published a series of interesting research results using a new convolutional neural network-based approach for language translation. Thanks to this new method, the team was able to improve the accuracy of machine translation by nine times compared to recurrent neural systems.
What are convolutional neural networks?
Convolutional neural networks are feed-forward artificial neural networks used extensively in image processing. Connections between the units do not form a cycle, unlike recurrent neural networks.
Interestingly, the CNN design reflects vision processing in living organisms. It includes multiple layers that process the output of the prior layer. These layers create neuron collections that process a part of the image. At the end, the outputs of these collections overlap, to obtain a higher-resolution representation of the original image.
Convolutional neural networks are becoming more and more popular among machine translation researchers thanks to their high accuracy.
Facebooks achieves state-of-the art translation accuracy using CNNs
Computers translate text in a linear order, reading one word at a time and identifying the word that conveys the same meaning in another language. This is a slow process because each word must wait until the network is done with the previous word.
In comparison, CNNs can compute all elements simultaneously. Moreover, information is processed hierarchically, allowing computers to detect complex relationships in the data. In this manner, machine translation algorithms can use context to produce better, more accurate translations.
FAIR created a new translation model design perfectly adapted to CNN. This new translation design relies on multi-hop attention. The network actually breaks down sentences, taking repeated “glimpses” at the sentence components to produce better translations. These glimpses depend on each other, allowing the algorithms to eliminate ambiguity.
Another essential element of the new translation architecture is gating. The system controls exactly which information should be transmitted to the next unit. When predicting the next word, the system takes into account the translation it has produced so far. Gating allows it to zoom in on a particular aspect of the translation and use the information to get a broader picture.
Here’s an example of how this method works:
As you can see, the computation is done simultaneously on two layers. At every step, the system glimpses the whole French sentence to decode the correct meaning of the words.
The sequence modeling toolkit (fairseq) source code and the trained systems are available under an open source license on GitHub.Follow The AI Center on social media:
Join me as I track the latest progress in AI research.
Latest posts by Maddie Blau (see all)
- True Emoji is an AI app that uses your expressions to create animated emojis - November 21, 2017
- Canada’s first AI exchange-traded fund enters the market - November 2, 2017
- Are you curious to see who’s the smartest AI in the world? - November 1, 2017