ARTICLES INDUSTRY ARTICLES - 2021
Natural Language Processing
SAS is the leader in analytics. Through innovative Analytics, Artificial Intelligence and Data Management software and services, SAS helps turn your data into better decisions.
Natural Language Processing
-------------------------------------------------------------------------------------------------------
(NLP) AI spoken language
AI contexts Processing
------------------------------------------------------------
Processing-English
Natural Language Processing
(NLP), i.e. understands naturally spoken language and contexts,
Summary:
“That is the basis of everything.” Without this foundation, it will not be possible to write helpful algorithms, and processes will hardly be able to achieve a previously unknown level of automation through machine learning. But at the Bavarian OEM, the job is taken over by an artificially intelligent process that was previously learned using pictures from the paint shop and is now improving its skills itself through machine learning. Another example: Whereas in the past, hosts of linguistically experienced employees had to check documents, instructions and contracts for weak, ambiguous or redundant formulations, a system that masters Natural Language Processing (NLP), i.e. understands naturally spoken language and contexts, is now taking over.
Item content:
__________________________________________________________________________________________________________________________
Big data, data lakes - all well and good. The only thing is that nothing has been gained from a flood of data that arises automatically in a networked, digitized company. "It is crucial to recognize that data quality is essential for digital transformation," says Kai Demtröder, Vice President Data Transformation, Artificial Intelligence, DevOps Platforms at BMW. “That is the basis of everything.” Without this foundation, it will not be possible to write helpful algorithms, and processes will hardly be able to achieve a previously unknown level of automation through machine learning. Only through the smart handling of big data does the much-cited oil well emerge, which allows new business models and services to bubble up and takes supply chains, production, and quality assurance to a new level. Quality control through artificial intelligence At BMW, for example, irregularities in the paint process are detected in almost real time based on production data, their cause is analyzed, reacts to them - and this is automated. This increases the quality and avoids additional costs. Not a trivial task, because the paintwork can show very different and, above all, minimal deviations that are barely noticeable depending on the viewing angle and lighting. Actually a classic case for a keen, trained human eye, but the job at the Bavarian OEM is taken over by an artificially intelligent process that was previously learned using pictures from the paint shop and is now improving its skills itself through machine learning. Now not only one error is recognized, but also the type of error is identified and the vehicle is automatically forwarded to the corresponding post-processing process. A similar process is now being used for overall visual quality control. “Here, AI works very well - in an industrialized manner,” Demtröder notes. He knows: “Showcases are easy to show, industrial implementation is more difficult.” Another example: Whereas in the past hordes of linguistically experienced employees had to check documents, instructions and contracts for weak, ambiguous or redundant formulations, a system now takes over the natural language processing (NLP), i.e. understands naturally spoken language and contexts. None of this would be possible without proper data preparation, especially with regard to unstructured data, how they arise in the everyday flow of speech. "Raw data are of little value, you have to process them so that they can also be used by those who are not IT-savvy and non-technical," says Demtröder. "That is why we do not see data lakes as a pure data sink, but rather as a platform for data assets."
the natural language processing (NLP), i.e. understands naturally spoken language and contexts,the everyday flow of speech. "Raw data are of little value, you have to process them so that they can also be used by those who are not IT-savvy and non-technical,"
_____________________________________________________________
Share text with friends: SEO Text
Item content:
Big data, data lakes - all well and good. The only thing is that nothing has been gained from a flood of data that arises automatically in a networked, digitized company. "It is crucial to recognize that data quality is essential for digital transformation," says Kai Demtröder, Vice President Data Transformation, Artificial Intelligence, DevOps Platforms at BMW. “That is the basis of everything.” Without this foundation, it will not be possible to write helpful algorithms, and processes will hardly be able to achieve a previously unknown level of automation through machine learning. Only through the smart handling of big data does the much-cited oil well emerge, which allows new business models and services to bubble up and takes supply chains, production and quality assurance to a new level. Quality control through artificial intelligence At BMW, for example, irregularities in the paint process are detected in almost real time based on production data, their cause is analyzed, reacts to them - and this is automated. This increases the quality and avoids additional costs. Not a trivial task, because the paintwork can show very different and, above all, minimal deviations that are barely noticeable depending on the viewing angle and lighting. Actually a classic case for a keen, trained human eye, but the job at the Bavarian OEM is taken over by an artificially intelligent process that was previously learned using pictures from the paint shop and is now improving its skills itself through machine learning. Now not only one error is recognized, but also the type of error is identified and the vehicle is automatically forwarded to the corresponding post-processing process. A similar process is now being used for overall visual quality control. “Here, AI works very well - in an industrialized manner,” Demtröder notes. He knows: “Showcases are easy to show, industrial implementation is more difficult.” Another example: Whereas in the past hordes of linguistically experienced employees had to check documents, instructions and contracts for weak, ambiguous or redundant formulations, a system now takes over the natural language processing (NLP), i.e. understands naturally spoken language and contexts. None of this would be possible without proper data preparation, especially with regard to unstructured data, how they arise in the everyday flow of speech. "Raw data are of little value, you have to process them so that they can also be used by those who are not IT-savvy and non-technical," says Demtröder. "That is why we do not see data lakes as a pure data sink, but rather as a platform for data
Writing for artificial intelligence.
SAS: Analytics, Artificial Intelligence and Data Management
sas.comSAS is the leader in analytics. Through innovative Analytics, Artificial Intelligence and Data Management software and services, SAS helps turn your data into better decisions.
-------------------------------------------------------------------------------------------------------
(NLP) AI spoken language
AI contexts Processing
------------------------------------------------------------
Item content: Natural Language Processing, a Brief History "Learning another language is not only learning different words for the same things, but learning another way to think about things." - Flora Lewis One of the first ideas in the field of NLP could be as early as in the 17th century. Descartes and Leibniz came up with a dictionary created by universal numerical codes used to translate text between different languages. An unambiguous universal language based on logic and iconography was then developed by Cave Beck, Athanasius Kircher, and Joann Joachim Becher. 1957, Noam Chomsky published Syntactic Structures. The monograph was considered one of the most significant studies in linguistics in the 20th century. The monograph constructed a formal linguistic structure with phrase structure rules and syntax trees to analyze English sentences. “Colorless green ideas sleep furiously.”, A famous sentence constructed based on the phase structure rule, is grammatically correct but makes no sense at all. “Colorless green ideas sleep furiously”, Image from steemitUp to the 1980s, most NLP systems were based on complex hand-written rules. Until after, machine learning algorithms started to kick in due to the increase in computational power and available training data. One of the most well-known machine learning algorithms being RNN, an architecture based on the neural networks. With the increasing demands in processing natural languages with machines, new models have been rapidly iterating throughout recent years. BERT, being the current SOTA, may be replaced in a couple of years or so, who knows? We have just gone over the history of Latin root languages such as English and Spanish. Other languages such as Chinese, Hindi, and Arabic are totally different stories. Different from English, which can be described by a set of simple rules, Chinese grammar is extremely complex and sometimes too blurry to be defined with logical elements.Example of Chinese Paragraphs, Image from 智 经 研究 中心 The complexity of Chinese grammar and the fact Modern machines are based on logic circuits, maybe the reason why popular programming languages are often in English. Several attempts on creating programming languages in Chinese, such as Yi and Wenyan was done in the past. These languages were very similar to our everyday programming languages such as Basic and C. The attempts couldn't prove Chinese to be a better language to be used as a programming language. Translations between the languages are also more difficult between languages of a different root. Translation between sign languages to our everyday languages also encounter numerous obstacles. Guess that is the price we have to pay for trying to build the tower of babel. In order to punish mankind who tried to build the tower, it was said that God decided to divide mankind by making us speak different languages.Tower of Babel, Image from iCRBut in time, we have conquered tons of obstacles to get where we are right now. As technology advances, the language barrier is becoming less and less of a problem. We can totally buy one of those translation sticks shown in the video below and book our travel to Japan, without knowing any Japanese in advance.And NLP algorithms are also helping us in many other areas such as automated subtitling, user experience studies, accessibility, and even the writing of this article as my English would suck so much without the help of Grammarly. Now, let's dive into the computer algorithms behind those awesome technologies.Abstract Syntax Tree, Context-Free Grammar and Compilers "Colorless green ideas sleep furiously." - Noam Chomsky Our modernized software industry is built based on NLP. One of the applications of the syntax tree is our compilers. Without it, we will have to deal with machine codes rather than easy-to-learn programming languages such as Python and JavaScript. Imagine coding our machine learning algorithms with binary machine instructions, yuck… Invented by Noam Chomsky, Abstract Syntax Tree (AST) and Context-Free Grammar (CFG) are used to describe and analyze programming languages we use to code with.Warning: Examples below are simplified for educational purposes. Please refer to a more professional documentation if you seriously wanted to know how exactly compilers work.Abstract Syntax Tree (AST) The compiler uses a CFG to interpret code written in human-readable programming languages. It will break down the logic of the code and interpret the recursive logic out of it. To understand how the code is broken down into recursive logic, it is better to represent the process as an AST. Below is an illustration of an AST constructed out from the code.Abstract Syntax Tree Interpreting a Line of CodeContext-Free Grammar (CFG) CFG is used to describe the rules converting between the input language and the output tokens. Inside the file that defines the compiler, it is usually written in the way shown below: if_stm: expr = left_bkt cond_stm right_bktcond_stm: expr = expr and_stm expr | expr or_stm expr | not_stm expr | num less_than num | num greater_than numnum: var | constThese CFG basically define the rules on how if and condition statements can look like. For example, a condition statement can be multiple condition statements (“expr” means itself, which is “cond_stm” in this case) linked with and statements (“and_stm”, which is represented as “&&” in the code), or simple numerically connected by comparators such as “>” or “<”. The compiler will iterate through the code and see if it matches the rule defined by its CFG, and then extract the recursive reasoning then try to convert it to machine code. A more formal definition of CFG can be found on Wikipedia, and more on compilers such as the famous Yet Another Compiler Compiler (YACC) can be found here.Neural Building Blocks for NLP: Word Embeddings, RNN, and LSTM “Mimicking the intricacies of the human brain, a neuro-inspired computer would work in a fashion similar to the way neurons and synapses communicate. It could potentially learn or develop memory. ”- Nayef Al-Rodhan Advanced NLP algorithms are built with variations of neural networks. I have a more detailed explanation of neural networks in my Alpha Go article, including histories related to neural networks. Neural networks are Directed Acyclic Graph (DAG) consists of connected layers of artificial neurons. The values in input layers propagate layer by layer into the output layer. These models based on how neuroscientists think our brain works have shown some competitive performance in recent years.Forward Propagation of Artificial Neural NetworkWord EmbeddingsNeural networks, as well as other machine learning models, generally takes inputs in the form of numerical vectors. But our English words are not numerical. That is why we have word embeddings. Word embeddings refer to language modeling and feature learning techniques converting words and phrases from the vocabulary into numerical vectors. One example may be running through thousands of paragraphs with a neural network to collect which words are more often seen with another, thus giving them closer values.One famous collection of word embedding models maybe Word2vect. The model is based on a shallow neural network and assumes words that are close to each other inside paragraphs also share similar semantic values. Below is a video explaining it. Understanding Word2Vec, Video from YoutubeRecurrent Neural Network (RNN) RNN is a variation of the neural networks, best at processing sequential data. Data such as sound waves, stock histories and natural languages are considered sequential. RNN is a neural network where the output is fed back into the network during the process of sequential data, allowing it to also take consideration of past states. In our natural languages, the context of a sentence is not always represented by the context in the words. The sentence “Other reviewers think the food was great but I think it was not good” for example, has 2 words that have a positive meaning (great and good). The artificial neural network will have no clue of the sequence and just sum everything up, but an RNN is able to catch the inversion words such as “but” and “not” inside the sentence and adjust to it.ANN Processing a SentenceRNN Processing a Sentence Long Short-Term Memory (LSTM) “Without forgetting it is quite impossible to live at all. ”- Friedrich Nietzsche In the RNN sentence processing example, you may be wondering how most of the words are irrelevant. This is exactly why we needed LSTM. LSTM introduces a “forget layer” that decides the information that should be kept or forget, making the model easier to train under large amounts of data. LSTM has 3 cell states, the forget state, the update state, and the output state, each serves different purposes. Forget State While the forget state, LSTM will retrieve information from the input and the previous state. A sigmoid function σ is then used to decide whether the previous state should be forgotten. The sigmoid function will output a value between 0 and 1. By multiplying it with the output from the previous state it will decide how much of the previous state should be forgotten.LSTM Forget StateUpdate StateDuring the update state, LSTM will try to update state value. A hyper tangent function tanh will softly force the value to lie between 0 and 1 so it does not accumulate into some crazy numbers. Another sigmoid function is used to determine how much of that will be added into the cell state.LSTM Update StateOutput StateFinally, the cell state will be processed by another hyper tangent function. Then a sigmoid function is used to determine how much of it will be feed into the output.LSTM Output StateMore on LSTM and its training with detailed explanations in math can be found here.Text Classification: GRU and HANCongratulations! We have finally made it to the point where we can go into the SOTA models such as HAN and BERT. We are going over HAN first not because one model is superior to the other, simply because HAN is easier to understand and may help with the understanding of BERT. Proposed by the Carnegie Mellon University (CMU) and Microsoft Research in 2016, HAN proved its ability in text classification. Being able to classify text such as Yelp reviews can help in various fields such as user experience studies and support ticket management. We will first start with GRU, which is the fundamental building block of HAN. Do not worry, it is very similar to LSTM we learned in the previous section. After that, we would be able to understand the architecture of HAN.Gated Recurrent Unit (GRU) GRU, Image from BUSTLE “I'm joking! Although it is true. Anyway, have a good one. " - GRUGRU stands for gated recurrent units. They are less powerful but simpler model compares to LSTM. In some cases, however, GRU can outperform LSTM, which we will get more into when we learn about HAN. This is why researchers usually experiment with both units to see which one works the best.GRU ExplainedGRU uses both input and previous state to make all the decisions through the sigmoid function. There are 2 decisions in total: The first decision is whether the previous state will be merged with the input state. The second decision is whether this mixture (or plain input state) or the previous state alone will go to the next state and as output. More on variants of GRU and their detailed math models can be found here. Hierarchical Attention Network (HAN) " Each letter of the alphabet is a steadfast loyal soldier in a great army of words, sentences, paragraphs, and stories. One letter falls, and the entire language falters. "- Vera Nazarian It is difficult for conventional RNN and LSTM to interpret large amounts of text where some of the keywords are wide-separated from each other. This is why the attention mechanism in HAN generates an importance weight α from the context vector u. This importance weight is then used to selectively filter out the output that is worth the attention.HAN utilizes 2 main layers to classify texts - the word layer and the sentence layer. The word vectors are first fed into encoders consist of bidirectional GRUs. Bidirectional GRU is just GRUs with opposite directions stacked on top of each other. The output is then used to calculate the attention along with the context vector, and the output will be feed into the encoders in the sentence layer to go through a similar process. The final output vector will be summed up and feed into a softmax layer.Hierarchical Attention Network (HAN) ExplainedFor a detailed explanation of HAN, its mathematical details and its performance on Yelp reviews, please refer to the original paper.Language Understanding: Transformer and BERT And finally, we have made it to what the title suggests. In order to understand this section fully, we may have to understand extensive mathematical reasonings. These can be found in the original paper for the Transformer and BERT. In this article, we will only be going over the basic architectures.Encoder-Decoder ArchitectureFirst, let us take a look at the encoder-decoder architecture. In machine translation, the encoder-decoder architecture is used quite often because the source text and the target text does not always have a one to one matching. An encoder is first used to produce an output value from the input message, and it will be taken by the decoder to generate the output message.Translation with Encoder-Decoder ArchitectureTransformer “Attention Is All You Need” - Title of the original paper of TransformerIn 2017, researchers from Google proposed an architecture solely based on self-attention. Self-attention means that instead of taking the attention score externally, the model decides the attention on its own by interpreting the input.There are 3 different types of vectors, the query vector Q, the key vector K, and the value vector V. These vectors can be understood as a data search mechanism. Query (Q) is the kind of information we look for. Key (K) is the relevance of the query. Value (V) is the actual input. Scaled Dot-Product Attention & Multi-Head Attention, Image from Original Paper of Transformer The Scaled Dot-Product Attention first multiplies Q and K, then multiplies it with V after some conversion shown in the diagram. The Multi-Head Attention concatenates the heads retrieved from the Scaled Dot-Product Attention. Mask is used to filter out some of the values. The mechanism is a bit complicated to explain shortly, here is an article that offers a more detailed explanation of the attention mechanism. The Transformer is an encoder-decoder architecture based on Multi-Head Attention. Since the Transformer uses a simplified model and no longer consider positional information like RNN, both Input Embedding and Positional Embeddings are applied to the inputs to ensure the model also catches positional information. Add & Norm are residual connections and batch normalizations. Feed Forward is just simple feed-forward neural networks reshaping the vectors.Transformer Model Architecture, Image from Original Paper of TransformerBidirectional Encoder Representations from Transformers (BERT) In 2018, researchers from Google introduced a language representation model called BERT. BERT model architecture is a multi-layer bidirectional transformer encoder. There are 2 steps in constructing the framework - pre-training and fine-tuning. Pre-Training: During pre-training, the model is trained on unlabeled data over different pre-training tasks. Fine-Tuning: During fine-tuning, the BERT model is first initialized with the pre-trained parameters, and all of the parameters are fine-tuned using labeled data from the downstream tasks. BERT Pre-training and Fine-Tuning Procedures, Image from Original Paper of BERT The model advanced the SOTA of 11 NLP tasks after the fine-tuning. For mode details on BERT please refer to the original paper.Words in the end… I planed to write articles on both natural language processing and computer vision. Since I have already written about the convolutional neural networks in my previous article, I decided to write something about recurrent neural networks first. There are still more on convolutional neural networks which I have elaborated in my computer vision article. I am also planning to write about generative models and automated machine learning. The field of artificial intelligence has numerous wonders, follow me to see more of it in a distant future! Words in the end ... I planed to write articles on both natural language processing and computer vision. Since I have already written about the convolutional neural networks in my previous article, I decided to write something about recurrent neural networks first. There are still more on convolutional neural networks which I have elaborated in my computer vision article. I am also planning to write about generative models and automated machine learning. The field of artificial intelligence has numerous wonders, follow me to see more of it in a distant future! Words in the end ... I planed to write articles on both natural language processing and computer vision. Since I have already written about the convolutional neural networks in my previous article, I decided to write something about recurrent neural networks first. There are still more on convolutional neural networks which I have elaborated in my computer vision article. I am also planning to write about generative models and automated machine learning. The field of artificial intelligence has numerous wonders, follow me to see more of it in a distant future! There are still more on convolutional neural networks which I have elaborated in my computer vision article. I am also planning to write about generative models and automated machine learning. The field of artificial intelligence has numerous wonders, follow me to see more of it in a distant future! There are still more on convolutional neural networks which I have elaborated in my computer vision article. I am also planning to write about generative models and automated machine learning. The field of artificial intelligence has numerous wonders, follow me to see more of it in a distant future!
write something about
computer vision article.
automated machine learning...
language processing
networks neural networks
see more
written about
networks neural networks
----------------------------------------------------
computer vision article
automated machine learning
see more of the distant future!neural networks
automated machine learning
machine learning. awesome technologies
automated machine learning
the keywords
large amounts of text
data search
data search
data search
----------------------------------------------------
relevance keywords
keywordsindustry is built based on NLP. One of the applications of the syntax tree is our compilers. Without it, we will have to deal with machine codes rather than easy-to-learn programming languages such as Python and JavaScript. Imagine coding our machine learning algorithms with binary machine
----------------------------------------------------
creating programming languages relevance keywords
planning
-training tasks
write articles on both natural language processing and computer vision
artificial intelligence
modernized software
----------------------------------------
Data Mining
machine learning
artificial intelligence
----------------------------------------
natural language processing
- industries
Natural language processing
Natural language processing
Intelligence and Data Management software and services, SAS helps turn your data into better decisions...SAS is the leader in analytics. and Data Management.Through innovative Analytics, Artificial Intelligence- Analytics,
sas.comSAS is the leader in analytics.Artificial .SAS: Analytics,Artificial .SAS:
Through innovative Analytics, Artificial Intelligence, and Data Management...
-------------------------------------------------
Artificial Intelligence data mining
deep learning
-------------------------------------------------
Text Generator - KI Text & Artikel erstellen | www.ArtikelSchreiber.com : SEO Text Generator - https://www.artikelschreiber.com/schreibenDE.php