In the early days of artificial intelligence, efforts were made to create machines similar to humans by imitating the human brain and way of thinking. In 1955, Marvin Minsky and others at Dartmouth College in the United States built the first neural network, the SNARC system. Around the same time, computer scientist Viktor Glushkov in the Soviet Union created the All-Union Automatic Information Processing System (OGAS). With the development of computer science, rather than implementing human intelligence itself, AI machines were made to efficiently solve real-world problems that can be broadly divided into four types of stages.

At the early stage of AI, a single layer neural network, perceptron, was used. However, due to the limitations of information processing capabilities, the applications of the perceptron were limited, and its popularity waned for a while. In 1974, Paul Warboss proposed a backpropagation algorithm that could solve multilayer neural networks, and research on AI using multilayer neural network models was actively conducted, resulting in visible results such as character recognition and speech recognition. However, until then, it was closer to an answering machine than a conversation with a human. In 2006, Geoffrey Hinton announced the deep confidence neural network that is capable of unsupervised learning methods which were considered impossible, and as a result, the methodology called deep learning replaced the higher-level concept of the artificial neural network and became the only methodology. In particular, in 2012, Hinton's disciples Alex Krizhevsky and Ilya Sutskever built AlexNet, a convolutional neural network architecture, and won the computer vision competition called "ILSVRC" with overwhelming performance, and deep learning became an overwhelming trend, surpassing the existing methodology. In 2016, Google DeepMind's AlphaGo popularized deep learning methods and showed results that surpassed human levels in several fields. AI research has continued to conduct innovative research to solve problems that only humans could do, such as natural language processing and complex mathematical problems, using computers. Awareness spread that AI could quickly surpass human capabilities in the field of weak artificial intelligence.

Artificial intelligence began to change significantly in 2022 with the emergence of generative AI. OpenAI's ChatGPT and Drawing AI, which are representative of the generative AI, have finally begun to be applied to actual personal hobbies and work applications, and the practical application of AI, which had seemed like a dream, has finally begun. The background of this generative AI was the transformer structure. However, generative AI has led to active discussions about AI, and among them, theories of caution and threats to AI have also begun to emerge.

For example, if we give a picture of a partially obscured puppy and ask the artificial intelligence whether the object in the image is a puppy, the artificial intelligence filters the signal for the picture according to its own criteria, which is the characteristics of each puppy remembered by each node. If it is a picture of a puppy with its eyes obscured, the node that learned the puppy's eyes determines that the picture is an incorrect image (signal) because there are no puppy eyes, and outputs 0, meaning false or incorrect. However, nodes that have learned other parts of the dog, such as the nose, mouth, ears, and legs, judge that the photo is the correct image and output 1, meaning true or correct. At this time, since the values ​​of 1 are produced more than 0 by parts other than the eyes, the AI ​​ultimately calculates the answer for the photo as 'puppy'. This characteristic of AI contributes to automatically ignoring errors and producing the correct answer, just like a human, even if there are typos or incorrect words in the question. That is why some experts call AI's answers probabilistic answers.

In modern times, research on probability and random algorithms is the most popular. In general, problems that can be determined as "if A, then B." can be approached relatively easily by computer programs. However, if there are cases where multiple answers can exist, such as 'art' can be 'art' or 'technology', the surrounding circumstances such as 'context' must be considered. But it is difficult to give a clear answer such as "if these words appear before and after, then it is 'art', otherwise it is 'technology'." This type of problem is solved using complex mathematics that deals with statistics and probability. In fact, modern artificial intelligence research is said to be proceeding by assigning categories to each word and interpreting the meaning of the sentence as a whole with more categories. As an extremely simple example, when 'Music is an art' is said, it guesses the category 'art' that includes the two meaningful words in the sentence, music and art, and interprets it according to the context. AlphaGo also belongs to this method.

In artificial intelligence, if a given problem can be solved, all techniques and technologies are used without distinction. If the quality of the results is excellent, technologies with no theoretical background are applied and recognized. Below is a list of only some of the famous technologies and techniques.

The fundamental problem is that the current von Neumann structure computer components were not initially designed for artificial neural networks. Current computers are separated from the CPU, which is a serial processing device, to the data storage and processing lines and auxiliary data processing devices. In particular, this auxiliary data processing device refers to the graphic processing units (GPU), which perform parallel processing. Since artificial neural networks are designed based on matrix mathematics, simultaneous calculations must be performed in each node (computational unit/coded neuron) that makes up the artificial neural network, and to do so, the parallel processing device must be the main device. This problem of inefficiency is directly linked to the problem of heat generation, so there is a limit to simply increasing the number of transistors through process miniaturization without changing the architecture to improve performance.

For this reason, many semiconductor companies are currently jumping into the development of AI chips to reduce power consumption and increase computational efficiency. The industry predicts that the landscape of AI development will change again when AI chips completely replace GPUs in the future. However, even if an AI chip is developed, it will take a considerable amount of time to establish a software ecosystem that can utilize the chip.

There are various machine learning methods for artificial intelligence, and models such as supervised learning (decision tree, Bayes classification, kNN, support vector machine, etc.) and unsupervised learning (k-means clustering, etc.) studied in chapters 6 to 8 of this book are widely used. And recently, the reinforcement learning method, which gives rewards instead of outcome values ​​and makes action selection, is also introduced using the Markovian Decision Process (MDP) theory. In reinforcement learning, a reward is given every time an action is taken in the current state, and learning proceeds in the direction of maximizing this reward. The video below shows how to teach a robotic arm to play table tennis using reinforcement learning.

\(\qquad \qquad\)
\(\qquad \qquad \quad\)https://www.youtube.com/watch?v=SH3bADiB7uQ

However, in 2006, Professor Geoffrey Hinton of the University of Toronto, Canada, solved the vanishing gradient problem through pretraining of neural networks and dropout data, and began calling the neural network that applied this algorithm as deep learning. The recent popularity of deep learning is due to the development of an algorithm that overcomes the limitations of the existing back-propagation, the increase in learning materials due to the accumulation of a large amount of data, and the development of graphic processing unit (GPU) hardware suitable for learning and calculation using neural networks. Deep learning has been popular since 2010, and there are several important events that led to this popularity.

These three major events confirmed the performance of deep learning and became an opportunity to be actively used in various fields of society.

A traditional method of searching for useful information in text databases is the keyword information retrieval method used in libraries. Libraries classify books or reference materials by subject or author and create a database system. When users query using the keywords of the information they are looking for, the computer system searches for documents or books containing these words. Recently, keyword search methods are also used when searching web documents on portal sites such as Google. However, as the amount of digital documents increases, keyword search methods can cause users to be confused about which documents are useful, as the number of web documents searched for a given query reaches thousands. Therefore, it became necessary to create measures of the importance or relevance of the documents being searched and to prioritize the information being searched. This type of research is an example of text mining. Let's discuss various issues for text data mining.

Similarity-based search uses the frequency and proximity of keywords to find similar documents. The following cosine distance measure is often used to measure the similarity of documents. Let's display a document as a set of terms excluding stop words. If the terms appearing in the entire document set are \(x_{1}, x_{2}, ... x_{m}\), then each document can be represented as a binary data vector indicating whether the terms appear (‘1’) or not (‘0’), for example, \(\boldsymbol x_{1} = (x_{1}=1, x_{2}=0, ... x_{m}=1)\) by the occurrence of each term. The cosine distance between two document vectors \(\boldsymbol x_{1}\) and \(\boldsymbol x_{2}\) is as follows. $$ d(\boldsymbol x_{1}, \boldsymbol x_{2}) \;=\; \frac{\boldsymbol x_{1} \cdot \boldsymbol x_{2}}{||\boldsymbol x_{1}|| ||\boldsymbol x_{2}||} $$ Here, \(\boldsymbol x_{1} \cdot \boldsymbol x_{2}\) is the inner product of two vectors, which means the number of terms that both documents contain, and \(||\boldsymbol x_{1}||\) which means the number of terms in document \(\boldsymbol x_{1}\). In other words, the cosine distance means the ratio of both terms included to the average number of terms in each document.

A document can be represented as a binary vector indicating the occurrence of a term, or as a term frequency vector indicating how many times a word appears. Or, it can be represented as a relative frequency of the occurrence frequency of all terms to analyze similarity. For more information, please refer to the references.

Recently, as the amount of newly created documents increases, the problem of classifying these documents into appropriate document sets has been gaining attention. To this end, a set of pre-classified documents is used as training data, and new documents are classified using the classification analysis.

All methods studied to solve such problems are called web data mining. Recently, portal sites such as Google have appeared that organize web pages well on behalf of users and search for necessary web pages, competing with users with their own search engines. The search method used is mainly a keyword-based search method used in text mining. As the web becomes larger, there is a problem that when a common keyword is given, thousands of web contents with low relevance are searched, or web contents with synonyms for the keyword are not searched. Web content mining, which searches for websites with good information that users want in web databases that have a more complex structure than text databases, is an important research field. In addition, there are various research fields such as web document classification and web log mining.

Many studies have been developed using hyperlinks or hub pages to search for reliable web content, and a useful search method called HITS (Hyperlink Induced Topic Search) is introduced.

The HITS search method has the disadvantage of relying only on links, but it is useful for web page search. It is used in Google's search engine and is reported to obtain better search results than other search engines.

In order to analyze web log data, preprocessing processes such as refining, simplifying, and converting the data studied in Chapter 3 are necessary. Then, a multidimensional analysis of IP addresses, URLs, time, and web page content information is performed to find potential customers. By studying association rules, sequential patterns, and web access trends, we can understand users' reactions and motivations, and use the results of these analyses to improve the system, such as page design. We can help build personalized web services for users and attempt to rank web pages.

Content-based searches for images based on visual characteristics such as color, texture, and shape of the image, and is widely applied in medical diagnosis, weather forecasting, and e-commerce. Content-based search can be divided into a method of finding similar image data in a database using image samples and a method of finding data with similar characteristics in a database by describing the characteristics of the image in detail, and the latter is more commonly used. The following methods are available for representing the characteristics of images:

In image association analysis, resolution can also be important. That is, there are cases where features appear to be the same up to a certain resolution but differ at a finer resolution. In this case, association analysis is first performed at a low resolution, and then association analysis is performed while gradually increasing the resolution for patterns that satisfy the minimum support criterion.

Spatial data often exists in various data formats and in different regions as databases. In such cases, it is recommended to create a spatial data cube after organizing a data warehouse by subject, time, or other classifications for analysis. Multidimensional analysis can be used to facilitate spatial data analysis.

Spatial data analysis can be used to describe, associate, classify, cluster, analyze spatial trends, and analyze spatial data. For example, spatial association analysis is to find an association rule that says, ‘If the average monthly income is 3 million won or more and there are many sports centers in an area with a high concentration of apartments.’ Spatial cluster analysis generalizes geographical points that are detailed into cluster areas such as commercial, residential, industrial, and agricultural areas according to land use. Spatial classification analysis can be used as an example to classify regions into high-income, low-income, etc. according to average income per household. This type of spatial classification analysis is usually classified in relation to spatial objects such as administrative districts, rivers, and highways. Spatial trend analysis studies changes in spatial data that change over time when there is a variable called time in spatial data and can examine population density according to economic development, etc.