The 7 families of artificial intelligence and their implementation in Python

Par: Abilian 13/03/2023 Tous les articles

Python has emerged in recent years as the most popular language for artificial intelligence (AI).

What are the main relevant technologies and application areas?

Based on our experience with Python and machine learning, as well as other key AI technologies, we will attempt to provide an overview of the field.

Outline of this post

We can distinguish seven main approaches and applications of artificial intelligence:

  1. Expert systems: they use formal logic rules to simulate human reasoning in a specific domain, by answering questions or solving problems.
  2. Supervised machine learning: these algorithms learn from labelled data, i.e. for which the expected answer is known in advance, to perform tasks such as pattern recognition, classification or prediction.
  3. Unsupervised machine learning: these algorithms discover structures or relationships in unlabelled data, without human supervision, to perform tasks such as segmentation or dimensionality reduction.
  4. Reinforcement learning: these algorithms learn from interactions with a dynamic environment, by maximising a reward or minimising a penalty, to perform tasks such as planning or decision making.
  5. Artificial neural networks: these are inspired by the workings of biological neurons and allow complex functions to be modelled from data, to perform tasks such as image recognition or natural language processing.
  6. Natural language processing: uses text analysis techniques to understand human language and perform tasks such as machine translation, text generation or query understanding.
  7. Multi-agent systems: simulate the behaviour of several agents, such as robots or people, to perform tasks cooperatively or competitively, using coordination, planning and negotiation techniques.

Expert Systems in Python

An expert system is a computer system that uses knowledge and rules to solve problems in a specific domain. They aim to mimic human reasoning by using decision rules based, on the one hand, on data and, on the other hand, on expert knowledge of a particular domain.

There are many interesting expert system projects in Python or usable from Python. They are usually based on rule engines and the Rete algorithm. Here are some examples:

  1. Experta: a Python library for building expert systems strongly inspired by CLIPS, a fork of Pyknow (cf. below).
  2. CLIPS: an expert system written in C, developed since 1985 by NASA. It is possible to use it from Python thanks to the Clispy project.
  3. pyknow: a rule-based expert system library for Python, easy to use and extend. This project is unfortunately not maintained since 2018.
  4. Nools: a JavaScript library for expert systems, but which also has a Python interface for developers who prefer to use Python. This project has not been maintained since 2018.
  5. pyke: a Python rules engine that provides features such as feedback, inheritance and template rules. This project has not been maintained since 2010.
  6. Intellect. This project has not been maintained since 2012.

Application areas

These projects can be used in many fields, including medicine, law, finance and engineering. They can be used to diagnose diseases, to assist in financial decision-making, to assist in construction planning, to assist in production planning in industry, and in many other cases where human expertise is needed.

Our comment

As can be seen, the projects identified above are mostly abandoned or stalled. However, the inference engine approach is still important for some application areas. We recommend using the CLIPS project, which is over 30 years old and still active, and interfacing it with Python.

References

  • Presentation: "Expert System With Python" - part 1 and part 2

Supervised Machine Learning

Supervised machine learning is a branch of artificial intelligence that involves building a model from labelled data. Labelled data is data that has been previously tagged with a known answer or class, so that the model can learn to make predictions from new data.

There are many supervised machine learning projects in Python. Here is a selection:

  1. Scikit-learn: Scikit-learn is a popular machine learning library in Python that supports several types of supervised machine learning models, such as decision trees, random forests, SVMs, neural networks, etc.
  2. TensorFlow: TensorFlow is an open source deep learning library developed by Google. It provides a wide variety of supervised machine learning models, such as neural networks, SVMs, decision trees, etc.
  3. Keras: Keras is a high-level deep learning library that builds on TensorFlow to provide easy-to-use and easy-to-configure supervised machine learning models.
  4. PyTorch: PyTorch is another popular deep learning library that provides supervised machine learning models for several tasks, such as classification, segmentation, image recognition, etc.
  5. XGBoost: XGBoost ("eXtreme Gradient Boosting") is a performance-optimised decision tree ensemble library used for classification and regression tasks.
  6. LightGBM: LightGBM is a fast and efficient decision tree ensemble library for classification and regression.
  7. CatBoost: CatBoost is a deep learning library for classification and regression tasks (GBDT) that can be used with categorical data.

Application areas

The main applications of supervised machine learning are as follows:

  1. Classification: aims to classify data into different categories or classes. For example, it can be used to classify emails as spam or non-spam, to classify images into different categories, or to predict whether a customer will buy a particular product.
  2. Regression: aims to predict numerical values from input data. For example, it can be used to predict the price of a house based on various characteristics such as size, location and amenities.
  3. Pattern recognition: such as handwriting recognition or speech recognition.
  4. Detection of anomalies or suspicious behaviour in data, which can be useful for detecting fraud or cyber attacks.
  5. Recommending products or services based on user preferences or purchase history.

These supervised machine learning projects are widely used in industry (e-commerce, healthcare, finance, security, predictive maintenance...) and research, and are regularly updated to incorporate the latest advances in machine learning.

Unsupervised Machine Learning Projects

Unsupervised machine learning is a branch of machine learning that involves discovering patterns and structures in data without any prior labels or supervision. Some common applications include:

  1. Text classification: classifying documents according to their content. Unsupervised machine learning algorithms can analyse large volumes of text data and discover categories of text (topic modeling) based on similarity and theme.
  2. Anomaly detection in data by finding patterns or data points that behave differently from normal.
  3. Segmenting data based on similar characteristics. This can be useful for segmenting customers into groups of similar behaviour, segmenting images based on their visual characteristics, or segmenting time sequences based on their trends.
  4. Reducing the dimensionality of data by finding the most important or representative features of the data.
  5. Recommending content to users based on their preferences and interests. Unsupervised machine learning algorithms can analyse user behaviour and profile data to suggest similar products or content.
  6. Clustering: grouping data into clusters based on similarity of characteristics. Clustering algorithms can be used to find groups of similar customers, to segment images based on their visual characteristics, or to group time sequences based on their trends.

Here are two general purpose Python library projects that implement the main unsupervised machine learning algorithms:

  1. Scikit-learn: in addition to the supervised learning features mentioned above, Scikit-learn offers a wide variety of unsupervised machine learning models, such as clustering (Affinity Propagation, Agglomerative Clustering, BIRCH, DBSCAN, K-Means, Mini-Batch K-Means, Mean Shift, OPTICS, Spectral Clustering, Mixture of Gaussians, etc. ), dimension reduction (PCA, t-SNE, etc.), etc.
  2. TensorFlow: similarly, TensorFlow offers unsupervised machine learning models, such as auto-encoder, clustering, dimension reduction, etc.

Reinforcement learning

Reinforcement learning is a branch of machine learning that aims to develop algorithms that can learn to make decisions by interacting with an environment.

In reinforcement learning, an agent learns to make decisions by observing the state of the environment it is in, performing actions and receiving rewards or penalties for its actions. The agent's objective is to maximise its reward in the long term.

The learning process takes place in several stages. First, the agent observes the state of the environment and makes a decision. Next, the environment evaluates the agent's action and provides it with a reward or penalty depending on the quality of the action. The agent then uses this information to adjust its decision strategy to optimise its reward in the long run.

The main projects we have identified are

  1. Gymnasium: Gymnasium is a platform for the development of reinforcement learning algorithms that offers a wide variety of learning environments, such as video games, physical simulations, etc.
  2. TensorFlow Agents: TensorFlow Agents is a Python library for implementing reinforcement learning algorithms based on TensorFlow.
  3. Stable Baselines: Stable Baselines is a Python library for reinforcement learning that provides a wide variety of reinforcement learning algorithms based on PyTorch.
  4. Dopamine: Dopamine is an open-source search library for reinforcement learning offered by Google, based on Tensorflow.
  5. PyBullet: PyBullet is an open-source physics simulation library that provides reinforcement learning.

Application areas

The main application areas are :

  1. Robotics: Reinforcement learning is used to teach robots to perform complex tasks, such as autonomous navigation, assembly of parts, manipulation of objects, etc.
  2. Games: Reinforcement learning is used to develop intelligent agents that can play games such as chess, Go, poker and video games.
  3. Online Advertising: to optimise online advertising strategies by learning how to maximise click-through rates, conversion rates and profits.
  4. Process Control: to control industrial processes such as power generation, waste management, inventory management, etc.
  5. Finance: to optimise investment portfolios, forecast financial asset prices and detect financial fraud.
  6. Health: to optimise personalised medical treatments, treatment planning, disease detection and patient monitoring.

Reinforcement learning is also often combined with techniques such as deep neural networks and dynamic programming to develop efficient algorithms.

Neural Networks

Principles

Artificial neural networks are a set of algorithms that aim to mimic the functioning of the human brain. The main principles are as follows:

  • Individual function: each neuron performs a weighted sum of inputs, then applies a non-linear activation function to produce an output. The weights of the connections between neurons are adjusted during training to optimise the performance of the network.

  • Architecture: they are composed of layers of interconnected neurons. Each layer receives inputs from the previous layer and produces an output for the next layer.

  • Learning: they learn from data by adjusting the weights of the connections between neurons to minimise a cost function that measures the difference between the predicted and actual outputs. The most common learning methods are gradient backpropagation and genetic algorithms.

  • Types of networks: There are several types of networks, including multi-layer neural networks (MLP), convolutional neural networks (CNN), recurrent neural networks (RNN) and short-term memory neural networks (LSTM).

Main projects

Here is a selection of the most active projects in this field:

  1. PyTorch: PyTorch is an open-source library for Python developed by Facebook, which allows the creation of neural networks and the training of machine learning models. PyTorch is particularly popular in the research field, where it is used for creating deep neural networks and training machine learning models.
  2. TensorFlow: TensorFlow is an open-source machine learning platform developed by Google. It offers a wide range of tools for creating and training neural networks, including convolutional, recurrent, and generative adversarial networks (GANs).
  3. Keras: Keras is an open-source library for deep learning, which provides an intuitive and user-friendly API on top of Tensorflow. It allows to create and train neural networks in a simple and fast way. It also provides modules for data pre-processing and tools for performance evaluation.
  4. MXNet: Apache MXNet is an open-source library for Python originally developed by Amazon. It allows the creation of deep neural networks for a variety of machine learning applications, including speech recognition, computer vision, and machine translation.
  5. scikit-learn offers a simple implementation of multi-layer neural networks for supervised learning.

Applications

These libraries are used in the following areas:

  1. Image recognition and computer vision: object recognition in images, face detection, image segmentation, scene understanding, etc.
  2. Natural Language Processing: automatic translation, text generation, speech recognition, sentiment analysis, automatic question answering, etc.
  3. Prediction and classification: as discussed above, neural networks are used for time series prediction, image classification, fraud detection, anomaly detection, and demand prediction, among others.
  4. Video games: real-time scene analysis, performance optimisation, AI behaviour generation, gesture recognition, etc.
  5. Robotics: environmental perception, trajectory planning, object recognition, decision making, etc.

Language Processing (NLP)

Natural Language Processing, or NLP, is a field of artificial intelligence that processes and analyses natural human language. NLP enables computers to understand, analyse, manipulate and generate natural language.

Areas of application

NLP is used in a wide variety of applications, such as machine translation, sentiment analysis, speech recognition, text generation, natural language understanding and information retrieval.

NLP involves the use of techniques such as named entity recognition, sentence segmentation, parsing, semantic analysis, topic modelling, text classification, natural language generation and deep learning to help computers understand and process natural language.

Main projects

Here are some open source projects in Python, which we have used in our projects:

  1. Natural Language Toolkit (NLTK): a mature library (project started in 2001) that provides tools for tokenization, stemming, lemmatization, named entity recognition and parsing.
  2. spaCy: a pipeline processing library that uses pre-trained models for specific tasks such as named entity recognition, parsing and lemmatisation. SpaCy is known for its processing speed and large-scale processing capability, thanks to its use of optimised algorithms and efficient data structures
  3. Gensim: a Python library specialising in topic modelling, document similarity and document vectorisation.
  4. TextBlob: provides a simple API for performing common NLP tasks, such as tokenization, lemmatization, sentiment analysis and language detection.

Multi-agent systems

Multi-agent systems (MAS) is a field of AI that deals with systems where several autonomous agents interact to solve complex problems.

Applications

MAS have many applications in various fields, including:

  1. Robotics : they are used in robot design to enable efficient communication and cooperation between multiple robots to accomplish complex tasks.
  2. Games : they are used in game design to enable interaction between multiple players and to create virtual opponents that act autonomously and intelligently.
  3. Finance : they are used in financial market modelling to simulate the behaviour of market participants and to predict market trends and developments.
  4. Supply Chain Management : they are used in supply chain management to enable efficient communication between different actors in the chain, such as suppliers, transporters and retailers.
  5. Intelligent transport : they are used in the design of intelligent transport systems to optimise traffic flow and to enable communication between autonomous vehicles and traffic management systems.
  6. Monitoring systems : they are used in security surveillance to enable effective communication and collaboration between different sensors and security officers.
  7. Military : they are used to simulate battle scenarios to optimise strategies and tactics and to train troops, to enable effective coordination between different actors on the battlefield and to improve decision making.

Some projects in Python

Here are some multi-agent system projects in Python:

  1. Mesa: an open source library for multi-agent simulation based on Python, which provides tools for multi-agent model creation, visualisation and analysis of results.
  2. SPADE (Smart Python Agent Development Environment): provides a set of Python libraries for the creation, simulation and deployment of autonomous agents.

References