Cybernetical intelligence : engineering cybernetics with machine intelligence / Kelvin K. L. Wong.
CYBERNETICAL INTELLIGENCE Highly comprehensive, detailed, and up-to-date overview of artificial intelligence and cybernetics, with practical examples and supplementary learning resources Cybernetical Intelligence: Engineering Cybernetics with Machine Intelligence is a comprehensive guide to the fiel...
Saved in:
Online Access: |
Full Text (via IEEE) |
---|---|
Main Author: | |
Format: | Electronic eBook |
Language: | English |
Published: |
Hoboken, NJ :
Wiley : IEEE Press,
[2024]
|
Subjects: |
Table of Contents:
- Preface
- About the Author
- About the Companion Website
- 1 Artificial Intelligence and Cybernetical Learning
- 1.1 Artificial Intelligence Initiative
- 1.2 Intelligent Automation Initiative
- 1.2.1 Benefits of IAI
- 1.3 Artificial Intelligence Versus Intelligent Automation
- 1.3.1 Process Discovery
- 1.3.2 Optimization
- 1.3.3 Analytics and Insight
- 1.4 The Fourth Industrial Revolution and Artificial Intelligence
- 1.4.1 Artificial Narrow Intelligence
- 1.4.2 Artificial General Intelligence
- 1.4.3 Artificial Super Intelligence
- 1.5 Pattern Analysis and Cognitive Learning
- 1.5.1 Machine Learning
- 1.5.1.1 Parametric Algorithms
- 1.5.1.2 Nonparametric Algorithms
- 1.5.2 Deep Learning
- 1.5.2.1 Convolutional Neural Networks in Advancing Artificial Intelligence
- 1.5.2.2 Future Advancement in Deep Learning
- 1.5.3 Cybernetical Learning
- 1.6 Cybernetical Artificial Intelligence
- 1.6.1 Artificial Intelligence Control Theory
- 1.6.2 Information Theory
- 1.6.3 Cybernetic Systems
- 1.7 Cybernetical Intelligence Definition
- 1.8 The Future of Cybernetical Intelligence
- Summary
- Exercise Questions
- Further Reading
- 2 Cybernetical Intelligent Control
- 2.1 Control Theory and Feedback Control Systems
- 2.2 Maxwell's Analysis of Governors
- 2.3 Harold Black
- 2.4 Nyquist and Bode
- 2.5 Stafford Beer
- 2.5.1 Cybernetic Control
- 2.5.2 Viable Systems Model
- 2.5.3 Cybernetics Models of Management
- 2.6 James Lovelock
- 2.6.1 Cybernetic Approach to Ecosystems
- 2.6.2 Gaia Hypothesis
- 2.7 Macy Conference
- 2.8 McCulloch-Pitts
- 2.9 John von Neumann
- 2.9.1 Discussions on Self-Replicating Machines
- 2.9.2 Discussions on Machine Learning
- Summary
- Exercise Questions
- Further Reading
- 3 The Basics of Perceptron
- 3.1 The Analogy of Biological and Artificial Neurons
- 3.1.1 Biological Neurons and Neurodynamics
- 3.1.2 The Structure of Neural Network
- 3.1.3 Encoding and Decoding
- 3.2 Perception and Multilayer Perceptron
- 3.2.1 Back Propagation Neural Network
- 3.2.2 Derivative Equations for Backpropagation
- 3.3 Activation Function
- 3.3.1 Sigmoid Activation Function
- 3.3.2 Hyperbolic Tangent Activation Function
- 3.3.3 Rectified Linear Unit Activation Function
- 3.3.4 Linear Activation Function
- Summary
- Exercise Questions
- Further Reading
- 4 The Structure of Neural Network
- 4.1 Layers in Neural Network
- 4.1.1 Input Layer
- 4.1.2 Hidden Layer
- 4.1.3 Neurons
- 4.1.4 Weights and Biases
- 4.1.5 Forward Propagation
- 4.1.6 Backpropagation
- 4.2 Perceptron and Multilayer Perceptron
- 4.3 Recurrent Neural Network
- 4.3.1 Long Short-Term Memory
- 4.4 Markov Neural Networks
- 4.4.1 State Transition Function
- 4.4.2 Observation Function
- 4.4.3 Policy Function
- 4.4.4 Loss Function
- 4.5 Generative Adversarial Network
- Summary
- Exercise Questions
- Further Reading
- 5 Backpropagation Neural Network
- 5.1 Backpropagation Neural Network
- 5.1.1 Forward Propagation
- 5.2 Gradient Descent
- 5.2.1 Loss Function
- 5.2.2 Parameters in Gradient Descent
- 5.2.3 Gradient in Gradient Descent
- 5.2.4 Learning Rate in Gradient Descent
- 5.2.5 Update Rule in Gradient Descent
- 5.3 Stopping Criteria
- 5.3.1 Convergence and Stopping Criteria
- 5.3.2 Local Minimum and Global Minimum
- 5.4 Resampling Methods
- 5.4.1 Cross-Validation
- 5.4.2 Bootstrapping
- 5.4.3 Monte Carlo Cross-Validation
- 5.5 Optimizers in Neural Network
- 5.5.1 Stochastic Gradient Descent
- 5.5.2 Root Mean Square Propagation
- 5.5.3 Adaptive Moment Estimation
- 5.5.4 AdaMax
- 5.5.5 Momentum Optimization
- Summary
- Exercise Questions
- Further Reading
- 6 Application of Neural Network in Learning and Recognition
- 6.1 Applying Backpropagation to Shape Recognition
- 6.2 Softmax Regression
- 6.3 K-Binary Classifier
- 6.4 Relational Learning via Neural Network
- 6.4.1 Graph Neural Network
- 6.4.2 Graph Convolutional Network
- 6.5 Cybernetics Using Neural Network
- 6.6 Structure of Neural Network for Image Processing
- 6.7 Transformer Networks
- 6.8 Attention Mechanisms
- 6.9 Graph Neural Networks
- 6.10 Transfer Learning
- 6.11 Generalization of Neural Networks
- 6.12 Performance Measures
- 6.12.1 Confusion Matrix
- 6.12.2 Receiver Operating Characteristic
- 6.12.3 Area Under the ROC Curve
- Summary
- Exercise Questions
- Further Reading
- 7 Competitive Learning and Self-Organizing Map
- 7.1 Principal of Competitive Learning
- 7.1.1 Step 1: Normalized Input Vector
- 7.1.2 Step 2: Find the Winning Neuron
- 7.1.3 Step 3: Adjust the Network Weight Vector and Output Results
- 7.2 Basic Structure of Self-Organizing Map
- 7.2.1 Properties Self-Organizing Map
- 7.3 Self-Organizing Mapping Neural Network Algorithm
- 7.3.1 Step 1: Initialize Parameter
- 7.3.2 Step 2: Select Inputs and Determine Winning Nodes
- 7.3.3 Step 3: Affect Neighboring Neurons
- 7.3.4 Step 4: Adjust Weights
- 7.3.5 Step 5: Judging the End Condition
- 7.4 Growing Self-Organizing Map
- 7.5 Time Adaptive Self-Organizing Map
- 7.5.1 TASOM-Based Algorithms for Real Applications
- 7.6 Oriented and Scalable Map
- 7.7 Generative Topographic Map
- Summary
- Exercise Questions
- Further Reading
- 8 Support Vector Machine
- 8.1 The Definition of Data Clustering
- 8.2 Support Vector and Margin
- 8.3 Kernel Function
- 8.3.1 Linear Kernel
- 8.3.2 Polynomial Kernel
- 8.3.3 Radial Basis Function
- 8.3.4 Laplace Kernel
- 8.3.5 Sigmoid Kernel
- 8.4 Linear and Nonlinear Support Vector Machine
- 8.5 Hard Margin and Soft Margin in Support Vector Machine
- 8.6 I/O of Support Vector Machine
- 8.6.1 Training Data
- 8.6.2 Feature Matrix and Label Vector
- 8.7 Hyperparameters of Support Vector Machine
- 8.7.1 The C Hyperparameter
- 8.7.2 Kernel Coefficient
- 8.7.3 Class Weights
- 8.7.4 Convergence Criteria
- 8.7.5 Regularization
- 8.8 Application of Support Vector Machine
- 8.8.1 Classification
- 8.8.2 Regression
- 8.8.3 Image Classification
- 8.8.4 Text Classification
- Summary
- Exercise Questions
- Further Reading
- 9 Bio-Inspired Cybernetical Intelligence
- 9.1 Genetic Algorithm
- 9.2 Ant Colony Optimization
- 9.3 Bees Algorithm
- 9.4 Artificial Bee Colony Algorithm
- 9.5 Cuckoo Search
- 9.6 Particle Swarm Optimization
- 9.7 Bacterial Foraging Optimization
- 9.8 Gray Wolf Optimizer
- 9.9 Firefly Algorithm
- Summary
- Exercise Questions
- Further Reading
- 10 Life-Inspired Machine Intelligence and Cybernetics
- 10.1 Multi-Agent AI Systems
- 10.1.1 Game Theory
- 10.1.2 Distributed Multi-Agent Systems
- 10.1.3 Multi-Agent Reinforcement Learning
- 10.1.4 Evolutionary Computation and Multi-Agent Systems
- 10.2 Cellular Automata
- 10.3 Discrete Element Method
- 10.3.1 Particle-Based Simulation of Biological Cells and Tissues
- 10.3.2 Simulation of Microbial Communities and Their Interactions
- 10.3.3 Discrete Element Method-Based Modeling of Biological Fluids and Soft Materials
- 10.4 Smoothed Particle Hydrodynamics
- 10.4.1 SPH-Based Simulations of Biomimetic Fluid Dynamic
- 10.4.2 SPH-Based Simulations of Bio-Inspired Engineering Applications
- Summary
- Exercise Questions
- Further Reading
- 11 Revisiting Cybernetics and Relation to Cybernetical Intelligence
- 11.1 The Concept and Development of Cybernetics
- 11.1.1 Attributes of Control Concepts
- 11.1.2 Research Objects and Characteristics of Cybernetics
- 11.1.3 Development of Cybernetical Intelligence
- 11.2 The Fundamental Ideas of Cybernetics
- 11.2.1 System Idea
- 11.2.2 Information Idea
- 11.2.3 Behavioral Idea
- 11.2.4 Cybernetical Intelligence Neural Network
- 11.3 Cybernetic Expansion into Other Fields of Research
- 11.3.1 Social Cybernetics
- 11.3.2 Internal Control-Related Theories
- 11.3.3 Software Control Theory
- 11.3.4 Perceptual Cybernetics
- 11.4 Practical Application of Cybernetics
- 11.4.1 Research on the Control Mechanism of Neural Networks
- 11.4.2 Balance Between Internal Control and Management Power Relations
- 11.4.3 Software Markov Adaptive Testing Strategy
- 11.4.4 Task Analysis Model
- Summary
- Exer.
- cise Questions
- Further Reading
- 12 Turing Machine
- 12.1 Behavior of a Turing Machine
- 12.1.1 Computing with Turing Machines
- 12.2 Basic Operations of a Turing Machine
- 12.2.1 Reading and Writing to the Tape
- 12.2.2 Moving the Tape Head
- 12.2.3 Changing States
- 12.3 Interchangeability of Program and Behavior
- 12.4 Computability Theory
- 12.4.1 Complexity Theory
- 12.5 Automata Theory
- 12.6 Philosophical Issues Related to Turing Machines
- 12.7 Human and Machine Computations
- 12.8 Historical Models of Computability
- 12.9 Recursive Functions
- 12.10 Turing Machine and Intelligent Control
- Summary
- Exercise Questions
- Further Reading
- 13 Entropy Concepts in Machine Intelligence
- 13.1 Relative Entropy of Distributions
- 13.2 Relative Entropy and Mutual Information
- 13.3 Entropy in Performance Evaluation
- 13.4 Cross-Entropy Softmax
- 13.5 Calculating Cross-Entropy
- 13.6 Cross-Entropy as a Loss Function
- 13.7 Cross-Entropy and Log Loss
- 13.8 Application of Entropy in Intelligent Control
- 13.8.1 Entropy-Based Control
- 13.8.2 Fuzzy Entropy
- 13.8.3 Entropy-Based Control Strategies
- 13.8.4 Entropy-Based Decision-Making
- Summary
- Exercise Questions
- Further Reading
- 14 Sampling Methods in Cybernetical Intelligence
- 14.1 Introduction to Sampling Methods
- 14.2 Basic Sampling Algorithms
- 14.2.1 Importance of Sampling Methods in Machine Intelligence
- 14.3 Machine Learning Sampling Methods
- 14.3.1 Random Oversampling
- 14.3.2 Random Undersampling
- 14.3.3 Synthetic Minority Oversampling Technique
- 14.3.4 Adaptive Synthetic Sampling
- 14.4 Advantages and Disadvantages of Machine Learning Sampling Methods
- 14.5 Advanced Sampling Methods in Cybernetical Intelligence
- 14.5.1 Ensemble Sampling Method
- 14.5.2 Active Learning
- 14.5.3 Bayesian Optimization in Sampling
- 14.6 Applications of Sampling Methods in Cybernetical Intelligence
- 14.6.1 Image Processing and Computer Vision
- 14.6.2 Natural Language Processing
- 14.6.3 Robotics and Autonomous Systems
- 14.7 Challenges and Future Directions
- 14.8 Challenges and Limitations of Sampling Methods
- 14.9 Emerging Trends and Innovations in Sampling Methods
- Summary
- Exercise Questions
- Further Reading
- 15 Dynamic System Control
- 15.1 Linear Systems
- 15.2 Nonlinear System
- 15.3 Stability Theory
- 15.4 Observability and Identification
- 15.5 Controllability and Stabilizability
- 15.6 Optimal Control
- 15.7 Linear Quadratic Regulator Theory
- 15.8 Time-Optimal Control
- 15.9 Stochastic Systems with Applications
- 15.9.1 Stochastic System in Control Systems
- 15.9.2 Stochastic System in Robotics and Automation
- 15.9.3 Stochastic System in Neural Networks
- Summary
- Exercise Questions
- Further Reading
- 16 Deep Learning
- 16.1 Neural Network Models in Deep Learning
- 16.2 Methods of Deep Learning
- 16.2.1 Convolutional Neural Networks
- 16.2.2 Recurrent Neural Networks
- 16.2.3 Generative Adversarial Networks
- 16.2.4 Deep Learning Based Image Segmentation Models
- 16.2.5 Variational Auto Encoders
- 16.2.6 Transformer Models
- 16.2.7 Attention-Based Models
- 16.2.8 Meta-Learning Models
- 16.2.9 Capsule Networks
- 16.3 Deep Learning Frameworks
- 16.4 Applications of Deep Learning
- 16.4.1 Object Detection
- 16.4.2 Intelligent Power Systems
- 16.4.3 Intelligent Control
- Summary
- Exercise Questions
- References
- Further Reading
- 17 Neural Architecture Search
- 17.1 Neural Architecture Search and Neural Network
- 17.2 Reinforcement Learning-Based Neural Architecture Search
- 17.3 Evolutionary Algorithms-Based Neural Architecture Search
- 17.4 Bayesian Optimization-Based Neural Architecture Search
- 17.5 Gradient-Based Neural Architecture Search
- 17.6 One-shot Neural Architecture Search
- 17.7 Meta-Learning-Based Neural Architecture Search
- 17.8 Neural Architecture Search for Specific Domains
- 17.8.1 Cybernetical Intelligent Systems: Neural Architecture Search in Real-World
- 17.8.2 Neural Architecture Search for Specific Cybernetical Control Tasks
- 17.8.3 Neural Architecture Search for Cybernetical Intelligent Systems in Real-World
- 17.8.4 Neural Architecture Search for Adaptive Cybernetical Intelligent Systems
- 17.9 Comparison of Different Neural Architecture Search Approaches
- Summary
- Exercise Questions
- Further Reading
- Final Notes on Cybernetical Intelligence
- Index.