### Linear and Nonlinear Modeling

### Linear and Nonlinear Modeling

#### Get Citation

*Select the Optimal Model for Interpreting Multivariate Data*

**Introduction to Multivariate Analysis: Linear and Nonlinear Modeling** shows how multivariate analysis is widely used for extracting useful information and patterns from multivariate data and for understanding the structure of random phenomena. Along with the basic concepts of various procedures in traditional multivariate analysis, the book covers nonlinear techniques for clarifying phenomena behind observed multivariate data. It primarily focuses on regression modeling, classification and discrimination, dimension reduction, and clustering.

The text thoroughly explains the concepts and derivations of the AIC, BIC, and related criteria and includes a wide range of practical examples of model selection and evaluation criteria. To estimate and evaluate models with a large number of predictor variables, the author presents regularization methods, including the *L*_{1} norm regularization that gives simultaneous model estimation and variable selection.

For advanced undergraduate and graduate students in statistical science, this text provides a systematic description of both traditional and newer techniques in multivariate analysis and machine learning. It also introduces linear and nonlinear statistical modeling for researchers and practitioners in industrial and systems engineering, information science, life science, and other areas.

**Introduction **

Regression Modeling

Classification and Discrimination

Dimension Reduction

Clustering

Linear Regression Models

Relationship between Two Variables

Relationships Involving Multiple Variables

Regularization

Nonlinear Regression Models

Modeling Phenomena

Modeling by Basis Functions

Basis Expansions

Regularization

Logistic Regression Models

Risk Prediction Models

Multiple Risk Factor Models

Nonlinear Logistic Regression Models

**Model Evaluation and Selection**

Criteria Based on Prediction Errors

Information Criteria

Bayesian Model Evaluation Criterion

Discriminant Analysis

Fisher’s Linear Discriminant Analysis

Classification Based on Mahalanobis Distance

Variable Selection

Canonical Discriminant Analysis

Bayesian Classification

Bayes’ Theorem

Classification with Gaussian Distributions

Logistic Regression for Classification

Support Vector Machines

Separating Hyperplane

Linearly Nonseparable Case

From Linear to Nonlinear

Principal Component Analysis

Principal Components

Image Compression and Decompression

Singular Value Decomposition

Kernel Principal Component Analysis

Clustering

Hierarchical Clustering

Nonhierarchical Clustering

Mixture Models for Clustering

**Appendix A: Bootstrap Methods **

**Appendix B: Lagrange Multipliers **

**Appendix C: EM Algorithm **

Bibliography

Index

*Select the Optimal Model for Interpreting Multivariate Data*

**Introduction to Multivariate Analysis: Linear and Nonlinear Modeling** shows how multivariate analysis is widely used for extracting useful information and patterns from multivariate data and for understanding the structure of random phenomena. Along with the basic concepts of various procedures in traditional multivariate analysis, the book covers nonlinear techniques for clarifying phenomena behind observed multivariate data. It primarily focuses on regression modeling, classification and discrimination, dimension reduction, and clustering.

The text thoroughly explains the concepts and derivations of the AIC, BIC, and related criteria and includes a wide range of practical examples of model selection and evaluation criteria. To estimate and evaluate models with a large number of predictor variables, the author presents regularization methods, including the *L*_{1} norm regularization that gives simultaneous model estimation and variable selection.

For advanced undergraduate and graduate students in statistical science, this text provides a systematic description of both traditional and newer techniques in multivariate analysis and machine learning. It also introduces linear and nonlinear statistical modeling for researchers and practitioners in industrial and systems engineering, information science, life science, and other areas.

**Introduction **

Regression Modeling

Classification and Discrimination

Dimension Reduction

Clustering

Linear Regression Models

Relationship between Two Variables

Relationships Involving Multiple Variables

Regularization

Nonlinear Regression Models

Modeling Phenomena

Modeling by Basis Functions

Basis Expansions

Regularization

Logistic Regression Models

Risk Prediction Models

Multiple Risk Factor Models

Nonlinear Logistic Regression Models

**Model Evaluation and Selection**

Criteria Based on Prediction Errors

Information Criteria

Bayesian Model Evaluation Criterion

Discriminant Analysis

Fisher’s Linear Discriminant Analysis

Classification Based on Mahalanobis Distance

Variable Selection

Canonical Discriminant Analysis

Bayesian Classification

Bayes’ Theorem

Classification with Gaussian Distributions

Logistic Regression for Classification

Support Vector Machines

Separating Hyperplane

Linearly Nonseparable Case

From Linear to Nonlinear

Principal Component Analysis

Principal Components

Image Compression and Decompression

Singular Value Decomposition

Kernel Principal Component Analysis

Clustering

Hierarchical Clustering

Nonhierarchical Clustering

Mixture Models for Clustering

**Appendix A: Bootstrap Methods **

**Appendix B: Lagrange Multipliers **

**Appendix C: EM Algorithm **

Bibliography

Index

*Select the Optimal Model for Interpreting Multivariate Data*

**Introduction to Multivariate Analysis: Linear and Nonlinear Modeling** shows how multivariate analysis is widely used for extracting useful information and patterns from multivariate data and for understanding the structure of random phenomena. Along with the basic concepts of various procedures in traditional multivariate analysis, the book covers nonlinear techniques for clarifying phenomena behind observed multivariate data. It primarily focuses on regression modeling, classification and discrimination, dimension reduction, and clustering.

The text thoroughly explains the concepts and derivations of the AIC, BIC, and related criteria and includes a wide range of practical examples of model selection and evaluation criteria. To estimate and evaluate models with a large number of predictor variables, the author presents regularization methods, including the *L*_{1} norm regularization that gives simultaneous model estimation and variable selection.

For advanced undergraduate and graduate students in statistical science, this text provides a systematic description of both traditional and newer techniques in multivariate analysis and machine learning. It also introduces linear and nonlinear statistical modeling for researchers and practitioners in industrial and systems engineering, information science, life science, and other areas.

**Introduction **

Regression Modeling

Classification and Discrimination

Dimension Reduction

Clustering

Linear Regression Models

Relationship between Two Variables

Relationships Involving Multiple Variables

Regularization

Nonlinear Regression Models

Modeling Phenomena

Modeling by Basis Functions

Basis Expansions

Regularization

Logistic Regression Models

Risk Prediction Models

Multiple Risk Factor Models

Nonlinear Logistic Regression Models

**Model Evaluation and Selection**

Criteria Based on Prediction Errors

Information Criteria

Bayesian Model Evaluation Criterion

Discriminant Analysis

Fisher’s Linear Discriminant Analysis

Classification Based on Mahalanobis Distance

Variable Selection

Canonical Discriminant Analysis

Bayesian Classification

Bayes’ Theorem

Classification with Gaussian Distributions

Logistic Regression for Classification

Support Vector Machines

Separating Hyperplane

Linearly Nonseparable Case

From Linear to Nonlinear

Principal Component Analysis

Principal Components

Image Compression and Decompression

Singular Value Decomposition

Kernel Principal Component Analysis

Clustering

Hierarchical Clustering

Nonhierarchical Clustering

Mixture Models for Clustering

**Appendix A: Bootstrap Methods **

**Appendix B: Lagrange Multipliers **

**Appendix C: EM Algorithm **

Bibliography

Index

*Select the Optimal Model for Interpreting Multivariate Data*

**Introduction to Multivariate Analysis: Linear and Nonlinear Modeling** shows how multivariate analysis is widely used for extracting useful information and patterns from multivariate data and for understanding the structure of random phenomena. Along with the basic concepts of various procedures in traditional multivariate analysis, the book covers nonlinear techniques for clarifying phenomena behind observed multivariate data. It primarily focuses on regression modeling, classification and discrimination, dimension reduction, and clustering.

*L*_{1} norm regularization that gives simultaneous model estimation and variable selection.

**Introduction **

Regression Modeling

Classification and Discrimination

Dimension Reduction

Clustering

Linear Regression Models

Relationship between Two Variables

Relationships Involving Multiple Variables

Regularization

Nonlinear Regression Models

Modeling Phenomena

Modeling by Basis Functions

Basis Expansions

Regularization

Logistic Regression Models

Risk Prediction Models

Multiple Risk Factor Models

Nonlinear Logistic Regression Models

**Model Evaluation and Selection**

Criteria Based on Prediction Errors

Information Criteria

Bayesian Model Evaluation Criterion

Discriminant Analysis

Fisher’s Linear Discriminant Analysis

Classification Based on Mahalanobis Distance

Variable Selection

Canonical Discriminant Analysis

Bayesian Classification

Bayes’ Theorem

Classification with Gaussian Distributions

Logistic Regression for Classification

Support Vector Machines

Separating Hyperplane

Linearly Nonseparable Case

From Linear to Nonlinear

Principal Component Analysis

Principal Components

Image Compression and Decompression

Singular Value Decomposition

Kernel Principal Component Analysis

Clustering

Hierarchical Clustering

Nonhierarchical Clustering

Mixture Models for Clustering

**Appendix A: Bootstrap Methods **

**Appendix B: Lagrange Multipliers **

**Appendix C: EM Algorithm **

Bibliography

Index

*Select the Optimal Model for Interpreting Multivariate Data*

**Introduction to Multivariate Analysis: Linear and Nonlinear Modeling** shows how multivariate analysis is widely used for extracting useful information and patterns from multivariate data and for understanding the structure of random phenomena. Along with the basic concepts of various procedures in traditional multivariate analysis, the book covers nonlinear techniques for clarifying phenomena behind observed multivariate data. It primarily focuses on regression modeling, classification and discrimination, dimension reduction, and clustering.

*L*_{1} norm regularization that gives simultaneous model estimation and variable selection.

**Introduction **

Regression Modeling

Classification and Discrimination

Dimension Reduction

Clustering

Linear Regression Models

Relationship between Two Variables

Relationships Involving Multiple Variables

Regularization

Nonlinear Regression Models

Modeling Phenomena

Modeling by Basis Functions

Basis Expansions

Regularization

Logistic Regression Models

Risk Prediction Models

Multiple Risk Factor Models

Nonlinear Logistic Regression Models

**Model Evaluation and Selection**

Criteria Based on Prediction Errors

Information Criteria

Bayesian Model Evaluation Criterion

Discriminant Analysis

Fisher’s Linear Discriminant Analysis

Classification Based on Mahalanobis Distance

Variable Selection

Canonical Discriminant Analysis

Bayesian Classification

Bayes’ Theorem

Classification with Gaussian Distributions

Logistic Regression for Classification

Support Vector Machines

Separating Hyperplane

Linearly Nonseparable Case

From Linear to Nonlinear

Principal Component Analysis

Principal Components

Image Compression and Decompression

Singular Value Decomposition

Kernel Principal Component Analysis

Clustering

Hierarchical Clustering

Nonhierarchical Clustering

Mixture Models for Clustering

**Appendix A: Bootstrap Methods **

**Appendix B: Lagrange Multipliers **

**Appendix C: EM Algorithm **

Bibliography

Index

*Select the Optimal Model for Interpreting Multivariate Data*

**Introduction to Multivariate Analysis: Linear and Nonlinear Modeling** shows how multivariate analysis is widely used for extracting useful information and patterns from multivariate data and for understanding the structure of random phenomena. Along with the basic concepts of various procedures in traditional multivariate analysis, the book covers nonlinear techniques for clarifying phenomena behind observed multivariate data. It primarily focuses on regression modeling, classification and discrimination, dimension reduction, and clustering.

*L*_{1} norm regularization that gives simultaneous model estimation and variable selection.

**Introduction **

Regression Modeling

Classification and Discrimination

Dimension Reduction

Clustering

Linear Regression Models

Relationship between Two Variables

Relationships Involving Multiple Variables

Regularization

Nonlinear Regression Models

Modeling Phenomena

Modeling by Basis Functions

Basis Expansions

Regularization

Logistic Regression Models

Risk Prediction Models

Multiple Risk Factor Models

Nonlinear Logistic Regression Models

**Model Evaluation and Selection**

Criteria Based on Prediction Errors

Information Criteria

Bayesian Model Evaluation Criterion

Discriminant Analysis

Fisher’s Linear Discriminant Analysis

Classification Based on Mahalanobis Distance

Variable Selection

Canonical Discriminant Analysis

Bayesian Classification

Bayes’ Theorem

Classification with Gaussian Distributions

Logistic Regression for Classification

Support Vector Machines

Separating Hyperplane

Linearly Nonseparable Case

From Linear to Nonlinear

Principal Component Analysis

Principal Components

Image Compression and Decompression

Singular Value Decomposition

Kernel Principal Component Analysis

Clustering

Hierarchical Clustering

Nonhierarchical Clustering

Mixture Models for Clustering

**Appendix A: Bootstrap Methods **

**Appendix B: Lagrange Multipliers **

**Appendix C: EM Algorithm **

Bibliography

Index