Problem of Inference & Prediction (Category:Concepts, Level:Basics)

Before Understanding problem of Inference and prediction , let us understand how Input and output variables are defined

The input variables are typically denoted using the variable output variable symbol X, with a subscript to distinguish them. The inputs go by different names, such as predictors, independent variables, features or sometimes just variables. The output variable is variable often called the response or dependent variable, and is typically denoted using the symbol Y .

More generally, suppose that we observe a quantitative response Y and p different predictors,
X1,X2, . . .,Xp. We assume that there is some relationship between Y and X = (X1,X2, . . .,Xp), which can be written in the very general form

Y = f(X) + e




Here f is some fixed but unknown function of X1, . . . , Xp, and e is a random error term, which is independent of X and has mean zero. These 2 properties are very important of error term. It should not have any relationship with any input variable and at the same time average of the error term for all observation should be approximately zero.


 In this formulation, f represents the systematic information that X provides about Y .  Statistical Learning is all about methods or approaches of estimating this function f.


Now when we estimate f we are usually solving two kinds of problems.




Prediction
In many situations, a set of inputs X are readily available, but the output Y cannot be easily obtained. In this setting, since the error term averages to zero, we can predict Y using

Y = ˆ f(X)

where f represents our estimate for f, and ^ represents the resulting prediction for Y . In this setting, ˆ f is often treated as a black box, in the sense that one is not typically concerned with the exact form of ˆ f, provided that it yields accurate predictions for Y .



Inference
We are often interested in understanding the way that Y is affected as X1, . . . , Xp change. In this situation we wish to estimate f, but our goal is not necessarily to make predictions for Y . We instead want to understand the relationship between X and Y , or more specifically, to understand how
Y changes as a function of X1, . . .,Xp. Now ˆ f cannot be treated as a black box, because we need to know its exact form. In this setting, one may be interested in answering the following questions:

• Which predictors are associated with the response? It is often the case that only a small fraction of the available predictors are substantially associated with Y . Identifying the few important predictors among a large set of possible variables can be extremely useful, depending on
the application.


• What is the relationship between the response and each predictor?
Some predictors may have a positive relationship with Y , in the sense that increasing the predictor is associated with increasing values of Y . Other predictors may have the opposite relationship. Depending on the complexity of f, the relationship between the response and a given predictor may also depend on the values of the other predictors.


• Can the relationship between Y and each predictor be adequately summarized
using a linear equation, or is the relationship more complicated?
Historically, most methods for estimating f have taken a linear form. In some situations, such an assumption is reasonable or even desirable. But often the true relationship is more complicated, in which case a linear model may not provide an accurate representation of the relationship between the input and output variables.



Thanks
Learner


Reference: Introduction to Statistical Learning, Statistics, Wiki