Regression Analysis - NU

1. Simple Linear Regression (SLR)The SLR Model: Defining the relationship Y = β₀ + β₁ X + ε, identifying independent (X) and dependent (Y) variables, and making standard Gauss-Markov assumptions about the error term (ε).Ordinary Least Squares (OLS): Deriving estimators for the intercept (β̂₀) and slope (β̂₁) by minimizing the sum of squared residuals.Hypothesis Testing & Inference: Conducting t-tests on the slope parameter to evaluate linear significance, alongside building confidence intervals for predictions.Goodness of Fit: Calculating and interpreting the Coefficient of Determination (R²) to evaluate model explanatory power.2. Multiple Linear Regression (MLR)The Matrix Approach: Formulating the multiple variable model Y = Xβ + ε and deriving the OLS normal equation estimator \(\hat{\beta} = (X^T X)^{-1} X^T Y\).Analysis of Variance (ANOVA): Partitioning the total sum of squares (SST) into regression sum of squares (SSR) and error sum of squares (SSE); executing overall model F-tests.Adjusted R²: Evaluating explanatory gains vs. degrees of freedom penalties when adding extra explanatory variables.3. Model Diagnostics and Remedial MeasuresResidual Analysis: Checking standard assumptions using residual vs. fitted plots to detect non-linearity, heteroscedasticity, and error dependencies.Multicollinearity: Identifying variance inflation among independent variables using Variance Inflation Factors (VIF) and implementing remedial adjustments.Outliers & Influential Points: Detecting unusual data coordinates utilizing leverage scores, Studentized residuals, and Cook’s Distance metrics.4. Polynomial and Qualitative RegressionPolynomial Models: Adjusting linear structures to model curved trends by introducing higher-order terms (X², X³).Dummy Variables: Incorporating qualitative, categorical predictors (e.g., gender, location) into the regression framework using binary indicators.