Trad-ML breadth review (regression, trees, ensembles, clustering)

Interview questions to prep

1. What's the monotonic-stack invariant, and how does each pop give you the answer?
2. Why is the total work O(n) when the inner loop looks O(n²)?

Interview questions to prep

1. Derive the OLS solution θ = (XᵀX)⁻¹Xᵀy. When is XᵀX not invertible?
2. Show that OLS = MLE under Gaussian noise.

Interview questions to prep

1. Walk through the four classical assumptions of linear regression and how to diagnose violations.
2. What's heteroscedasticity and how do you fix it?

Interview questions to prep

1. Compare MSE, MAE, and Huber loss — what do you use when outliers matter?
2. Why is Huber loss differentiable and robust at the same time?

Interview questions to prep

1. Implement a vectorized linear regression forward pass for X @ w + b and state the expected tensor shapes.
2. Implement one gradient-descent training step for linear regression and explain loss vs cost vs prediction error.
3. Does sklearn's LinearRegression use gradient descent or ordinary least squares? Why does that matter in an interview?

Interview questions to prep

1. Walk me through how GBM fits each new tree on the negative gradient of the loss.
2. Why is GBM more sensitive to learning rate than Random Forest?

Interview questions to prep

1. What does XGBoost do differently from vanilla GBM that made it dominate Kaggle?
2. How does XGBoost handle missing values automatically?

Interview questions to prep

1. When would you pick LightGBM over XGBoost?
2. What problem does CatBoost's ordered boosting solve?