ML interviews test both theoretical foundations and practical judgment. The questions below cover the concepts that come up most consistently, with the kind of depth interviewers actually want to hear.
What is the bias-variance tradeoff?
Bias is the error from incorrect assumptions in the learning algorithm — a high-bias model is too simple and underfits the data. Variance is the error from sensitivity to small fluctuations in the training set — a high-variance model is too complex and overfits. The tradeoff is that reducing one typically increases the other. The goal is to find the model complexity that minimizes total error on unseen data. Regularization, cross-validation, and ensemble methods are all tools for managing this tradeoff.
What is the difference between L1 and L2 regularization?
Both add a penalty term to the loss function to discourage large weights. L2 (Ridge) adds the sum of squared weights, which shrinks all weights toward zero but rarely to exactly zero. L1 (Lasso) adds the sum of absolute values of weights, which tends to produce sparse solutions where some weights are exactly zero — effectively performing feature selection. L2 is preferred when you believe most features are relevant; L1 when you suspect many features are irrelevant and want automatic selection.
How does gradient descent work?
Gradient descent is an optimization algorithm that iteratively updates model parameters in the direction that reduces the loss function. At each step, it computes the gradient of the loss with respect to each parameter and moves the parameters by a small amount (the learning rate) in the negative gradient direction. Batch gradient descent uses the full dataset per update. Stochastic gradient descent (SGD) uses one sample. Mini-batch gradient descent — the most common variant in practice — uses a small random batch, balancing the noisy but fast updates of SGD against the stable but slow updates of full-batch.
What is overfitting and how do you prevent it?
Overfitting happens when a model learns the training data too well, including its noise, and fails to generalize to new data. Prevention strategies include: getting more training data, simplifying the model architecture, applying regularization (L1/L2 or dropout), using early stopping during training, and cross-validation to detect when validation loss starts diverging from training loss.
Explain precision, recall, and F1 score
Precision is the fraction of positive predictions that are actually positive: TP / (TP + FP). Recall is the fraction of actual positives that the model correctly identifies: TP / (TP + FN). F1 is the harmonic mean of the two: 2 * (precision * recall) / (precision + recall). Precision matters when false positives are costly (spam detection — you do not want to delete real email). Recall matters when false negatives are costly (cancer screening — you do not want to miss a case). F1 balances both and is useful when the class distribution is uneven.
What is cross-validation?
Cross-validation is a technique to estimate how well a model will generalize to an independent dataset. In k-fold cross-validation, the training data is split into k equal subsets. The model is trained k times, each time using k-1 folds for training and the remaining fold for validation. The final performance metric is averaged across all k runs. This gives a more reliable estimate than a single train-test split, especially on small datasets.
What is the difference between a generative and discriminative model?
A discriminative model learns the decision boundary between classes — it models P(y|x) directly. Logistic regression and SVMs are discriminative. A generative model learns the distribution of each class — it models P(x|y) and uses Bayes' theorem to compute P(y|x). Naive Bayes and Gaussian Mixture Models are generative. Generative models can generate new data samples; discriminative models typically cannot. Discriminative models usually achieve better classification accuracy when labeled data is sufficient.
What is the curse of dimensionality?
As the number of features increases, the volume of the feature space grows exponentially. Data becomes increasingly sparse — any fixed number of training samples covers a smaller and smaller fraction of the space. This causes distance-based algorithms like k-NN to break down because all points appear roughly equidistant in high dimensions. It also makes overfitting more likely. Principal component analysis, feature selection, and regularization are common responses.
How does a random forest work?
A random forest is an ensemble of decision trees trained on random subsets of the training data (bootstrap sampling) with a random subset of features considered at each split. The final prediction is the majority vote (classification) or average (regression) across all trees. The randomness ensures the trees are decorrelated, so their errors tend to cancel out. Random forests are robust to overfitting, handle high-dimensional data well, and provide feature importance estimates.
What is the vanishing gradient problem?
In deep neural networks trained with backpropagation, gradients are multiplied layer by layer as they flow backward through the network. When using activation functions like sigmoid or tanh, whose derivatives are at most 0.25, the gradient shrinks exponentially as it passes through layers. By the time it reaches early layers, the gradient is nearly zero and those weights stop updating — the network fails to learn long-range dependencies. ReLU activations, residual connections (ResNets), batch normalization, and careful weight initialization (He or Xavier) are all strategies developed to address this.