This project implements logistic regression from scratch to predict credit card default risk using the UCI Credit Default dataset.
The model was built without using machine learning libraries such as sklearn. All core components including the sigmoid function, cost computation, gradient descent optimization, and regularization were implemented manually.
To build a binary classification model that predicts whether a customer will default on their credit card payment based on financial and payment history features.
The following components were implemented from scratch:
- Sigmoid activation function
- Cross-entropy loss function
- Gradient descent optimization
- Train-test split
- Feature scaling (standardization)
- L2 regularization
- Prediction thresholding
- Cost vs Iterations visualization
Hypothesis:
h(x) = 1 / (1 + e^(-z))
Cost Function with L2 Regularization:
J = (1/m) * Σ log-loss + (λ / 2m) * Σ w²
Gradient Update:
dw = (1/m) Xᵀ(h - y) + (λ/m)w db = (1/m) Σ(h - y)
Bias term was not regularized.
- Test Accuracy: ~80%
- Smooth convergence observed in cost vs iterations plot
- Regularization applied to prevent overfitting
- Python
- NumPy
- Pandas
- Matplotlib
- Understanding logistic regression at a mathematical level
- Implementing gradient descent from scratch
- Importance of feature scaling
- Effect of regularization on model performance
- Difference between training and test evaluation
- Add precision, recall, and confusion matrix
- Compare with sklearn implementation
- Hyperparameter tuning
- Add regularization tuning experiments