Today, the rise of online financial activities has ushered in unprecedented convenience but has also given rise to a looming threat – credit card fraud. As financial transactions grow more intricate, cybercriminals continuously refine their tactics to exploit vulnerabilities.
This blog briefly goes into the realm of Credit Card Fraud Detection, examining how advanced Ensemble Machine Learning techniques act as the frontline defence against fraudulent activities.
The objectives are simple, like any approach to Machine Learning.
Perform basic data preprocessing in Python
Model a binary classification task using the Scikit-Learn Python APIs
Train Support Vector Machine and Decision Tree models using Scikit-Learn and Snap ML
Run inference and assess the quality of the trained models
Building the Model.
In a financial setting, the dataset is usually highly unbalanced, and the target variable is not usually represented properly, thus making us give special attention to the training phase when building the model. A simple way to do this is to make the model biased to the minority class of the training data. The model would then take account of the class weights of the training data. In your data, the predictor variable or dependent variable must have two values: 0 (the credit card transaction is legitimate) and 1 (the credit card transaction is fraudulent) to ensure it successfully performs as a binary classification task.
On your dataset, ensure you do the following
Preprocess the dataset such as scaling and normalization.
Since our dataset has monetary values, standardize features by removing the mean and scaling to unit variance
Here is a Python code snippet that describes this
# Importing StandardScaler from scikit-learn
from sklearn.preprocessing import StandardScaler
# Assuming financial_data is a pandas DataFrame containing financial data
num_columns = len(financial_data.columns)
# Scaling columns 1 to num_columns of the DataFrame
# The iloc[:, 1:num_columns] selects all rows and columns from index 1 to nth
# StandardScaler() standardizes features by removing the mean and scaling to unit variance
financial_data.iloc[:, 1:num_columns] = StandardScaler().fit_transform(financial_data.iloc[:, 1:num_columns])
# Converting the entire DataFrame into a NumPy array
# The .values attribute of a DataFrame returns the underlying data as a NumPy array
data_matrix = financial_data.values
Then define your dependent and independent variables X is the independent variable and y is the dependent variable.
X = data_matrix[:, 1:num_columns]
# y: labels vector
y = data_matrix[:, num_columns]
# data normalization
X = normalize(X, norm="l1")
Now that your dataset is fully ready for classification, it is best practice to divide your data into train/test splits in order to prevent overfitting and have high out-of-sample accuracy. Here is how to do this using 70% of our data for training and 30% for testing and using a random state to ensure reproducibility.
# Splitting the dataset into training (70%) and testing (30%) sets. 'X' and 'y' are features and labels, respectively.
# 'random_state' ensures reproducibility of the split, and 'stratify=y' ensures distribution of labels is consistent in both sets.
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y)
Now we balance class weights inversely proportional to class frequencies in the input data y_train. This helps to address class imbalance by giving more weight to less frequent classes during training. Then we create a Decision-Tree classifier and fit/train the data into the classifier
Training the model.
# compute the sample weights to be used as input to the train routine so that
# it takes into account the class imbalance present in this dataset
w_train = compute_sample_weight('balanced', y_train)
# import the Decision Tree Classifier Model from scikit-learn
from sklearn.tree import DecisionTreeClassifier
# for reproducible output across multiple function calls, set random_state to a given integer value
sklearn_dt = DecisionTreeClassifier(max_depth=4, random_state=35)
# train the model
sklearn_dt.fit(X_train, y_train, sample_weight=w_train)
Now we create a Support Vector Machine classifier and fit/train the data into the classifier
# import the linear Support Vector Machine (SVM) model from Scikit-Learn
from sklearn.svm import LinearSVC
# instatiate a scikit-learn SVM model
# to indicate the class imbalance at fit time, set class_weight='balanced'
# for reproducible output across multiple function calls, set random_state to a given integer value
sklearn_svm = LinearSVC(class_weight='balanced', random_state=31, loss="hinge", fit_intercept=False)
# train the model
sklearn_svm.fit(X_train, y_train)
Evaluating the model.
After training both models, use an evaluation metric (like accuracy, precision, recall, F1 score, etc.) to assess their performance. You would typically use the test dataset (not used in training) to get an unbiased evaluation, then get probabilities or decision functions for each model
Convert the averaged predictions into final class decisions.
Evaluate these final decisions against the true labels from the test set using an appropriate metric like accuracy.
from sklearn.metrics import accuracy_score
# Get probabilities or decision functions for each model
dt_probs = sklearn_dt.predict_proba(X_test)[:, 1] # Probabilities for the positive class
svm_probs = sklearn_svm.predict_proba(X_test)[:, 1]
# Average the probabilities
avg_probs = (dt_probs + svm_probs) / 2
# Convert averaged probabilities to binary predictions (e.g., using 0.5 as a threshold)
ensemble_predictions = np.where(avg_probs > 0.5, 1, 0)
# Calculate and print ensemble accuracy
ensemble_accuracy = accuracy_score(y_test, ensemble_predictions)
print(f"Ensemble Accuracy: {ensemble_accuracy}")
Final thoughts.
In our exploration of combating credit card fraud using machine learning, we employed an ensemble technique that leverages the strengths of both Decision Tree Classifiers and Support Vector Machines (SVM). This approach demonstrates the power of ensemble methods in enhancing predictive performance, particularly in complex and sensitive applications like fraud detection.
Key Takeaways:
Addressing Class Imbalance:
- Credit card fraud detection often deals with highly imbalanced datasets. Our approach used weighted training for the Decision Tree and class weighting for the SVM to effectively manage this imbalance, ensuring that our models did not overlook the less frequent, yet critical, fraudulent transactions.
Leveraging Different Model Strengths:
- Decision Trees are known for their interpretability and ability to capture non-linear patterns, while SVMs are effective in high-dimensional spaces. By combining these models, our ensemble method potentially captures a broader range of fraud indicators than either model alone.
Ensemble Strategy – Averaging Predictions:
- We averaged the probabilistic outputs of both models to make final predictions. This simple yet effective technique can often lead to better performance as it balances out individual model biases and variances.
Performance Evaluation:
- The ensemble method showed promising results, as indicated by the accuracy metric. However, in fraud detection, other metrics like precision, recall, and the area under the ROC curve are also crucial for understanding the effectiveness of the model in identifying fraudulent transactions accurately.
Future Directions
Exploring Advanced Ensemble Techniques:
- Beyond simple averaging, more sophisticated ensemble methods like stacking or blending could be explored to potentially yield better results.
Fine-Tuning and Hyperparameter Optimization:
- Further tuning of model parameters and employing techniques like cross-validation could enhance model performance.
Real-time Application and Scalability:
- Considering the implementation of this model in a real-time fraud detection system and assessing its scalability and adaptability to new and evolving fraud patterns is a crucial next step.
Ethical Considerations and False Positives:
- It's important to balance sensitivity to fraudulent transactions with the need to minimize false positives, as these can significantly impact customer experience.
In conclusion, the use of an ensemble ML technique in credit card fraud detection illustrates a promising path forward. This approach harnesses the complementary strengths of different algorithms, offering a robust tool against the ever-evolving challenge of financial fraud. As with any machine learning application, continuous evaluation and adaptation are key to maintaining its effectiveness in the dynamic landscape of financial transactions.
Follow my blog for more posts regarding Software Engineering / Computer Science and AI Engineering.