Real-Time Fraud Detection ML Pipeline with SageMaker & Databricks

Summary

A fintech startup in Singapore, needed to move beyond legacy fraud detection based on hourly batch jobs. I worked remotely with their ML team to build a real-time machine learning pipeline that detects fraudulent mobile wallet transactions under 300ms using streaming, serverless, and cloud-native tools.

Business Challenge

As scaled across Southeast Asia, their fraud risk increased—especially during promotions. Their existing fraud detection system, based on hourly batch processing, had significant limitations:

• Only detected ~70% of fraud cases, with 15% missed entirely.
• High false positive rate (~18%) impacted user experience.
• Delayed alerts prevented real-time intervention or transaction blocking.

The goal was to replace batch detection with an ML-powered scoring system capable of handling real-time transaction streams with high precision and low latency.

Solution and Architecture

We designed a real-time fraud detection pipeline using AWS and Databricks components. Transactions streamed via Kafka were enriched with user behavior and blacklist checks, then scored via a SageMaker ML endpoint—all within milliseconds.

• Kafka: Ingested real-time transactions from mobile wallet app.
• Databricks: Performed batch and streaming feature engineering across 2M+ historical transactions.
• SageMaker: Trained and hosted a LightGBM model with 0.94 ROC AUC.
• Lambda: Queried DynamoDB + S3 and called SageMaker for sub-300ms scoring.
• Athena & CloudWatch: Monitored fraud capture rate, latency, and false positives.

Steps Taken

• Risk Profiling: Collaborated with client ML engineer to define fraud types (velocity, account takeover, device/IP mismatch).
• Data Ingestion: Pulled and labeled six months of transactions and behavioral logs.
• Feature Engineering: Created time-based and categorical features using PySpark in Databricks.
• Model Training: Trained and tuned XGBoost and LightGBM models; LightGBM selected for production.
• Real-Time Pipeline: Lambda triggered by Kafka events enriched data from DynamoDB/S3 and scored via SageMaker.
• Monitoring & Feedback: Logged predictions and outcomes to Athena for continuous retraining and dashboard tracking.

Impact and Results

• Increased fraud detection recall from 70% to 95%.
• Reduced false positive rate from 18% to 5%.
• Scoring latency averaged 280ms—suitable for real-time payment decisions.
• Estimated $1.2M in annual fraud loss prevention.
• Enabled retraining every 2 weeks based on new fraud trends.

This solution combined low-latency architecture with advanced ML modeling to enable PaySure to respond to fraud threats in real time—improving both security and customer trust.