Data Scientist, Engineer, AI, Educator
This project implements an automated MLOps pipeline for deploying a machine learning model, focusing on operational efficiency and security. It integrates Jenkins for CI/CD, Docker for containerization, and AWS ECS for scalable deployment, ensuring a repeatable and reliable workflow. Project Overview The pipeline automates the deployment of a Flask-based ML application, emphasizing Continuous Integration and …
This project demonstrates the deployment of a machine learning model using Docker and Kubernetes on Google Cloud Platform (GCP), highlighting skills in MLOps, containerization, and cloud infrastructure management. The process integrates a pre-trained PyCaret model into a Flask application, containerized with Docker, and orchestrated with Kubernetes for scalability and reliability. Project Overview Key Steps Outcomes …
This project is a hands-on implementation of a real-time algorithmic trading system that processes market data and news sentiment to make automated trading decisions. It integrates Apache Flink for stream processing, Redpanda for data streaming, and the Alpaca API for market data and trade execution, with sentiment analysis driving the strategy. Technologies Used How It …
This project is a real-time anomaly detection system for stock trading data, built with Redpanda and Quix. It processes streaming trade data and flags unusual patterns using both rule-based and machine learning techniques. Purpose The goal is to detect anomalies—like sudden price jumps or high-volume trades—as they happen, using a lightweight, practical setup. Technologies Used …
This project involves the development of a real-time analytics dashboard designed to process and visualize voting data for an election simulation involving three political parties: Party A, Party B, and Party C. The system integrates multiple technologies to handle data streaming, processing, and visualization, demonstrating proficiency in modern data engineering tools. For a detailed overview, …
This project demonstrates the development of a real-time data pipeline for e-commerce sales analytics using Apache Kafka, Apache Flink, PostgreSQL, and Elasticsearch. The pipeline ingests high-throughput transaction data, processes it for real-time insights, and stores it for querying and visualization, addressing needs such as inventory optimization and fraud detection. For more details, refer to the …
In today’s data-driven world, real-time monitoring of database changes is critical for applications like fraud detection and live analytics dashboards. This project demonstrates a robust real-time data pipeline using PostgreSQL, Debezium, and Kafka, orchestrated with Docker, to capture and stream database changes efficiently. The project includes: Features Technology Used Learn More Link: Read the full …
This repo shows how to use AWS EMR and Apache Spark to process big data in the cloud. It includes scripts and notes from my Medium guide on setting up and running Spark jobs. Key Features Tech Stack Link: Read the full article on Medium Code: https://github.com/shj37/AWS-EMR-Scalable-Spark-Processing
This project demonstrates the creation of an employee churn prediction pipeline using Google BigQuery, PyCaret, and Looker Studio. The goal is to predict which employees might leave based on historical data and offer insights to boost retention. Technologies Used Key Features Link: Read the full article on Medium Code: https://github.com/shj37/Employee-Churn-Prediction-with-Looker-Studio-BigQuery-and-PyCaret
This project is a real-time data pipeline that keeps tabs on YouTube video stats—like likes and views—using Kafka and ksqlDB. It pulls data from the YouTube Data API, processes it on the fly, and sends me a Telegram message when something noteworthy changes. How It Works Tech Stack Why It MattersIt’s a hands-on showcase of …