Data Scientist, Engineer, AI, Educator
This project involves the development of a real-time analytics dashboard designed to process and visualize voting data for an election simulation involving three political parties: Party A, Party B, and Party C. The system integrates multiple technologies to handle data streaming, processing, and visualization, demonstrating proficiency in modern data engineering tools. For a detailed overview, …
This repo shows how to use AWS EMR and Apache Spark to process big data in the cloud. It includes scripts and notes from my Medium guide on setting up and running Spark jobs. Key Features Tech Stack Link: Read the full article on Medium Code: https://github.com/shj37/AWS-EMR-Scalable-Spark-Processing
This project is a real-time data pipeline that pulls data from an external API, processes it with Apache Kafka and Spark, and stores it in Apache Cassandra. Apache Airflow handles the scheduling, and Docker ties it all together for a clean, reproducible setup. Details: What I Learned:I got practical experience with real-time data tools, from …