Project description in details at Medium.com:
- Part 1: Environment Setup and Ingestion https://medium.com/@jushijun/migrating-on-premise-sql-to-azure-using-medallion-architecture-part-1-setup-and-ingestion-6385406c3e49
- Part 2: Data Transformation and Pipeline Orchestration https://medium.com/@jushijun/migrating-on-premise-sql-to-azure-using-medallion-architecture-part-2-data-transformation-pipeline-bb80338e617a
- Part 3: Data Loading, Reporting, and Security https://medium.com/@jushijun/migrating-on-premise-sql-to-azure-using-medallion-architecture-part-3-data-loading-reporting-c0c8388ba9f9
- Environment Setup (Part 1): Configure Azure services and connect to the on-premise SQL database.
2. Data Ingestion (Part 1): Extract data into Azure Data Lake Storage.
3. Data Transformation (Part 2): Clean and enrich data using Azure Databricks.
4. Data Pipeline (Part 2): Orchestrate workflows with Azure Data Factory.
5. Data Loading (Part 3): Load transformed data into Azure Synapse Analytics.
6. Data Reporting (Part 3): Visualize insights with reporting tools.
7. Security and Governance (Part 3): Implement access control.
