A detailed case study of migrating Spark workloads from AWS EMR to Kubernetes, including challenges, solutions, and measurable improvements in scalability, reliability, and cost-effectiveness