Kaya KupferschmidtFlowman — A Declarative ETL Framework powered by Apache SparkDon’t reinvent the wheel by writing more boilerplate code. Focus on critical business logic and delegate the tricky details to a clever…·9 min read·Jun 2, 2023--4--4
Kaya KupferschmidtinTowards Data ScienceRethinking the Roles of Data Scientists, Engineers and ArchitectsWhat the wording tells about the roles — and why some companies should rethink their approach and expectations from data projects.·18 min read·Jan 20, 2021----
Kaya KupferschmidtinTowards Data ScienceUsing Permutation Tests to proof the Climate ChangeA simple statistical test shows that average temperatures are very unlikely to increase due to “bad luck”.·9 min read·Dec 23, 2020----
Kaya KupferschmidtinTowards Data ScienceData Engineering at ScaleHow to speed up building your Big Data ETL pipelines and getting them into production·7 min read·Dec 17, 2020----
Kaya KupferschmidtinTowards Data ScienceInvestigating the Climate Change with Python and Spark, Part 3Create your own Insights on Global Warming using publicly available Data.·12 min read·Dec 15, 2020--1--1
Kaya KupferschmidtinTowards Data ScienceUsing Python and Spark to research the Climate Change, Part 2Create your own Insights on Global Warming using publicly available Data.·13 min read·Dec 11, 2020----
Kaya KupferschmidtinTowards Data ScienceUsing Python and Spark to research the Climate Change, Part 1Create your own Insights on Global Warming using publicly available Data.·11 min read·Dec 8, 2020----
Kaya KupferschmidtinTowards Data ScienceDo I need Big Data? And if so, how much?Many companies follow the hype of big data without understanding the implications of the technology.·9 min read·Nov 16, 2020----
Kaya KupferschmidtinTowards Data ScienceSpark vs Pandas, part 4— RecommendationsWhy neither Spark nor Pandas is better than the other. Or: Always chose the right tool for the right job.·6 min read·Nov 14, 2020--2--2
Kaya KupferschmidtinTowards Data ScienceSpark vs Pandas, part 3 — Scala vs PythonWhy programming languages matter·11 min read·Oct 26, 2020----