Data Science & Data Engineering blogs
Thanks for reading. Here you will find a huge range of information in text, audio and video on topics such as Data Science, Data Engineering, Machine Learning Engineering, DataOps and much more. The show notes for “Data Science in Production” are also collated here.
At the Data + AI Summit, Simon delivered a session on “Achieving Lakehouse Models with Spark 3.0“. During the session there were a load of great questions. Here the questions and the answers from Simon. Drop a comment if you were in the session and have any follow up questions!
Read MoreOne of the first of many big announcements at the 2020 Spark and AI Summit was the official release of Koalas 1.0, the pandas API on top of Apache Spark.
This blog will explore how Koalas differs from PySpark.
Read MoreEveryone’s talking about Azure Synapse Analytics, but does it sometimes feel like they’re talking about different things? Are you having trouble following where Azure SQL Datawarehouse is these days? In this post, Simon attempts to clarify the marketing message and talk about what’s actually coming and where we should be thinking about using it.
Read MoreThe thought of learning Scala fills many with fear, its very name often causes feelings of terror. This suggests it’s either doing something very good, or very bad! The truth is Scala can be used for many things; from a simple web application to complex ML (Machine Learning). Moreover, it unusually fully incorporates two programming paradigms: OOP (Object Orientated Programming) and FP (Functional programming).
In this blog series I will break down some of the barriers and myths hindering mere mortals from learning Scala, and hopefully set some of you on the path to FP enlightenment within your data projects
Read MoreWe spent the best part of October speaking at various conferences in Europe, but we took special time out to attend the Spark & AI Summit in Amsterdam – two days of keynotes, spark deep-dives, machine learning use cases and some fantastic conversations.
Read More
