Data Science & Data Engineering blogs
Thanks for reading. Here you will find a huge range of information in text, audio and video on topics such as Data Science, Data Engineering, Machine Learning Engineering, DataOps and much more. The show notes for “Data Science in Production” are also collated here.
In this video Simon takes you through how to size a cluster. We look at what happens when you take 3GB of data and cache it on a 2 node cluster. How do you see the distribution of data? How many partitions are there on each node?. How do we achieve workload isolation? All these questions are answered.
Read MoreIn this video Simon takes you through the creation of a cluster in Azure Databricks, explore the difference between the various cluster types and Databricks run-times and options to get you started with your first cluster.
Read MoreIn this video Terry takes you through a tour of the Databricks workspace.
Read MoreWith the term Data Engineer growing exponentially, it can be difficult to pin down what exactly the role is, and where did it come from? This post dissects the history of the data engineer, how it relates to data science and business intelligence and asks the question… is it more than just ETL?
Read MoreIn this video we look at adding a new resource in Azure, exploring the managed resource group and a quick look at the ARM template for deploying Azure Databricks.
Read MoreIn this video we look to understand the core concepts of Apache Spark to understand how Databricks works. From there we look at the history of Databricks and describe the fundamentals.
Read MoreAll this month we sharing a video expanding on a core concept in Azure Databricks and Apache Spark. Subscribe on Youtube for more content, and to make sure you do not miss a video.
Read MoreWe’re thrilled to announce that both of our directors have been renewed as Microsoft Most Valuable Professionals - they would both like to share a quick message to the community!
Read MoreThe ETL window is part and parcel of how BI developers build their solutions - but is it an outdated concept? With event-driven processes, it’s fairly straight forward to move past this as a concept!
Read MoreKimball warehouses have been around for an age, but technology has changed massively since they were designed. So why are people still using it? Does it have a purpose? This blog discusses some of the background behind the questioning of Kimball
Read MoreAdvancing Analytics are heading to Seattle in November for the PASS Summit. We will be delivering a full day training day on Azure Databricks - Practical Azure Databricks: Engineering & Warehousing at Scale.
Read MoreDatawarehousing is a slow, unreactive thing, and it’s missing out on many opportunities because of it. How do you structure your modern data warehouse to react quickly and unlock value within your data, faster?
Read MoreI am super pleased to introduce the new Director of Engineering Simon Whiteley
Read MoreDocker is a great option for Production Machine Learning. We have been exploring how to build an application, let’s now look at building a Machine Learning model in Python ready for deployment with Docker.
Read MoreOn a recent training course we were asked 60 great questions. I an hoping that they may help others. We have re-worded them and answered each in more detail. This is part 6 or 6.
Read MoreOn a recent training course we were asked 60 great questions. I an hoping that they may help others. We have re-worded them and answered each in more detail. This is part 5 or 6.
Read More
