Advancing Analytics
Data Science | AI | DataOps | Engineering
backgroundblue.png

Modern Data Warehouse

Modern Data Warehouse

Azure modern data WAREHOUSING

The data landscape has changed drastically in the past two years, and what it means to build data platforms has evolved alongside it. The tools, languages and architectural patterns have all shifted to take advantage of emerging technologies, cloud scalability and advancements in Data Science… but how to you evolve alongside it?

 The typical problems of "Big Data" are becoming more mainstream:
- Data volumes are growing
- There is a boom in sensor & event data
- Exotic data types are now part of many third party integrations
- Business opportunities need immediate reaction and cannot wait on a release cycle that counts in months

We have been building Modern Data Warehouses for the last few years, adapting as tools came and went. The architectural choice is vast and constantly changing but we have a simplified reference architecture, and we want to teach you to master it. We have to embrace a polyglot architecture and understand how an orchestrate multiple tools and languages together, but it doesn't have to be a complex mess. Understanding each tool is hard, getting them to work together is an art. Get it wrong and it can cost you more than the time and resources!

Advancing Analytics use their consultancy experience and close relationships with Microsoft to deliver the most up-to-date Modern Data Warehouse course available. We will show you what works, what does not work and most importantly how to build a Modern Data Warehouse for your business. We use hands-on labs to make sure you’re getting the most out of the course and you'll build a solution to take away with you.

We have a stock Modern Data Warehouse course which teaches you & your team the core principles. We much prefer to spend a bit of time getting to know your business and the problems you have, then shape the course to meet your needs.

The course will be delivered by Terry McCann, Data Platform MVP (Microsoft MVP). Terry is recognised for his ability to convert deep technical material in to bite sized understandable chunks.

2019-04-12_08-38-06.jpg
Certs2.png

Cost / logistics

Remote delivery: £850** per delegate (minimum 5 delegates)
On-site delivery: £1000** per delegate (minimum 8 delegates)
Tailored course: POA
** If you have attended a public course let me know and we will reduce the price by 33% for one attendee.

This is a 3-day course. With an optional add-on days. Each day is charged additional to the Modern Data Warehouse course and builds on the MDW content.
Machine Learning add-on day: £850/£1000
Reporting architectures and design add-on day: £850/£1000
Advanced Streaming in a day add-on day: £850/£1000


Very good and understandable explanations of quite complex stuff. A very good kickoff to get started with Databricks. Great combination of theory and real life experiences.
— Previous course attendee

Stats.png

Agenda

(A Full agenda is available upon request)

Introduction

General introduction

  • How to use the cloud to build a Modern Data warehouse

Intro to Big Data Processing

  • Introduction to Big Data Processing - why we do what we do.
  • Introduce you to the skills required
  • How to move from On-prem to the cloud
  • They hybrid approach

Exploring the Modern Data Warehouse Architecture

  • An introduction to the tools we will cover

An introduction to Azure

  • How to use the cloud
  • Extract-Transform-Load
  • Extract-Load-Transform
  • Working with Azure
  • Getting all the tools you need set up

Data Lakes in Azure

  • An introduction to Hadoop
  • Blob Storage
  • Data Lake Gen 1
  • Data Lake Gen 2

Orchestration in Azure

  • Azure Data Factory
  • Azure Automation

Introduction to Databricks

  • The languages (Scala/Python/R/Java)
  • Introduction to Scala
  • Introduction to PySpark
  • PySpark deep dive
  • Working with the additional Spark APIs

EXPLORING AZURE DATABRICKS

  • Getting set up
  • Exploring Azure Databricks

MANAGING DATABRICKS

  • Managing Secrets
  • Orchestrating Pipelines
  • Troubleshooting Query Performance
  • Source Controlling Notebooks
  • Cluster Sizing
  • Installing packages on our cluster / All clusters
  • DATA ENGINEERING
  • Cloud ETL Patterns
  • Design patterns
  • Loading Data
  • Schema Management
  • Transforming Data
  • Storing Data
  • Managing Lakes

Databricks Delta Tables

  • Introduction to Delta, What is is how it works
  • Datalake management
  • Problems with Hadoop based lakes
  • Creating a Delta Table
  • The Transaction Log
  • Managing Schema change
  • Time travelling

Bring it all back together

  • How this all fits in to a wider architecture.
  • Projects we have worked on.
  • Managing Databricks in production
  • Deploying with Azure DevOps

Intro to Azure SQLDW

  • MPP systems
  • Distibuting data
  • CTAS

Advanced ADF Patterns

  • Automation
  • Code generation
  • Source control

ADF Mapping Data Flows

  • What are mapping tools
  • How to get started
  • Deployment
  • Patterns & more

Add-on day: Machine Learning in Azure Databricks

In this add-on day we will explore howyou can use the data in your Modern Data Warehouse to look forwad with predictive analytics.

Introduction to Machine learning

  • Getting started
  • Team Data Science Process
  • Shallow Vs Deep Learning
  • Unsupervised Vs Supervised learning
  • Training models
  • Scoring models
  • Model lifecycle

The matematics of machine learning

  • Enough maths to get you started

Machine learning in Python

  • Creating your first model
  • Clustering models
  • Regression models
  • Classification models

Machine Learning in Azure Databricks

  • MLLib in Azure Databricks
  • Batch vs Interactive machine learning

Deployment scenarios

  • Azure ML Services
  • Docker & Kubernetes

Add-on day: Advanced Streaming Scenarios

In this add-on day we will explore how to stream data using a variety of different tools and techniques in Azure.

Streaming in Azure Databricks

  • Reading from streams
  • File-streaming
  • Pushing data to a stream
  • Databricks Delta streaming

Reporting on a stream of data

  • How to set up Power BI for streaming

Introduction to Streaming

  • Streaming in Azure
  • Event Hubs/IoT hubs & Service buses

Intro to Stream Analytics & Lambda

  • Realtime Streaming with Stream Analytics
  • Lambda Architecture
  • Kappa Architecture
  • Stream Analytics

Add-on day: Power BI and Reporting in Azure

In this add-on day we will explore how access the data in your modern data warehouse. Accessing large amounts of data means you need to desing up from for aggregations and read speed.


If you would like to arrange a call regarding training please use the link below:

Alternatively, please fill out the following form and someone will get back to you.

Name *
Name