Workshop – Continuous Data Warehousing: A Disciplined Hybrid Method
Data warehouse (DW) teams are increasingly asked to do more, do it quicker, and produce higher quality while doing so. Data – more accurately the right data in the right format at the right time – is critical for decision makers across your organization. These “decision makers” include both people as well as machine learning (ML) applications, both of whom expect your team to provide high quality data regardless of how it is sourced. Your teams need to learn how to succeed at agile data warehousing and agile business intelligence.
This interactive workshop teaches data professionals a proven approach to building and evolving DW solutions. It works through a disciplined hybrid approach that adopts ways of working (WoW) and ways of thinking (WoT) from agile, lean, and traditional sources. Students will work through the DW lifecycle from beginning to end and then back again.
What You Will Learn in Continuous Data Warehousing
-
- Identify the roles on a continuous data warehousing/business intelligence (BI) team
- Discover how to organize and lead a continuous data warehousing team
- Understand practical strategies for overcoming common challenges on continuous data warehousing teams
- Learn an effective way of working (WoW) for your team, combining agile, lean, and traditional strategies to do so
- Understand how to take an optimal approach to initial requirements modeling, architectural modeling, and planning that fits your context
- Learn how to write question stories, a form of user story that is tailored for DW/BI development
- Explore DataOps practices and see how to apply them in practice
- Discover how to successfully deliver the first release of your DW solution
- Learn how to adopt a continuous delivery approach for subsequent releases
- Learn how lean governance, including data governance, strategies are required for continuous DW teams to succeed
Logistics
-
- Length: 2 days – 16 hours class time (exclusive of breaks)
- Delivery strategy: On premises
Audience
This workshop is aimed at data professionals who want to learn about continuous data warehousing through the entire DevOps/DataOps lifecycle.
Prerequisites
-
- An understanding of data and database fundamentals is required. Students are expected to be familiar with basic data terminology and techniques, including the ability to read a data model.
- An understanding of Scrum is required. We will do a very brief review of Scrum concepts but we assume that students have taken a Scrum workshop before attending this one.
- An understanding of Agile and Lean concepts is very useful. We will review key concepts, particularly those from Disciplined Agile (DA), that are applicable to DW teams. You may want to read the short book Choose Your WoW! 2/e to learn about the fundamentals of DA
- Knowledge of DataVault2 (DV2) is useful. In the workshop we will review key architecture and design heuristics from DV2.
Detailed Outline of Continuous Data Warehousing
Foundational Concepts
-
-
- Defining DW, business intelligence (BI), Agile, Lean, and Continuous DW/BI
- Agile and lean ways of thinking (WoT) for DW
- Explore common challenges faced by agile DW teams
-
Roles on Continuous Datawarehousing Teams
-
-
- Primary and supporting roles
- Responsibilities
- How to build an effective team
-
Choosing the Right Lifecycle
-
-
- Comparing traditional, Scrum, and Kanban-based approaches
- Comparing project and continuous delivery approaches
- The traditional and agile project lifecycles for data warehouses
- The lean continuous delivery lifecycle for data warehouses
-
Initial Requirements for Agile Data Warehousing (DW)
-
-
- Question stories
- Agile data mapping/lineage
- Initial agile data modeling
- Identifying when you are “done” for now
-
Initial Architecture for Agile DW
-
-
- Overview of the DataVault 2 architecture strategy
- Legacy data source modeling
- Clean data architecture
- Initial sizing and scheduling: Guesstimation
-
Construction – The first few sprints
-
-
- Thin/vertical slicing
- The user-facing BI solution
- Clean database design
-
Working in Sprints: Overcoming the challenges
-
-
- Common Scrum ceremonies
- Look-ahead analysis
- Look-ahead planning: Sizing and scheduling
-
DataOps Practices for Development
-
-
- Database refactoring to pay down data debt
- Automated database testing
- Continuous database integration
- Continuous database deployment
-
Putting it All together: A Disciplined Sprint
-
-
- Making sprints work in practice
- Lean data governance
-
Deployment
-
-
- Getting ready to deploy
- Continuous database deployment
- Deploying data infrastructure updates
-
Evolving Your Production DW
-
-
- From project to product
- Beyond agile DW: Lean continuous delivery for DW
-
Wrap Up
-
-
- Revisiting the challenges with agile and DW
-
Public Offerings of Agile Data Warehousing
Taught by Me
No public offerings are currently scheduled.
Taught by Authorized Trainers
The Continuous Data Warehousing workshop is offered in Europe on a regular basis by Scalefree International GmbH.