Agile Database Techniques overviews ways of working (WoW) and ways of thinking (WoT) that enable agile database development.

This book overviews the myriad of development techniques that support agile database development.  This includes the fundamental data skills and knowledge that everyone on an agile project team should have, how to take an evolutionary approach to database development, and fundamental development skills associated with working with data. 

This book captured the initial version of The Agile Data method.

Book Organization

Part 1 describes the fundamental data skills and knowledge that everyone on an agile project team should have. This includes the basics of object orientation, relational databases, the object-relational impedance mismatch, data modeling, and how to deal with legacy data issues. Without this common base of knowledge it is very difficult for application developers and data professionals to work together effectively. A significant problem in the IT industry is that most data books do not cover object-oriented development issues and most object books seem to ignore data issues. This needs to stop.

• Chapter 1: Data Can Be Agile. Explores how application developers, database administrators (DBAs), enterprise architects, and data administrators can work together effectively in an agile environment.
• Chapter 2: From Uses Cases to Databases – Real-World UML. Object technology is the norm for modern projects, therefore it is critical for everyone to understand the basics of object orientation and the Unified Modeling Language (UML) 2.x (including UML data modeling).
• Chapter 3: Data Modeling 101. Data modeling is a fundamental skill that all software professionals, including object professionals, require if they wish to store data effectively. UML data modeling is covered in detail.
• Chapter 4: Data Normalization. Normalization is a collection of design strategies that ensure data is stored in one place and one place only, promoting the design of highly cohesive and loosely coupled data schemas.
• Chapter 5: Class Normalization. The concepts of normalization can be applied to object schemas, a complementary technique to design patterns and programming idioms.
• Chapter 6: Relational Database Technology, Like it or Not. Relational databases (RDBs) have been the dominant technology for persisting business objects and will likely remain so, therefore you need to understand technology.
• Chapter 7: Overcoming The Object-Relational Impedance Mismatch.Object technology and relational technology are based on different paradigms, presenting us with a technical impedance mismatch that we must overcome. Worse yet is the cultural mismatch between object professionals and data professionals that must also be bridged.
• Chapter 8: Legacy Databases – Everything You Need to Know But Are Afraid to Deal With.Virtually every project team finds that it needs to work with legacy data sources, and when doing so quickly discover serious data quality, database design, and data architecture problems that they need to overcome.

Part 2 focuses on how to take an evolutionary approach to data. This section sets the foundation for a model-driven development (MDD) approach, or more accurately an agile model-driven development (AMDD) approach where your application code and database schemas are based on agile models. This isn’t the only way to work, you may decide to take a test-driven development (TDD) approach instead or better yet combine it with AMDD. Both methods support evolutionary development but because MDD is very common within the data community and I suspect that they will gravitate more towards an AMDD approach rather than a TDD approach. However, some agile developers, particularly extreme programmers, prefer TDD over AMDD.Luckily the two approaches work very well together so it really doesn’t matter. The implication is that TDD will become more important to data professionals in the coming years.This section also describes database refactoring, an evolutionary technique that enables you to improve your database design in small steps. In many ways database refactoring is normalization after the fact. Chapters describing mapping objects to relational databases, performance tuning, database encapsulation, and supporting tools are included in this part because they enable evolutionary development

• Chapter 9: Vive La Evolution. This chapter argues that agile software development is real and here to stay, that data is an important aspect of most systems, and that all agile methodologies take an evolutionary approach to development.Therefore if data professionals wish to remain relevant they must embrace evolutionary development.
• Chapter 10: Agile Model Driven Development (AMDD). Agile Modeling (AM) is a chaordic collection of practices for the effective development of models and documents. AM defines a streamlined approach to evolutionary modeling for agile developers.
• Chapter 11: Test-Driven Development (TDD). Test-driven development (TDD) is a development approach where developers add a test before they add new functional code. This minimizes the feedback loop while providing developers with the confidence to proceed in small, evolutionary steps.
• Chapter 12: Database Refactoring. A database refactoring is a small change to a database schema that improves its design. The process of database refactoring enables you to evolve your data schema in step with the evolution of the systems that access that data.
• Chapter 13: Database Encapsulation Strategies. Encapsulating access to data sources enables developers to evolve both the data schema as well as their application schemas independently of one another.
• Chapter 14: Mapping Objects to Relational Databases. When working with object and relational technologies you must map your object schema to your database schema, evolving it over time as your two schemas evolve.
• Chapter 15: Performance Tuning. The need to ensure sufficient system performance is often a primary motivator of evolutionary changes late in the lifecycle for traditional projects.On agile projects it motivates changes almost from the beginning.
• Chapter 16: Tools for Evolutionary Database Development. An overview of testing tools, …

Part 3 focuses on implementation techniques and strategies such as concurrency control, security access control, finding objects in relational databases, referential integrity, and the effective use of XML. An important observation is that many of these topics are traditionally thought of as data issues, but as you’ll see there is far more to them than this – it isn’t a black and white world.

• Chapter 17: Implementing Concurrency Control. Concurrency control deals with the issues involved with allowing multiple people simultaneous access to shared entities, be they objects, data records, or some other representation.
• Chapter 18: Finding Objects in Relational Databases. A common programming task is retrieve one or more objects from a database, perhaps to display a list of people that work in a department, to list available inventory items that meet user-defined search criteria, or simple to implement a report.
• Chapter 19: Implementing Referential Integrity and Shared Business Logic. When one entity references another then that other entity should exist.Referential integrity is an issue when a row references another row, an object references another object, or an object represents a data within a database (or vice versa).
• Chapter 20: Implementing Security Access Control. Your system must ensure that authenticated users access only what they are authorized to.The complexities of modern development require developers to understand a wide range of options to do exactly this.
• Chapter 21: Implementing Reports. There are several common strategies to support reporting functionality within your system and/or to integrate with a common reporting application that crosses a variety of systems.
• Chapter 22: Advanced XML? No, Just Realistic XML.XML is a robust and growing set of technologies. Unfortunately many people seem to have forgotten the data community’s hard-earned lessons and seem to think that XML is the panacea that will solve all of their problems. This is hardly the case.XML has a very bright future but level setting is required.

Part 4 describes strategies for adopting agile database techniques.These chapter provides advice for individuals who want to become agile software developers and for organizations that want to adopt agile techniques.

• Chapter 23: How to Become Agile. As an individual you need to decide to become more agile in the way that you work. First you need to adopt “the agile attitude” and then you need to actively work towards gaining the skills to make you a valued member on an agile project team.
• Chapter 24: Bringing Agility Into Your Organization. People, and therefore organizations, resist change. To adopt agile database techniques you need to overcome this resistance to change.

Updated Material

Over the years I have applied the original techniques presented in this book in a wide variety of situations, learning as I went.  I shared these learnings in updated descriptions of the techniques at AgileData.org, which was the site I developed when originally writing the book.