Amazon DynamoDB Data Modeling for Architects & Developers
Note:
-
This course does NOT use AWS console for showing you basic DynamoDB operations
-
This course does NOT focus on just coding
This course primarily focusses on Data Modeling practices and patterns for Amazon DynamoDB. You will learn DynamoDB best practices, tips and techniques for building modern and cost effective applications/microservices. At a high level, course is divided into 2 parts:
Part-1 Objective: Build a solid understanding of DynamoDB
-
How DynamoDB works under the covers?
-
Hands-on exercises to try out the DynamoDB API(s) using CLI and Python code
-
Feature/capability comparison with relational database
Part-2 Objective: Learn to apply DynamoDB design patterns
-
Learn DynamoDB data modeling process
-
How to apply patterns to address business requirements
-
Analyze the access patterns and identify the right pattern to use
-
Hands-on exercises to build a data model for a banking use case (refer: ACME case study)
Tools in use
-
NoSQL workbench
-
Local DynamoDB for model development & testing
-
AWS cloud account for testing (minimal use)
-
An integrated development environment (like Visual Studio Code)
-
Python – if you would like to run the code samples on your machine
-
Git client
Is this course for me?
Yes – if you are an Architect responsible for designing apps on DynamoDB
Yes – if you are a Developer and aspiring to become an architect
Yes – if you are a DBA and looking to expand your skills to NoSQL databases
Yes – if you are a Technologist who is curious on how DynamoDB works under the covers
No – if you are just looking for basic “How-to” tutorial on DynamoDB
No – if you are looking for coding guidance DynamoDB apps
Course approach
Course does not assume that you already know DynamoDB. For that reasons, I am first covering the basics of DynamoDB. But unlike other courses, I will not just show you how-to use DynamoDB rather I will show you how DynamoDB works under the covers !! I strongly believe that to effectively use a database, an architect MUST understand how database technology works under the covers.
Data modeling is an iterative process. I am sharing a prescriptive guidance on the process and then showing step-by-step how to apply the process. As a case study I am using an example of ACME bank (a fictitious retail bank). Throughout the course we will be applying the process to ACME bank use cases to evolve a full blown DynamoDB data model.
Case study : ACME Bank
A fictitious bank that has embarked on a journey to modernize its retail banking systems. You are the Lead Architect for the retail banking modernization initiative. It is your responsibility:
-
To work with business owners to understand the requirements
-
To model the data for DynamoDB
-
Prove/Disprove that DynamoDB will meet the application needs
-
Educate stakeholders on DynamoDB capabilities
-
Help ACME Bank CTO to make a decision on whether to use DynamoDB or not
Introduction
Raj's introduction !!
I will provide you an outline of the course and make suggestions on how to get the most out of this course
Tools, Local Development Environment
Setup local development environment
You will learn about NoSQL Workbench capabilities
You will get hands on experience with using CLI and Workbench against the local DynamoDB
Setup VSC and clone the course repository
Amazon DynamoDB Fundamentals
In this lesson I will provide you a logical view of the DynamoDB service and introduce you to basic terminology.
Learn how DynamoDB pricing works. You learn the terms WCU/RCU/WRU/RRU
Learn about the two capacity modes supported by DynamoDB.
In this exercise you will experiment with capacity modes
DynamoDB is NOT type-less, it supports multiple types of data
In this exercise you will create a simple model
Learn about the control & data plane operations
Learn the DynamoDB Control Plane API
CRUD: Create API
Learn how Item Writes work under the covers
Learn about the data plane API
Set BatchWriteItem API in action
CRUD: Retrieve API
Learn how Reads work under the covers
Checkout the GetItem, Scan and BatchGetItem in action
Learn how queries are processed and how to use the query operation
CRUD: Update & Delete API
Learn how Update and Deletes work under the covers
Try out the UpdateItem and DeleteItem API
Try the DeleteItem and BatchWriteItem API
CRUD : PartiQL Queries
Learn how to use PartiQL queries with DynamoDB
DynamoDB Secondary Indexes
Learn the differences between Relational database indexes and DynamoDB indexes. Learn about the types of indexes and which one to use when.
Deep dive into LSI
Try out an LSI
Deep dive into GSI
Try out a GSI
Single Table Design
Learn the ideas behind Single Table Design and why it is important to use it for DynamoDB
Learn the process of Single Table Design/modeling process
Case Study : ACME Bank
ACME Bank is a fictitious company that has decided to use DynamoDB for their retail banking systems. You are the lead architect who is given the responsibility to design the DynamoDB data model.
In this lesson we will go deeper into ERD and Access patterns that need to be addressed with DynamoDB model
Modeling : Entities
Learn how to model entities
Apply the entity model to ACME retail banking data model
Modeling : Primary Keys
Learn the patterns for defining the primary key for items
Address the issue with primary key defined for the item in earlier iteration
Modeling : One to Many Relationships
Learn the patterns used for designing one-to-many relationship between entities
Analyze the relationship between customer and account entity
Apply an appropriate pattern to ACME data model - design the relationship between account and customer
Modeling : Secondary Indexes
Address some of the customer access patterns with secondary indexes
Model the transaction entity
Analyze the remaining access patterns to see if you can take advantage of Sparse indexes.
Implement a Sparse GSI for the Cust & Txn access pattern identified in earlier lesson
Analyze the remaining pattern to see if we can use index overloading for saving costs
Add an overloaded index
Modeling : Many-to-Many Relationship
Learn about the adjacency list pattern
Analyze ACME bank use case for applicability of adjacency pattern
Modeling : Preventing Hot Partitions
What are hot partitions? and patterns for addressing the issue with patterns
Iteration#9 Apply hot partition pattern to ACME state distribution access pattern
Modeling : Efficient Scans
Sequential scans are in-efficient from cost and performance perspective. How can it be addressed?
(Iteration#10) Address scan performance challenge with a report for ACME bank
ACID Transactions
Learn how ACID transactions are supported by DynamoDB
Learn how to use TransactWrite API
Try out the TransactWrite API
What does idempotency mean in the context of DynamoDB
Try out the TransactGetItem
Modeling : Transactions
(Iteration#11.1) Design a counter at Account level
(Iteration#11.2) Design the credit transaction
Iteration# 11.3) Design the debit transaction
DynamoDB Streams
You will learn (a) what are DyanamoDB streams (b) How the work under the covers
You will learn about shards and how applications read stream record events from the stream
In this hands on exercise you will try out the stream API
In this hands on exercise you will use Python code to read the stream events
21. Time to Live (TTL)
In this lesson you will learn about TTL and how it works with streams
You will gain hands on experience with the TTL feature
Modeling : DynamoDB Streams
(Iteration# 12) ACME bank has a need for a daily stats report. In this lesson you will take care of the requirement using Stream + Lambda function
ACME bank transactions are archived after 7 years. In this exercise you will model this requirement.
