Hi, this is a page related to a new project I am starting to work on. A while ago I wrote a post on Data Mesh, how to use a domain driven approach when creating a data platform. So I wondered what if I wanted to combine domain driver design, with microservices and data mesh thinking?
I could learn some new things and expand my knowledge in areas I do not know or am familiar enough.
I started with the following key concepts and things I wanted to learn and improve upon:
- Domain Driven Design
- Vertical Slice Architecture
- Unified API
- Federated Gateways
- Microservices, starting small with container instances with docker in Azure, then moving to Kubernetes in AKS
- Data Mesh
- Data platforms and Data warehousing
- GraphQL Mesh
- Hybrid Cloud solution between Azure (main platform) and AWS
- Infrastructure using Terraform
- Jenkis for CI/CD (although I will start probably with something easier, like using Bitbucket pipelines)
- Identity management through AWS Cognito
- Write operations safety with AWS SQS
- Backend APIs with Spring Boot and Java
- Frontend apps using most likely React
- Caching with Redis
- ELK Stack for logging and logic metrics
What I want to do is to create a system that allows me to experiment with different technologies, especially with data ingestion, processing, loading then taking all this and exposing this through domain specific APIs internally before exposing data to the outside world through and API gateway and/or application.
Here is the first version of the architecture diagram:
System Architecture Diagram V3
I will keep updating this page when new content arises and the project moves forward. It’s a long haul project.
For the love of challenges :) 