Unified APIs and Federated Gateways – Part 1: Unified API, Monolith and Microservices Architecture

Introduction

Hello,

In this blog post, I will start to open up some thoughts on things and directions I want to take the over all architecture for my side project Element where I study and try out things to learn. I’m going to focus on the following approaches and technologies:

I am primarily going to focus on two aspects of a distributed system:

  1. External Communication with your system and parts of it
  2. Internal Communication between parts of your system

For the External Communication I will explore the Unified API approach with GraphQL Mesh, Apollo GraphQL, Netflix Federated Gateway and API Gateways.

Internal Communication between services can be tough and can be done in different ways. I will use the tools described above with other approaches as Domain Driven Design, bounded contexts, Data Mesh and events.

I will start by exploring these tools, technologies and approaches before I finally create and present an idea of how I might approach this in my own project.

I am still testing things out and learning, so for more info to any unanswered questions, look at the end of this post for links to different sources I used for this blog post.

A good picture what I am trying to achieve is what I saw with Netflix GraphQL federation and their evolution in their system architecture. I think the following image shows quite well different kinds of internet based solutions look between the client and server side. In this blog post, the end result that I will try to achieve will be a federated gateway.

Evolution of an API Architecture
https://netflixtechblog.com/how-netflix-scales-its-api-with-graphql-federation-part-1-ae3557c187e2

Monoliths vs. Microservices

Before I go on, I would like to point out one thing, if you don’t need a distributed system architecture, don’t do it :). The complexity of it is large and in my own current opinion the best use cases are for large solutions and services that server high volume of users and/or data.

Although you can implement microservices or similar distributed systems’ architecture with reasonable resources and time, you have to do things from different perspectives and automate as much as possible.

When it comes to monolith architecture, I would say if you are working on a small to medium size project this might be a good approach. Although it is funny to note that due to the different issues that arise with a monolith approach, your internal code structure and design might look similar to that of a distributed system’s architecture like microservices. Example: Domain Driven Design approach or Vertical Slice Architecture. Notice that it is not the same thing as a microservice architecture, since things like a database is most likely shared between the entire monolith solution.

But let’s look at a good list of comparison between monolith and microservices architecture:

Monoliths’ Advantages

Easy Development

Easy Testing, with everything you need in one package – from UI to database.

Easy Deployment, all you have to do is package the entire project & deploy one binary file

Easy to scale horizontally, by adding multiple instances on multiple machines.

https://www.pentalog.com/blog/it-development-technology/microservices-spring-boot-netflix-oss

Monoliths’ Disadvantages

The application can become very large, very quickly – and it can become difficult for a developer to maintain a mental image of what the application is doing.

The size of the application can slow down start-up times.

On each iteration you must deploy the entire application, even if there are changes only in a small part of the app.

Every change can introduce more bugs. The impact is not fully understood because of the fuzzy mental image.

Reliability – a breaking bug in one part of the application can bring down the entire app and not just one functionality.

As the project becomes larger and the team increases, making it sometimes hard for developers to work independently & without conflict.

Monoliths are not flexible in terms of team diversity or technologies used and are skeptical about adopting new technologies or updating libraries since any change will affect the entire application at tremendous costs.

https://www.pentalog.com/blog/it-development-technology/microservices-spring-boot-netflix-oss

Microservices’ Advantages

Reduced complexity of an application by dividing it into smaller problems. (Divide et Impera)

Calls for independent, flexible teams free to choose the best technology stack that best suits them and the product.

Easily integrated with CI/CD environments.

Can be scaled independently. So, if the business needs just one feature to be scaled, microservices can scale just that part – not the entire monolith.

When a bug occurs, it will affect just one or a few systems – instead of the entire application.

https://www.pentalog.com/blog/it-development-technology/microservices-spring-boot-netflix-oss

Microservices’ Disadvantages:

Added complexity with inter-service communication, separated database pattern & testing.

Harder Database transactions, because changes need to be made in databases owned by multiple services.

Testing can be more difficult.

Harder to implement changes across multiple modules because of coordination across multiple teams.

Deploying is harder because there are multiple binaries, but this process is made easier with modern cloud infrastructures.

https://www.pentalog.com/blog/it-development-technology/microservices-spring-boot-netflix-oss

Polyglot Architecture

Polyglot architecture simply put is about having multiple technologies, languages, frameworks, libraries that can achieve the same thing. You use these to create your services. This applies more to a distributed system than a monolithic one.

The good:

  • Not being depended on a particular set of technologies, if something breaks or is not supported you can easily drop it and move to something familiar within the system
  • Being able to create a solution that can work from the start with different technologies. Also, having people with different talents can bring new ways of solving things not thought before using X set of technologies.
  • Having a broader set of tools available
  • Certain autonomy provided to the team that can manage their own tech stack

The bad:

  • Needing a diverse set of talent to upkeep and create new functionality, you need to keep this talent, and you need to train the new people to what you are doing or find people who are good that the same technology stack you need
  • You might need to duplicate the same kind of functionalities over and over again for different languages and frameworks, things like logging, auditing, security, messaging, communication, database access etc.
  • Existing developers might not want to learn new or diversify outside they key expertise, making things hard to maintain or create quality solutions
  • Complexity increases by having to manage multiple different technologies

My advice is if you would choose to use a polyglot architecture look at your needs, your teams amounts, your team sizes and skills sets. Start small and simple, select a few different technologies, two or three maybe but don’t overdo it. Specialize in these and add new ones only if they do something really specific not being able to do with something else or you plan to replace something old.

In my situation, I am going to choose at least three different technologies and approaches to do things. This is mostly for the learning process.

Unified API

A unified API is a single entry point to your distributed system that hides away your complexity and makes things easier to be used by different solutions and parties.

But while complexity is hidden from an outside point of view, it still exists, and you are adding additional layers of complexity upon the existing or future architecture. Still, if the Unified API added complexity solves a greater complex issue, then the cost is acceptable.

The more important question is do you need this? Probably no, I would say start simple, build, extract, diversify, refactor and add complexity as the solution grows. Leave room for growth, but creating the most future-proof solution from the start is probably going to be the bad approach in most real life situations cases.

A Unified API is an abstraction layer that easily handles communication with many APIs and backend data models. You can think of a Unified API as an airport hub that maintains relationships with airlines and helps route passengers to their destination of choice. The beauty of a Unified API is its ability to offer a consistent API design and developer experience.

https://blog.apideck.com/what-is-a-unified-api

Unified APIs aggregate cloud providers in specific software categories, offering a single endpoint to areas like banking, accounting, CRM, leads, calendar, or email. By standardizing resources and data models across a single vertical, Unified APIs combine many services into a single integration — significantly decreasing redundancy.

https://blog.apideck.com/what-is-a-unified-api

A Unified API can also augment the data in transit and introduce PII (personal identifiable information) masking, field-level encryption, and an additional security gateway. This could provide a standard OAuth and scope management mechanism while allowing the flexibility to use unique Client IDs and secrets. Such an abstraction layer could also generate valuable support materials, such as libraries and SDKs, to streamline consumption. It could even convert a REST API to GraphQL without you doing any of the heavy lifting.

https://blog.apideck.com/what-is-a-unified-api

Benefits of Unified APIs

No integration pains: Unified APIs lower integration hurdles and makes the painful process of debugging APIs a thing of the past.

Consolidated docs: No more reading a gazillion developer docs to make integrations. We all know the feeling. 🕵🏻‍♀️

Solve compliance issues: A Unified API layer could help transform or mask potentially sensitive data (Personally Identifiable Information) to meet regulatory standards like HIPAA, PSD2, GDPR.

Centralized logging: Unified APIs can centralize logging, increasing visibility and getting full observability out-of-the-box.

Resource aliasing: Traditional integration development faces a fragmented taxonomy. A Unified API layer can create aliases to standardize terminologies across organizations, enabling you to use a common name per resource type.

Normalized data: Unified APIs normalize data into a consumable format for easier ingestion.

Domain model discovery: By mapping domain models across API vendors, it’s much easier for developers to grasp specific APIs’ scope and potential.

Proxying requests: The ability to have a gateway that allows you to perform validation upfront and capture traffic going through all APIs.

Simplifying webhooks: By standardizing events, a Unified API can make event-driven architecture more seamless.

Standardized pagination and filtering: You know what you’re going to get. Unified APIs make pagination, filtering, and sorting more consistent. They could also help avoid under-fetching and over-fetching.

Lower maintenance cost: A unified API takes care of versioning downstream APIs to facilitate ongoing maintenance.

Future-proofing: A Unified API could help future-proof connections, especially for AI use cases. Through machine-readable standards like schema.org, an aggregator could extract helpful metadata and even pinpoint real-life objects.

API Augmentation: With common libraries and SDKs, Unified APIs can support more developer preferences and programming languages.

https://blog.apideck.com/what-is-a-unified-api

Potential Downsides of Unified APIs

Latency: Adding an extra hop into the mix will introduce extra milliseconds.

Vendor lock-in: You’re adding another dependency to your stack. Conversely, a single API integration could decrease future migration concerns.

Data privacy: Adding another data processor increases liability, especially if sharing confidential data with a third party. Unified APIs must be GDPR-compliant and encrypt data in transit to meet PSD2, CCPA, and other regulations.

Lowest common denominator effect: If you’re choosing commonalities across many APIs, you lose intricacies of downstream payloads. A robust unified API should enable access to the full payload for power users and support custom fields.

Data standardization qualms: To support the Unified API concept, you need a smart abstraction layer. Common data models for specific software categories must be impeccably designed.

Reliability: According to Postman’s 2020 report, 71.6% of developers say reliability is the most important factor of API integration. Abstraction layers upon APIs must not introduce outages.

Cost: An abstraction layer could cost money. However, you gain velocity and time to market, equatable to revenue.

https://blog.apideck.com/what-is-a-unified-api

Features you would want in a Unified API

To have a functional and good Unified API, you should consider adding the following features:

  • Load Balancing
  • Security
  • Discoverability
  • Monitoring
  • Registration
  • API Management
  • Fault Tolerance

Unified API Examples: https://unifiedapis.io/

Sources for this blog post

https://www.apollographql.com/docs/

https://the-guild.dev/blog/graphql-mesh

https://netflixtechblog.com/how-netflix-scales-its-api-with-graphql-federation-part-1-ae3557c187e2

https://netflixtechblog.com/how-netflix-scales-its-api-with-graphql-federation-part-2-bbe71aaec44a

https://docs.dapr.io/concepts/service-mesh/

https://www.apollographql.com/docs/federation/

https://moonhighway.com/apollo-federation

https://graphcms.com/blog/graphql-vs-rest-apis

https://www.apollographql.com/blog/graphql/basics/graphql-vs-rest/

https://medium.com/@skpallewatta92/microservices-with-netflix-oss-spring-boot-and-non-jvm-applications-2d762768921a
https://structure101.com/2020/02/05/polyglot-architecture-control-and-its-gotchas/

https://movio.co/blog/building-a-new-api-platform-for-movio/

https://www.apollographql.com/blog/graphql/basics/the-anatomy-of-a-graphql-query/

https://graphcms.com/blog/graphql-vs-rest-apis

https://itnext.io/a-guide-to-graphql-schema-federation-part-1-995b639ac035

https://itnext.io/a-guide-to-graphql-schema-federation-part-2-872a820510ff

https://www.sanity.io/guides/graphql-vs-rest-api-comparison

https://tyk.io/blog/an-introduction-to-graphql-federation/

https://hasura.io/blog/the-ultimate-guide-to-schema-stitching-in-graphql-f30178ac0072/

2 thoughts on “Unified APIs and Federated Gateways – Part 1: Unified API, Monolith and Microservices Architecture

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.