My Tech Sharing

 If you have been working on API implementation for awhile and try to make your API resilient & bulletproof, the topic today might be something you will find interesting. Today, nobody can guarantee the microservices that you built will not run into trouble. When issue occurred, we often hope that the simplest way to solve the issue is to retry and invoke the API again. Retry can be an easy mechanism handled by your middleware/ API orchestration product. The challenge for doing so will always be on the business logic of the API if the nature of the record can gracefully managed the duplication through something like Primary Key. (e.g. create a car record with car registration number) You probably won't hit a problem when performing record update (if multiple updates is not a problem. e.g. update status of your car record from "available" to "sold"),  but it will be tricky for scenario of record creation.(e.g. create a sales record for a car) Making microserv...

 API versioning is one of the key concept when we consider API implementation. It's useful when we need to maintain coexistence of multiple versions of the API implementation. It happens when you need to enhance the functionalities and at the same time support existing API consumer which still rely on the current implementation.  Typically, you do not need to maintain API versioning when the change can be gracefully handled by the application logic and the implementation won't result to any breaking change. Breaking change refers to implementation which affects the request or response of your existing API. API consumer can no longer consume the API as the API integration contract has been changed.     Implementation of API versioning There are many approaches can be used to handle API versioning. In generally, API service provider needs an indicator in API request to understand which version the API consumer need. I will cover two of the common approaches that y...

  As many applications have started the journey of modernisation from monolithic to microservices architecture, observability tool has now become essential to troubleshoot and understand issues happened in application. Business logic will no longer sit within the monolithic code, but has now spread across multiple microservices which can be hard to trace if we don't have a proper tool. Jaeger ( https://www.jaegertracing.io/ ) is an open source project developed by Uber Technologies which is meant for us to perform distributed tracing across APIs. In today's sharing, I am going to implement Jaeger distributed tracing via microservices developed using Node.JS. I am going to use threadfin-http (  https://github.com/joshua-lim/threadfin-http ) ; a simple Node.JS http server to illustrate this. If you are not a developer, just want to explore the features of Jaeger without doing much coding, you can download the hot r.o.d example ( https://github.com/jaegertracing/jaeger/tree/...

Build a Batch Framework with Go Recently, I have just picked up Golang and decided to use what I learnt to build a Batch Framework with Go. Batch Framework - Batch103 I called this project Batch103 ( S tarted as Batch101. 103 is the 3rd version I decided to publish ) ; the project is a simple batch framework inspired by SpringBatch - a popular batch framework in Java world. If you familiar with SpringBatch, you may find many similar features you used to see in SpringBatch. The source code of the framework is available at github  (https://github.com/pengyeng/batch103.git )  I will illustrates the high level  architecture of Batch103 using a diagram. (Please refer the overall architecture diagram) The framework comes with a Job Launcher which provides  a implementation class to launch the batch job. Job Launcher needs 3 key components to operate; which are : Reader - handling the input of batch job Processor - handling the logic to process the data Writer - handling th...

Prometheus is probably one of the most popular monitoring tools for microservices architecture. With Prometheus, microservices environment like docker can be easily monitor by connecting Prometheus to docker swarm cluster and docker daemon. There are many articles available to talk about setting up Prometheus. The question I have is how about the microservice deployed in environment such docker swarm? how can we effectively monitor these applications? I have recently found my answer through a npm library called prom-client (  https://www.npmjs.com/package/prom-client  ). This is an amazing client library which allows me to connect my Node.JS application with Prometheus. To share this with you, I am going to take you through a demo by creating a 2 nodes docker swarm and deploy my threadfin-http as a docker service. On top of that, I also going to deploy prometheus and grafana to enable monitoring and dashboard for visualization. Well… to kick start this, I am going to use threa...