Microservices, gRPC and Protocol Buffers — Hello World
In the last decade, A huge update in the development and deployment of different web based services has occurred. We have seen a wide variety of projects which moved towards the advanced “Microservice Architecture”, replacing the old traditional “Monolithic Architecture”. This shift of technology has already done wonders for us. As we move towards microservices, we are inclined towards the fact that “distributed in nature and smaller chunks are much easier to handle”, so here we have microservices that can be handled conveniently. For interaction among different microservices the HTTP API calls were being used, which seems great.
From REST to gRPC
Talking about microservices and the communication between them, we came to the concept of Synchronous (APIs) and asynchronous communications (message queues). REST APIs are a pretty straightforward way to create a communication channel. We’ve a wide variety of frameworks for the development of REST APIs. It is a HTTP service and HTTP is a protocol built over TCP. When we create a REST connection we’re opening a TCP connection to the server, we send the request payload, we receive the response, and we close the connection. We are well aware of the fact that big payloads over TCP connection results in slow response time. Here comes the gRPC in the game and with gRPC we create a persistent connection between client and server (instead of opening and closing connection like REST) and also we use a binary payload to reduce the size improving the performance. So, gRPC does not provide a set of guidelines on how to create web APIs; it basically enforces the set of rules. This is what is the main advantage over REST, gRPC in most cases is way faster and more robust, as it defines a specific set of rules each request and response should adhere to.
About gRPC?
gRPC is actually developed by Google. But the “g” in gRPC is still not confirmed by “google”. the letter “g” in every release keeps changing and the names are like “gambit”, “goose”, “gold”, “green” etc. According the official documentation;
“gRPC is a modern open source high performance Remote Procedure Call (RPC) framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication. It is also applicable in last mile of distributed computing to connect devices, mobile applications and browsers to backend services.”
Let’s look closely at the concept of remote procedure calls (RPC), In simple words the messages that the server sends to the remote system in order to get execute the the method or the task done. Google’s based RPC is actually designed and developed for the efficient and smooth communication between the services. Few important aspects to remember while looking at the concept of gRPC.
HTTP/2: It uses HTTP/2 protocol instead of HTTP 1.1, which allows multiple bidirectional streams that can be created and sent over TCP connections.
Language platforms independent: Two services may be written in different languages, say Python and Golang. gRPC ensures smooth communication between them.
Integrated Auth and load balancing: gRPC provides all these features, making it a secure and reliable option to choose.
Protocol Buffers: It uses protocol buffers for the definition of the type of data, as the message interchange format. It is an open source mechanism for serializing structured data (although it can be used with other data formats such as JSON), which is also developed by google.
Building APIs with gRPC
If we talk about the implementation of a simple gRPC service, we will be defining three files:-
- Client: responsible for making the gRPC call to the server.
- Server: responsible for serving the client requests.
- Proto file: Proto file comprises the declaration of the service that is used to generate stubs (<package_name>_pb2.py and <package_name>_pb2_grpc.py). It can be understood as a step. We actually don’t need to define it when we are working with HTTP APIs (we use JSON and we forget the rest)
syntax = "proto3";
package grpctest;
service Grpctest{
// A simple RPC.
//
// Obtains the MessageResponse at a given position.
rpc GetServerResponse(Message) returns (MessageResponse) {}
}
message Message{
string message = 1;
}
message MessageResponse{
string message = 1;
bool received = 2;
}Here we have created a service named Grpctest. In this service we have GetServerResponse() which simply takes an input of type Message and returns a MessageResponse. This .proto file will be now used to generate the stubs using the following command;
$ pip3 install grpcio grpcio-tools (for installing gRPC stuff)$ python3 -m grpc_tools.protoc --proto_path=. ./grpctest.proto --python_out=. --grpc_python_out=.
Two files are generated named grpctest_pb2.py and grpctest_pb2_grpc.py
Writing Client and Server
In order to write the code for client and server, here are both of the client and server code chunks.
import grpc
from concurrent import futures
import time
import grpctest.grpctest_pb2_grpc as pb2_grpc
import grpctest.grpctest_pb2 as pb2
class GrpctestService(pb2_grpc.GrpctestServicer):
def __init__(self, *args, **kwargs):
pass
def GetServerResponse(self, request, context):
# get the string from the incoming request
message = request.message
result = f'Hello I am up and running received "{message}" message from you'
result = {'message': result, 'received': True}
return pb2.MessageResponse(**result)
def serve():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
pb2_grpc.add_GrpctestServicer_to_server(GrpctestService(), server)
server.add_insecure_port('[::]:50051')
server.start()
server.wait_for_termination()
if __name__ == '__main__':
serve()Writing the code for client;
import grpc
import grpctest.grpctest_pb2_grpc as pb2_grpc
import grpctest.grpctest_pb2 as pb2
class grpctestClient(object):
"""
Client for gRPC functionality
"""
def __init__(self):
self.host = 'localhost'
self.server_port = 50051
# instantiate a channel
self.channel = grpc.insecure_channel(
'{}:{}'.format(self.host, self.server_port))
# bind the client and the server
self.stub = pb2_grpc.GrpctestStub(self.channel)
def get_url(self, message):
"""
Client function to call the rpc for GetServerResponse
"""
message = pb2.Message(message=message)
print(f'{message}')
return self.stub.GetServerResponse(message)
if __name__ == '__main__':
client = grpctestClient()
result = client.get_url(message="Hello Server you there?")
print(f'{result}')Execution of the Server and Client
In order to execute the server and client, we will be doing the following steps;
$ python3 grpctest_server.py$ python3 grpctest_client.py
In a Nutshell
gRPC is a fresh and new RPC framework that makes communication between microservices faster and efficient. Yes, it seems to be the new thing to learn and have a look, but it cannot very easily replace the REST framework. We can say that gRPC is more structured and usually faster. It can be widely used when doing the inter communications between your microservices. I have put the code here in order to execute it and see the results. Enjoy learning!!!
What’s Next
Diving deep into the documentation gRPC and try to learn in which use cases it can be applied. Moreover going through the documentation in different environments with Go, Python or Node.
I went through few of the blogs and docs and after that I just jot this article down.
References:
https://grpc.io/
https://towardsdatascience.com/reasons-to-choose-grpc-over-rest-and-how-to-adopt-it-into-your-python-apis-197ac28e22b4
