gRPC, introduced in August 2016 by Google, gradually booming in the market due to its unique capabilities like Binary data transfer over HTTP/2 protocol, uni-directional & bi-directional streaming. Many microservice architects are nowadays preferring gRPC on top of REST-based communication for better performance for service-2-service data transfer.
gRPC also provides a unique capability for bi-directional streaming which could be used for any use cases starting from Chat server, persistent service-to-service connection, and lots more.
Today we will learn how to implement bi-directional streaming using gRPC with NodeJS. The codebase has been shared on GitHub for your convenience as follows: https://github.com/techunits/bidirectional_grpc_sample_node
Prerequisite:
In order to implement the following tutorial, we need to have NodeJS 14+ installed on your system.
Use case:
We will build a bi-directional streaming server and client to create an article management application that will allow us to create articles in bulk using streaming.
Build Proto:
The first step to implement any gRPC service is to build the data contract in the form of a protocol buffer or proto file. Here is our proto file which will enable us to set up streaming using the “stream” keyword.
The RPC method “createBulkEntries” will allow developers to communicate with the gRPC service using bi-directional streaming as both request and response parameters are wrapped with the “stream” keyword.
Let’s also assume the filename is article.proto, which we will refer into our implementation
syntax = "proto3";
package ArticleManagerPackage;
message ArticleCreateRequest {
string title = 1;
string description = 2;
}
message ArticleResponse {
string id = 1;
string title = 2;
string description = 3;
string created_on = 4;
}
service ArticleService {
rpc createBulkEntries(stream ArticleCreateRequest) returns (stream ArticleResponse) {}
}
Install required packages:
We will need the following packages as a part of the implementation which should be installed via npm command:
$ npm install @grpc/grpc-js # core grpc library
$ npm install @grpc/proto-loader # core grpc proto manager
$ npm install faker # to generate fake data for testing
$ npm install elogger # advanced logging library
$ npm install uuid # UUID generation library
$ npm install slugify # convert some string to slug
Build the server:
After installation of all the above packages, we can start with our server script as follows:
Load required packages:
In the below code snippet, we are loading a custom node package named “handler.js” that will be required to manipulate the data.
// load required packages
const logger = require("elogger");
const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader");
const handlerObj = require("./handler");
Load “article.proto” to load the gRPC data contract:
const packageDefinition = protoLoader.loadSync("./article.proto", {
keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true
});
const articleManagerProto = grpc.loadPackageDefinition(packageDefinition).ArticleManagerPackage;
Initialize server and register handlers for the respective RPC methods:
const server = new grpc.Server();
server.addService(articleManagerProto.ArticleService.service, {
createBulkEntries: handlerObj.createBulkEntries
});
Bind & start the server process to port 9111:
const bindEndpoint = `0.0.0.0:9111`;
server.bindAsync(bindEndpoint, grpc.ServerCredentials.createInsecure(), (err, response) => {
if(err) {
logger.error(err);
}
else {
server.start();
logger.info(`Article manager gRPC service started on grpc://${bindEndpoint}`);
}
});
Deep dive into “handler.js” file:
“handler.js” is responsible for handler the gRPC streaming calls and saving the data to the respective medium. In our example, we have mocked the save and returned the id and created_on timestamp to the response stream.
const logger = require("elogger");
const uuid = require("uuid");
exports.createBulkEntries = (call) => {
logger.debug(`gRPC ${call.call.handler.path}`);
// handle the data stream
call.on("data", async (payload) => {
console.log(payload);
payload.id = uuid.v4();
payload.created_on = new Date().getTime();
call.write(payload);
});
// if server encouters event request to end the stream
call.on("end", async (payload) => {
call.end();
});
};
The above snippet will start our gRPC server on port 9111. The output should be as follows:
$ node server.js
INFO: Mon Feb 01 2021 17:54:15 GMT+0530 (India Standard Time) Article manager gRPC service started on grpc://0.0.0.0:9111