Our guest post today comes from Brandon Phillips of CoreOS. CoreOS builds open source projects and products for Linux Containers. Their flagship products for consensus and discovery (etcd) and their container engine rkt are early adopters gRPC, an RPC framework based on protobufs and HTTP/2, that can make building and consuming APIs easier and more performant. Since many clients speak http/1.1 plus json, the interoperability with JSON and Open API is very important. For users who are more comfortable with HTTP/1.1+JSON-based APIs and the Open API Initiative Specs (formerly swagger) APIs they are using a combination of open source libraries to make their gRPC services available in that form as well. They have built API multiplexers to give users the best of both worlds. Let’s dive into the details and find out how they did it!
This post was originally published at the CoreOS blog (link). We are reproducing here with some edits.
One of the key reasons CoreOS chose gRPC is because it uses HTTP/2, enabling applications to present both a HTTP 1.1 REST+JSON API and an efficient gRPC interface on a single TCP port.(available for Go) This gives developers compatibility with the REST web ecosystem while advancing a new, high-efficiency RPC protocol. With Go 1.6 recently released, Go ships with a stable net/http2 package by default.
A gRPC application called EchoService
In this post we will build a small proof-of-concept gRPC application from a gRPC API definition, add a REST service gateway, and finally serve it all on a single TLS port. The application is called EchoService, and is the web equivalent of the shell command echo: the service returns, or “echoes”, whatever text is sent to it.
First, let’s define the arguments to EchoService in a protobuf message called EchoMessage, which includes a single field called value. We will define this message in a protobuf “.proto” file called service.proto. Here is our EchoMessage:
message EchoMessage { string value = 1; }
In this same protobuf file, we define a gRPC service that takes this data structure and returns it:
service EchoService { rpc Echo(EchoMessage) returns (EchoMessage) { } }
Running this service.proto file through the Protocol Buffer compiler protoc generates a stub gRPC service in Go, along with clients in various languages. But gRPC alone isn’t as useful as a service that also exposes a REST interface, so we won’t stop with the gRPC service stub.
Next, we add the gRPC REST Gateway. This library will build a RESTful proxy atop the gRPC EchoService. To build this gateway, we add metadata to the EchoService proto to indicate that the Echo RPC maps to a RESTful POST method with all RPC parameters mapped to a JSON body. The gateway can map RPC parameters to URL paths and query parameters, but we omit those complications here for brevity.
service EchoService { rpc Echo(EchoMessage) returns (EchoMessage) { option (google.api.http) = { post: “/v1/echo” body: “*” }; } }
In practical terms this means the gateway, once generated by protoc, can now accept a request from curl like this:
curl -X POST -k https://localhost:10000/v1/echo -d ‘{“value”: “CoreOS is hiring!”}’
The whole system so far looks like this, with a single service.proto file generating both a gRPC server and a REST proxy:
gRPC API with REST gateway
To bring this all together, the echo service creates a Go http.Handler to detect if the protocol is HTTP/2 and the Content-Type is “application/grpc” and sends such requests to the gRPC server. Everything else is routed to the REST gateway. The code looks something like this:
if r.ProtoMajor == 2 && strings.Contains(r.Header.Get(“Content-Type”), “application/grpc”) { grpcServer.ServeHTTP(w, r) } else { otherHandler.ServeHTTP(w, r) }
To try it out, all you need is a working Go 1.6 development environment and the following simple incantations:
$ go get -u github.com/philips/grpc-gateway-example $ grpc-gateway-example serve
With the server running you can attempt requests on both HTTP 1.1 and gRPC interfaces:
grpc-gateway-example echo Take a REST from REST with gRPC curl -X POST -k https://localhost:10000/v1/echo -d ‘{“value”: “CoreOS is hiring!”}’
One last bonus: Because we have an Open API specification, you can browse the Open API UI running at https://localhost:10000/swagger-ui/#!/EchoService/Echo if you have the server above running on your laptop.
gRPC/REST Open API document
We’ve taken a look at how to use gRPC to bridge to the world of REST. If you want to take a look at the complete project, check out the repo on GitHub. We think this pattern of using a single protobuf to describe an API leads to an easy to consume, flexible API framework, and we’re excited to leverage it in more of our projects.
—
About The Author
Brandon Phillips (CoreOS)
Brandon Philips is helping to build modern Linux server infrastructure at CoreOS as CTO. Prior to CoreOS, he worked at Rackspace hacking on cloud monitoring and was a Linux kernel developer at SUSE. As a graduate of Oregon State’s Open Source Lab he is passionate about open source technologies.