Search for content

Protobuf format

Protocol Buffers (Protobuf), originally developed by Google, is a free and open-source cross-platform data format used to serialize structured data (see here). 

It is useful in developing applications that communicate with each other over a network or for storing data. 

The method is based on: 

  • An interface description language that describes the data structure (called message), defined in a proto definition file (.proto).

  • An application (protoc) used for:

    • generating a stream of bytes starting from the .proto description

    • parsing a stream of bytes to generate a .proto definition file

Example

You have:

  • a file your.proto, stored into your directory ./protos/

  • principal message: YourMessage

  • binary file: response.protobuf
     

The command to get the de-serialized content form the response.protobuf is:

protoc --decode=YourMessage \--proto_path=./protos \ your.proto < response.protobuf

 

forecast_data_message.proto

The protobuf binary files can be handled with the forecast_data_message.proto file shown below:

//model.statistical_forecast
syntax = "proto3";
import "google/protobuf/timestamp.proto";
package com.ptvgroup.mlf.common.messagehandling.forecast;

message ForecastData {
    message StreetForecast {  
        int32 id =1;  
        int32 fromNode=2;  
        repeated Forecast forecast=3;
        string openLRcode=4;
    }
    message Forecast {
        google.protobuf.Timestamp start=1;
        google.protobuf.Timestamp end=2;
        bool flow_valid = 3;
        double flow=4;
        bool speed_valid= 5;
        double speed=6;
        string timezone = 7;
    }
    repeated StreetForecast streetForecast=1;
}

 

How to decode a response in protobuf format

The command to be used is based on these elements:

  • protoc-3.15.8-win64\bin\protoc.exe: It is the version of the protoc command
  • com.ptvgroup.mlf.common.messagehandling.forecast.ForecastData: It is the package path associated to the .proto file definition
  • forecast_data_message.proto: It is the .proto file shown above
  • response: It is the API endpoint response in protobuf (binary) format
  • response.json: It is the decoded JSON file, containing the response in readable format

The complete command is:

protoc-3.15.8-win64\bin\protoc.exe --decode=com.ptvgroup.mlf.common.messagehandling.forecast.ForecastData forecast_data_message.proto < response > response.json