Search for content

Protobuf format

Protocol Buffers (Protobuf), originally developed by Google, is a free and open-source cross-platform data format used to serialize structured data (see here).

It is useful in developing applications that communicate with each other over a network or for storing data. 

The method is based on: 

  • An interface description language that describes the data structure (called message), defined in a proto definition file (.proto).

  • An application (protoc) used for:

    • generating a stream of bytes starting from the .proto description

    • parsing a stream of bytes to generate a .proto definition file

Example

You have:

  • a file your.proto, stored into your directory ./protos/

  • principal message: YourMessage

  • binary file: response.protobuf
     

The command to get the de-serialized content form the response.protobuf is:

protoc --decode=YourMessage \--proto_path=./protos \ your.proto < response.protobuf

 

flows_network_v1.proto

The protobuf binary files can be handled with the flows_network_v1.proto file shown below:

syntax = "proto3";
package com.ptvgroup.archimedes.api.protobuf.network.v1;
option java_package = "com.ptvgroup.archimedes.api.protobuf.network.v1";
option csharp_namespace = "PTVGroup.Flows.Protos.API";
message Network {
    string map_version = 1; 
    string coordinate_reference_system = 2; // mandatory WGS-84
    repeated Street street = 3;
    repeated string tile = 4;  // map data is for these specific tiles.
}
message Street {
 int32 id = 1; // Id of the street
 int32 from_node_id = 2; // Id of the tail Visum Node 
 bytes openlr = 3; // binary format for openlr
 int32 functional_road_class = 4; // Functional road class defined by TomTom. Possible values are in the range [0..7] as defined in OpenLR's whitepaper https://download.tomtom.com/open/banners/openlr-whitepaper_v1.5.pdf
 int32 form_of_way = 5; // Form of way defined by TomTom. Possible values are in the range [0..7] as defined in OpenLR's whitepaper https://download.tomtom.com/open/banners/openlr-whitepaper_v1.5.pdf
 double free_flow_speed_kmph = 6; // The street free flow speed [Km/h]
 bytes shape = 7; // The street geometry in WKBLineString format https://www.ibm.com/docs/en/db2-warehouse?topic=formats-well-known-binary-wkb-format
 string name = 8; // The street name
 int32 to_node_id = 9; // Id of the head Visum Node 
 double length = 10; //The length of the street geometry [Km]
}

 

How to decode a response in protobuf format

The command to be used is based on these elements:

  • protoc-3.15.8-win64\bin\protoc.exe: It is the version of the protoc command
  • com.ptvgroup.archimedes.api.protobuf.network.v1: It is the package path associated to the .proto file definition
  • flows_network_v1.proto: It is the .proto file shown above
  • response: It is the API endpoint response in protobuf (binary) format
  • response.json: It is the decoded JSON file, containing the response in readable format

The complete command is:

protoc-3.15.8-win64\bin\protoc.exe --decode=com.ptvgroup.archimedes.api.protobuf.network.v1.Network flows_network_v1.proto < response > response.json