Added map sorting to binary and text encoders.

For the binary encoder, sorting is off by default.
For the text encoder, sorting is on by default.
Both defaults can be explicitly overridden.

This grows code size a bit. I think we could potentially
shave this (and other map-related code size) by having
the generated code inject a function pointer to the map-related
parsing/serialization code if maps are present.

    FILE SIZE        VM SIZE
 --------------  --------------
   +86% +1.07Ki   +71%    +768    upb/msg.c
    [NEW]    +391  [NEW]    +344    _upb_mapsorter_pushmap
    [NEW]    +158  [NEW]    +112    _upb_mapsorter_cmpstr
    [NEW]    +111  [NEW]     +64    _upb_mapsorter_cmpbool
    [NEW]    +110  [NEW]     +64    _upb_mapsorter_cmpi32
    [NEW]    +110  [NEW]     +64    _upb_mapsorter_cmpi64
    [NEW]    +110  [NEW]     +64    _upb_mapsorter_cmpu32
    [NEW]    +110  [NEW]     +64    _upb_mapsorter_cmpu64
    -3.6%      -8  -4.3%      -8    _upb_map_new
  +9.5%    +464  +9.2%    +424    upb/text_encode.c
    [NEW]    +656  [NEW]    +616    txtenc_mapentry
     +15%     +32   +20%     +32    upb_text_encode
   -20.1%    -224 -20.7%    -224    txtenc_msg
  +5.7%    +342  +5.3%    +296    upb/encode.c
    [NEW]    +344  [NEW]    +304    encode_mapentry
    [NEW]    +246  [NEW]    +208    upb_encode_ex
    [NEW]     +41  [NEW]     +16    upb_encode_ex.ch
    +0.7%      +8  +0.7%      +8    encode_scalar
    -1.0%     -32  -1.0%     -32    encode_message
    [DEL]     -38  [DEL]     -16    upb_encode.ch
    [DEL]    -227  [DEL]    -192    upb_encode
  +2.0%    +152  +2.2%    +152    upb/decode.c
     +44%    +128   +44%    +128    [section .rodata]
    +3.4%     +24  +3.4%     +24    _GLOBAL_OFFSET_TABLE_
  +0.6%    +107  +0.3%     +48    upb/def.c
    [NEW]    +100  [NEW]     +48    upb_fielddef_descriptortype
    +7.1%      +7  [ = ]       0    upb_fielddef_defaultint32
  +2.9%     +24  +2.9%     +24    [section .dynsym]
  +1.2%     +24  [ = ]       0    [section .symtab]
  +3.2%     +16  +3.2%     +16    [section .plt]
    [NEW]     +16  [NEW]     +16    memcmp@plt
  +0.5%     +16  +0.6%     +16    tests/conformance_upb.c
    +1.5%     +16  +1.6%     +16    DoTestIo
  +0.1%     +16  +0.1%     +16    upb/json_decode.c
    +0.4%     +16  +0.4%     +16    jsondec_wellknown
  +3.0%      +8  +3.0%      +8    [section .got.plt]
    +3.0%      +8  +3.0%      +8    _GLOBAL_OFFSET_TABLE_
  +1.6%      +7  +1.6%      +7    [section .dynstr]
  +1.8%      +4  +1.8%      +4    [section .hash]
  +0.5%      +3  +0.5%      +3    [LOAD #2 [RX]]
  +2.8%      +2  +2.8%      +2    [section .gnu.version]
 -60.0% -1.74Ki  [ = ]       0    [Unmapped]
  +0.3%    +496  +1.4% +1.74Ki    TOTAL
14 files changed
tree: dc46cea932a86f96e154628bb95572b7f78039d0
  1. .bazelci/
  2. bazel/
  3. benchmarks/
  4. cmake/
  5. examples/
  6. kokoro/
  7. tests/
  8. third_party/
  9. tools/
  10. upb/
  11. upbc/
  12. .bazelrc
  13. .gitignore
  14. BUILD
  15. CONTRIBUTING.md
  16. DESIGN.md
  17. LICENSE
  18. README.md
  19. WORKSPACE
README.md

μpb - a small protobuf implementation in C

PlatformBuild Status
macOSBuild Status
ubuntuBuild Status

μpb (often written ‘upb’) is a small protobuf implementation written in C.

upb generates a C API for creating, parsing, and serializing messages as declared in .proto files. upb is heavily arena-based: all messages always live in an arena (note: the arena can live in stack or static memory if desired). Here is a simple example:

#include "conformance/conformance.upb.h"

void foo(const char* data, size_t size) {
  upb_arena *arena;

  /* Generated message type. */
  conformance_ConformanceRequest *request;
  conformance_ConformanceResponse *response;

  arena = upb_arena_new();
  request = conformance_ConformanceRequest_parse(data, size, arena);
  response = conformance_ConformanceResponse_new(arena);

  switch (conformance_ConformanceRequest_payload_case(request)) {
    case conformance_ConformanceRequest_payload_protobuf_payload: {
      upb_strview payload = conformance_ConformanceRequest_protobuf_payload(request);
      // ...
      break;
    }

    case conformance_ConformanceRequest_payload_NOT_SET:
      fprintf(stderr, "conformance_upb: Request didn't have payload.\n");
      break;

    default: {
      static const char msg[] = "Unsupported input format.";
      conformance_ConformanceResponse_set_skipped(
          response, upb_strview_make(msg, sizeof(msg)));
      break;
    }
  }

  /* Frees all messages on the arena. */
  upb_arena_free(arena);
}

API and ABI are both subject to change! Please do not distribute as a shared library for this reason (for now at least).

Using upb in your project

Currently only Bazel is supported (CMake support is partial and incomplete but full CMake support is an eventual goal).

To use upb in your Bazel project, first add upb to your WORKSPACE file, either as a git_repository() or as a new_local_repository() with a Git Submodule. (For an example, see `examples/bazel/ in this repo).

# Add this to your WORKSPACE file.
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

git_repository(
    name = "upb",
    remote = "https://github.com/protocolbuffers/upb.git",
    commit = "d16bf99ac4658793748cda3251226059892b3b7b",
)

load("@upb//bazel:workspace_deps.bzl", "upb_deps")

upb_deps()

Then in your BUILD file you can add upb_proto_library() rules that generate code for a corresponding proto_library() rule. For example:

# Add this to your BUILD file.
load("@upb//bazel:upb_proto_library.bzl", "upb_proto_library")

proto_library(
    name = "foo_proto",
    srcs = ["foo.proto"],
)

upb_proto_library(
    name = "foo_upbproto",
    deps = [":foo_proto"],
)

cc_binary(
    name = "test_binary",
    srcs = ["test_binary.c"],
    deps = [":foo_upbproto"],
)

Then in your .c file you can #include the generated header:

#include "foo.upb.h"

/* Insert code that uses generated types. */

Old “handlers” interfaces

This library contains several semi-deprecated interfaces (see BUILD file for more info about which interfaces are deprecated). These deprecated interfaces are still used in some significant projects, such as the Ruby and PHP C bindings for protobuf in the main protobuf repo. The goal is to migrate the Ruby/PHP bindings to use the newer, simpler interfaces instead. Please do not use the old interfaces in new code.

Lua bindings

This repo has some Lua bindings for the core library. These are experimental and very incomplete. These are currently included in order to validate that the C API is suitable for wrapping. As the project matures these Lua bindings may become publicly available.

Contact

Author: Josh Haberman (jhaberman@gmail.com, haberman@google.com)