commit	c642e43a5a7b9d1487e7207720bb2f247b2d7b6d	[log] [tgz]
author	Joshua Haberman <haberman@google.com>	Wed Mar 29 15:03:36 2023 -0700
committer	Copybara-Service <copybara-worker@google.com>	Wed Mar 29 15:04:19 2023 -0700
tree	ae6aa5a3ade7bf415bc203cf2ab7e1835a624af7
parent	3b0c9261c6c1abc554395678da71dab6d8ed9802 [diff]

Allow fuse/fuse races, so that upb_Arena is fully thread-compatible.

Previously upb_Arena was not thread-compatible when `upb_Arena_Fuse(a, b)` and `upb_Arena_Fuse(c, d)` executed in parallel if `b` and `c` were previously fused.  This CL fixed that by allowing `upb_Arena_Fuse()` to run in parallel without limitations.

Details on the design of the algorithm are captured in comments.

The CL slightly improves the performance of `upb_Arena_Fuse()`.

```
name                                           old cpu/op   new cpu/op   delta
BM_ArenaOneAlloc                                 20.0ns ±19%  17.5ns ± 4%  -12.30%  (p=0.000 n=19+17)
BM_ArenaInitialBlockOneAlloc                     6.65ns ± 4%  5.17ns ± 3%  -22.23%  (p=0.000 n=18+17)
BM_ArenaFuseUnbalanced/2                         69.1ns ± 7%  68.5ns ± 4%     ~     (p=0.327 n=18+19)
BM_ArenaFuseUnbalanced/8                          542ns ± 3%   513ns ± 4%   -5.25%  (p=0.000 n=18+18)
BM_ArenaFuseUnbalanced/64                        5.04µs ± 8%  4.74µs ± 4%   -5.93%  (p=0.000 n=17+17)
BM_ArenaFuseUnbalanced/128                       10.1µs ± 4%   9.6µs ± 4%   -4.80%  (p=0.000 n=18+17)
BM_ArenaFuseBalanced/2                           71.8ns ± 7%  68.4ns ± 6%   -4.75%  (p=0.000 n=17+17)
BM_ArenaFuseBalanced/8                            541ns ± 3%   519ns ± 3%   -4.21%  (p=0.000 n=18+17)
BM_ArenaFuseBalanced/64                          5.00µs ± 7%  4.86µs ± 4%   -2.78%  (p=0.003 n=17+18)
BM_ArenaFuseBalanced/128                         10.0µs ± 4%   9.7µs ± 4%   -2.68%  (p=0.001 n=16+18)
BM_LoadAdsDescriptor_Upb<NoLayout>               5.52ms ± 2%  5.54ms ± 4%     ~     (p=0.707 n=16+19)
BM_LoadAdsDescriptor_Upb<WithLayout>             6.18ms ± 3%  6.15ms ± 3%     ~     (p=0.501 n=18+18)
BM_LoadAdsDescriptor_Proto2<NoLayout>            11.8ms ± 7%  11.7ms ± 5%     ~     (p=0.330 n=16+18)
BM_LoadAdsDescriptor_Proto2<WithLayout>          11.9ms ± 3%  11.8ms ± 3%     ~     (p=0.303 n=18+17)
BM_Parse_Upb_FileDesc<UseArena, Copy>            12.2µs ± 4%  12.3µs ± 4%     ~     (p=0.935 n=17+18)
BM_Parse_Upb_FileDesc<UseArena, Alias>           11.3µs ± 6%  11.3µs ± 3%     ~     (p=0.873 n=16+17)
BM_Parse_Upb_FileDesc<InitBlock, Copy>           12.1µs ± 4%  12.1µs ± 3%     ~     (p=0.501 n=18+18)
BM_Parse_Upb_FileDesc<InitBlock, Alias>          11.1µs ± 4%  11.1µs ± 2%     ~     (p=0.297 n=18+16)
BM_Parse_Proto2<FileDesc, NoArena, Copy>         24.2µs ± 3%  25.6µs ±16%     ~     (p=0.177 n=17+20)
BM_Parse_Proto2<FileDesc, UseArena, Copy>        11.6µs ± 3%  11.7µs ± 4%     ~     (p=0.232 n=17+18)
BM_Parse_Proto2<FileDesc, InitBlock, Copy>       11.5µs ± 7%  11.4µs ± 4%     ~     (p=0.707 n=18+19)
BM_Parse_Proto2<FileDescSV, InitBlock, Alias>    12.8µs ± 5%  13.0µs ±14%     ~     (p=0.782 n=18+17)
BM_SerializeDescriptor_Proto2                    5.69µs ± 5%  5.76µs ± 6%     ~     (p=0.143 n=18+18)
BM_SerializeDescriptor_Upb                       10.2µs ± 4%  10.2µs ± 3%     ~     (p=0.613 n=18+17)

name                                           old time/op             new time/op             delta
BM_ArenaOneAlloc                                 20.0ns ±19%             17.6ns ± 4%  -12.37%        (p=0.000 n=19+17)
BM_ArenaInitialBlockOneAlloc                     6.66ns ± 4%             5.18ns ± 3%  -22.24%        (p=0.000 n=18+17)
BM_ArenaFuseUnbalanced/2                         69.2ns ± 7%             68.6ns ± 4%     ~           (p=0.343 n=18+19)
BM_ArenaFuseUnbalanced/8                          543ns ± 3%              515ns ± 4%   -5.21%        (p=0.000 n=18+18)
BM_ArenaFuseUnbalanced/64                        5.05µs ± 8%             4.75µs ± 4%   -5.93%        (p=0.000 n=17+17)
BM_ArenaFuseUnbalanced/128                       10.1µs ± 4%              9.6µs ± 4%   -4.78%        (p=0.000 n=18+17)
BM_ArenaFuseBalanced/2                           72.0ns ± 7%             68.6ns ± 6%   -4.73%        (p=0.000 n=17+17)
BM_ArenaFuseBalanced/8                            543ns ± 3%              520ns ± 3%   -4.20%        (p=0.000 n=18+17)
BM_ArenaFuseBalanced/64                          5.01µs ± 7%             4.87µs ± 4%   -2.78%        (p=0.004 n=17+18)
BM_ArenaFuseBalanced/128                         10.0µs ± 3%              9.8µs ± 4%   -2.67%        (p=0.001 n=16+18)
BM_LoadAdsDescriptor_Upb<NoLayout>               5.53ms ± 2%             5.56ms ± 4%     ~           (p=0.707 n=16+19)
BM_LoadAdsDescriptor_Upb<WithLayout>             6.20ms ± 3%             6.17ms ± 2%     ~           (p=0.424 n=18+18)
BM_LoadAdsDescriptor_Proto2<NoLayout>            11.8ms ± 7%             11.7ms ± 5%     ~           (p=0.297 n=16+18)
BM_LoadAdsDescriptor_Proto2<WithLayout>          11.9ms ± 3%             11.9ms ± 3%     ~           (p=0.351 n=18+17)
BM_Parse_Upb_FileDesc<UseArena, Copy>            12.3µs ± 4%             12.3µs ± 4%     ~           (p=1.000 n=17+18)
BM_Parse_Upb_FileDesc<UseArena, Alias>           11.3µs ± 6%             11.3µs ± 3%     ~           (p=0.845 n=16+17)
BM_Parse_Upb_FileDesc<InitBlock, Copy>           12.1µs ± 4%             12.1µs ± 3%     ~           (p=0.542 n=18+18)
BM_Parse_Upb_FileDesc<InitBlock, Alias>          11.1µs ± 4%             11.2µs ± 2%     ~           (p=0.330 n=18+16)
BM_Parse_Proto2<FileDesc, NoArena, Copy>         24.2µs ± 3%             25.7µs ±17%     ~           (p=0.167 n=17+20)
BM_Parse_Proto2<FileDesc, UseArena, Copy>        11.6µs ± 3%             11.7µs ± 3%     ~           (p=0.232 n=17+18)
BM_Parse_Proto2<FileDesc, InitBlock, Copy>       11.5µs ± 7%             11.4µs ± 4%     ~           (p=0.799 n=18+19)
BM_Parse_Proto2<FileDescSV, InitBlock, Alias>    12.8µs ± 5%             13.0µs ±14%     ~           (p=0.807 n=18+17)
BM_SerializeDescriptor_Proto2                    5.71µs ± 5%             5.78µs ± 6%     ~           (p=0.143 n=18+18)
BM_SerializeDescriptor_Upb                       10.2µs ± 4%             10.2µs ± 3%     ~           (p=0.613 n=18+17)

name                                           old allocs/op           new allocs/op           delta
BM_ArenaOneAlloc                                   1.00 ± 0%               1.00 ± 0%     ~     (all samples are equal)
BM_ArenaFuseUnbalanced/2                           2.00 ± 0%               2.00 ± 0%     ~     (all samples are equal)
BM_ArenaFuseUnbalanced/8                           8.00 ± 0%               8.00 ± 0%     ~     (all samples are equal)
BM_ArenaFuseUnbalanced/64                          64.0 ± 0%               64.0 ± 0%     ~     (all samples are equal)
BM_ArenaFuseUnbalanced/128                          128 ± 0%                128 ± 0%     ~     (all samples are equal)
BM_ArenaFuseBalanced/2                             2.00 ± 0%               2.00 ± 0%     ~     (all samples are equal)
BM_ArenaFuseBalanced/8                             8.00 ± 0%               8.00 ± 0%     ~     (all samples are equal)
BM_ArenaFuseBalanced/64                            64.0 ± 0%               64.0 ± 0%     ~     (all samples are equal)
BM_ArenaFuseBalanced/128                            128 ± 0%                128 ± 0%     ~     (all samples are equal)
BM_LoadAdsDescriptor_Upb<NoLayout>                6.05k ± 0%              6.05k ± 0%     ~     (all samples are equal)
BM_LoadAdsDescriptor_Upb<WithLayout>              6.36k ± 0%              6.36k ± 0%     ~     (all samples are equal)
BM_LoadAdsDescriptor_Proto2<NoLayout>             83.4k ± 0%              83.4k ± 0%     ~     (all samples are equal)
BM_LoadAdsDescriptor_Proto2<WithLayout>           84.4k ± 0%              84.4k ± 0%   -0.00%        (p=0.013 n=19+20)
BM_Parse_Upb_FileDesc<UseArena, Copy>              7.00 ± 0%               7.00 ± 0%     ~     (all samples are equal)
BM_Parse_Upb_FileDesc<UseArena, Alias>             7.00 ± 0%               7.00 ± 0%     ~     (all samples are equal)
BM_Parse_Proto2<FileDesc, NoArena, Copy>            765 ± 0%                765 ± 0%     ~     (all samples are equal)
BM_Parse_Proto2<FileDesc, UseArena, Copy>          8.00 ± 0%               8.00 ± 0%     ~     (all samples are equal)

name                                           old peak-mem(Bytes)/op  new peak-mem(Bytes)/op  delta
BM_ArenaOneAlloc                                    336 ± 0%                328 ± 0%   -2.38%        (p=0.000 n=20+20)
BM_ArenaFuseUnbalanced/2                            672 ± 0%                656 ± 0%   -2.38%        (p=0.000 n=20+20)
BM_ArenaFuseUnbalanced/8                          2.69k ± 0%              2.62k ± 0%   -2.38%        (p=0.000 n=20+20)
BM_ArenaFuseUnbalanced/64                         21.5k ± 0%              21.0k ± 0%   -2.38%        (p=0.000 n=20+20)
BM_ArenaFuseUnbalanced/128                        43.0k ± 0%              42.0k ± 0%   -2.38%        (p=0.000 n=20+20)
BM_ArenaFuseBalanced/2                              672 ± 0%                656 ± 0%   -2.38%        (p=0.000 n=20+20)
BM_ArenaFuseBalanced/8                            2.69k ± 0%              2.62k ± 0%   -2.38%        (p=0.000 n=20+20)
BM_ArenaFuseBalanced/64                           21.5k ± 0%              21.0k ± 0%   -2.38%        (p=0.000 n=20+20)
BM_ArenaFuseBalanced/128                          43.0k ± 0%              42.0k ± 0%   -2.38%        (p=0.000 n=20+20)
BM_LoadAdsDescriptor_Upb<NoLayout>                10.0M ± 0%               9.9M ± 0%   -0.05%        (p=0.000 n=20+20)
BM_LoadAdsDescriptor_Upb<WithLayout>              10.0M ± 0%              10.0M ± 0%   -0.05%        (p=0.000 n=20+20)
BM_LoadAdsDescriptor_Proto2<NoLayout>             6.62M ± 0%              6.62M ± 0%     ~     (all samples are equal)
BM_LoadAdsDescriptor_Proto2<WithLayout>           6.66M ± 0%              6.66M ± 0%   -0.01%        (p=0.013 n=19+20)
BM_Parse_Upb_FileDesc<UseArena, Copy>             36.5k ± 0%              36.5k ± 0%   -0.02%        (p=0.000 n=20+20)
BM_Parse_Upb_FileDesc<UseArena, Alias>            36.5k ± 0%              36.5k ± 0%   -0.02%        (p=0.000 n=20+20)
BM_Parse_Proto2<FileDesc, NoArena, Copy>          35.8k ± 0%              35.8k ± 0%     ~     (all samples are equal)
BM_Parse_Proto2<FileDesc, UseArena, Copy>         65.3k ± 0%              65.3k ± 0%     ~     (all samples are equal)

name                                           old speed               new speed               delta
BM_LoadAdsDescriptor_Upb<NoLayout>              137MB/s ± 2%            137MB/s ± 4%     ~           (p=0.707 n=16+19)
BM_LoadAdsDescriptor_Upb<WithLayout>            122MB/s ± 3%            123MB/s ± 3%     ~           (p=0.501 n=18+18)
BM_LoadAdsDescriptor_Proto2<NoLayout>          64.2MB/s ± 7%           64.7MB/s ± 5%     ~           (p=0.330 n=16+18)
BM_LoadAdsDescriptor_Proto2<WithLayout>        63.6MB/s ± 3%           63.9MB/s ± 3%     ~           (p=0.303 n=18+17)
BM_Parse_Upb_FileDesc<UseArena, Copy>           614MB/s ± 4%            613MB/s ± 4%     ~           (p=0.935 n=17+18)
BM_Parse_Upb_FileDesc<UseArena, Alias>          665MB/s ± 6%            667MB/s ± 3%     ~           (p=0.873 n=16+17)
BM_Parse_Upb_FileDesc<InitBlock, Copy>          624MB/s ± 4%            622MB/s ± 3%     ~           (p=0.501 n=18+18)
BM_Parse_Upb_FileDesc<InitBlock, Alias>         681MB/s ± 4%            675MB/s ± 2%     ~           (p=0.297 n=18+16)
BM_Parse_Proto2<FileDesc, NoArena, Copy>        311MB/s ± 3%            296MB/s ±15%     ~           (p=0.177 n=17+20)
BM_Parse_Proto2<FileDesc, UseArena, Copy>       649MB/s ± 3%            644MB/s ± 3%     ~           (p=0.232 n=17+18)
BM_Parse_Proto2<FileDesc, InitBlock, Copy>      656MB/s ± 7%            659MB/s ± 4%     ~           (p=0.707 n=18+19)
BM_Parse_Proto2<FileDescSV, InitBlock, Alias>   587MB/s ± 5%            576MB/s ±16%     ~           (p=0.584 n=18+18)
BM_SerializeDescriptor_Proto2                  1.32GB/s ± 5%           1.31GB/s ± 7%     ~           (p=0.143 n=18+18)
BM_SerializeDescriptor_Upb                      737MB/s ± 4%            737MB/s ± 7%     ~           (p=0.839 n=18+18)
```

PiperOrigin-RevId: 520452349

4 files changed

tree: ae6aa5a3ade7bf415bc203cf2ab7e1835a624af7

README.md

μpb: small, fast C protos

μpb (often written ‘upb’) is a small protobuf implementation written in C.

upb is the core runtime for protobuf languages extensions in Ruby, PHP, and Python.

While upb offers a C API, the C API & ABI are not stable. For this reason, upb is not generally offered as a C library for direct consumption, and there are no releases.

Features

upb has comparable speed to protobuf C++, but is an order of magnitude smaller in code size.

Like the main protobuf implementation in C++, it supports:

a generated API (in C)
reflection
binary & JSON wire formats
text format serialization
all standard features of protobufs (oneofs, maps, unknown fields, extensions, etc.)
full conformance with the protobuf conformance tests

upb also supports some features that C++ does not:

optional reflection: generated messages are agnostic to whether reflection will be linked in or not.
no global state: no pre-main registration or other global state.
fast reflection-based parsing: messages loaded at runtime parse just as fast as compiled-in messages.

However there are a few features it does not support:

text format parsing
deep descriptor verification: upb's descriptor validation is not as exhaustive as protoc.

Install

For Ruby, use RubyGems:

$ gem install google-protobuf

For PHP, use PECL:

$ sudo pecl install protobuf

For Python, use PyPI:

$ sudo pip install protobuf

Alternatively, you can build and install upb using vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install upb

The upb port in vcpkg is kept up to date by microsoft team members and community contributors.

If the version is out of date, please create an issue or pull request on the vcpkg repository.

Contributing

Please see CONTRIBUTING.md.