Name: Protocol Buffers
Short Name: protobuf
URL: https://github.com/google/protobuf
License: BSD
License File: LICENSE
Version: 21.12
CPEPrefix: cpe:/a:google:protobuf:21.12
Revision: f0dc78d7e6e331b8c6bb2d5283e06aa26883ca7c
Security Critical: yes
Shipped: yes
Steps used to create the current version (or roll a newer one):
0. Nuke third_party/protobuf from orbit (rm -rf `third_party/protobuf`). Note
that this removes this README.chromium so you may want it open in code search
or remember the steps up until 2. where you get it back.
1. Pull the release from https://github.com/google/protobuf/releases (Source
code .tar.gz). Extract to `third_party/protobuf` (without version suffix),
i.e. you may `$ mv protobuf-21.12 third_party/protobuf` or similar.
2. Add back chromium's non-upstream files (from this directory):
$ git checkout main BUILD.gn DEPS DIR_METADATA OWNERS README.chromium \
gen_chromium_file_lists.py gen_extra_chromium_files.py proto_library.gni \
patches/
Note: Ideally we wouldn't do this but instead have a
`third_party/protobuf/src/` subdirectory where upstream code goes directly
into. That will require updating things outside this repository (at least
`third_party/webrtc`) to agree on where the source code lives. I wouldn't be
surprised if that also required Google-internal uses of protobuf from webrtc
to also agree on `third_party/protobuf/src` so maybe this'll never happen?
Here be dragons. Probably.
Probably worth making a local `git commit` here before applying patches (next
step).
3. Apply patches in patches/ (see the description below):
$ for patch in patches/*; do patch -s -p1 < $patch; done
When a patch fails, it's probably best to check out previous state (from 3)
for any changes not inside the patches/ folder, fix the patch and then try to
reapply everything. It's not a good idea to fix the error, forget to generate
a patch and leave patches/ in a bad state.
If a patch no longer applies it may not be needed, if so remove it and its
description in this README.chromium file. Similarly try to eventually
upstream any patches out of band from unblocking the roll so we can
eventually remove it here as well.
Probably worth committing locally here too.
4. Generate `proto_sources.gni` using the script `./gen_chromium_file_lists.py`.
Add and commit `proto_sources.gni`,
5. Generate descriptor_pb2.py using the script "gen_extra_chromium_files.py" in
the same directory as this file.
$ ./gen_extra_chromium_files.py -C ../../out/Default
Add and commit `python/google/protobuf/descriptor_pb2.py` and
`python/google/protobuf/compiler/plugin_pb2.py` as well.
6. Update Version, CPEPrefix and Revision in this file.
Notes (hints) for the next roll(er):
* This is probably harder to roll than most things you've rolled in the past.
Good luck! Also please reach out for help proactively. The current roll as of
writing this required help from multiple very-senior people to even figure out
what was broken in the first place. `protobuf` rolling is not in a good place.
* Update README.chromium as you go so you don't forget (after step 2. above, so
you don't nuke those changes).
* Rolling one version of protobuf likely introduces a large amount of changes.
We should possibly figure out if it's reasonable to stay close to tip of tree
and roll frequently rather than use released versions. Sharp corners here
likely include that the version is encoded in protobuf output and I (pbos@)
don't know if that version tag changes immediately on format incompatibilities
or get bumped every time something format incompatible is introduced.
* After v3.20 protobuf-javascript/ split out from upstream protobuf/. You may
need to be aware of both repositories. Also upstream protobuf/ seems to move
faster / out of sync with protobuf-javascript/. Hopefully that doesn't cause
format incompatibilities in the future (protobuf/ supports something we need
that protobuf-javascript/ doesn't).
* Any argument changes to `protoc_wrapper` in `third_party/protobuf/BUILD.gn`
may require updating `build/config/siso/protoc_wrapper.star` as well.
* We've seen instances where instead of rolling protobuf a local copy of a new
built-in type has been checked in. When that happens you may see ODR
violations as your built-in type is defined in multiple files. If that happens
you may want to move the copy that should never have been checked in into its
own namespace, complete the roll and then cleanup the local copy.
* third_party/mediapipe checks in a weird modified copy of
`third_party/protobuf/src/google/protobuf/any.pb.cc` called `any_lite.pb.cc`.
This precompiled protobuf will get invalidated on any protobuf version bumps
and you'll need to either painstakingly generated a new one (do a diff of the
checked-in `any_lite.pb.{cc,h}` and current `any.pb.{cc,h}` to aid you in the
process.
WARNING: This item is a lot larger than it looks. We should really
either fix mediapipe upstream to build with `MEDIAPIPE_PROTO_LITE` and
`MEDIAPIPE_PROTO_THIRD_PARTY` both set, but it looks like that configuration
has bit rotted.
Note about vpython:
Some Python scripts end up mixing protoc output from this copy of protobuf with
the google.protobuf module from vpython's protobuf. If Python scripts break due
to the vpython copy of protobuf, you may need to update the version in
//.vpython3. See https://crbug.com/1320047.
Description of the patches:
- 0004-fix-shared-library-exports.patch
This patch allows exporting protobuf symbols in Linux .so libraries, so
that protobuf can be built as a component (see http://crrev.com/179806).
- 0008-uninline_get_empty_string.patch
- 0010-uninline-generated-code.patch
These patches uninline some functions, resulting in a significant reduction
(somewhere between 500 KB and 1 MB) of binary size.
- 0021-Fix-protobuf-s-library-.gitinore-file.patch
Un-ignores python/google/protobuf/descriptor_pb2.py and
python/google/protobuf/compiler/plugin_pb2.py
- 0022-Allow-deprecated-fields.patch
Allows deprecated fields to be used without extra C++ compiler warnings.
- 0026-remove-sprintf.patch
Imports
https://github.com/protocolbuffers/protobuf/commit/c0fc2e881bc36aafb0bf539bf41889611370f60c
to remove use of sprintf.
- 0027-no-noreturn.patch
Removes an instance of [[noreturn]]. The attribute is correct, but the way
protobuf's GOOGLE_LOG(FATAL) is defined, the compiler can't see this and it
trips -Winvalid-noreturn. See https://github.com/protocolbuffers/protobuf/issues/9817
- 0029-make-initializers-optimizable.patch
Makes the InitProtobufDefaults() static initializer optimizable by Clang when
built with libc++. It patches out the OnShutdownDestroyString call, which we
do not need, and removes the thread-safe initialization. Thread safety is only
needed if a static initializer spawns a thread which then calls
InitProtobufDefaults() without synchronizing with the start of main().
(Anything which happens after main() starts can rely on the initializer
running.)
- 0030-workaround-window-constinit.patch
Disables PROTOBUF_CONSTINIT in generated code in Windows shared library
builds. Protobuf's default instances take pointers to a dllimport variable,
fixed_address_empty_string. This is not constinit on Windows. This is a bug in
protobuf as the default instance was intended to be constant-initialized. But
the components build in Chromium is a developer configuration, so we tolerate
an initializer as long as the build works, until protobuf has a proper fix.
See https://github.com/protocolbuffers/protobuf/issues/10159.
- 0031-workaround-cfi-unrelated-cast.patch
A workaround for Clang's Control Flow Integrity check for casting pointers to
memory that his not yet initialized to be of that type for empty arrays, does
not work, and still fails. This patch removes the workaround and instead
disables cfi-unrelated-cast for the affected methods and simplifies them.
See https://github.com/protocolbuffers/protobuf/issues/10186.
See https://bugs.chromium.org/p/chromium/issues/detail?id=1294200#c26.
- 0032-cxx20.patch
Fixes necessary to build in --std=c++20 mode.
Imports https://critique.corp.google.com/cl/451177197 (a portion of
https://github.com/protocolbuffers/protobuf/commit/6dd8af4ecfa7987bddb309862932886b84f1e4ef
).
- 0033-no-enum-conversion-warn.patch
Avoid hitting the clang error -Wenum-constexpr-conversion by specifying width
of the enum.
Imports https://critique.corp.google.com/cl/466986872.
- 0034-change-macro-to-avoid-pedantic-warning.patch
Avoid hitting -Wextra-semi.
Imports (rebased):
https://github.com/protocolbuffers/protobuf/commit/def602dd07b7eae1cac6823705975317b5607fc3
- 0035-fix-shared-library-constants
Fixes for component build when building with MediaPipe. AnyMetadata::PackFrom
is called which require string constants to be exported.
- 0036-fix-undefined-memcpy-call.patch
Fixes an undefined call to memcpy which may be passed NULL, 0. NULL, 0 is
forbidden in C, due to a mistake in the C language specification. This
partially integrates
https://github.com/protocolbuffers/protobuf/commit/b2e1d7d8a1b1959c782595cd815c22fb690ac8e2
from upstream and may be removed when protobuf is updated past that commit.
- 0037-fix-wextra-semi.patch
Removes an extra semicolon, fixed in upstream
https://github.com/protocolbuffers/protobuf/commit/3e6f8ee152db71ae49cc95649feab50d2faacdfa
- 0038-silence-implicit-precision-loss.patch
Silences -Wshorten-64-to-32 for now. Fixed in upstream
https://github.com/protocolbuffers/protobuf/commit/d85c9944c55fb38f4eae149979a0f680ea125ecb
- 0039-generate-mediapipe-anylite.patch
Workaround to allow generating protobuf code compatible with
third_party/mediapipe. This replaces checked-in precompiled .pb.cc files. See
https://crbug.com/332939935, which when/if resolved should make this patch
obsolete.
- 0040-fix-ndk-major.patch
Fix Android compile error that affects webrtc. Fixed in upstream
https://github.com/protocolbuffers/protobuf/commit/9471a882256eaf01027524e8b6c15cbb5e42bef6
Codebase Browser
chromium