Fix raw_logging.h to allow deterministic symbols, and regenerate .defs.
__FILE__ is replaced by a string constant, whose type is const char[n],
where 'n' is the length of the actual filename. Because
ABSL_INTERNAL_LOG() passes __FILE__ to the templated function
AtomicHook::operator()(Args&...), the value of 'n' is encoded into the
mangled name of the generated operator. As a result, generated .def
files containing an entry for this operator would result in "symbol not
found" errors during link on machines which used a different-length
filename for such a call.
This was not previously a problem because for some reason this
particular symbol was omitted from the generated .defs. As part of
trying to fix the win32-archive-dbg builder, which failed due to
not-found symbols after crrev.com/793680, I saw that no updates to these
files had been applied during the last couple of Abseil rolls, suspected
they might be out of date, and regenerated the files. This caused more
builds to break rather than fewer since this symbol was now being
included.
It's possible this has something to do with absolute paths (see
blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html ),
but I don't think so; we should be using relative paths to compile.
Plus, the failed builds were on configurations that did not enable
is_official_build, and when that's not enabled, we should be setting
strip_absolute_paths_from_debug_symbols as well. (That might be
irrelevant; I'm overly ignorant of how object files encode info.) I am
guessing the core problem is that if multiple files invoke
ABSL_INTERNAL_LOG(), then even with relative file paths, the generated
symbol effectively depends on which file ends up instantiating this
first, or gets selected by the linker, or whatever.
So this patch modifies the macro to explicitly cast __FILE__ to a
pointer (the type that will be passed to the ultimate destination
anyway), making the symbol name for the operator deterministic. If we
think this patch is a good idea, we should try to upstream it.
The .def files have been subsequently regenerated (and got one other
symbol added, so maybe they did omit something important?), and I added
a note to README.chromium about running this script, along with trying
to update that file more generally.
Bug: 1103706
Change-Id: I39ba72b4dd688340cacf0a814d6f461a96f891e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2333649
Commit-Queue: Peter Kasting <pkasting@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#794391}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2a15c32108ed6c80bb75b529e027860c17219553
diff --git a/README.chromium b/README.chromium
index cb4beff..076f0b2 100644
--- a/README.chromium
+++ b/README.chromium
@@ -20,21 +20,14 @@
2. Copy the content of the Abseil git repo to //third_party/abseil-cpp.
-3. From //third_party/abseil-cpp/ launch ./rename_annotations.sh.
- This script will rewrite dynamic_annotations and thread_annotations
- macros and function inside Abseil in order to avoid ODR violations
- and macro clashing with Chromium
- (see: https://github.com/abseil/abseil-cpp/issues/122).
+3. From your source root run third_party/abseil-cpp/generate_def_files.py to
+ regenerate Windows symbol definition files.
Local Modifications:
* absl/copts.bzl has been translated to //third_party/absl-cpp/BUILD.gn. Both
files contain lists of compiler flags in order to reduce duplication.
-* All the BUILD.bazel files has been translated to BUILD.gn files.
-
-* Functions and macros in absl/base/dynamic_annotations.{h,cc} and
- absl/base/thread_annotations.h have been renamed to avoid ODR
- violations and macro clashes with Chromium (see step 3).
+* All the BUILD.bazel files have been translated to BUILD.gn files.
* Patches from //third_party/abseil-cpp/patches have been applied.
diff --git a/absl/base/internal/raw_logging.h b/absl/base/internal/raw_logging.h
index 51551ba..4d93a22 100644
--- a/absl/base/internal/raw_logging.h
+++ b/absl/base/internal/raw_logging.h
@@ -72,10 +72,11 @@
//
// The API is a subset of the above: each macro only takes two arguments. Use
// StrCat if you need to build a richer message.
-#define ABSL_INTERNAL_LOG(severity, message) \
- do { \
- ::absl::raw_logging_internal::internal_log_function( \
- ABSL_RAW_LOGGING_INTERNAL_##severity, __FILE__, __LINE__, message); \
+#define ABSL_INTERNAL_LOG(severity, message) \
+ do { \
+ ::absl::raw_logging_internal::internal_log_function( \
+ ABSL_RAW_LOGGING_INTERNAL_##severity, \
+ static_cast<const char*>(__FILE__), __LINE__, message); \
} while (0)
#define ABSL_INTERNAL_CHECK(condition, message) \
diff --git a/generate_def_file.py b/generate_def_files.py
old mode 100644
new mode 100755
similarity index 90%
rename from generate_def_file.py
rename to generate_def_files.py
index 996baf0..1961ac4
--- a/generate_def_file.py
+++ b/generate_def_files.py
@@ -1,6 +1,10 @@
-"""Script to generate Chromium's Abseil .def file at roll time.
+#!/usr/bin/env python
-This script generates //third_party/abseil-app/absl/symbols_x64.def at Abseil
+# NOTE: This script requires python 3.
+
+"""Script to generate Chromium's Abseil .def files at roll time.
+
+This script generates //third_party/abseil-app/absl/symbols_*.def at Abseil
roll time.
Since Abseil doesn't export symbols, Chromium is forced to consider all
@@ -11,6 +15,9 @@
Unless you are on a Windows machine, you need to set up your Chromium
checkout for cross-compilation by following the instructions at
https://chromium.googlesource.com/chromium/src.git/+/master/docs/win_cross.md.
+If you are on Windows, you may need to tweak this script to run, e.g. by
+changing "gn" to "gn.bat", changing "llvm-nm-9" to the name of your copy of
+llvm-nm, etc.
"""
import fnmatch
diff --git a/patches/0002-deterministic-logging-invocation.patch b/patches/0002-deterministic-logging-invocation.patch
new file mode 100644
index 0000000..798d55d
--- /dev/null
+++ b/patches/0002-deterministic-logging-invocation.patch
@@ -0,0 +1,20 @@
+diff --git a/third_party/abseil-cpp/absl/base/internal/raw_logging.h b/third_party/abseil-cpp/absl/base/internal/raw_logging.h
+index 51551bafff48..4d93a22fe628 100644
+--- a/third_party/abseil-cpp/absl/base/internal/raw_logging.h
++++ b/third_party/abseil-cpp/absl/base/internal/raw_logging.h
+@@ -72,10 +72,11 @@
+ //
+ // The API is a subset of the above: each macro only takes two arguments. Use
+ // StrCat if you need to build a richer message.
+-#define ABSL_INTERNAL_LOG(severity, message) \
+- do { \
+- ::absl::raw_logging_internal::internal_log_function( \
+- ABSL_RAW_LOGGING_INTERNAL_##severity, __FILE__, __LINE__, message); \
++#define ABSL_INTERNAL_LOG(severity, message) \
++ do { \
++ ::absl::raw_logging_internal::internal_log_function( \
++ ABSL_RAW_LOGGING_INTERNAL_##severity, \
++ static_cast<const char*>(__FILE__), __LINE__, message); \
+ } while (0)
+
+ #define ABSL_INTERNAL_CHECK(condition, message) \
diff --git a/symbols_arm64_dbg.def b/symbols_arm64_dbg.def
index a67b055..fb78f79 100644
--- a/symbols_arm64_dbg.def
+++ b/symbols_arm64_dbg.def
@@ -191,6 +191,7 @@
??$?RAEAY0O@$$CBDPEAVCondVar@absl@@@?$AtomicHook@P6AXPEBDPEBX@Z@base_internal@absl@@QEBAXAEAY0O@$$CBD$$QEAPEAVCondVar@2@@Z
??$?RAEA_J@?$AtomicHook@P6AX_J@Z@base_internal@absl@@QEBAXAEA_J@Z
??$?RPEAVSpinLock@base_internal@absl@@AEB_K@?$AtomicHook@P6AXPEBX_J@Z@base_internal@absl@@QEBAX$$QEAPEAVSpinLock@12@AEB_K@Z
+ ??$?RW4LogSeverity@absl@@PEBDHAEAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@?$AtomicHook@P6AXW4LogSeverity@absl@@PEBDHAEBV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z@base_internal@absl@@QEBAX$$QEAW4LogSeverity@2@$$QEAPEBD$$QEAHAEAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z
??$?XH@Duration@absl@@QEAAAEAV01@H@Z
??$Append@V?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@$0A@@Cord@absl@@QEAAX$$QEAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z
??$AppendImpl@AEBVCord@absl@@@Cord@absl@@AEAAXAEBV01@@Z
@@ -1669,6 +1670,7 @@
?GetAllocator@?$AllocationTransaction@V?$allocator@USubRange@absl@@@__1@std@@@inlined_vector_internal@absl@@QEAAAEAV?$allocator@USubRange@absl@@@__1@std@@XZ
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K@Z
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K_K@Z
+ ?GetCachedTID@base_internal@absl@@YAIXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@H@__1@std@@@inlined_vector_internal@absl@@QEAAAEA_KXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@PEAUCordRep@cord_internal@absl@@@__1@std@@@inlined_vector_internal@absl@@QEAAAEA_KXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@PEBUCordRep@cord_internal@absl@@@__1@std@@@inlined_vector_internal@absl@@QEAAAEA_KXZ
diff --git a/symbols_arm64_rel.def b/symbols_arm64_rel.def
index 98a8803..1ce997c 100644
--- a/symbols_arm64_rel.def
+++ b/symbols_arm64_rel.def
@@ -368,6 +368,7 @@
?FromUniversal@absl@@YA?AVTime@1@_J@Z
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K@Z
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K_K@Z
+ ?GetCachedTID@base_internal@absl@@YAIXZ
?GetCurrentTimeNanos@absl@@YA_JXZ
?GetFlatAux@Cord@absl@@CA_NPEAUCordRep@cord_internal@2@PEAVstring_view@2@@Z
?GetId@GraphCycles@synchronization_internal@absl@@QEAA?AUGraphId@23@PEAX@Z
diff --git a/symbols_x64_dbg.def b/symbols_x64_dbg.def
index 4ddceb6..b767bda 100644
--- a/symbols_x64_dbg.def
+++ b/symbols_x64_dbg.def
@@ -191,6 +191,7 @@
??$?RAEAY0O@$$CBDPEAVCondVar@absl@@@?$AtomicHook@P6AXPEBDPEBX@Z@base_internal@absl@@QEBAXAEAY0O@$$CBD$$QEAPEAVCondVar@2@@Z
??$?RAEA_J@?$AtomicHook@P6AX_J@Z@base_internal@absl@@QEBAXAEA_J@Z
??$?RPEAVSpinLock@base_internal@absl@@AEB_K@?$AtomicHook@P6AXPEBX_J@Z@base_internal@absl@@QEBAX$$QEAPEAVSpinLock@12@AEB_K@Z
+ ??$?RW4LogSeverity@absl@@PEBDHAEAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@?$AtomicHook@P6AXW4LogSeverity@absl@@PEBDHAEBV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z@base_internal@absl@@QEBAX$$QEAW4LogSeverity@2@$$QEAPEBD$$QEAHAEAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z
??$?XH@Duration@absl@@QEAAAEAV01@H@Z
??$Append@V?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@$0A@@Cord@absl@@QEAAX$$QEAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z
??$AppendImpl@AEBVCord@absl@@@Cord@absl@@AEAAXAEBV01@@Z
@@ -1671,6 +1672,7 @@
?GetAllocator@?$AllocationTransaction@V?$allocator@USubRange@absl@@@__1@std@@@inlined_vector_internal@absl@@QEAAAEAV?$allocator@USubRange@absl@@@__1@std@@XZ
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K@Z
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K_K@Z
+ ?GetCachedTID@base_internal@absl@@YAIXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@H@__1@std@@@inlined_vector_internal@absl@@QEAAAEA_KXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@PEAUCordRep@cord_internal@absl@@@__1@std@@@inlined_vector_internal@absl@@QEAAAEA_KXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@PEBUCordRep@cord_internal@absl@@@__1@std@@@inlined_vector_internal@absl@@QEAAAEA_KXZ
diff --git a/symbols_x64_rel.def b/symbols_x64_rel.def
index 1e301a4..1c620a5 100644
--- a/symbols_x64_rel.def
+++ b/symbols_x64_rel.def
@@ -368,6 +368,7 @@
?FromUniversal@absl@@YA?AVTime@1@_J@Z
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K@Z
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K_K@Z
+ ?GetCachedTID@base_internal@absl@@YAIXZ
?GetCurrentTimeNanos@absl@@YA_JXZ
?GetFlatAux@Cord@absl@@CA_NPEAUCordRep@cord_internal@2@PEAVstring_view@2@@Z
?GetId@GraphCycles@synchronization_internal@absl@@QEAA?AUGraphId@23@PEAX@Z
diff --git a/symbols_x64_rel_asan.def b/symbols_x64_rel_asan.def
index 90417be..ff4257a 100644
--- a/symbols_x64_rel_asan.def
+++ b/symbols_x64_rel_asan.def
@@ -377,6 +377,7 @@
?FromUniversal@absl@@YA?AVTime@1@_J@Z
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K@Z
?GetAppendRegion@InlineRep@Cord@absl@@QEAAXPEAPEADPEA_K_K@Z
+ ?GetCachedTID@base_internal@absl@@YAIXZ
?GetCurrentTimeNanos@absl@@YA_JXZ
?GetFlatAux@Cord@absl@@CA_NPEAUCordRep@cord_internal@2@PEAVstring_view@2@@Z
?GetId@GraphCycles@synchronization_internal@absl@@QEAA?AUGraphId@23@PEAX@Z
diff --git a/symbols_x86_dbg.def b/symbols_x86_dbg.def
index 8d50a8e..a9c8727 100644
--- a/symbols_x86_dbg.def
+++ b/symbols_x86_dbg.def
@@ -191,6 +191,7 @@
??$?RAAY0O@$$CBDPAVCondVar@absl@@@?$AtomicHook@P6AXPBDPBX@Z@base_internal@absl@@QBEXAAY0O@$$CBD$$QAPAVCondVar@2@@Z
??$?RAA_J@?$AtomicHook@P6AX_J@Z@base_internal@absl@@QBEXAA_J@Z
??$?RPAVSpinLock@base_internal@absl@@AB_K@?$AtomicHook@P6AXPBX_J@Z@base_internal@absl@@QBEX$$QAPAVSpinLock@12@AB_K@Z
+ ??$?RW4LogSeverity@absl@@PBDHAAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@?$AtomicHook@P6AXW4LogSeverity@absl@@PBDHABV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z@base_internal@absl@@QBEX$$QAW4LogSeverity@2@$$QAPBD$$QAHAAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z
??$?XH@Duration@absl@@QAEAAV01@H@Z
??$Append@V?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@$0A@@Cord@absl@@QAEX$$QAV?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@@Z
??$AppendImpl@ABVCord@absl@@@Cord@absl@@AAEXABV01@@Z
@@ -1668,6 +1669,7 @@
?GetAllocator@?$AllocationTransaction@V?$allocator@USubRange@absl@@@__1@std@@@inlined_vector_internal@absl@@QAEAAV?$allocator@USubRange@absl@@@__1@std@@XZ
?GetAppendRegion@InlineRep@Cord@absl@@QAEXPAPADPAI@Z
?GetAppendRegion@InlineRep@Cord@absl@@QAEXPAPADPAII@Z
+ ?GetCachedTID@base_internal@absl@@YAIXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@H@__1@std@@@inlined_vector_internal@absl@@QAEAAIXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@PAUCordRep@cord_internal@absl@@@__1@std@@@inlined_vector_internal@absl@@QAEAAIXZ
?GetCapacity@?$AllocationTransaction@V?$allocator@PBUCordRep@cord_internal@absl@@@__1@std@@@inlined_vector_internal@absl@@QAEAAIXZ
diff --git a/symbols_x86_rel.def b/symbols_x86_rel.def
index 861f1ac..ad76f93 100644
--- a/symbols_x86_rel.def
+++ b/symbols_x86_rel.def
@@ -368,6 +368,7 @@
?FromUniversal@absl@@YA?AVTime@1@_J@Z
?GetAppendRegion@InlineRep@Cord@absl@@QAEXPAPADPAI@Z
?GetAppendRegion@InlineRep@Cord@absl@@QAEXPAPADPAII@Z
+ ?GetCachedTID@base_internal@absl@@YAIXZ
?GetCurrentTimeNanos@absl@@YA_JXZ
?GetFlatAux@Cord@absl@@CA_NPAUCordRep@cord_internal@2@PAVstring_view@2@@Z
?GetId@GraphCycles@synchronization_internal@absl@@QAE?AUGraphId@23@PAX@Z
