Hi,
On 2025-11-21 11:48:10 +0100, Peter Eisentraut wrote:
> Currently, headerscheck and cpluspluscheck are very slow, and they defeat
> use of ccache. I have fixed that, and now they are much faster. :-)
>
> The problem was (I think) that the test files are created in a
> randomly-named directory (`mktemp -d /tmp/$me.XXXXXX`), and this directory
> is named on the compiler command line, which is part of the cache key.
>
> My solution is to create the test files in the build directory. For
> example, for src/include/storage/ipc.h I generate
>
> headerscheck_src_include_storage_ipc_h.c (or .cpp)
>
> Now ccache works. (And it's also a bit easier to debug everything with this
> naming.)
I applaud this effort. I think long-term I want this stuff properly
integrated into the build system, but this is a good step on its own.
> The observed speedup on Cirrus CI for headerscheck plus cpluspluscheck is
> from about 1min 20s to only 20s. In local use, the speedups are similar.
>
> (I noticed that on Cirrus CI, the first uncached run with this patch was
> quite a bit slower. I don't know if this was because of the additional
> cache population that was happening, or if it was a fluke. Maybe others can
> try it in their environments. Maybe it's not a problem in the long run.)
That doesn't surprise me - one of the major throttling factors on CI is the
number of iops. Filling an empty cache drastically increases iops, due to
having to create all the cache files (there are multiple for each cache
entry).
Hm. I wonder whether we would gain performance if we changed the linux CI
image to use data=writeback instead of the default data=ordered, that can
really help with metadata heavy workloads.
Bilal, any chance you would try modifying the CI image generation to change
the default option for the root filesystem using tune2fs -o
journal_data_writeback (that's easier IME than changing the mount option, as
the root filesystem is mounted very early, in the initramfs, before fstab is
known). Or perhaps just do it interactively in a VM (needs a reboot though)?
Greetings,
Andres Freund
