Re: Mingw task for Cirrus CI - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: Mingw task for Cirrus CI
Date
Msg-id d1499118-56ed-75db-d14e-d55b96b55339@dunslane.net
Whole thread Raw
In response to Re: Mingw task for Cirrus CI  (Andrew Dunstan <andrew@dunslane.net>)
List pgsql-hackers
On 4/7/22 16:48, Andrew Dunstan wrote:
> On 4/7/22 13:10, Andres Freund wrote:
>> Hi,
>>
>> On 2022-04-06 11:03:37 -0400, Andrew Dunstan wrote:
>>> On 3/30/22 20:26, Andres Freund wrote:
>>>> Could you try using dash to invoke configure here, and whether it makes configure faster?
>>> I got weird failures re libxml/parser.h when I tried with dash. See
>>> <https://cirrus-ci.com/task/5963254039052288> (It would be nice if we
>>> could see config.log on failure.)
>> Since dash won't help us to get the build time down sufficiently, and the
>> tests don't pass without a separate build tree, I looked at what makes
>> config/prep_buildtree so slow.
>>
>> It's largely just bad code. The slowest part are spawning one expr and mkdir
>> -p for each directory. One 'cmp' for each makefile doesn't help either.
>>
>> The expr can be replaced with
>>   subdir=${item#$sourcetree}
>> that's afaics posix syntax ([1]), not bash.
>>
>> Spawning one mkdir for each directory can be replaced by a single mkdir
>> invocation with all the directories. On my linux workstation that gets the
>> time for the first loop down from 1005ms to 38ms, really.
>>
>> That has the danger of the commandline getting too long. But since we rely on
>> the final link of the backend to be done in a single command, I don't think
>> it's making things worse? We could try to use xargs otherwise, iirc that's in
>> posix as well.
>>
>> Using parameter substitution in the second loop takes it down from 775ms to
>> 533ms. Not calling cmp when the file doesn't exist cuts it down to 337ms.
>>
>> I don't know of a way to batch the call to ln. The time with ln replaced with
>> : is 151ms, fwiw.
>
> AFAIK Msy2s 'ln -s' by default copies a non-directory rather than
> actually symlinking it. If we want real symlinks, then we need
> MSYS=|winsymlinks:nativestrict set. The is will fail unless the calling
> user is an Administrator or has the SeCreateSymbolicLink privilege. See
> |


Sometimes I hate Thunderbird. Of course the | is spurious above, we
would need

MSYS=winsymlinks:nativestrict

set.


cheers

andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com




pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: Mingw task for Cirrus CI
Next
From: Andrew Dunstan
Date:
Subject: Re: Can we automatically add elapsed times to tap test log?