Re: Cirrus CI (Windows help wanted) - Mailing list pgsql-hackers

On 1/5/21 7:18 AM, Andrew Dunstan wrote:
> On 1/4/21 10:58 PM, Thomas Munro wrote:
>> Hi,
>>
>> My new favourite CI is Cirrus CI, because it has 4 operating systems,
>> generous enough quotas to handle 250+ branches in a single account,
>> and public build/test log URLs.  I flipped cfbot.cputube.org (mostly)
>> over to that and it seems to work well so far -- fingers crossed.
>> I've also been using it for my own development branches that involve
>> some systems hacking-heavy work that uses different kernel interfaces
>> on all 4 of those OSes.
>>
>> There's one thing I'm stuck on, though: Windows.  If anyone wants to
>> help figure out how to get PostgreSQL to build on Cirrus's Windows,
>> I'd be very interested.  To play with this stuff, you need a public
>> Github repo, and you need to add Cirrus CI from "Marketplace" (it's
>> free for public/open source), and then you add a .cirrus.yml file such
>> as https://github.com/macdice/cfbot/blob/master/cirrus/.cirrus.yml to
>> the top level of a PostgreSQL branch.  When you push, you should see
>> build results on the Github web UI.
>>
>> For a similar example that works on Windows on another CI, see
>> https://github.com/macdice/cfbot/blob/master/appveyor/appveyor.yml
>> (note that it also references a couple of other files; it'd be nice to
>> be able to do that stuff without the need for separate files, possibly
>> by using Power Shell).  That's what cfbot is using for Windows for
>> now, which works really well, but it'd be nice to have more
>> options/choices.  For another example of Windows builds working on
>> another CI, see the Github Actions patch I posted earlier when I was
>> considering that for cfbot[1].  I think what's different is that those
>> other CIs have images with MSVC on them, but Cirrus wants you to
>> figure out how to install the right toolchain yourself (and then, as a
>> next step after it's actually working, it also provides a way to
>> define what you want in a way that captures the resulting image using
>> Docker voodoo, so that you get fast startup times).  Or something.
>>
>> [1]
https://www.postgresql.org/message-id/flat/CA%2BhUKG%2By_SHVQcU3CPokmJxuHp1niebCjq4XzZizf8SR9ZdQRQ%40mail.gmail.com
>>
>>
> Here's what I use for MS Tools in automated setups, which gives you all
> you should need:
>
>     choco install -y --no-progress --limit-output
>     visualstudio2019-workload-vctools --package-parameters
>     "--includeOptional"
>
> Your PATH adjustment should add this:
>
>     C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\bin
>
> There might be other environment settings needed - see drongo's config on the buildfarm. LMK how you get on.
>
> Constructing an image where you don't have to do that every build would be super nice.
>
>

OK, I have dug into this quite a bit. The way cirrus works is in fact
somewhat fragile. Anyway, here are the highlights:


The image you're using comes with these choco packages pre-installed:


    Chocolatey v0.10.15
    7zip 19.0
    7zip.install 19.0
    chocolatey 0.10.15
    chocolatey-core.extension 1.3.5.1
    chocolatey-dotnetfx.extension 1.0.1
    chocolatey-visualstudio.extension 1.8.1
    chocolatey-windowsupdate.extension 1.0.4
    cmake 3.19.1
    cmake.install 3.19.1
    dotnetfx 4.8.0.20190930
    git 2.29.2.3
    git.install 2.29.2.3
    KB2919355 1.0.20160915
    KB2919442 1.0.20160915
    KB2999226 1.0.20181019
    KB3033929 1.0.5
    KB3035131 1.0.3
    mingw 8.1.0
    vcredist140 14.28.29325.2
    visualstudio-installer 2.0.1
    visualstudio2019-workload-vctools 1.0.0
    visualstudio2019buildtools 16.8.2.0


However, sadly the vctools package above isn't installed with all its
optional packages, so some crucial things are missing. I cured that by
forcing a reinstall with the optional components enabled. Sadly, that
takes a huge amount of time, over 20 minutes. We either need to find an
image where this isn't necessary or find out how to make one to use,
assuming that's possible. Or maybe we can ask cirrus to modify their
buildscripts just a tad to include the required parameter.

The other issue I ran into was one with the ActivePerl install script. I
got around that by installing StrawberryPerl instead. It should work
fine I believe.

So here is the .cirrus.yml file I came up with. The last line fails as
expected because I have been testing in an environment without postgres
sources, but other than that it works. Note that we use VS's vcvarsall
script to set the environment properly for us.

    task:
      name: Windows
      windows_container:
        image: cirrusci/windowsservercore:cmake
      install_script:
        - choco list --localonly
        - choco install -y winflexbison diffutils
        - rename c:\ProgramData\chocolatey\bin\win_flex.exe flex.exe
        - rename c:\ProgramData\chocolatey\bin\win_bison.exe bison.exe
        - choco install -y strawberryperl
        - choco install --force -y visualstudio2019-workload-vctools
    --package-parameters "--includeOptional"
        - choco list --localonly
      build_script:
          - cd
          - set
          - cd "c:/Program Files (x86)/Microsoft Visual
    Studio/2019/BuildTools/VC/Auxiliary/Build"
          - vcvarsall x64
          - echo on
          - cd
    C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build
          - set
          - perl -v
          - msbuild pgsql.sln

HTH


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com