Thread: Commitfest 2020-11 is closed
Hi, hackers! Commitfest 2020-11 is officially closed now. Many thanks to everyone who participated by posting patches, reviewing them, committing and sharing ideas in discussions! Today, me and Georgios will move the remaining items to the next CF or return them with feedback. We're planning to leave Ready For Committer till the end of the week, to make them more visible and let them get the attention they deserve. And finally in the weekend I'll gather and share some statistics. -- Anastasia Lubennikova Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On Tue, Dec 01, 2020 at 01:05:54PM +0300, Anastasia Lubennikova wrote: > Commitfest 2020-11 is officially closed now. > Many thanks to everyone who participated by posting patches, reviewing them, > committing and sharing ideas in discussions! Thanks Anastasia and Georgios for working on that! -- Michael
Attachment
Anastasia Lubennikova <a.lubennikova@postgrespro.ru> writes:
> Commitfest 2020-11 is officially closed now.
> Many thanks to everyone who participated by posting patches, reviewing
> them, committing and sharing ideas in discussions!
Thanks for all the hard work!
> Today, me and Georgios will move the remaining items to the next CF or
> return them with feedback. We're planning to leave Ready For Committer
> till the end of the week, to make them more visible and let them get the
> attention they deserve.
This is actually a bit problematic, because now the cfbot is ignoring
those patches (or if it's not, I don't know where it's displaying the
results). Please go ahead and move the remaining open patches, or
else re-open the CF if that's possible.
regards, tom lane
On Thu, Dec 3, 2020 at 10:00 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > This is actually a bit problematic, because now the cfbot is ignoring > those patches (or if it's not, I don't know where it's displaying the > results). Please go ahead and move the remaining open patches, or > else re-open the CF if that's possible. As of quite recently, Travis CI doesn't seem to like cfbot's rate of build jobs. Recently it's been taking a very long time to post results for new patches and taking so long to get around to retesting patches for bitrot that the my "delete old results after a week" logic started wiping out some results while they are still the most recent, leading to the blank spaces where the results are supposed to be. D'oh. I'm looking into a couple of options.
On 12/2/20 4:15 PM, Thomas Munro wrote: > On Thu, Dec 3, 2020 at 10:00 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: >> This is actually a bit problematic, because now the cfbot is ignoring >> those patches (or if it's not, I don't know where it's displaying the >> results). Please go ahead and move the remaining open patches, or >> else re-open the CF if that's possible. > > As of quite recently, Travis CI doesn't seem to like cfbot's rate of > build jobs. Recently it's been taking a very long time to post > results for new patches and taking so long to get around to retesting > patches for bitrot that the my "delete old results after a week" logic > started wiping out some results while they are still the most recent, > leading to the blank spaces where the results are supposed to be. > D'oh. I'm looking into a couple of options. pgBackRest test runs have gone from ~17 minutes to 3-6 hours over the last two months. Also keep in mind that travis-ci.org will be shut down at the end of the month. Some users who have migrated to travis-ci.com have complained that it is not any faster, but I have not tried myself (yet). Depending on how you have Github organized migrating to travis-ci.com may be bit tricky because it requires full access to all private repositories in your account and orgs administrated by your account. Regards, -- -David david@pgmasters.net
On Thu, Dec 3, 2020 at 10:51 AM David Steele <david@pgmasters.net> wrote: > On 12/2/20 4:15 PM, Thomas Munro wrote: > > On Thu, Dec 3, 2020 at 10:00 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > >> This is actually a bit problematic, because now the cfbot is ignoring > >> those patches (or if it's not, I don't know where it's displaying the > >> results). Please go ahead and move the remaining open patches, or > >> else re-open the CF if that's possible. > > > > As of quite recently, Travis CI doesn't seem to like cfbot's rate of > > build jobs. Recently it's been taking a very long time to post > > results for new patches and taking so long to get around to retesting > > patches for bitrot that the my "delete old results after a week" logic > > started wiping out some results while they are still the most recent, > > leading to the blank spaces where the results are supposed to be. > > D'oh. I'm looking into a couple of options. > > pgBackRest test runs have gone from ~17 minutes to 3-6 hours over the > last two months. Ouch. > Also keep in mind that travis-ci.org will be shut down at the end of the > month. Some users who have migrated to travis-ci.com have complained > that it is not any faster, but I have not tried myself (yet). Oh. > Depending on how you have Github organized migrating to travis-ci.com > may be bit tricky because it requires full access to all private > repositories in your account and orgs administrated by your account. I'm experimenting with Github's built in CI. All other ideas welcome.
On 12/2/20 5:13 PM, Thomas Munro wrote: > On Thu, Dec 3, 2020 at 10:51 AM David Steele <david@pgmasters.net> wrote: >> On 12/2/20 4:15 PM, Thomas Munro wrote: >>> On Thu, Dec 3, 2020 at 10:00 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: >>>> This is actually a bit problematic, because now the cfbot is ignoring >>>> those patches (or if it's not, I don't know where it's displaying the >>>> results). Please go ahead and move the remaining open patches, or >>>> else re-open the CF if that's possible. >>> >>> As of quite recently, Travis CI doesn't seem to like cfbot's rate of >>> build jobs. Recently it's been taking a very long time to post >>> results for new patches and taking so long to get around to retesting >>> patches for bitrot that the my "delete old results after a week" logic >>> started wiping out some results while they are still the most recent, >>> leading to the blank spaces where the results are supposed to be. >>> D'oh. I'm looking into a couple of options. >> >> pgBackRest test runs have gone from ~17 minutes to 3-6 hours over the >> last two months. > > Ouch. Yeah. >> Also keep in mind that travis-ci.org will be shut down at the end of the >> month. Some users who have migrated to travis-ci.com have complained >> that it is not any faster, but I have not tried myself (yet). > > Oh. Yeah. >> Depending on how you have Github organized migrating to travis-ci.com >> may be bit tricky because it requires full access to all private >> repositories in your account and orgs administrated by your account. > > I'm experimenting with Github's built in CI. All other ideas welcome. We're leaning towards Github actions ourselves. The only thing that makes us want to stay with Travis (at least for some tests) is the support for the ppc64le, arm64, and s390x architectures. s390x in particular since it is the only big-endian architecture we have access to. Regards, -- -David david@pgmasters.net
On 12/2/20 5:13 PM, Thomas Munro wrote: > > I'm experimenting with Github's built in CI. All other ideas welcome. I'd look very closely at gitlab. cheers andrew
st 2. 12. 2020 v 23:36 odesílatel Andrew Dunstan <andrew@dunslane.net> napsal: > > > On 12/2/20 5:13 PM, Thomas Munro wrote: > > > > I'm experimenting with Github's built in CI. All other ideas welcome. > > > > I'd look very closely at gitlab. I was about to respond before with the same idea. Feel free to ping regarding another CI. Also there is possibility to move to GitHub Actions (free open source CI). I got some experience with that as well. > > cheers > > > andrew > > >
On 12/02/20 16:51, David Steele wrote: > Depending on how you have Github organized migrating to travis-ci.com may be > bit tricky because it requires full access to all private repositories in > your account and orgs administrated by your account. PL/Java just began using travis-ci.com this summer at the conclusion of our GSoC project, and Thomas had been leery of the full-access-to-everything requirement, but that turned out to have been an old way it once worked. The more current way involves installation as a GitHub app into a particular repository, and it did not come with excessive access requirements. That being said, we got maybe three months of use out of it all told. On 2 November, they announced a "new pricing model"[1],[2], and since 24 November it has no longer run PL/Java tests, logging a "does not have enough credits" message[3] instead. So I rather hastily put a GitHub Actions workflow together to plug the hole. Apparently there is a way for OSS projects to ask nicely for an allocation of some credits that might be available. Regards, -Chap [1] https://www.jeffgeerling.com/blog/2020/travis-cis-new-pricing-plan-threw-wrench-my-open-source-works [2] https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing [3] https://travis-ci.com/github/tada/pljava/requests
On Thu, Dec 3, 2020 at 12:02 PM Josef Šimánek <josef.simanek@gmail.com> wrote: > st 2. 12. 2020 v 23:36 odesílatel Andrew Dunstan <andrew@dunslane.net> napsal: > > On 12/2/20 5:13 PM, Thomas Munro wrote: > > > I'm experimenting with Github's built in CI. All other ideas welcome. > > > > I'd look very closely at gitlab. > > I was about to respond before with the same idea. Feel free to ping > regarding another CI. Also there is possibility to move to GitHub > Actions (free open source CI). I got some experience with that as > well. I spent today getting something working on Github just to try it out, and started a new thread[1] about that. I've not tried Gitlab and have no opinion about that; if someone has a working patch similar to that, I'd definitely be interested to take a look. I've looked at some others. For what cfbot is doing (namely: concentrating the work of hundreds into one "account" via hundreds of branches), the spin-up times and concurrency limits are a bit of a problem on many of them. FWIW I think they're all wonderful for offering this service to open source projects and I'm grateful that they do it! [1] https://www.postgresql.org/message-id/flat/CA%2BhUKG%2By_SHVQcU3CPokmJxuHp1niebCjq4XzZizf8SR9ZdQRQ%40mail.gmail.com
On 02.12.2020 23:59, Tom Lane wrote: > Anastasia Lubennikova <a.lubennikova@postgrespro.ru> writes: >> Commitfest 2020-11 is officially closed now. >> Many thanks to everyone who participated by posting patches, reviewing >> them, committing and sharing ideas in discussions! > Thanks for all the hard work! > >> Today, me and Georgios will move the remaining items to the next CF or >> return them with feedback. We're planning to leave Ready For Committer >> till the end of the week, to make them more visible and let them get the >> attention they deserve. > This is actually a bit problematic, because now the cfbot is ignoring > those patches (or if it's not, I don't know where it's displaying the > results). Please go ahead and move the remaining open patches, or > else re-open the CF if that's possible. > > regards, tom lane Oh, I wasn't aware of that. Thank you for the reminder. Now all patches are moved to the next CF. -- Anastasia Lubennikova Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On 2020-12-02 23:13, Thomas Munro wrote: > I'm experimenting with Github's built in CI. All other ideas welcome. You can run Linux builds on AppVeyor, too.
On 12/3/20 4:54 AM, Anastasia Lubennikova wrote: > On 02.12.2020 23:59, Tom Lane wrote: >> Anastasia Lubennikova <a.lubennikova@postgrespro.ru> writes: >>> Commitfest 2020-11 is officially closed now. >>> Many thanks to everyone who participated by posting patches, reviewing >>> them, committing and sharing ideas in discussions! >> Thanks for all the hard work! >> >>> Today, me and Georgios will move the remaining items to the next CF or >>> return them with feedback. We're planning to leave Ready For Committer >>> till the end of the week, to make them more visible and let them get >>> the >>> attention they deserve. >> This is actually a bit problematic, because now the cfbot is ignoring >> those patches (or if it's not, I don't know where it's displaying the >> results). Please go ahead and move the remaining open patches, or >> else re-open the CF if that's possible. >> >> regards, tom lane > > Oh, I wasn't aware of that. Thank you for the reminder. > > Now all patches are moved to the next CF. > Maybe this needs to be added to the instructions for CF managers so the workflow is clear. cheers andrew
On Wed, Dec 2, 2020 at 2:36 PM Andrew Dunstan <andrew@dunslane.net> wrote:
On 12/2/20 5:13 PM, Thomas Munro wrote:
>
> I'm experimenting with Github's built in CI. All other ideas welcome.
I'd look very closely at gitlab.
+1.
Why:
- having a great experience for more than 2 years
- if needed, there is an open-source version of it
- it's possible to set up your own CI [custom] runners even when you're working with their SaaS
- finally, it's on Postgres itself
On Fri, Dec 4, 2020 at 1:29 AM Peter Eisentraut <peter.eisentraut@enterprisedb.com> wrote: > On 2020-12-02 23:13, Thomas Munro wrote: > > I'm experimenting with Github's built in CI. All other ideas welcome. > > You can run Linux builds on AppVeyor, too. Yeah. This would be the easiest change to make, because I already have configuration files, cfbot already knows how to talk to Appveyor to collect results, and the build results are public URLs. So I'm leaning towards this option in the short term, so that cfbot keeps working after December 31. We can always review the options later. Appveyor's free-for-open-source plan has no cap on minutes, but limits concurrent jobs to 1. Gitlab's free-for-open-source plan is based on metered CI minutes, but cfbot is a bit too greedy for the advertised limits. An annual allotment of 50,000 minutes would run out some time in February assuming we rebase each of ~250 branches every few days. GitHub Actions has very generous resource limits, nice UX and API, etc etc, so it's tempting, but its build log URLs can only be accessed by people with accounts. That limits their usefulness when discussing test failures on our public mailing list. I thought about teaching cfbot to pull down the build logs and host them on its own web server, but that feels like going against the grain.