Thread: No Issue Tracker - Say it Ain't So!

Kam Lasater wrote:

> I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
> that order). There are tens to hundreds of other great ones out there,
> I'm sure one of them would also work.

If you install debbugs and feed it from our lists, maybe enough of us
would jump into the bandwagon enough to get it off the ground.  I'm
unsure that it would work to maintain something that's too removed from
the mailing lists.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

> > I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
> > that order). There are tens to hundreds of other great ones out there,
> > I'm sure one of them would also work.
>
> If you install debbugs and feed it from our lists, maybe enough of us
> would jump into the bandwagon enough to get it off the ground.  I'm
> unsure that it would work to maintain something that's too removed from
> the mailing lists.

Alvaro,

Thanks for the suggestion. However, an issue tracker is not a
replacement for mailing list(s) and vice versa. They are both
necessary for success.

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:
> Kam Lasater wrote:
>
> > I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
> > that order). There are tens to hundreds of other great ones out there,
> > I'm sure one of them would also work.
>
> If you install debbugs and feed it from our lists, maybe enough of us
> would jump into the bandwagon enough to get it off the ground.  I'm
> unsure that it would work to maintain something that's too removed from
> the mailing lists.

The difficulty with this is that someone has to actually offer up to
maintain it.  Bug trackers do not maintain themselves.

Personally, I do like debbugs and it would likely be the least offensive
approch to those who ike the current mailing list.  I don't know offhand
how much effort it would be to set up, perhaps I'll ask around.

Further, the distributions which most of our users use do have their own
bug trackers (Debian, Ubuntu, RedHat) which include tracking bugs in
PostgreSQL.

Thanks!

Stephen

On Wed, Sep 23, 2015 at 2:30 PM, Kam Lasater <ckl@seekayel.com> wrote:
> Thanks for the suggestion. However, an issue tracker is not a
> replacement for mailing list(s) and vice versa. They are both
> necessary for success.

I venture to say that we are succeeding as it is, although of course
we might have more success if we did some things better, including
this.  However, as Stephen says, the problem with an issue tracker is
that, unless some person or persons committed to keep it up to date,
it would just fill up with crap. We have an issue tracker for database
server issues here at EnterpriseDB, and keeping it up to date is a ton
of work.  If nobody's volunteering to do that work in the PostgreSQL
community, an issue tracker is going to end up not being useful,
because it will just be wrong all the time.

If somebody does do the work, then we get to the next question: if we
had an accurate list of open bugs, would anybody who currently doesn't
work on fixing those bugs step up to help fix them?  I hope so, but I
don't know.  If not, we might not feel that the effort of maintaining
the bug tracker paid much of a dividend.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 09/23/2015 11:23 AM, Alvaro Herrera wrote:
> Kam Lasater wrote:
>
>> I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
>> that order). There are tens to hundreds of other great ones out there,
>> I'm sure one of them would also work.
>
> If you install debbugs and feed it from our lists, maybe enough of us
> would jump into the bandwagon enough to get it off the ground.  I'm
> unsure that it would work to maintain something that's too removed from
> the mailing lists.

There needs to be community buy in on this idea else it is just a waste 
of resources. The hackers need say, "Hey... you know, all those other 
projects may be on to something. Perhaps we should learn from them and 
do better by our users." This is more than just an issue/bug tracker 
discussion. It is a ideology shift.

Until that happens asking anyone to put resources into this idea is just 
not worth it.

Sincerely,

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/23/2015 11:33 AM, Stephen Frost wrote:
> * Alvaro Herrera (alvherre@2ndquadrant.com) wrote:
>> Kam Lasater wrote:
>>
>>> I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
>>> that order). There are tens to hundreds of other great ones out there,
>>> I'm sure one of them would also work.
>>
>> If you install debbugs and feed it from our lists, maybe enough of us
>> would jump into the bandwagon enough to get it off the ground.  I'm
>> unsure that it would work to maintain something that's too removed from
>> the mailing lists.
>
> The difficulty with this is that someone has to actually offer up to
> maintain it.  Bug trackers do not maintain themselves.

If we integrate it into the process it gets easier. This is especially 
true if we use a bug tracker that can be managed via email as well as a 
web interface.

Sincerely,

JD


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/23/2015 11:18 AM, Kam Lasater wrote:
> 
> At this point not having one is borderline negligent. I'd suggest:
> Github Issues, Pivotal Tracker or Redmine (probably in that order).
> There are tens to hundreds of other great ones out there, I'm sure one
> of them would also work.

First, understand that the Postgres project was created before bug
trackers existed. And people are very slow to change their habits,
especially since not having a bug tracker was actually a benefit up
until around 2005.  It's not anymore, but I'm sure people will argue
with my statement on that.

We have to use something OSS; open source projects depending on
closed-source infra is bad news.  Out of what's available, I'd actually
choose Bugzilla; as much as BZ frustrates the heck out of me at times,
it's the only OSS tracker that's at all sophisticated.

The alternative would be someone building a sophisticated system on top
of RequestTracker, which would also let us have tight mailing list
integration given RT's email-driven model.  However, that would require
someone with the time to build a custom workflow system and web UI on
top of RT.  It's quite possible that Best Practical would be willing to
help here.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 09/23/2015 11:43 AM, Robert Haas wrote:
> If somebody does do the work, then we get to the next question: if we
> had an accurate list of open bugs, would anybody who currently doesn't
> work on fixing those bugs step up to help fix them?  I hope so, but I
> don't know.  If not, we might not feel that the effort of maintaining
> the bug tracker paid much of a dividend.

I don't anticipate that getting additional bug fixers would be a benefit
of having a bug tracker, at least not in the first year.  In fact, I
would say that we don't need a bug tracker to fix most significant bugs
at all.  We're pretty good at that.

What we need a bug tracker for is:

1. so users and downstream projects know where to report bugs (and no,
our idiosyncratic bug form doesn't fit into anyone's workflow).

2. so that users know when a bug is fixed, and what release it's fixed
in, rather than depending on "ask someone on IRC".

3. so that we don't completely lose track of low-importance, hard-to-fix
bugs and trivial bugs, which we currently certainly do.

4. so that we can have a clearer idea more immediately that we've fixed
all known bugs in upcoming postgresql releases, instead of depending on
Bruce catching up on his email.

5. so that we have a place to track bugs which require hard, multi-step
fixes and don't lose track of some of the steps like we did with Multixact.

Those are the main reasons to have a BT.  Offering a place for new
hackers to get started with trivial code fixes might be a side benefit,
but isn't a good enough reason to have one.

Obviously, everything said about "who's going to maintain this" is
completely valid.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

* Josh Berkus (josh@agliodbs.com) wrote:
> On 09/23/2015 11:18 AM, Kam Lasater wrote:
> > At this point not having one is borderline negligent. I'd suggest:
> > Github Issues, Pivotal Tracker or Redmine (probably in that order).
> > There are tens to hundreds of other great ones out there, I'm sure one
> > of them would also work.
>
> First, understand that the Postgres project was created before bug
> trackers existed. And people are very slow to change their habits,
> especially since not having a bug tracker was actually a benefit up
> until around 2005.  It's not anymore, but I'm sure people will argue
> with my statement on that.
>
> We have to use something OSS; open source projects depending on
> closed-source infra is bad news.  Out of what's available, I'd actually
> choose Bugzilla; as much as BZ frustrates the heck out of me at times,
> it's the only OSS tracker that's at all sophisticated.
>
> The alternative would be someone building a sophisticated system on top
> of RequestTracker, which would also let us have tight mailing list
> integration given RT's email-driven model.  However, that would require
> someone with the time to build a custom workflow system and web UI on
> top of RT.  It's quite possible that Best Practical would be willing to
> help here.

Yeah, even if we got past the "do we want a bug tracker?" question, any
project would probably end up stalling indefinitely on "well then, which
one?"

In the end, we should probably just throw something up based on whatever
the folks who are going to be actually maintaining it want and call it a
'beta' and see what happens with it.  The above-referenced individuals
would be the bug tracking system curators, of course.  Unless it's got
serious technical issues, the infrastructure team will do our best to
support the choice.  On the other hand, some of us would likely be
involved in bug curation also.

Thanks!

Stephen

> We have to use something OSS; open source projects depending on
> closed-source infra is bad news.  Out of what's available, I'd actually
> choose Bugzilla; as much as BZ frustrates the heck out of me at times,
> it's the only OSS tracker that's at all sophisticated.

There are several OSS projects that use closed-source trackers. 
I (personally) don't see a real problem with that.  Atlassian offers free
hosting 
for open source projects (I'm sure Postgres would qualify) and Confluence
Jira 
is one of the best trackers I have worked with. I does have a mail gateway
were 
issues can be created and maintained by sending emails (rather than editing
them in 
the web front end) 

They also support Postgres as their backend (and you do find hints here and
there 
that it is the recommended open source DBMS for them - but they don't 
explicitly state it like that). We are using Jira at the company I work for
and 
all Jira installations run on Postgres there. 

https://www.atlassian.com/software/views/open-source-license-request

Thomas




--
View this message in context: http://postgresql.nabble.com/No-Issue-Tracker-Say-it-Ain-t-So-tp5867020p5867046.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.

Joshua D. Drake wrote:

> Until that happens asking anyone to put resources into this idea is just not
> worth it.

I wonder if you still have this conversation archived:

Date: Thu, 10 May 2007 11:30:55 -0400
From: Andrew Dunstan <andrew@dunslane.net>
To: "Joshua D. Drake", Magnus Hagander, Alvaro Herrera, Robert Treat, Lukas Smith, Andrew Sullivan, David Fetter
Subject: tracker

There was some very good input there -- it would be a shame to have to
repeat that all over again.  Hey, it's only been 8 years ...

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

> > We have to use something OSS; open source projects depending on
> > closed-source infra is bad news.  Out of what's available, I'd actually
> > choose Bugzilla; as much as BZ frustrates the heck out of me at times,
> > it's the only OSS tracker that's at all sophisticated.

Josh,

I'm not sure I agree here on the BT needing to be OSS. That said not
sure its my call :)

> ... The above-referenced individuals
> would be the bug tracking system curators, of course.  Unless it's got
> serious technical issues, the infrastructure team will do our best to
> support the choice.  On the other hand, some of us would likely be
> involved in bug curation also.

Stephen,

In digging around more I found this wiki page that seems to be the
closest thing to a BT: https://wiki.postgresql.org/wiki/Todo

Is the curation already being done? If the contents of that wiki page
were injected into a BT, would that be enough of a start?

Kam,

* Kam Lasater (ckl@seekayel.com) wrote:
> > ... The above-referenced individuals
> > would be the bug tracking system curators, of course.  Unless it's got
> > serious technical issues, the infrastructure team will do our best to
> > support the choice.  On the other hand, some of us would likely be
> > involved in bug curation also.
>
> Stephen,
>
> In digging around more I found this wiki page that seems to be the
> closest thing to a BT: https://wiki.postgresql.org/wiki/Todo
>
> Is the curation already being done? If the contents of that wiki page
> were injected into a BT, would that be enough of a start?

If you look at what happens on the -bugs mailing list, you'll see very
quickly that a bug tracker and the Todo wiki page are very different.
Perhaps the Todo could be squeezed into a bug tracker, but many of the
items on there might well end up as 'wontfix' (to use the debbugs
lingo).

That is certainly not the same curation as what a BT would require.

Thanks!

Stephen

Jeff Janes wrote:

> For whatever it is worth, one of the frustrations I've had with projects
> (other than PostgreSQL) of which I am a casual users is that reporting a
> single bug meant signing up for yet another account on yet another site and
> learning yet another bug tracking system.

Right.

> I think the email based system is more friendly for the casual user.  You
> don't even have to sign up for the bugs mail list as long as people keep
> you CC'ed.  I don't think that having a tracker just for the sake of having
> one is going to attract new contributors.

Yay, another vote for debbugs!

> I'd rather, say, put some more work into cleaning the kruft out of the
> To-Do list, then put that effort into migrating the kruft to a fancier
> filing cabinet.

Casual users would need a community account in order to file bugs in the
Todo wiki page.  I don't think a wiki page qualifies (by a long mile) as
a bug tracker, anyway.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Szymon Lipiński wrote:

> Then I need to read through the emails. This is not user friendly too, as I
> need to click through the email tree, and if an email has multiple replies,
> it is usually hard not to omit some of them, as after going into a reply, I
> need to click to get to the parent mail again.

Evidently, the "flat" link is easy to miss.  Give it a try.

The bug tracker is not intended as a feature-request tracker, anyway.
Those two things are very different, even if many projects just conflate
the two things.

> Personally I'd also change sending patches in emails to github pull
> requests :).

That won't happen, at least not this decade.

> ... or maybe the difference is more in the data structure, the email
> discussion is a tree (with a horrible interface to the archive) while in a
> bug tracker, the discussion is linear, and easier to follow.

FWIW in my opinion our mailing list archives interface is the best there
is --- and I disagree that the linear discussion is easy to follow,
except for trivial discussions.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Szymon,

* Szymon Lipiński (mabewlun@gmail.com) wrote:
> a couple of days ago I was reading through the tickets in the Django bug
> tracker. It was much easier to find any information about the things to
> work on than currently for Postgres.

I tend to doubt that a bug tracker would change that situation.

> >From my point of view, for Postgres, there is just a not updated too often
> list of things to implement on the wiki. If I need to find some additional
> information, then I can find there just some links to mails from the mail
> groups.

As far as 'new features' go, I don't know that we'd put those on the bug
tracker anyway.  Perhaps some of them should go on the todo list, such
as the discussion around restrictive RLS policies, but the canonical
list is really developer oriented and less project oriented.  That's
probably not a good thing, but I don't think trying to use a bug tracker
to track features is the answer there either.

I will say that something easier than the todo list (aka the wiki) to
work with would be nice for tracking new feature thoughts.

> Then I need to read through the emails. This is not user friendly too, as I
> need to click through the email tree, and if an email has multiple replies,
> it is usually hard not to omit some of them, as after going into a reply, I
> need to click to get to the parent mail again.

This is solved by the 'flat' view, which gives you a single page with
all emails for the thread.  Go to any email in the archives and click on
'flat', at the end of the Message-ID line.

> What's more, there are some things on the wiki, and when I asked about
> that, it turned out that "oh, there was some discussion long time ago, that
> it is not doable".

Right, that's one of the challenges with the current todo list.

> It would be also worth storing the information that someone is working on
> something, so the work won't be doubled.

Thankfully, that doesn't seem to happen too much in this community as we
all communicate a fair bit.  I agree that risk exists, but I don't
believe it's a reason for a bug or feature tracker by itself.

> Personally I'd also change sending patches in emails to github pull
> requests :).

Don't get your hopes up.

> ... or maybe the difference is more in the data structure, the email
> discussion is a tree (with a horrible interface to the archive) while in a
> bug tracker, the discussion is linear, and easier to follow.

The interface is entirely open source and I'm sure Magnus would be
happen to hear specific ideas for improvement, or, even better, pull
requests. :)

Thanks!

Stephen

On 9/23/15 3:12 PM, Thomas Kellerer wrote:
> They also support Postgres as their backend (and you do find hints here and
> there
> that it is the recommended open source DBMS for them - but they don't
> explicitly state it like that). We are using Jira at the company I work for
> and
> all Jira installations run on Postgres there.

I'll second Jira as well. It's the only issue tracker I've seen that you 
can actually use for multiple different things without it becoming a 
mess. IE: it could track Postgres bugs, infrastructure issues, and the 
TODO list if we wanted, allow issues to reference each other 
intelligently, yet still keep them as 3 separate bodies.

They're also based here in Austin so we've got community folks that can 
interface with them directly if that's ever needed.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

>> Personally I'd also change sending patches in emails to github pull
>> requests :).
>
> That won't happen, at least not this decade.

FWIW, a year ago I might have agreed that a github pull-request would
be preferable.  However, since, I have grown to really like the patch
via email approach.  I can see a lot of value in keeping patch
submission decoupled from a specific service/technology/workflow in
this way.

>> ... or maybe the difference is more in the data structure, the email
>> discussion is a tree (with a horrible interface to the archive) while in a
>> bug tracker, the discussion is linear, and easier to follow.
>
> FWIW in my opinion our mailing list archives interface is the best there
> is --- and I disagree that the linear discussion is easy to follow,
> except for trivial discussions.

In my experience, following other mailing lists, I really appreciate
our interface.  I'm not sure that I'd call it the best, but I've
certainly seen far worse and I have no real complaints about it.  What
I think I like best about it is that it has an community "official"
status, meaning we don't depend on some other mirror/archive site to
support it, like gmane or spinics.  This is just my opinion though.

-Adam

-- 
Adam Brightwell - adam.brightwell@crunchydatasolutions.com
Database Engineer - www.crunchydatasolutions.com

On 9/23/15 3:29 PM, Alvaro Herrera wrote:
> Joshua D. Drake wrote:
>
>> Until that happens asking anyone to put resources into this idea is just not
>> worth it.
>
> I wonder if you still have this conversation archived:
>
> Date: Thu, 10 May 2007 11:30:55 -0400
> From: Andrew Dunstan <andrew@dunslane.net>
> To: "Joshua D. Drake", Magnus Hagander, Alvaro Herrera, Robert Treat, Lukas Smith, Andrew Sullivan, David Fetter
> Subject: tracker
>
> There was some very good input there -- it would be a shame to have to
> repeat that all over again.  Hey, it's only been 8 years ...

Ha! Good to know our scheduling is consistent at least!
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 09/23/2015 03:05 PM, Jim Nasby wrote:
> On 9/23/15 3:12 PM, Thomas Kellerer wrote:
>> They also support Postgres as their backend (and you do find hints
>> here and
>> there
>> that it is the recommended open source DBMS for them - but they don't
>> explicitly state it like that). We are using Jira at the company I
>> work for
>> and
>> all Jira installations run on Postgres there.
> 
> I'll second Jira as well. It's the only issue tracker I've seen that you
> can actually use for multiple different things without it becoming a
> mess. IE: it could track Postgres bugs, infrastructure issues, and the
> TODO list if we wanted, allow issues to reference each other
> intelligently, yet still keep them as 3 separate bodies.

Speaking as someone who uses Jira for commericial work, I'm -1 on them.I simply don't find Jira to be superior to OSS
BTsystems, and inferior
 
in several ways (like that you can't have more than one person assigned
to a bug).  And email integration for Jira is nonexistant.

When we discussed this 8 years ago, Debian said debbugs wasn't ready for
anyone else to use.  Has that changed?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Thu, Sep 24, 2015 at 11:33 AM, Josh Berkus <josh@agliodbs.com> wrote:
> On 09/23/2015 03:05 PM, Jim Nasby wrote:
>> On 9/23/15 3:12 PM, Thomas Kellerer wrote:
>>> They also support Postgres as their backend (and you do find hints
>>> here and
>>> there
>>> that it is the recommended open source DBMS for them - but they don't
>>> explicitly state it like that). We are using Jira at the company I
>>> work for
>>> and
>>> all Jira installations run on Postgres there.
>>
>> I'll second Jira as well. It's the only issue tracker I've seen that you
>> can actually use for multiple different things without it becoming a
>> mess. IE: it could track Postgres bugs, infrastructure issues, and the
>> TODO list if we wanted, allow issues to reference each other
>> intelligently, yet still keep them as 3 separate bodies.
>
> Speaking as someone who uses Jira for commericial work, I'm -1 on them.
>  I simply don't find Jira to be superior to OSS BT systems, and inferior
> in several ways (like that you can't have more than one person assigned
> to a bug).  And email integration for Jira is nonexistant.
>
> When we discussed this 8 years ago, Debian said debbugs wasn't ready for
> anyone else to use.  Has that changed?

Do you think it would make any sense to consider evolving what we have
already?  At the moment, we have a bug form, and when you submit it it
does this (if I'm looking at the right thing, please correct me if I'm
not):

http://git.postgresql.org/gitweb/?p=pgweb.git;a=blob;f=pgweb/misc/views.py

That is, the database interaction is limited to using a SEQUENCE to
generate a new bug ID, and then an email is sent to pgsql-bugs.  What
if we also created a bug record for that ID to track its status, and
allowed anyone with a community account to edit the bug record and
change the status?  There could be a simple page that lets you see and
search for bugs, with a link to the flat mail archive thread where
discussion is held.  All actual discussion would continue on mailing
lists.  That would be similar to the commitfest.

I suppose some forms of cross-reference would also be useful: when
viewing the bug's page, you might want to see any commitfest items or
pgsql-committers messages that relate to that bug.  Perhaps we could
automatically create those links when bug IDs are recognised in those
messages, so that no extra workflow/maintenance is required in
straightforward cases.  To continue that line of thinking it would
also be possible for bug statuses to be changed when certain words are
spotted in either commit messages (which doesn't have to be a commit
hook, it could be taken from pgsql-committers messages) or pgsql-bugs
messages.

Cf github commit message parsing:
https://help.github.com/articles/closing-issues-via-commit-messages/

-- 
Thomas Munro
http://www.enterprisedb.com

On 09/23/2015 05:21 PM, Thomas Munro wrote:
> Do you think it would make any sense to consider evolving what we have
> already?  At the moment, we have a bug form, and when you submit it it
> does this (if I'm looking at the right thing, please correct me if I'm
> not):
>
> http://git.postgresql.org/gitweb/?p=pgweb.git;a=blob;f=pgweb/misc/views.py
>
> That is, the database interaction is limited to using a SEQUENCE to
> generate a new bug ID, and then an email is sent to pgsql-bugs.  What
> if we also created a bug record for that ID to track its status, and
> allowed anyone with a community account to edit the bug record and
> change the status?  There could be a simple page that lets you see and
> search for bugs, with a link to the flat mail archive thread where
> discussion is held.  All actual discussion would continue on mailing
> lists.  That would be similar to the commitfest.
>
> I suppose some forms of cross-reference would also be useful: when
> viewing the bug's page, you might want to see any commitfest items or
> pgsql-committers messages that relate to that bug.  Perhaps we could
> automatically create those links when bug IDs are recognised in those
> messages, so that no extra workflow/maintenance is required in


It would be nice if you could essentially promote a bug into a
commitfest item, maybe through a status change.


> straightforward cases.  To continue that line of thinking it would
> also be possible for bug statuses to be changed when certain words are
> spotted in either commit messages (which doesn't have to be a commit
> hook, it could be taken from pgsql-committers messages) or pgsql-bugs
> messages.
>
> Cf github commit message parsing:
> https://help.github.com/articles/closing-issues-via-commit-messages/

I was thinking along the same lines. If we could paste a bug reference
number into the commit message and have that change the bug status it
would go a long way to making this workable I think.

Joe

--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development

Josh Berkus wrote:

> When we discussed this 8 years ago, Debian said debbugs wasn't ready for
> anyone else to use.  Has that changed?

Emacs uses debbugs now, so there's at least one non-Debian user.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Thu, Sep 24, 2015 at 1:31 PM, Joe Conway <mail@joeconway.com> wrote:
> On 09/23/2015 05:21 PM, Thomas Munro wrote:
>> Do you think it would make any sense to consider evolving what we have
>> already?  At the moment, we have a bug form, and when you submit it it
>> does this (if I'm looking at the right thing, please correct me if I'm
>> not):
>>
>> http://git.postgresql.org/gitweb/?p=pgweb.git;a=blob;f=pgweb/misc/views.py
>>
>> That is, the database interaction is limited to using a SEQUENCE to
>> generate a new bug ID, and then an email is sent to pgsql-bugs.  What
>> if we also created a bug record for that ID to track its status, and
>> allowed anyone with a community account to edit the bug record and
>> change the status?  There could be a simple page that lets you see and
>> search for bugs, with a link to the flat mail archive thread where
>> discussion is held.  All actual discussion would continue on mailing
>> lists.  That would be similar to the commitfest.
>>
>> I suppose some forms of cross-reference would also be useful: when
>> viewing the bug's page, you might want to see any commitfest items or
>> pgsql-committers messages that relate to that bug.  Perhaps we could
>> automatically create those links when bug IDs are recognised in those
>> messages, so that no extra workflow/maintenance is required in
>
>
> It would be nice if you could essentially promote a bug into a
> commitfest item, maybe through a status change.
>
>
>> straightforward cases.  To continue that line of thinking it would
>> also be possible for bug statuses to be changed when certain words are
>> spotted in either commit messages (which doesn't have to be a commit
>> hook, it could be taken from pgsql-committers messages) or pgsql-bugs
>> messages.
>>
>> Cf github commit message parsing:
>> https://help.github.com/articles/closing-issues-via-commit-messages/
>
> I was thinking along the same lines. If we could paste a bug reference
> number into the commit message and have that change the bug status it
> would go a long way to making this workable I think.

The two most common interactions could go something like this:

1.  User enters bug report via form, creating an issue in NEW state
and creating a pgsql-bugs thread.  Someone responds by email that this
is expected behaviour, not a bug, not worth fixing or not a Postgres
issue etc using special trigger words.  The state is automatically
switched to WORKS_AS_DESIGNED or WONT_FIX.  No need to touch the web
interface: the only change from today's workflow is awareness of the
right wording to trigger the state change.

2.  User enters bug report via form, creating issue #1234 in NEW
state.   Someone responds by email to acknowledge that that may indeed
be an issue, and any response to an issue in NEW state that doesn't
reject it switches it to UNDER_DISCUSSION.  Maybe if a commitfest item
references the same thread (or somehow references the issue number?)
its state is changed to IN_COMMITFEST, or maybe as you say there could
be a way to generate the commitfest item from the issue, not sure
about that.  Eventually a commit log message says "Fixes bug #1234"
and the state automatically goes to FIXED.

Other interactions (curation of unresolved issues, reopening disputed
issues, resolving issues where the committer or responder forgot to
use the right magic words, etc etc) could be done via the web
interface which would initially have only a couple of pages: one for
paging through issues in different states and one for viewing/editing
them.  (Obviously you could go further and deal with assigning issues
to people, and adding more states etc etc).

I don't know much about pgweb and friends but it seems like we already
have a bunch of Python machinery processing all mailing list traffic,
a database and a webapp doing something pretty closely related.  How
hard would it be to teach it to track issues this way, building on the
existing infrastructure, compared to rolling out a new separate
product, and could the result be better integrated?

-- 
Thomas Munro
http://www.enterprisedb.com

>  And email integration for Jira is nonexistant.

That is not true. We do have an email integration where customers can create
issues by sending an email to a specific "Jira Email" address. And as far as
I know this is a standard module from Atlassian. I _think_ it can also be
configured that you can reply to the notification emails and the reply will
be added as a comment to the issue.

> like that you can't have more than one person assigned to a bug

That is true, there is only one "Assignee" for an issue - the person who is
responsible for it. 

But given the flexibility of Jira I'm pretty sure one could configure an
additional field (e.g. "is being worked on by" that can be assigned multiple
users. 

One thing that is indeed still missing is a Git integration the way the
Subversion integration works. Jira scans the commit messages for ticket
numbers and automatically links an issue to the commits. 

Thomas

(Note that I'm not in any way affiliated with Atlassian - just to avoid that
impression)



--
View this message in context: http://postgresql.nabble.com/No-Issue-Tracker-Say-it-Ain-t-So-tp5867020p5867093.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.

On 24.09.2015 01:33, Josh Berkus wrote:
> On 09/23/2015 03:05 PM, Jim Nasby wrote:
>> On 9/23/15 3:12 PM, Thomas Kellerer wrote:
>>> They also support Postgres as their backend (and you do find hints
>>> here and
>>> there
>>> that it is the recommended open source DBMS for them - but they don't
>>> explicitly state it like that). We are using Jira at the company I
>>> work for
>>> and
>>> all Jira installations run on Postgres there.
>>
>> I'll second Jira as well. It's the only issue tracker I've seen that you
>> can actually use for multiple different things without it becoming a
>> mess. IE: it could track Postgres bugs, infrastructure issues, and the
>> TODO list if we wanted, allow issues to reference each other
>> intelligently, yet still keep them as 3 separate bodies.
>
> Speaking as someone who uses Jira for commericial work, I'm -1 on them.

Full ACK. -1 from me too.

Greetings,
Torsten

On 23.09.2015 20:43, Robert Haas wrote:
> On Wed, Sep 23, 2015 at 2:30 PM, Kam Lasater <ckl@seekayel.com> wrote:
>> Thanks for the suggestion. However, an issue tracker is not a
>> replacement for mailing list(s) and vice versa. They are both
>> necessary for success.
>
> I venture to say that we are succeeding as it is, although of course
> we might have more success if we did some things better, including
> this.  However, as Stephen says, the problem with an issue tracker is
> that, unless some person or persons committed to keep it up to date,
> it would just fill up with crap. We have an issue tracker for database
> server issues here at EnterpriseDB, and keeping it up to date is a ton
> of work.  If nobody's volunteering to do that work in the PostgreSQL
> community, an issue tracker is going to end up not being useful,
> because it will just be wrong all the time.

I would volunteering to do that work if the community decides to get a 
bug tracker.

Greetings,
Torsten

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:
> Josh Berkus wrote:
> > When we discussed this 8 years ago, Debian said debbugs wasn't ready for
> > anyone else to use.  Has that changed?
>
> Emacs uses debbugs now, so there's at least one non-Debian user.

Oh?  Nice.  debbugs matches up largely with what Thomas Munro was just
suggesting regarding the workflow also.

Thanks!

Stephen

* Thomas Kellerer (spam_eater@gmx.net) wrote:
> >  And email integration for Jira is nonexistant.
>
> That is not true. We do have an email integration where customers can create
> issues by sending an email to a specific "Jira Email" address. And as far as
> I know this is a standard module from Atlassian. I _think_ it can also be
> configured that you can reply to the notification emails and the reply will
> be added as a comment to the issue.

I've tried to make email integration work also and it's really painful.
I agree it exists but it's not nearly as good as debbugs.

> One thing that is indeed still missing is a Git integration the way the
> Subversion integration works. Jira scans the commit messages for ticket
> numbers and automatically links an issue to the commits.

Not hard to do w/ debbugs.

Thanks!

Stephen

On Thu, Sep 24, 2015 at 1:25 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
> The two most common interactions could go something like this:
>
> 1.  User enters bug report via form, creating an issue in NEW state
> and creating a pgsql-bugs thread.  Someone responds by email that this
> is expected behaviour, not a bug, not worth fixing or not a Postgres
> issue etc using special trigger words.  The state is automatically
> switched to WORKS_AS_DESIGNED or WONT_FIX.  No need to touch the web
> interface: the only change from today's workflow is awareness of the
> right wording to trigger the state change.
>
> 2.  User enters bug report via form, creating issue #1234 in NEW
> state.   Someone responds by email to acknowledge that that may indeed
> be an issue, and any response to an issue in NEW state that doesn't
> reject it switches it to UNDER_DISCUSSION.  Maybe if a commitfest item
> references the same thread (or somehow references the issue number?)
> its state is changed to IN_COMMITFEST, or maybe as you say there could
> be a way to generate the commitfest item from the issue, not sure
> about that.  Eventually a commit log message says "Fixes bug #1234"
> and the state automatically goes to FIXED.
>
> Other interactions (curation of unresolved issues, reopening disputed
> issues, resolving issues where the committer or responder forgot to
> use the right magic words, etc etc) could be done via the web
> interface which would initially have only a couple of pages: one for
> paging through issues in different states and one for viewing/editing
> them.  (Obviously you could go further and deal with assigning issues
> to people, and adding more states etc etc).
>
> I don't know much about pgweb and friends but it seems like we already
> have a bunch of Python machinery processing all mailing list traffic,
> a database and a webapp doing something pretty closely related.  How
> hard would it be to teach it to track issues this way, building on the
> existing infrastructure, compared to rolling out a new separate
> product, and could the result be better integrated?

I think all this sounds pretty cool, frankly.  The astute among you
will have picked up on the fact that bug-trackers are not my absolute
favorite piece of technology, but it seems like something of this sort
could be a significant advance over the status quo.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Sep 23, 2015 at 04:33:33PM -0700, Josh Berkus wrote:
> On 09/23/2015 03:05 PM, Jim Nasby wrote:
> > On 9/23/15 3:12 PM, Thomas Kellerer wrote:
> >> They also support Postgres as their backend (and you do find hints
> >> here and
> >> there
> >> that it is the recommended open source DBMS for them - but they don't
> >> explicitly state it like that). We are using Jira at the company I
> >> work for
> >> and
> >> all Jira installations run on Postgres there.
> > 
> > I'll second Jira as well. It's the only issue tracker I've seen that you
> > can actually use for multiple different things without it becoming a
> > mess. IE: it could track Postgres bugs, infrastructure issues, and the
> > TODO list if we wanted, allow issues to reference each other
> > intelligently, yet still keep them as 3 separate bodies.
> 
> Speaking as someone who uses Jira for commericial work, I'm -1 on them.
>  I simply don't find Jira to be superior to OSS BT systems, and inferior
> in several ways (like that you can't have more than one person assigned
> to a bug).  And email integration for Jira is nonexistant.
> 
> When we discussed this 8 years ago, Debian said debbugs wasn't ready for
> anyone else to use.  Has that changed?
> 

I do not think using a commercial system is a good idea. Currently, Jira
is free for open-source, but there is no guarantee. That could change at
anytime and result in possibly an expensive license cost or port to another
system. We use Jira/Confluence and the random loss of support for various
plugins caused by forced security-based upgrades has resulted in a lot of
unexpected work to maintain the system.

Regards,
Ken

On 09/24/2015 10:28 AM, ktm@rice.edu wrote:
> On Wed, Sep 23, 2015 at 04:33:33PM -0700, Josh Berkus wrote:
>> On 09/23/2015 03:05 PM, Jim Nasby wrote:
>>> On 9/23/15 3:12 PM, Thomas Kellerer wrote:
>>>> They also support Postgres as their backend (and you do find hints
>>>> here and
>>>> there
>>>> that it is the recommended open source DBMS for them - but they don't
>>>> explicitly state it like that). We are using Jira at the company I
>>>> work for
>>>> and
>>>> all Jira installations run on Postgres there.
>>> I'll second Jira as well. It's the only issue tracker I've seen that you
>>> can actually use for multiple different things without it becoming a
>>> mess. IE: it could track Postgres bugs, infrastructure issues, and the
>>> TODO list if we wanted, allow issues to reference each other
>>> intelligently, yet still keep them as 3 separate bodies.
>> Speaking as someone who uses Jira for commericial work, I'm -1 on them.
>>   I simply don't find Jira to be superior to OSS BT systems, and inferior
>> in several ways (like that you can't have more than one person assigned
>> to a bug).  And email integration for Jira is nonexistant.
>>
>> When we discussed this 8 years ago, Debian said debbugs wasn't ready for
>> anyone else to use.  Has that changed?
>>
> I do not think using a commercial system is a good idea. Currently, Jira
> is free for open-source, but there is no guarantee. That could change at
> anytime and result in possibly an expensive license cost or port to another
> system. We use Jira/Confluence and the random loss of support for various
> plugins caused by forced security-based upgrades has resulted in a lot of
> unexpected work to maintain the system.
>



+1

Regardless of the quality of any non-OSS tracker, about which I have no 
comment, I firmly believe that as an OSS project we should use OSS 
infrastructure.

About 10 years ago I helped get Bugzilla over the hurdle of database 
mono-culturism (basically by coming up with the initial version of this: 
<https://github.com/bugzilla/bugzilla/commit/b8793ea28e3e03b2452bac119f2adcd3758e7260>). 
Part of my motivation was to have a tracker to support the PostgreSQL 
project that would run on PostgreSQL. We can see how well that worked 
out :-)

cheers

andrew

On 09/23/2015 10:25 PM, Thomas Munro wrote:
> On Thu, Sep 24, 2015 at 1:31 PM, Joe Conway <mail@joeconway.com> wrote:
>> On 09/23/2015 05:21 PM, Thomas Munro wrote:
>>> Do you think it would make any sense to consider evolving what we have
>>> already?  At the moment, we have a bug form, and when you submit it it
>>> does this (if I'm looking at the right thing, please correct me if I'm
>>> not):

I know we're big on reinventing the wheel here, but it would really be a
better idea to use an established product than starting over from
scratch. Writing a bug tracker is a lot of work and maintenance.

> The two most common interactions could go something like this:
> 
> 1.  User enters bug report via form, creating an issue in NEW state
> and creating a pgsql-bugs thread.  Someone responds by email that this
> is expected behaviour, not a bug, not worth fixing or not a Postgres
> issue etc using special trigger words.  The state is automatically
> switched to WORKS_AS_DESIGNED or WONT_FIX.  No need to touch the web
> interface: the only change from today's workflow is awareness of the
> right wording to trigger the state change.
> 
> 2.  User enters bug report via form, creating issue #1234 in NEW
> state.   Someone responds by email to acknowledge that that may indeed
> be an issue, and any response to an issue in NEW state that doesn't
> reject it switches it to UNDER_DISCUSSION.  Maybe if a commitfest item
> references the same thread (or somehow references the issue number?)
> its state is changed to IN_COMMITFEST, or maybe as you say there could
> be a way to generate the commitfest item from the issue, not sure
> about that.  Eventually a commit log message says "Fixes bug #1234"
> and the state automatically goes to FIXED.

I don't know debbugs, but I know that it would be possible to program RT
to do all of the above, except add the item to the commitfest.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

* Josh Berkus (josh@agliodbs.com) wrote:
> I know we're big on reinventing the wheel here, but it would really be a
> better idea to use an established product than starting over from
> scratch. Writing a bug tracker is a lot of work and maintenance.

I tend to agree.

> > The two most common interactions could go something like this:
> >
> > 1.  User enters bug report via form, creating an issue in NEW state
> > and creating a pgsql-bugs thread.  Someone responds by email that this
> > is expected behaviour, not a bug, not worth fixing or not a Postgres
> > issue etc using special trigger words.  The state is automatically
> > switched to WORKS_AS_DESIGNED or WONT_FIX.  No need to touch the web
> > interface: the only change from today's workflow is awareness of the
> > right wording to trigger the state change.
> >
> > 2.  User enters bug report via form, creating issue #1234 in NEW
> > state.   Someone responds by email to acknowledge that that may indeed
> > be an issue, and any response to an issue in NEW state that doesn't
> > reject it switches it to UNDER_DISCUSSION.  Maybe if a commitfest item
> > references the same thread (or somehow references the issue number?)
> > its state is changed to IN_COMMITFEST, or maybe as you say there could
> > be a way to generate the commitfest item from the issue, not sure
> > about that.  Eventually a commit log message says "Fixes bug #1234"
> > and the state automatically goes to FIXED.
>
> I don't know debbugs, but I know that it would be possible to program RT
> to do all of the above, except add the item to the commitfest.

debbugs does most of the above by default, no programming needed...  I'm
sure we could get it to integrate with the commitfest and have a git
commit hook which sends the appropriate email to it also.

That the emacs folks are using it makes me *much* more interested in the
idea of getting debbugs up and running..

Thanks!

Stephen

On 09/24/2015 10:08 AM, Stephen Frost wrote:
> debbugs does most of the above by default, no programming needed...  I'm
> sure we could get it to integrate with the commitfest and have a git
> commit hook which sends the appropriate email to it also.
>
> That the emacs folks are using it makes me *much* more interested in the
> idea of getting debbugs up and running..

I'm not familiar with debbugs myself, but given that description it
sounds to me like it would be worth giving it a try.

Joe

--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development

I promised myself I'd stay out of this discussion, but ...

Josh Berkus <josh@agliodbs.com> writes:
> I know we're big on reinventing the wheel here, but it would really be a
> better idea to use an established product than starting over from
> scratch. Writing a bug tracker is a lot of work and maintenance.

Yeah; "let's write our own bug tracker" is a good way to make sure nothing
comes of this.  On the other hand, "let's get all the Postgres hackers to
change their habits" is an equally good way to make sure nothing comes of
this.  (We tried that once already, with I-forget-which tracker; the
results were, um, forgettable.)  I think the only approach that has any
chance of success is to connect up some existing tracker software to more
or less our existing work patterns.  So somebody who wants to make this
happen needs to sit down and do that.

FWIW, I concur with the opinions that it needs to be an OSS tracker.
This project has been around for twenty years and I have every expectation
that it will survive for another twenty.  I have no confidence in any
closed-source product still being available (and free) in 2035.  We need
something with an active development/support community, too, so it doesn't
end up being us supporting it on our ownsome ten years out.  Other than
that, I'm agnostic as to what gets picked.
        regards, tom lane

* Joe Conway (mail@joeconway.com) wrote:
> On 09/24/2015 10:08 AM, Stephen Frost wrote:
> > debbugs does most of the above by default, no programming needed...  I'm
> > sure we could get it to integrate with the commitfest and have a git
> > commit hook which sends the appropriate email to it also.
> >
> > That the emacs folks are using it makes me *much* more interested in the
> > idea of getting debbugs up and running..
>
> I'm not familiar with debbugs myself, but given that description it
> sounds to me like it would be worth giving it a try.

It started out as Debian's bug tracking system, but apparently others
are using it now also.

Here's an example.

The main view for a particular package:

https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=util-linux;dist=unstable

A specific bug:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=786804

Note that the interface for working with the bug tracker is primairly
email, while everything is available for display over the web.  There is
an interface at the bottom of the 'main' package page for submitting a
bug and, iirc, there's a system-wide bug submission form somewhere too.

You'll notice that, at the bottom of the bug referenced above, is a
message which is auto-generated from a Debian Developer uploading a new
version that included 'Closes: #786804' in the changelog.  Having
something similar work for commits wouldn't be hard at all.

Including the bug's email address (all bugs have their own email
address) on the thread is also an excellent way to keep track of all of
the discussion which happened around a certain bug, even through subject
changes or whatever.

Not saying it's perfect, of course, but it's probably the best option
for minimizing impact on our existing process.

Thanks!

Stephen

On 09/24/2015 10:24 AM, Stephen Frost wrote:
> * Joe Conway (mail@joeconway.com) wrote:
>> On 09/24/2015 10:08 AM, Stephen Frost wrote:
>>> debbugs does most of the above by default, no programming needed...  I'm
>>> sure we could get it to integrate with the commitfest and have a git
>>> commit hook which sends the appropriate email to it also.
>>>
>>> That the emacs folks are using it makes me *much* more interested in the
>>> idea of getting debbugs up and running..
>>
>> I'm not familiar with debbugs myself, but given that description it
>> sounds to me like it would be worth giving it a try.
> 
> It started out as Debian's bug tracking system, but apparently others
> are using it now also.
> 
> Here's an example.
> 
> The main view for a particular package:
> 
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=util-linux;dist=unstable
> 
> A specific bug:
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=786804

I adore "Toggle useless messages" as a feature.  ;-)


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Thu, Sep 24, 2015 at 6:16 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Yeah; "let's write our own bug tracker" is a good way to make sure nothing
> comes of this.  On the other hand, "let's get all the Postgres hackers to
> change their habits" is an equally good way to make sure nothing comes of
> this.


I think the way to achieve something is to set a limited scope and
agree what we're going to use the tool to accomplish. If some people
want to drive the entire development workflow off the tool and others
are using it to track issues and others to track bugs then the three
groups will all end up dissatisfied and give up.

My personal feeling is that we should use it basically like a
spreadsheet listing the current pending bugs that we don't want to
forget about. Generally with Postgres there are maybe a half-dozen
such bugs at any time that are being actively worked on and perhaps a
few dozen other bugs that we consider wishlist items that live on
people's todo lists.

What we should NOT do is

a) drive the entire workflow off a trouble ticketing system which is
what tools like Jira are intended for. That will be frustrating
because some people will spend a lot of effort entering information
and then get annoyed that others are ignoring it and working on what
they want. There's no need to mark bug "owners" or keep track of
"milestone targets" to track who is going to do what when about the
bug -- the goal is just that we don't forget about it.

b) try to keep a massive database to search through of every user
report that ever came in. This seems to be what Bugzilla is designed
for -- the default screen doesn't show all the bug reports instead it
has lots of tools for tagging and curating your bugs and searching
through them. Every bugzilla I'm familiar with is full of thousands of
bugs that get ignored until the release they were reported on gets
EOL'd and lots of effort goes into curating this list.

I think those two antipatterns are what most people are afraid of in
this community. I think Debbugs was the best culture fit in that it
isn't designed for either of these two usage models and is email
drive. I think github issues might be a good alternative though.

-- 
greg

On Thu, Sep 24, 2015 at 12:10:07PM -0600, Ryan Pedela wrote:
> Kam Lasater wrote:
> > I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
> > that order). There are tens to hundreds of other great ones out there,
> > I'm sure one of them would also work.
> 
> Why not just use Github issues?

Is Github issues something we can run ourselves?

If not, it's a proprietary system which has a proprietor whose
existence even next month is not guaranteed, and whose interests are
not guaranteed to align with ours into an indefinite future.

In some very important sense, it does not matter what features a
system has if it isn't one we can control.  At a minimum, this means
we need to run it in its entirety on resources we control, and we need
to be able to patch any piece of it on our own say-so.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

All,

* Stephen Frost (sfrost@snowman.net) wrote:
> Not saying it's perfect, of course, but it's probably the best option
> for minimizing impact on our existing process.

I discussed the current state of debbugs with Don Armstrong (one of the
main individuals behind it) and his opinion is that debbugs could
certainly work.  Further, he's been working to add support for
PostgreSQL to it, which would certainly be nice.

* Josh Berkus (josh@agliodbs.com) wrote:
> I adore "Toggle useless messages" as a feature.  ;-)

Ditto. :)

There are a few questions regarding just how it would work and what the
structure would be, but my thinking is that we'd handle it in much the
same way that Debian already does, therefore, without thinking about it
too much, I imagine we'd have:

'source' packages which map up to individual git repos from
git.postgresql.org.

'binary' packages (for the 'postgres' source package; other source
packages can make their own decisions) which map up to each binary we
have (psql, pg_dump, etc).  That's a bit more granular than Debian does
but I don't think that's a bad thing.

I'm sure there are a number of other bits regarding the setup that need
to be considered and how it integrates with our existing mailing lists,
etc, etc, some of which will probably involve discussion with Don as we
work through them (based on my sub-1-hour response from him today, I
don't anticipate that being an issue), but the next big question is:

Are there any objections to pginfra standing up bugs.postgresql.org with
debbugs?  Obviously, it'd be more-or-less beta as we play with it, and
we could set it up as beta-bugs.p.o, if there's concern about that.

Thanks!

Stephen

Stephen Frost <sfrost@snowman.net> writes:
> Are there any objections to pginfra standing up bugs.postgresql.org with
> debbugs?  Obviously, it'd be more-or-less beta as we play with it, and
> we could set it up as beta-bugs.p.o, if there's concern about that.

I agree with the idea that we don't yet want to give the impression that
this is the official bug tracker.  However, "beta-bugs" could give the
impression that it was specifically for bugs about 9.5beta, without
dispelling the idea that it is official.  Maybe "bugs-test.p.o"?

(But let's not spend too much time bikeshedding the site name.)
        regards, tom lane

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > Are there any objections to pginfra standing up bugs.postgresql.org with
> > debbugs?  Obviously, it'd be more-or-less beta as we play with it, and
> > we could set it up as beta-bugs.p.o, if there's concern about that.
>
> I agree with the idea that we don't yet want to give the impression that
> this is the official bug tracker.  However, "beta-bugs" could give the
> impression that it was specifically for bugs about 9.5beta, without
> dispelling the idea that it is official.  Maybe "bugs-test.p.o"?

Works for me.

> (But let's not spend too much time bikeshedding the site name.)

Agreed.

Thanks!

Stephen

On 09/24/2015 07:03 PM, Josh Berkus wrote:
> On 09/23/2015 10:25 PM, Thomas Munro wrote:
>> On Thu, Sep 24, 2015 at 1:31 PM, Joe Conway <mail@joeconway.com> wrote:
>>> On 09/23/2015 05:21 PM, Thomas Munro wrote:
>>>> Do you think it would make any sense to consider evolving what we have
>>>> already?  At the moment, we have a bug form, and when you submit it it
>>>> does this (if I'm looking at the right thing, please correct me if I'm
>>>> not):
> 
> I know we're big on reinventing the wheel here, but it would really be a
> better idea to use an established product than starting over from
> scratch. Writing a bug tracker is a lot of work and maintenance.
> 
>> The two most common interactions could go something like this:
>>
>> 1.  User enters bug report via form, creating an issue in NEW state
>> and creating a pgsql-bugs thread.  Someone responds by email that this
>> is expected behaviour, not a bug, not worth fixing or not a Postgres
>> issue etc using special trigger words.  The state is automatically
>> switched to WORKS_AS_DESIGNED or WONT_FIX.  No need to touch the web
>> interface: the only change from today's workflow is awareness of the
>> right wording to trigger the state change.
>>
>> 2.  User enters bug report via form, creating issue #1234 in NEW
>> state.   Someone responds by email to acknowledge that that may indeed
>> be an issue, and any response to an issue in NEW state that doesn't
>> reject it switches it to UNDER_DISCUSSION.  Maybe if a commitfest item
>> references the same thread (or somehow references the issue number?)
>> its state is changed to IN_COMMITFEST, or maybe as you say there could
>> be a way to generate the commitfest item from the issue, not sure
>> about that.  Eventually a commit log message says "Fixes bug #1234"
>> and the state automatically goes to FIXED.
> 
> I don't know debbugs, but I know that it would be possible to program RT
> to do all of the above, except add the item to the commitfest.

well.... minus the fact that the commitfest process looked
different/non-existent back in 2007/2008 this is basically how the BZ
based PoC I did back than worked (hooked into the bug tracking form to
allow BZ to "learn" about an issue/bug/thread and keep tracking it
automatically afterwards. You could send it simply commands as part as
the mail or click in the gui (or do nothing at all - though it would
close the bug if it found the bug id in a commit).

Even adding something to the commitfest should be fairly easy to do in
most tools because they all have hooks to send stuff via email, to
twitter, hipchat, IRC or whatnot.


Stefan

On 09/24/2015 12:55 PM, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
>> Are there any objections to pginfra standing up bugs.postgresql.org with
>> debbugs?  Obviously, it'd be more-or-less beta as we play with it, and
>> we could set it up as beta-bugs.p.o, if there's concern about that.
> 
> I agree with the idea that we don't yet want to give the impression that
> this is the official bug tracker.  However, "beta-bugs" could give the
> impression that it was specifically for bugs about 9.5beta, without
> dispelling the idea that it is official.  Maybe "bugs-test.p.o"?

I'd suggest instead just having a big banner up in the page header which
says "this system is currently beta and not yet the canonical source for
postgres bug information".  That way, if it does become the canonical
source, we won't go breaking everyone's links when we change the domain
name.


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

Josh Berkus <josh@agliodbs.com> writes:
> On 09/24/2015 12:55 PM, Tom Lane wrote:
>> I agree with the idea that we don't yet want to give the impression that
>> this is the official bug tracker.  However, "beta-bugs" could give the
>> impression that it was specifically for bugs about 9.5beta, without
>> dispelling the idea that it is official.  Maybe "bugs-test.p.o"?

> I'd suggest instead just having a big banner up in the page header which
> says "this system is currently beta and not yet the canonical source for
> postgres bug information".  That way, if it does become the canonical
> source, we won't go breaking everyone's links when we change the domain
> name.

Works for me, if it's not too hard to do.
        regards, tom lane

On 24.09.2015 20:23, David Fetter wrote:
> On Thu, Sep 24, 2015 at 12:10:07PM -0600, Ryan Pedela wrote:
>> Kam Lasater wrote:
>>> I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
>>> that order). There are tens to hundreds of other great ones out there,
>>> I'm sure one of them would also work.
>>
>> Why not just use Github issues?
>
> Is Github issues something we can run ourselves?
>
> If not, it's a proprietary system which has a proprietor whose
> existence even next month is not guaranteed, and whose interests are
> not guaranteed to align with ours into an indefinite future.
>
> In some very important sense, it does not matter what features a
> system has if it isn't one we can control.  At a minimum, this means
> we need to run it in its entirety on resources we control, and we need
> to be able to patch any piece of it on our own say-so.

+ 1

Instead of Github maybe GitLab is a good choice. There is an open source 
community edition you have the full control over. And it is used by more 
than 100.000 organizations:

https://en.wikipedia.org/wiki/GitLab

Greetings,
Torsten

2015-09-25 9:57 GMT+02:00 Torsten Zuehlsdorff <mailinglists@toco-domains.de>:
>
>
> On 24.09.2015 20:23, David Fetter wrote:
>>
>> On Thu, Sep 24, 2015 at 12:10:07PM -0600, Ryan Pedela wrote:
>>>
>>> Kam Lasater wrote:
>>>>
>>>> I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
>>>> that order). There are tens to hundreds of other great ones out there,
>>>> I'm sure one of them would also work.
>>>
>>>
>>> Why not just use Github issues?
>>
>>
>> Is Github issues something we can run ourselves?
>>
>> If not, it's a proprietary system which has a proprietor whose
>> existence even next month is not guaranteed, and whose interests are
>> not guaranteed to align with ours into an indefinite future.
>>
>> In some very important sense, it does not matter what features a
>> system has if it isn't one we can control.  At a minimum, this means
>> we need to run it in its entirety on resources we control, and we need
>> to be able to patch any piece of it on our own say-so.
>
>
> + 1
>
> Instead of Github maybe GitLab is a good choice. There is an open source
> community edition you have the full control over. And it is used by more
> than 100.000 organizations:
>
> https://en.wikipedia.org/wiki/GitLab
>

Of course everyone has their own preferences. Just wanted to point you
to roundup [1].

It's:

- Simple
- Python
- Very easy to extend
- Integrates well with e-mail
- Complete web interface

We use it for the Tryton [2] project and the source code we use is
available here [3] so you can take a look how easy it is to add new
fields and other stuff.

We use mercurial but commits automatically close the bug reports and
we have links with the codereview.

[1] http://roundup.sourceforge.net/docs/features.html
[2] https://bugs.tryton.org
[3] http://hg.tryton.org/bugs.tryton.org/

-- 
Albert Cervera i Areny
Tel. 93 553 18 03
@albertnan
www.NaN-tic.com

On 25.09.2015 10:04, Albert Cervera i Areny wrote:
> 2015-09-25 9:57 GMT+02:00 Torsten Zuehlsdorff <mailinglists@toco-domains.de>:
>> On 24.09.2015 20:23, David Fetter wrote:
>>>
>>> On Thu, Sep 24, 2015 at 12:10:07PM -0600, Ryan Pedela wrote:
>>>>
>>>> Kam Lasater wrote:
>>>>>
>>>>> I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
>>>>> that order). There are tens to hundreds of other great ones out there,
>>>>> I'm sure one of them would also work.
>>>>
>>>>
>>>> Why not just use Github issues?
>>>
>>>
>>> Is Github issues something we can run ourselves?
>>>
>>> If not, it's a proprietary system which has a proprietor whose
>>> existence even next month is not guaranteed, and whose interests are
>>> not guaranteed to align with ours into an indefinite future.
>>>
>>> In some very important sense, it does not matter what features a
>>> system has if it isn't one we can control.  At a minimum, this means
>>> we need to run it in its entirety on resources we control, and we need
>>> to be able to patch any piece of it on our own say-so.
>>
>> + 1
>>
>> Instead of Github maybe GitLab is a good choice. There is an open source
>> community edition you have the full control over. And it is used by more
>> than 100.000 organizations:
>>
>> https://en.wikipedia.org/wiki/GitLab
>
> Of course everyone has their own preferences. Just wanted to point you
> to roundup [1].

You're right. Though GitLab is not my own preference, but i'm currently 
porting it to FreeBSD. ;) But it is a good choice if you like Github.

There is a greater list of tools to for source code hosting:
https://en.wikipedia.org/wiki/Comparison_of_source_code_hosting_facilities

Greetings,
Torsten

Hello,

I accidently sent this directly to TGL so here we go:

Hello,

I am pretty sure RT can do what I am about to suggest but I know Redmine 
can do it. Consider the following situation:

Robert Haas posts: Parallelized Joins patch    New issue is created (via email)    Patch is automatically attached to
issue
TGL reponds: Looks good but fix XYZ    Issue updated
Haas posts new patch    Issue updated    New patch, with date automatically appended to issue
Grittner responds: Tested this, looks good, committing    In body of email Grittner writes:        status: committed
Issueupdated    Status set to committed    Issue closed
 

Now, take the same thing and apply it to a bug. We even have the ability 
to say who can assign something committed. For example, we could set it 
so that only the email address of the git-hook can assign a ticket 
committed. Other things we can do:

* Assign to specific releases
* Move issues to different trackers (not a bug, but a feature request)
* Assign issues to different committers/reviewers/users
* Relatively easy to make work as an issue/bug/commit/discussion tracker
* Easy to move patches/feature requests/bugs between trackers/releases
* East to have different trackers for different releases
* Full role and rights customization
* Interfaces with GIT
* Built in WIKI
* Runs on PostgreSQL
* Has an API
* Already works with PostgreSQL community accounts
* Hugely active community that isn't centric
* Manageable via email (requirement for most hackers)

In short, although we are talking about an issue tracker this is 
something that can be integrated into our existing workflow, habits of 
the current hackers would have to adapt not completely change.

Joshua D. Drake

EDIT: Note that I am not trying to start an argument about a bunch of 
different issues here. I am trying to illustrate how a properly managed 
system could not only integrate into but enhance our current workflow.

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

Simon Riggs <simon@2ndQuadrant.com> writes:
> I have frequently been the agent of change in matters of process, but I see
> no useful change here, just lots of wasted time. But then why are we even
> talking about change? What thing is broken that needs to be fixed? Why is
> adopting a new package a fix for that?

Fair questions indeed.  I think the core points here are:

1. We don't have a good process for making sure things don't "slip through
the cracks".  I think everyone more or less relies on Bruce to run through
his mailbox periodically and nag them about threads that don't seem to
have been closed off.  The deficiencies of that are obvious.

2. There's no visibility for outsiders as to what issues are open or
recently fixed.  Not being outsiders, I'm not sure that we are terribly
well qualified to describe this problem precisely or identify a good
solution --- but I grant that there's a problem there.

I do not know how much emphasis the project should place on point #2.
By definition, fixing that will not return any direct benefit to us.
However, point #1 is clearly an issue and I think a low-overhead fix
for it would be a good thing.  If we can get some answer for #2 out
of it at the same time, even better.
        regards, tom lane

On 09/25/2015 09:32 AM, Tom Lane wrote:
> 2. There's no visibility for outsiders as to what issues are open or
> recently fixed.  Not being outsiders, I'm not sure that we are terribly
> well qualified to describe this problem precisely or identify a good
> solution --- but I grant that there's a problem there.
>
> I do not know how much emphasis the project should place on point #2.
> By definition, fixing that will not return any direct benefit to us.

I would argue that there is some benefit for us in terms of advocacy. We
certainly get criticized for our lack of a bug tracker. I don't know of
anyone who specifically rejected Postgres because of that fact, but
would not be surprised if has happened.

Joe

--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development

On 09/25/2015 09:55 AM, Joe Conway wrote:
> On 09/25/2015 09:32 AM, Tom Lane wrote:
>> 2. There's no visibility for outsiders as to what issues are open or
>> recently fixed.  Not being outsiders, I'm not sure that we are terribly
>> well qualified to describe this problem precisely or identify a good
>> solution --- but I grant that there's a problem there.
>>
>> I do not know how much emphasis the project should place on point #2.
>> By definition, fixing that will not return any direct benefit to us.
>
> I would argue that there is some benefit for us in terms of advocacy. We
> certainly get criticized for our lack of a bug tracker. I don't know of
> anyone who specifically rejected Postgres because of that fact, but
> would not be surprised if has happened.

I don't know if it has happened but I know that they have at least 
considered it. I also know that when standing in front of a room full of 
PostgreSQL users (Professional developers) and they find out we don't 
have a tracker? The only word for the response is: mortified.

JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

Joe Conway <mail@joeconway.com> writes:
> On 09/25/2015 09:32 AM, Tom Lane wrote:
>> I do not know how much emphasis the project should place on point #2.
>> By definition, fixing that will not return any direct benefit to us.

> I would argue that there is some benefit for us in terms of advocacy. We
> certainly get criticized for our lack of a bug tracker. I don't know of
> anyone who specifically rejected Postgres because of that fact, but
> would not be surprised if has happened.

Yeah.  I should have put in something to the effect that there may be
indirect benefits, but they're pretty hard to quantify.
        regards, tom lane

On 09/25/2015 10:11 AM, Simon Riggs wrote:    >
>     > I do not know how much emphasis the project should place on point #2.
>     > By definition, fixing that will not return any direct benefit to us.
>
>     I would argue that there is some benefit for us in terms of advocacy.
>
>
> There are also some negatives in terms of advocacy.

I know we are just being thorough here but I think the idea that there 
are negatives in the transparency of our project is simply untrue. The 
only argument I can think of is, "Well then people are going to know 
everything that is broken." To which I respond, "Good, that is a net 
positive".

JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/25/2015 10:27 AM, Simon Riggs wrote:

>     2. There's no visibility for outsiders as to what issues are open or
>     recently fixed.  Not being outsiders, I'm not sure that we are terribly
>     well qualified to describe this problem precisely or identify a good
>     solution --- but I grant that there's a problem there.
>
>
> If they can perform "git log" they can view what has happened recently.
> Tracking what might happen is much harder for active contributors.

>
> I've never had a user ask me for such a list. All I here is compliments
> that our software is incredibly robust.
>
> The only time this info is required is for people that provide a Support
> service based upon PostgreSQL, yet are not themselves sufficiently
> involved to know what bugs have been reported and are as yet unfixed. I
> expect such people are extremely interested in getting other people to
> do things that will help their business.
>
> I don't see a sustainable mechanism that can support the manpower
> resources required to provide that information to those people, apart
> from become an active contributor, or pay someone who is.

I think you are thinking of this from a very small view point. In my 
mind this goes way beyond a bug tracker (I assume there is a reason the 
individual posted and said "issue tracker").

JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Fri, Sep 25, 2015 at 11:32 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Simon Riggs <simon@2ndQuadrant.com> writes:
>> I have frequently been the agent of change in matters of process, but I see
>> no useful change here, just lots of wasted time. But then why are we even
>> talking about change? What thing is broken that needs to be fixed? Why is
>> adopting a new package a fix for that?
>
> Fair questions indeed.  I think the core points here are:
>
> 1. We don't have a good process for making sure things don't "slip through
> the cracks".  I think everyone more or less relies on Bruce to run through
> his mailbox periodically and nag them about threads that don't seem to
> have been closed off.  The deficiencies of that are obvious.
>
> 2. There's no visibility for outsiders as to what issues are open or
> recently fixed.  Not being outsiders, I'm not sure that we are terribly
> well qualified to describe this problem precisely or identify a good
> solution --- but I grant that there's a problem there.

This maybe understates the ability of google to match up problem
scenarios with the relevant fix and commentary.  I'm somewhat
skeptical that issue trackers are much of an improvement upon informal
processes.  Bruce will simply have to run through a different systems
to do the same amount of nagging he's always done.

merlin

On 09/25/2015 10:27 AM, Simon Riggs wrote:
> On 25 September 2015 at 11:32, Tom Lane <tgl@sss.pgh.pa.us
>     1. We don't have a good process for making sure things don't "slip
>     through
>     the cracks".  I think everyone more or less relies on Bruce to run
>     through
>     his mailbox periodically and nag them about threads that don't seem to
>     have been closed off.  The deficiencies of that are obvious.
> 
> 
> I don't rely on that myself. That sounds like a personal viewpoint only.
> I welcome more discussion amongst Committers with regard to
> coordination, but formal systems aren't what I think will help there.
> That situation has recently improved anyway, so no further change needed
> at present, IMHO.

??? Improved how?

>     2. There's no visibility for outsiders as to what issues are open or
>     recently fixed.  Not being outsiders, I'm not sure that we are terribly
>     well qualified to describe this problem precisely or identify a good
>     solution --- but I grant that there's a problem there.
> 
> 
> If they can perform "git log" they can view what has happened recently.
> Tracking what might happen is much harder for active contributors.

It takes a lot of technical knowledge of PostgreSQL to relate a commit
message to a bug report, given that the bug report may not be
referenced, and the report and the commit often use completely different
terminology.  Also, users are often wanting to look for bug fixes from
months or even years ago, and git log has crappy searchability.

I can't say how many times I've had a conversation like this:

"Oh, that's a known issue.  It was fixed later in the 9.3 series."

"Really?  In which release specificially?"

"Ummm.... lemme search the release notes ... I know it's here somewhere ..."

> 
> I've never had a user ask me for such a list. All I here is compliments
> that our software is incredibly robust.

I have. I've had users ask for it, I've had customers ask for it, I've
had companies thinking of adopting PostgreSQL ask for it ... and for a
few of them, our lack of an issue tracker was a deciding factor in
saying that PostgreSQL "wasn't mature enough".  Certainly it was major
points off when Forrester rated us.

Also, members of downstream projects would really like us to get a bug
tracker they can kick up bugs to if they're determined to be in
Postgres.   There are bugs we're not even hearing about because it's too
confusing for someone who has a lot of projects to cover to figure out
our idiosyncratic system.

Today, having an issue tracker is considered "normal" for any
significant OSS project. The fact that we don't have one is regarded as
abberant, and not in a good way ... we're at the point where if we're
not going to adopt one, we'd better have a darned good reason which we
can explain clearly to the public.

> The only time this info is required is for people that provide a Support
> service based upon PostgreSQL, yet are not themselves sufficiently
> involved to know what bugs have been reported and are as yet unfixed. I
> expect such people are extremely interested in getting other people to
> do things that will help their business.

That has absolutely nothing to do with any reason for the PostgreSQL
project to have a bug tracker.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Thursday 24 September 2015 12:10:07 Ryan Pedela wrote:
> Kam Lasater wrote:
> > I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
> > that order). There are tens to hundreds of other great ones out there,
> > I'm sure one of them would also work.
> 
> Why not just use Github issues?

I will also vote for github.  We have github mirror now. 

In a pinch, you can use gitlab.

PS mail lists outdated IMHO.
-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Mon, Sep 28, 2015 at 03:50:29PM +0300, YUriy Zhuravlev wrote:
> On Thursday 24 September 2015 12:10:07 Ryan Pedela wrote:
> > Kam Lasater wrote:
> > > I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably
> > > in that order). There are tens to hundreds of other great ones
> > > out there, I'm sure one of them would also work.
> > 
> > Why not just use Github issues?
> 
> I will also vote for github.  We have github mirror now. 
> 
> In a pinch, you can use gitlab.

Github is a company, and companies go out of business all the time,
frequently without warning.  However healthy it may look right now,
this is a scenario for which we need to have plans.

What is your plan for this contingency?

I assure you that when it's happening, the very last thing on the mind
of the people winding down the company is whether some project's bug
tracker gets properly exported and its functionality ported to another
compatible system.

> PS mail lists outdated IMHO.

They may well be, but until we decide it's worth the switching costs
to move to a totally different way of doing things, that system will
stay in place.

There are a good many decisions I would make differently if I were
waving a magic wand over a green field project.  Neither magic wands
nor a green field project are in operation here.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On Monday 28 September 2015 08:23:46 David Fetter wrote:
> They may well be, but until we decide it's worth the switching costs
> to move to a totally different way of doing things, that system will
> stay in place.
Until we decide we're wasting time

>Neither magic wands nor a green field project are in operation here.
Now any stick will help. IMHO

-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Mon, Sep 28, 2015 at 07:14:40PM +0300, YUriy Zhuravlev wrote:
> On Monday 28 September 2015 08:23:46 David Fetter wrote:
> > They may well be, but until we decide it's worth the switching
> > costs to move to a totally different way of doing things, that
> > system will stay in place.
> Until we decide we're wasting time

Great!

Since you're convinced that this is an unqualified win, please put
together a project plan for switching from our current system to
Github.  Make sure to account for all current functionality, even if
your plan is to drop it without archiving.  For bonus points, you can
do a side-by-side comparison of the system we have with the system you
envision, including the running and switching costs.  While
people-hours isn't a great metric, you could start with that, or
propose a different one.

> >Neither magic wands nor a green field project are in operation here.
> Now any stick will help. IMHO

That is an assertion that you now have an opportunity to back with
some kind of plan based on our current system and the future system
you envision.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On 2015-09-28 09:41:18 -0700, David Fetter wrote:
> Since you're convinced that this is an unqualified win, please put
> together a project plan for switching from our current system to
> Github.

Err, no. That's a waste of all our time.

It has been stated pretty clearly in this thread by a number of senior
community people that we're not going to use a closed source system.

On 09/28/2015 05:50 AM, YUriy Zhuravlev wrote:
> On Thursday 24 September 2015 12:10:07 Ryan Pedela wrote:
>> Kam Lasater wrote:
>>> I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in
>>> that order). There are tens to hundreds of other great ones out there,
>>> I'm sure one of them would also work.
>>
>> Why not just use Github issues?
>
> I will also vote for github.  We have github mirror now.
>
> In a pinch, you can use gitlab.
>
> PS mail lists outdated IMHO.
>

Github is not an option, period. .Org should as much as reasonably 
possible rely on Open Source tools and contributing companies/persons. 
The idea that we would use Github was a non-starter before the first 
person typed Gi...

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 9/28/15 11:43 AM, Andres Freund wrote:
> On 2015-09-28 09:41:18 -0700, David Fetter wrote:
>> Since you're convinced that this is an unqualified win, please put
>> together a project plan for switching from our current system to
>> Github.
>
> Err, no. That's a waste of all our time.
>
> It has been stated pretty clearly in this thread by a number of senior
> community people that we're not going to use a closed source system.

GitLab OTOH is released under a MIT license, so it is an option. I don't 
know how it compares to other suggested options, but if YUriy wants to 
propose it it's at least a viable option.

[1] https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
> On 9/28/15 11:43 AM, Andres Freund wrote:
>> It has been stated pretty clearly in this thread by a number of senior
>> community people that we're not going to use a closed source system.

> GitLab OTOH is released under a MIT license, so it is an option. I don't 
> know how it compares to other suggested options, but if YUriy wants to 
> propose it it's at least a viable option.

I think a more accurate summary of what's been said is that we won't
consider putting any important functionality on proprietary platforms,
of which closed-source tools would be a subset.  The intention of the
community is that we'll be around for as long as there's a critical mass
of people interested in maintaining Postgres.  We will not be dependent
on any one company, and that's why e.g. github is out.  (A lot of smaller
open-source projects don't have the luxury of rejecting such options ...
but we do, and we will.)

Now, running gitlab on community-owned hardware would potentially be an
option, if we find gitlab attractive from a functionality standpoint.
The question I'd have about that is whether it has a real development
community, or is open-source in name only.  If github did go belly up,
would we find ourselves maintaining the gitlab code all by ourselves?
That might not be the end of the world, but it wouldn't be a good use
of community time either.

Fundamentally, we're playing the long game here.  We do not want to make
a choice of tools that we're going to regret ten years from now.
        regards, tom lane

Tom Lane wrote:

> Now, running gitlab on community-owned hardware would potentially be an
> option, if we find gitlab attractive from a functionality standpoint.
> The question I'd have about that is whether it has a real development
> community, or is open-source in name only.  If github did go belly up,
> would we find ourselves maintaining the gitlab code all by ourselves?
> That might not be the end of the world, but it wouldn't be a good use
> of community time either.
> 
> Fundamentally, we're playing the long game here.  We do not want to make
> a choice of tools that we're going to regret ten years from now.

We already made a similar choice some years ago when we started
depending on the then-recently open sourced SourceForge code for
pgFoundry.  That didn't turn out all that well in the long run.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 09/28/2015 03:40 PM, Alvaro Herrera wrote:
> Tom Lane wrote:
>
>> Now, running gitlab on community-owned hardware would potentially be an
>> option, if we find gitlab attractive from a functionality standpoint.
>> The question I'd have about that is whether it has a real development
>> community, or is open-source in name only.  If github did go belly up,
>> would we find ourselves maintaining the gitlab code all by ourselves?
>> That might not be the end of the world, but it wouldn't be a good use
>> of community time either.
>>
>> Fundamentally, we're playing the long game here.  We do not want to make
>> a choice of tools that we're going to regret ten years from now.
>
> We already made a similar choice some years ago when we started
> depending on the then-recently open sourced SourceForge code for
> pgFoundry.  That didn't turn out all that well in the long run.

I think we need to look at long standing FOSS projects with a large and 
extended user base (Redmine, RT). Anything that is fairly community 
specific (Debbugs) likely will cause more heartache than it is worth in 
the long run.

JD


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/28/2015 03:40 PM, Alvaro Herrera wrote:
> Tom Lane wrote:
> 
>> Now, running gitlab on community-owned hardware would potentially be an
>> option, if we find gitlab attractive from a functionality standpoint.
>> The question I'd have about that is whether it has a real development
>> community, or is open-source in name only.  If github did go belly up,
>> would we find ourselves maintaining the gitlab code all by ourselves?
>> That might not be the end of the world, but it wouldn't be a good use
>> of community time either.
>>
>> Fundamentally, we're playing the long game here.  We do not want to make
>> a choice of tools that we're going to regret ten years from now.
> 
> We already made a similar choice some years ago when we started
> depending on the then-recently open sourced SourceForge code for
> pgFoundry.  That didn't turn out all that well in the long run.

No kidding.

Anyway, we don't have to have this discussion because the Github Issue
model is insufficiently sophisticated for our usage:

* crappy-to-nonexistant email integration
* flat "tag" categorization system
* no concept of releases
* too-simple two-level permissions model
* poor search tools
* no ability to add new fields to extend
* dependency on markup-based cross-referencing
* inability to flag issues for specific people's attention
* no workflow other than open/closed
* no support for attachments

... in short, Github issues is great for a small 6-dev project, but is
utterly inadequate for a project the size of PostgreSQL.

Now, if those issues were common to other tools we could find, then
maybe it would be worth fixing them.  But we already have access to
other tools which are more mature, so why would we bother?

The infra team seems to be good with debbugs, and several committers
seem to like it, why not go with it?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

Josh Berkus <josh@agliodbs.com> writes:
> The infra team seems to be good with debbugs, and several committers
> seem to like it, why not go with it?

It certainly seems like debbugs is the proposal to beat at this point.

In the end though, what matters is somebody doing the dogwork to make
it happen: connecting some tool up to our workflow and populating it
with an initial collection of items.  Until that happens, we're just
arguing in a vacuum with no way to see whether a proposal will really
work.
        regards, tom lane

On 29/09/15 11:54, Joshua D. Drake wrote:
> On 09/28/2015 03:40 PM, Alvaro Herrera wrote:
>> Tom Lane wrote:
>>
>>> Now, running gitlab on community-owned hardware would potentially be an
>>> option, if we find gitlab attractive from a functionality standpoint.
>>> The question I'd have about that is whether it has a real development
>>> community, or is open-source in name only.  If github did go belly up,
>>> would we find ourselves maintaining the gitlab code all by ourselves?
>>> That might not be the end of the world, but it wouldn't be a good use
>>> of community time either.
>>>
>>> Fundamentally, we're playing the long game here.  We do not want to 
>>> make
>>> a choice of tools that we're going to regret ten years from now.
>>
>> We already made a similar choice some years ago when we started
>> depending on the then-recently open sourced SourceForge code for
>> pgFoundry.  That didn't turn out all that well in the long run.
>
> I think we need to look at long standing FOSS projects with a large 
> and extended user base (Redmine, RT). Anything that is fairly 
> community specific (Debbugs) likely will cause more heartache than it 
> is worth in the long run.
>
> JD
>
>
Linux kernel project uses bugzilla (https://bugzilla.kernel.org)
and so does LibreOffice (https://bugs.documentfoundation.org)

I think they are both fairly big projects in for the long haul.


Cheers,
Gavin

On 09/28/2015 04:08 PM, Gavin Flower wrote:

>> JD
>>
>>
> Linux kernel project uses bugzilla (https://bugzilla.kernel.org)
> and so does LibreOffice (https://bugs.documentfoundation.org)
>
> I think they are both fairly big projects in for the long haul.

I am not anti-bugzilla, just not all that familiar with it recently.

JD

>
>
> Cheers,
> Gavin
>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 9/28/15 5:34 PM, Tom Lane wrote:
> Now, running gitlab on community-owned hardware would potentially be an
> option, if we find gitlab attractive from a functionality standpoint.
> The question I'd have about that is whether it has a real development
> community, or is open-source in name only.  If github did go belly up,
> would we find ourselves maintaining the gitlab code all by ourselves?
> That might not be the end of the world, but it wouldn't be a good use
> of community time either.

FWIW, Gitlab appears to be a completely separate company from GitHub. 
According to https://about.gitlab.com/about/, over 800 people have 
contributed to it. It actually started as an open source project.

> Fundamentally, we're playing the long game here.  We do not want to make
> a choice of tools that we're going to regret ten years from now.

Agreed.

In this case it's a question of whether we want (or think in the future 
we might want) the advanced features that Gitlab offers. Things like 
commenting directly on "patches" (really, pull requests), direct 
integration with the issue tracker, a CI framework, etc.

Perhaps there's not a strong desire for those features today, but 
tomorrow could be a very different case. I'm honestly rather shocked 
(plesantly) that the community is seriously considering a bug tracker, 
given the reaction that's happened every time in the past. I think it'd 
be a real shame if a few years from now the community might consider, 
say, pull requests instead of emailed patches... except that would mean 
we need Yet Another Tool. Another example is CI. Yes, we have the 
buildfarm, but that does nothing to prevent bitrot in patches.

Actually, now that I've poked around the site a bit, it might well be 
worth adopting Gitlab just to replace some of our current stand-alone 
tools with a single integrated solution, depending on how much time we 
spend maintaining all the separate stuff.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 9/28/15 6:06 PM, Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
>> The infra team seems to be good with debbugs, and several committers
>> seem to like it, why not go with it?
>
> It certainly seems like debbugs is the proposal to beat at this point.
>
> In the end though, what matters is somebody doing the dogwork to make
> it happen: connecting some tool up to our workflow and populating it
> with an initial collection of items.  Until that happens, we're just
> arguing in a vacuum with no way to see whether a proposal will really
> work.

Note that since they also offer a hosted solution we should use that to 
play with instead of trying to install it at this point.

Integrating the issue tracker looks like it's just a call to this API: 
http://doc.gitlab.com/ce/api/issues.html#new-issue. I don't normally do 
web development myself so I'd rather not figuring out how to setup a 
copy of the website to hack on, but if no one else wants to try it I can 
take a stab at it.

Presumably mirroring our git repository would work the same as it does 
for mirroring to GitHub. My guess is that would be enough to get the 
basic git/issue tracker integration working.

Commitfest could be tied in as well. Presumably each commitfest would be 
a milestone (http://doc.gitlab.com/ce/api/milestones.html) and each 
submission an issue.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

* Ryan Pedela (rpedela@datalanche.com) wrote:
> I haven't used Gitlab extensively, but it has a feature set similar to
> Github and then some [1]. The OSS project does seem active [2], but it is
> still relatively new.

I've set it up and used it for a relatively small environment and was
*not* impressed with it.  Perhaps it's come a long way in a year or two,
but I doubt it.

Thanks!

Stephen

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Josh Berkus <josh@agliodbs.com> writes:
> > The infra team seems to be good with debbugs, and several committers
> > seem to like it, why not go with it?
>
> It certainly seems like debbugs is the proposal to beat at this point.

Agreed.

> In the end though, what matters is somebody doing the dogwork to make
> it happen: connecting some tool up to our workflow and populating it
> with an initial collection of items.  Until that happens, we're just
> arguing in a vacuum with no way to see whether a proposal will really
> work.

Setting up debbugs is on my list of things to do in the not-too-distant
future, but don't expect much before beta1 as I've got a few items in
the hopper for that which are clearly higher priority than the bug
tracker we've lived without for 20 years.

I'll be sure to update the thread once there is progress to report.

Thanks!

Stephen

JD,

* Joshua D. Drake (jd@commandprompt.com) wrote:
> On 09/28/2015 03:40 PM, Alvaro Herrera wrote:
> >We already made a similar choice some years ago when we started
> >depending on the then-recently open sourced SourceForge code for
> >pgFoundry.  That didn't turn out all that well in the long run.
>
> I think we need to look at long standing FOSS projects with a large
> and extended user base (Redmine, RT). Anything that is fairly
> community specific (Debbugs) likely will cause more heartache than
> it is worth in the long run.

debbugs is being used by Debian, and has been since before our first
release (I believe- according to wikipedia, it started in 1994..).
Further, it's now being used by the GNU project for things as important
(well, to some ;) as Emacs.

It also requires a minimum of customization to integrate with our
existing workflow.

That's good enough for us to at least test it out, in my view.

I've run both Redmine (ugh) and RT (it's good, but I don't feel it's
quite as good as debbugs, for us).

Further, to your specific point, neither of those have the kind of FOSS
community backing that debbugs has.  I have no issue with BestPractical
but I don't know that there's 100 individuals ready to jump in and help
keep RT going if they go south.  Redmine seems a bit better in that
regard, but it's painful to maintain and is only 9 years old, while
debbugs is old enough to drink in the US at this point.

Thanks!

Stephen

On 9/28/15 9:03 PM, Stephen Frost wrote:
> * Ryan Pedela (rpedela@datalanche.com) wrote:
>> I haven't used Gitlab extensively, but it has a feature set similar to
>> Github and then some [1]. The OSS project does seem active [2], but it is
>> still relatively new.
>
> I've set it up and used it for a relatively small environment and was
> *not* impressed with it.  Perhaps it's come a long way in a year or two,
> but I doubt it.

2 years ago is when they first released the enterprise edition, which 
according to [1] had "The most important new feature is that you can now 
add members to groups of projects."

So looking at it now I'd say it's come a long way in 2 years.

[1] 
https://about.gitlab.com/2013/08/22/introducing-gitlab-6-0-enterprise-edition/
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

* Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
> 2 years ago is when they first released the enterprise edition,
> which according to [1] had "The most important new feature is that
> you can now add members to groups of projects."

It needed a lot more than a single feature.

> So looking at it now I'd say it's come a long way in 2 years.

Perhaps, but I'm not anxious to go down that road right now.  Let's give
debbugs a shot and see how that goes.

Thanks!

Stephen

On 09/28/2015 07:18 PM, Stephen Frost wrote:
> JD,
>

> debbugs is being used by Debian, and has been since before our first
> release (I believe- according to wikipedia, it started in 1994..).
> Further, it's now being used by the GNU project for things as important
> (well, to some ;) as Emacs.

I think if your level of expectation is set by Debian and Emacs, you are 
looking in the wrong direction.


> It also requires a minimum of customization to integrate with our
> existing workflow.

Prove it (Not being antagonistic)

>
> That's good enough for us to at least test it out, in my view.
>
> I've run both Redmine (ugh) and RT (it's good, but I don't feel it's
> quite as good as debbugs, for us).

My experience is with RT and Redmine, Redmine works, out of the box. 
RT.... Had some issues with but could see where it would work.

>
> Further, to your specific point, neither of those have the kind of FOSS
> community backing that debbugs has.

I disagree entirely. I have multiple customers that run Redmine or RT. I 
have exactly zero that run debbugs.

> keep RT going if they go south.  Redmine seems a bit better in that
> regard, but it's painful to maintain and is only 9 years old, while
> debbugs is old enough to drink in the US at this point.

So is Windows.

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

Stephen Frost <sfrost@snowman.net> writes:
> * Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
>> 2 years ago is when they first released the enterprise edition,
>> which according to [1] had "The most important new feature is that
>> you can now add members to groups of projects."

> It needed a lot more than a single feature.

Just going to their primary web page, and noting that the first line gives
links to "Features" and "Pricing" (but not "Documentation"), and the
second line makes it clear that there's a big difference between the
"Community Edition" and the "Enterprise Edition", is already enough to
turn me off.  We've seen that model before (mysql...) and it doesn't bode
well in the long run.

Further poking shows no evidence of any decent email integration, to name
just one of the things that have been mentioned as requirements in this
thread.  On the other hand, they are big on LDAP logins, and even
two-factor authentication.  (We need this for an issue tracker that's
supposed to provide visibility and easier reporting to people outside the
project?)  And JIRA integration, which seems to be an anti-feature to some
on this thread.  And they'd sure love to be in charge of our code repo.
And the main, possibly only, metadata they can track about an issue is
"assignee" and "milestone".

I can't imagine that this is what we want.
        regards, tom lane

Hello,

On 29.09.2015 00:34, Tom Lane wrote:
> Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
>> On 9/28/15 11:43 AM, Andres Freund wrote:
>>> It has been stated pretty clearly in this thread by a number of senior
>>> community people that we're not going to use a closed source system.
>
>> GitLab OTOH is released under a MIT license, so it is an option. I don't
>> know how it compares to other suggested options, but if YUriy wants to
>> propose it it's at least a viable option.
>
> I think a more accurate summary of what's been said is that we won't
> consider putting any important functionality on proprietary platforms,
> of which closed-source tools would be a subset.  The intention of the
> community is that we'll be around for as long as there's a critical mass
> of people interested in maintaining Postgres.  We will not be dependent
> on any one company, and that's why e.g. github is out.  (A lot of smaller
> open-source projects don't have the luxury of rejecting such options ...
> but we do, and we will.)
>
> Now, running gitlab on community-owned hardware would potentially be an
> option, if we find gitlab attractive from a functionality standpoint.
> The question I'd have about that is whether it has a real development
> community, or is open-source in name only.  If github did go belly up,
> would we find ourselves maintaining the gitlab code all by ourselves?
> That might not be the end of the world, but it wouldn't be a good use
> of community time either.

I ported GitLab to run on FreeBSD. From this progress i can say, that 
there is an active community. Many of the features (and bugfixes) came 
directly from the community.

I'm not for or against GitLab. I just wanted to point this out.

As mentioned in another Thread Bugzilla is used by LibreOffice and 
Linux. I want to add FreeBSD to the list.

Greetings,
Torsten

On 29.09.2015 05:54, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
>> * Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
>>> 2 years ago is when they first released the enterprise edition,
>>> which according to [1] had "The most important new feature is that
>>> you can now add members to groups of projects."
>
>> It needed a lot more than a single feature.
>
> Just going to their primary web page, and noting that the first line gives
> links to "Features" and "Pricing" (but not "Documentation"), and the
> second line makes it clear that there's a big difference between the
> "Community Edition" and the "Enterprise Edition", is already enough to
> turn me off.  We've seen that model before (mysql...) and it doesn't bode
> well in the long run.
>
> Further poking shows no evidence of any decent email integration, to name
> just one of the things that have been mentioned as requirements in this
> thread.

That is a fair point. First steps into this direction are done with 
version 8.0. This was released a week ago.

> On the other hand, they are big on LDAP logins, and even
> two-factor authentication.  (We need this for an issue tracker that's
> supposed to provide visibility and easier reporting to people outside the
> project?)

Login methods are well supported. There are various login strategies 
supported.

> And JIRA integration, which seems to be an anti-feature to some
> on this thread.

It is not only JIRA. Jira is one of a long list. Many like the Jenkins 
integration to support CI for example.

> And they'd sure love to be in charge of our code repo.

Mh - i'm not a native speaker. I didn't understand this line.

> And the main, possibly only, metadata they can track about an issue is
> "assignee" and "milestone".

Indeed - GitLab is *not* a bugtracker. It's an web based git repository 
manager. It also offers issue tracking, but this is not the main idea of 
GitLab. Therefore i doubt that its the best choice for the community, too.

Greetings,
Torsten

On Tue, Sep 29, 2015 at 07:01:16AM -0700, Steve Crawford wrote:
> On Mon, Sep 28, 2015 at 4:41 PM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
> 
> > Note that since they also offer a hosted solution we should use that to
> > play with instead of trying to install it at this point.
> >
> > Integrating the issue tracker looks like it's just a call to this API:
> > http://doc.gitlab.com/ce/api/issues.html#new-issue. I don't normally do
> > web development myself so I'd rather not figuring out how to setup a copy
> > of the website to hack on, but if no one else wants to try it I can take a
> > stab at it.
> >
> > Presumably mirroring our git repository would work the same as it does for
> > mirroring to GitHub. My guess is that would be enough to get the basic
> > git/issue tracker integration working.
> >
> > Commitfest could be tied in as well. Presumably each commitfest would be a
> > milestone (http://doc.gitlab.com/ce/api/milestones.html) and each
> > submission an issue.
> >
> >
> One of the issues identified with  Github is that it is closed and
> commercial which goes against the expressed desires of the PostgreSQL
> community. As such, it's important to note that Gitlab seems to be in the
> "freemium" model with a "community" and an "enterprise" version so for
> comparison purposes we should only look at the features in the open-source
> community version:
> https://about.gitlab.com/features/#compare

Pardon me for getting off in the weeds here, but the PostgreSQL
community is just fine with proprietary closed-source software.  There
are probably more billion-dollar proprietary closed-source forks of
PostgreSQL than of any other open source software, certainly if you
normalize to the number of contributors.  We fully intend to continue
spawning those proprietary closed-source forks.

What we're not fine with is depending on a proprietary system, no
matter what type of license, as infrastructure.  We've been burned
that way before, and we have no intention of getting burned again that
way again.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On Wed, Sep 23, 2015 at 1:18 PM, Kam Lasater <ckl@seekayel.com> wrote:
> Hello,
>
> Last night I heard that Postgres had no issue/bug tracker. At first I
> thought the guy was trolling me. Seriously, how could this be. Certainly a
> mature open source project that is the database for a measurable percentage
> of the internet would have an issue tracker.
>
> Sadly, I was not being trolled. I'm new around here so I will keep the
> preaching to a minimum and cut right to the chase...
>
> ___It is time for an issue tracker___

This thread is depressing me.  We use all these fancy tools at $work
and I'm not sure they are much of an improvement over informal
processes run by capable people.   I regularly advocate, pretty much
to the wind, to use JIRA less and email *more*.   The main benefit of
the system, various reports to corporate taskmasters, seems pretty
much irrelevant here.  If you're advocating introduction of new
tooling with all the associated processes and complexities, can you
point to specific breakdowns in the project and exactly how said
tooling would have helped the situation?

merlin

On 09/29/2015 10:55 AM, Steve Crawford wrote:
> On Tue, Sep 29, 2015 at 7:16 AM, David Fetter <david@fetter.org 
> <mailto:david@fetter.org>> wrote:
>
>     ...What we're not fine with is depending on a proprietary system, no
>     matter what type of license, as infrastructure...
>
>
> Exactly. Which is why I was warning about latching onto features only 
> available in the closed enterprise version.
>
>
>

Like Tom, I more or less promised myself not to get terribly involved in 
this discussion. Oh, well.

I'm not a fan of the "free sample" model of software. What happens when 
you want a feature that's in the not-free edition of the software? I 
think gitlab simply doesn't suit us for a number of reasons, and that 
seems to be the emerging consensus.

The only viable possibilities seem to me to be bugzilla and debbugs. 
Both are dedicated trackers, unquestionably open source, have long 
pedigrees, are very likely to stay around, and are or can be integrated 
with email systems. I have not personally used debbugs, so I favour 
bugzilla simply on the ground of familiarity, but I know other people 
dislike it. I will tell a small story about it - about 14 years ago I 
was given responsibility for an extra team following a corporate merger. 
They had been using a proprietary bug tracker while we had been using 
bugzilla. We decided to switch them to bugzilla so they sould integrate 
with the tem from the company I had been working for, and they bitched 
and moaned about it something fierce. Later the company decided to 
standardize on the proprietary system, and the same people bitched and 
moaned far more loudly at being made to give up bugzilla, which they 
found much more friendly. And in those days it wasn't as nice or capable 
as it is now.

cheers

andrew

Corey Huinker <corey.huinker@gmail.com> writes:
>>> And they'd sure love to be in charge of our code repo.

>> Mh - i'm not a native speaker. I didn't understand this line.

> Tom was saying that the JIRA/Atlassian people would happily volunteer to
> host our code repository for no cost (in money) to us.

Not that so much as that the gitlab code really wants to be connected up
to our code repo.  That makes complete sense in terms of its goals as
stated by Torsten upthread, namely to be a git repo manager.  But it's
not what we're looking for here.  We want an issue tracker, and we don't
particularly need it to decide that it's in charge of repo access
authorization for instance.  There's a whole bunch of functionality there
that at best is useless to us, and at worst will get in the way.
        regards, tom lane

Seems to me that there are a bunch of agendas here.

I read about not wanting to be trapped into a proprietary system.  You 
can be trapped in any software you depend upon.  Compilers, Platforms, 
SCM, issue tracking are all places to be trapped.

Postgres and Postgresql have been around a very long time for the 
software world.  It has outlived several of the systems it has depended 
upon over those many years.  I hope and expect that Postgres will 
continue to outlive some of these platforms.

So do not get hung up on having been 'burned' in the past.  Expect to be 
'burned' again.  Take steps to minimize that pain in the future.

For an issue tracker, open source or proprietary, I would want raw 
database dumps and schema information.  Postgres is a database after 
all.  If you truly have the data, you should be able to migrate it.

Also, does the system you adopt use Postgres?  You are your best open 
source software.  If performance starts to go downhill, you are in the 
best position to fix it if you understand and control it.  How 
responsive will whatever system be to your changing needs?  If you 
partner with an external group, the two groups will benefit from each 
other if they are truly sharing the technologies.

Jeff Anton

On 2015-09-29 13:27:15 -0400, Tom Lane wrote:
> Corey Huinker <corey.huinker@gmail.com> writes:
> >>> And they'd sure love to be in charge of our code repo.
> 
> >> Mh - i'm not a native speaker. I didn't understand this line.
> 
> > Tom was saying that the JIRA/Atlassian people would happily volunteer to
> > host our code repository for no cost (in money) to us.
> 
> Not that so much as that the gitlab code really wants to be connected up
> to our code repo.  That makes complete sense in terms of its goals as
> stated by Torsten upthread, namely to be a git repo manager.  But it's
> not what we're looking for here.  We want an issue tracker, and we don't
> particularly need it to decide that it's in charge of repo access
> authorization for instance.  There's a whole bunch of functionality there
> that at best is useless to us, and at worst will get in the way.

I don't have any opinion WRT gitlab, but I'm fairly certain it'd be
unproblematic to configure automatic mirroring into it from
gitmaster.

It imo can make a fair amount of sense to give the bugtracker
et al access to the commits so it can automatically track mentions of
individual bugs and such.

Andres

Andres Freund <andres@anarazel.de> writes:
> On 2015-09-29 13:27:15 -0400, Tom Lane wrote:
>> Not that so much as that the gitlab code really wants to be connected up
>> to our code repo.  That makes complete sense in terms of its goals as
>> stated by Torsten upthread, namely to be a git repo manager.  But it's
>> not what we're looking for here.  We want an issue tracker, and we don't
>> particularly need it to decide that it's in charge of repo access
>> authorization for instance.  There's a whole bunch of functionality there
>> that at best is useless to us, and at worst will get in the way.

> I don't have any opinion WRT gitlab, but I'm fairly certain it'd be
> unproblematic to configure automatic mirroring into it from
> gitmaster.

I think you missed my point: gitlab would then believe it's in charge of,
eg, granting write access to that repo.  We could perhaps whack it over
the head till it only does what we want and not ten other things, but
we'd be swimming upstream.
        regards, tom lane

On 09/29/2015 07:25 AM, Merlin Moncure wrote:
> On Wed, Sep 23, 2015 at 1:18 PM, Kam Lasater <ckl@seekayel.com> wrote:
>> Hello,
>>
>> Last night I heard that Postgres had no issue/bug tracker. At first I
>> thought the guy was trolling me. Seriously, how could this be. Certainly a
>> mature open source project that is the database for a measurable percentage
>> of the internet would have an issue tracker.
>>
>> Sadly, I was not being trolled. I'm new around here so I will keep the
>> preaching to a minimum and cut right to the chase...
>>
>> ___It is time for an issue tracker___
>
> This thread is depressing me.  We use all these fancy tools at $work
> and I'm not sure they are much of an improvement over informal
> processes run by capable people.   I regularly advocate, pretty much
> to the wind, to use JIRA less and email *more*.   The main benefit of
> the system, various reports to corporate taskmasters, seems pretty
> much irrelevant here.  If you're advocating introduction of new
> tooling with all the associated processes and complexities, can you
> point to specific breakdowns in the project and exactly how said
> tooling would have helped the situation?
From my perspective this is about more than bugs it is about 
development as a whole. Here is a very simple problem we have:

I come to hackers to discuss an idea
idea gets sign off
I submit a patch
*I am told to go over to this commitfest app thing and upload it there*

Why? That's stupid. I should have submitted the patch to hackers and it 
should have been automatically part of my existing thread.

Someone reviews the patch, decides it needs to be pushed to the next 
commitfest

In theory, at some point I will be informed of that or I will see that 
it was pushed to the next commitfest.

If we were running a properly integrated tracker, I would know that 
instantly because the issue would have been updated to the affect and 
marked for the next commitfest.

The next commitfest comes around, and I can't get to the patch changes 
in time so it gets pushed to the next release. With a properly 
integrated issue tracker, I would immediately see that, be able to 
comment on it and be able to see the entire history of the patch.

Oh... and this can be done all from email as long as the tracker is 
properly integrated.

Bugs can certainly be handled the same way and in some ways better.

JD

P.S. I am not complaining about the commitfest process, I am making 
remarks about the tools we are using to manage it.


>
> merlin
>
>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 2015-09-29 13:40:28 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I don't have any opinion WRT gitlab, but I'm fairly certain it'd be
> > unproblematic to configure automatic mirroring into it from
> > gitmaster.
> 
> I think you missed my point: gitlab would then believe it's in charge of,
> eg, granting write access to that repo.  We could perhaps whack it over
> the head till it only does what we want and not ten other things, but
> we'd be swimming upstream.

We today already have a github mirror, where exactly the same thing
exists, no?

Andres

Andres Freund <andres@anarazel.de> writes:
> On 2015-09-29 13:40:28 -0400, Tom Lane wrote:
>> I think you missed my point: gitlab would then believe it's in charge of,
>> eg, granting write access to that repo.  We could perhaps whack it over
>> the head till it only does what we want and not ten other things, but
>> we'd be swimming upstream.

> We today already have a github mirror, where exactly the same thing
> exists, no?

Sure, there's a mirror out there somewhere.  It has nothing to do with
our core development processes.
        regards, tom lane

Oleg Bartunov wrote:
> me too, but I have to mention one problem we should have in mind - it's
> independency from political games (sanctions).  Think about developers from
> Russia, who one day may be blocked by Github, for example.

That's a very good point.  I think Github and other sites are already
blocked in countries like India and Cuba.  This becomes more pressing as
commercial entities are formed in countries like Russia.  Surely we do
not want PostgresPro developers to be unable to interact with our
bugtracker ...

We've done an excellent job of keeping our servers far away from any
individual jurisdictions, going back many years ago when Marc Fournier
decided to host our stuff in Panama for precisely this reason.  Nowadays
for us it is reasonably simple to move stuff around in case there's
trouble in any particular country.  I have a very hard time believing we
would tie ourselves down for a bug tracker; hosting whatever in our own
infrastructure is probably the only reasonable option for us at this
point.  As Tom said, lesser projects cannot afford this luxury, but
we're not giving that away in a jiffy.

IANAL

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Sep 29, 2015 at 12:42 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
> On 09/29/2015 07:25 AM, Merlin Moncure wrote:
>>
>> On Wed, Sep 23, 2015 at 1:18 PM, Kam Lasater <ckl@seekayel.com> wrote:
>>>
>>> Hello,
>>>
>>> Last night I heard that Postgres had no issue/bug tracker. At first I
>>> thought the guy was trolling me. Seriously, how could this be. Certainly
>>> a
>>> mature open source project that is the database for a measurable
>>> percentage
>>> of the internet would have an issue tracker.
>>>
>>> Sadly, I was not being trolled. I'm new around here so I will keep the
>>> preaching to a minimum and cut right to the chase...
>>>
>>> ___It is time for an issue tracker___
>>
>>
>> This thread is depressing me.  We use all these fancy tools at $work
>> and I'm not sure they are much of an improvement over informal
>> processes run by capable people.   I regularly advocate, pretty much
>> to the wind, to use JIRA less and email *more*.   The main benefit of
>> the system, various reports to corporate taskmasters, seems pretty
>> much irrelevant here.  If you're advocating introduction of new
>> tooling with all the associated processes and complexities, can you
>> point to specific breakdowns in the project and exactly how said
>> tooling would have helped the situation?
>
>
> From my perspective this is about more than bugs it is about development as
> a whole. Here is a very simple problem we have:
>
> I come to hackers to discuss an idea
> idea gets sign off
> I submit a patch
> *I am told to go over to this commitfest app thing and upload it there*
>
> Why? That's stupid. I should have submitted the patch to hackers and it
> should have been automatically part of my existing thread.
>
> Someone reviews the patch, decides it needs to be pushed to the next
> commitfest
>
> In theory, at some point I will be informed of that or I will see that it
> was pushed to the next commitfest.
>
> If we were running a properly integrated tracker, I would know that
> instantly because the issue would have been updated to the affect and marked
> for the next commitfest.
>
> The next commitfest comes around, and I can't get to the patch changes in
> time so it gets pushed to the next release. With a properly integrated issue
> tracker, I would immediately see that, be able to comment on it and be able
> to see the entire history of the patch.
>
> Oh... and this can be done all from email as long as the tracker is properly
> integrated.
>
> Bugs can certainly be handled the same way and in some ways better.

I've read this email about three times now and it's not clear at all
to me what a issue/bug tracker brings to the table.  First, I find it
pretty hard to believe that a hypothetical patch author would not know
that a patch was or was not submitted in the current framework.
Putting that aside, if you insisted an automatic status change
notifications, that could be worked into the current commit fest
application, right? Note, once done, you'd get that notification with
zero extra process effort beyond what we currently do.

I also openly wonder how big this problem really is.  I haven't
submitted all that many patches, but for those I have it's never
really been in doubt as to what status they've were in at the time.
So, I'll repeat the question: if you want to accomplish with this
thread, let's give *specific examples* of project breakdowns we're
trying to solve with this tooling.  Did a user lose status of a patch?
If so, which user?...what patch?

merlin

Merlin Moncure wrote:

> I've read this email about three times now and it's not clear at all
> to me what a issue/bug tracker brings to the table.

In my opinion, this thread is about a bug tracker, not a patch tracker.
We already have a patch tracking system which works very well.  (We are
not able to process all the things we get, but that's a problem of
workload, not tooling).

Let's not mix things, as that only adds to the confusion.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 09/29/2015 03:08 PM, Merlin Moncure wrote:
> I've read this email about three times now and it's not clear at all
> to me what a issue/bug tracker brings to the table.

Here are the problems I'd like to solve:

1. "Was this issue fixed in a Postgres update?  Which one?"

2. Not losing track of minor bugs.

3. Having a better way to track bugs which require multi-part solutions
(e.g. multixact).

4. Having a place for downstream projects/packagers to report bugs.

5. Not answering this question ever again: "Why doesn't your project
have a bug tracker?"

Note that all of the above requires a bug *tracker*, that is, a tool
which tracks the bug activity which was happening anyway, just makes it
more visible.  Rather than an Issue Resolution System, which would be
intended to remake our workflow.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Merlin Moncure wrote:
>> I've read this email about three times now and it's not clear at all
>> to me what a issue/bug tracker brings to the table.

> In my opinion, this thread is about a bug tracker, not a patch tracker.
> We already have a patch tracking system which works very well.

Mumble ... the CF app works, but I'm not sure if it works "very well".
There's some ease-of-use issues I think, though we've now damped them
down to the point where the only really major stumbling block is getting
a patch into the CF app initially.

One thing to think about if we do adopt a bug tracker is how will it
interact with the CF app.  Ideally, patches that represent responses
to issues in the BT would be tracked more or less automatically by
both apps, if indeed we don't merge them altogether.
        regards, tom lane

On 09/29/2015 03:46 PM, Tom Lane wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>> Merlin Moncure wrote:
>>> I've read this email about three times now and it's not clear at all
>>> to me what a issue/bug tracker brings to the table.
>
>> In my opinion, this thread is about a bug tracker, not a patch tracker.
>> We already have a patch tracking system which works very well.
>
> Mumble ... the CF app works, but I'm not sure if it works "very well".
> There's some ease-of-use issues I think, though we've now damped them
> down to the point where the only really major stumbling block is getting
> a patch into the CF app initially.
>
> One thing to think about if we do adopt a bug tracker is how will it
> interact with the CF app.  Ideally, patches that represent responses
> to issues in the BT would be tracked more or less automatically by
> both apps, if indeed we don't merge them altogether.

That was kind of my point, although I obviously am not making it clear. 
The idea that we need just a bug tracker is flawed and narrow in 
thinking. We want something that will integrate into our entire 
development work flow with minimal disruption. Otherwise we are just 
building a lot of infrastructure in different places for no reason. 
Properly configured an issue tracker would service our existing work 
flow including enhancing the commitfest process as well as helping us 
track bugs.

JD


>
>             regards, tom lane
>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 9/29/15 3:36 PM, Oleg Bartunov wrote:
>              ...What we're not fine with is depending on a proprietary
>         system, no
>              matter what type of license, as infrastructure...
>
>
>         Exactly. Which is why I was warning about latching onto features
>         only available in the closed enterprise version.
>
>     Like Tom, I more or less promised myself not to get terribly
>     involved in this discussion. Oh, well.
>
>
> me too, but I have to mention one problem we should have in mind - it's
> independency from political games (sanctions).  Think about developers
> from Russia, who one day may be blocked by Github, for example.

No one is suggesting we depend on proprietary or closed anything.

Community GitLab is NOT a "free sample". There are literally hundreds[1] 
of people that have submitted code to it.

The only thing I did suggest is that the easiest way to kick the tires 
on it would probably be to just plug into their cloud service. If people 
like it then we'd run our own.

I wish people would at least consider this as an option because it 
integrates a ton of different features together. It has *the potential* 
to eliminate our need to keep maintaining CommitFest and buildfarm and 
could also replace mediawiki.

If people are hell-bent on every tool being separate then fine, but I 
get the distinct impression that everyone is discarding GitLab out of 
hand based on completely bogus information.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 9/29/15 12:46 PM, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2015-09-29 13:40:28 -0400, Tom Lane wrote:
>>> I think you missed my point: gitlab would then believe it's in charge of,
>>> eg, granting write access to that repo.  We could perhaps whack it over
>>> the head till it only does what we want and not ten other things, but
>>> we'd be swimming upstream.
>
>> We today already have a github mirror, where exactly the same thing
>> exists, no?
>
> Sure, there's a mirror out there somewhere.  It has nothing to do with
> our core development processes.

There are APIs as well, so it could be driven that way. I suspect it's 
unnecessary though.

BTW, the docs are at http://doc.gitlab.com/ce/.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

Pavan Deolasee wrote:
> On Wed, Sep 30, 2015 at 2:16 AM, Alvaro Herrera <alvherre@2ndquadrant.com>
> wrote:
> 
> > That's a very good point.  I think Github and other sites are already
> > blocked in countries like India and Cuba.
> 
> Github is not blocked in India and was never as far as I know. Well our
> government recently blocked some porn sites, but (thankfully) they went
> only that far.

This is what I remember:
http://www.zdnet.com/article/india-blocks-32-websites-including-github-internet-archive-pastebin-vimeo/

> But I see Oleg's point. Many things including Google are
> blocked in China for example.

Right.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Sep 29, 2015 at 6:39 PM, Josh Berkus <josh@agliodbs.com> wrote:
> On 09/29/2015 03:08 PM, Merlin Moncure wrote:
>> I've read this email about three times now and it's not clear at all
>> to me what a issue/bug tracker brings to the table.
>
> Here are the problems I'd like to solve:
>
> 1. "Was this issue fixed in a Postgres update?  Which one?"
>
> 2. Not losing track of minor bugs.
>
> 3. Having a better way to track bugs which require multi-part solutions
> (e.g. multixact).
>
> 4. Having a place for downstream projects/packagers to report bugs.
>
> 5. Not answering this question ever again: "Why doesn't your project
> have a bug tracker?"

Merlin,

I'm not sure if you are trolling me/us. I'm going to assume not and
interpret the comment from the prospective of: "the current process
works for those currently using the process"

That may be true (I'll leave it to someone more familiar with the
process to address). What that comment doesn't address is the needs of
those who are not currently involved or those who are not on this
email list. Just as "read the code" is an insufficient answer to a
user who is looking to use a feature, "read the mailing list" is an
insufficient answer to a query from a user about the state of bugs
past and present.

Given that, in addition to Josh's five points from an insider's
perspective I would add some from an outsider's perspective:

1/ Is the issue I'm having a known bug that can be fixed by an upgrade
to a more recent version, if so, which one?

2/ This project must be disorganized and/or not truly mature w/o a
central tracker

3/ No hints or help on what might be an easier place to start contributing

Hope that helps.

-Kam Lasater
@seekayel

On Tue, Sep 29, 2015 at 12:08:56PM +1300, Gavin Flower wrote:
> Linux kernel project uses bugzilla (https://bugzilla.kernel.org)

AIUI this is not mandatory for kernel hackers, and more opt-in from a
some/many/a few(?) subsystem maintainers.  Some parts use it more, some
less or not at all.

> and so does LibreOffice (https://bugs.documentfoundation.org)

Thas is true, however. 

Same for freedesktop.org and the Gnome project, I believe.


Michael

On Wed, Sep 30, 2015 at 7:17 AM, Kam Lasater <ckl@seekayel.com> wrote:
> On Tue, Sep 29, 2015 at 6:39 PM, Josh Berkus <josh@agliodbs.com> wrote:
>> On 09/29/2015 03:08 PM, Merlin Moncure wrote:
>>> I've read this email about three times now and it's not clear at all
>>> to me what a issue/bug tracker brings to the table.
>>
>> Here are the problems I'd like to solve:
>>
>> 1. "Was this issue fixed in a Postgres update?  Which one?"
>>
>> 2. Not losing track of minor bugs.
>>
>> 3. Having a better way to track bugs which require multi-part solutions
>> (e.g. multixact).
>>
>> 4. Having a place for downstream projects/packagers to report bugs.
>>
>> 5. Not answering this question ever again: "Why doesn't your project
>> have a bug tracker?"
>
> I'm not sure if you are trolling me/us. I'm going to assume not and
> interpret the comment from the prospective of: "the current process
> works for those currently using the process"
>
> That may be true (I'll leave it to someone more familiar with the
> process to address). What that comment doesn't address is the needs of
> those who are not currently involved or those who are not on this
> email list. Just as "read the code" is an insufficient answer to a
> user who is looking to use a feature, "read the mailing list" is an
> insufficient answer to a query from a user about the state of bugs
> past and present.
>
> Given that, in addition to Josh's five points from an insider's
> perspective I would add some from an outsider's perspective:
>
> 1/ Is the issue I'm having a known bug that can be fixed by an upgrade
> to a more recent version, if so, which one?
>
> 2/ This project must be disorganized and/or not truly mature w/o a
> central tracker
>
> 3/ No hints or help on what might be an easier place to start contributing

I'm not trolling in any way.  I'm just challenging you to back up your
blanket assertions with evidence.  For example, you're assertion that
mailing lists are insufficient is simply stated and expected to be
taken on faith: *How* is it insufficient and *what* do things like in
the new world?  Be specific: glossing over these details doesn't
really accomplish anything and avoids the careful examination that may
suggest small tweaks to the current processes that could get similar
results with a lot less effort.  In this entire massive thread, so far
only Josh has come up with what I'd consider to be actionable problem
cases.

Josh's point, "2. Not losing track of minor bugs." is an example of
what's bugging (pun intended) me.  Do you think issues don't get lost
in issue trackers?  They absolutely can, and do, even (where I work)
with a full time paid PM who spends her entire day in JIRA managing
this stuff.  Sure, you can do things like run aging reports and all
that but that immediately suggests the questions, 'who is running the
report?'  and 'what are the expected outputs of that report?'.  So I'm
putting it back on you: what "minor bugs" have been missed, and please
clearly explain how an issue tracker would improve the situation with
a little more detail than "Issue tracker, therefore, better".  This
would allow for objective examination of how things might look after
making major process changes.

As I noted upthread google is incredibly efficient at tying up a
observed issue with the relevant fix and commentary, all based on
search engine integration with the mailing list that you've summarily
dismissed (without any evidence whatsoever) as ineffective.  You're
just assuming it's better without any examination of the costs
involved.  For example, implementation and training aside, I expect
one of the immediate downsides of moving away from google + mailing
list is that you're going to be suffering from a deluge of duplicate
issues.

Note, I'm not picking on Josh here.   The points pertaining to
querying issues are certainly better than wading through the release
notes which I've always felt to be kind of a pain.  What I'm driving
at is that you should identify actual pain points in the process and
explain clearly how things would improve.  Also, consider low impact
solutions first (for example what low tech method makes bug
identification to release easier?) and move into a big tooling change
only after discarding them.

merlin

On 09/30/2015 07:44 AM, Merlin Moncure wrote:

> I'm not trolling in any way.  I'm just challenging you to back up your
> blanket assertions with evidence.  For example, you're assertion that
> mailing lists are insufficient is simply stated and expected to be
> taken on faith: *How* is it insufficient and *what* do things like in
> the new world?

I am short on time today but I will take this specific one:

Mailing lists are great for discourse however they do not:

1. Provide easy access to archived informationSearching google isn't an answer it is a band-aid
2. Provide proper access to valid informationEver get an answer, check the link, find out the solution references a 
5 year old version of PostgreSQL and then find out the problem is fixed 
in the 9.4 but not 9.3. You are running 9.3.(an issue tracker could track this, easily)
3. Provide properly linked information across threadsMy favourite is this:    SUBJECT: Help (was no longer wanting
help)Nownothing makes sense on the thread. It should be a new issue.
 
4. Using a recent submission as an example:        josh@idealist.org just submitted 6 patches. They are all based 
around making basebackups more useful (specifically pg_basebackup). This 
is awesome, but he has created 6 different threads with different 
discussions which will likely cause intercommunication between threads.
Using an issue tracker the first patch would be a parent issue and the 
subsequent patches would be child issues (that whole dependency thing). 
A single click would provide all the information required to correctly 
determine what is going on with the series of interrelated patches. A 
mailing list does not provide that.

I could go on for a long time with specific examples that our current 
model does not serve.

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/30/2015 12:02 AM, Jim Nasby wrote:

> If people are hell-bent on every tool being separate then fine, but I
> get the distinct impression that everyone is discarding GitLab out of
> hand based on completely bogus information.

Right, we need to stop thinking that every task is not interrelated. 
They all are. Although I am not a big fan of the gitlab idea but that is 
more out of ignorance of the software/service than anything else. My 
core focus on this discussion is to educate the -hackers that don't 
understand that all of this is related and to have a bug tracker, and a 
separate commitfest app, and a isolated git server that doesn't interact 
with any of them except through a commit message is broken.

If we can come to a solution that properly links the processes together 
(without outright throwing them out the window), that is the best 
solution. A "bug" tracker doesn't do that. It just adds another piece. 
An issue tracker (as everything including this discussion is an issue) 
works because an issue can be classified and tracked for its purpose.

JD





-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/30/2015 07:44 AM, Merlin Moncure wrote:
> I'm not trolling in any way.  I'm just challenging you to back up your
> blanket assertions with evidence.  For example, you're assertion that
> mailing lists are insufficient is simply stated and expected to be
> taken on faith: *How* is it insufficient and *what* do things like in
> the new world?  Be specific: glossing over these details doesn't
> really accomplish anything and avoids the careful examination that may
> suggest small tweaks to the current processes that could get similar
> results with a lot less effort.  In this entire massive thread, so far
> only Josh has come up with what I'd consider to be actionable problem
> cases.

I don't see any way to make small tweaks to the existing process which
would fix any of these problems.  I think if that were possible, we'd
already have done it.  Suggestions welcome, of course.

For example, "just use the wiki for this" has been mentioned as an
alternative.  But we've tried "just using the wiki" for a number of
things, and it doesn't really work.  For example, using the wiki as a
way of breaking down the various multixact issues manifestly didn't
work.  A big part of the problem there is that there's no good way for
the wiki to notify people when there's been an update; a smaller part is
that the formatting gets messed up and impossible to follow.

> Josh's point, "2. Not losing track of minor bugs." is an example of
> what's bugging (pun intended) me.  Do you think issues don't get lost
> in issue trackers? 

More accurately: losing track of *fewer* minor bugs.

> As I noted upthread google is incredibly efficient at tying up a
> observed issue with the relevant fix and commentary, all based on
> search engine integration with the mailing list that you've summarily
> dismissed (without any evidence whatsoever) as ineffective.

In my experience it has not been effective.  Generally when a client
asks me the question about which release a particular bug is fixed in,
it takes me 15-30 minutes to determine the answer using
google/list/commitlog.  The client would not be able to determine it for
themselves at all.  While I appreciate the billable hours, it doesn't
seem like a good use of the customer's money, you know?

> You're
> just assuming it's better without any examination of the costs
> involved.  For example, implementation and training aside, I expect
> one of the immediate downsides of moving away from google + mailing
> list is that you're going to be suffering from a deluge of duplicate
> issues.

As opposed to duplicate emails, which we already get?

> Note, I'm not picking on Josh here.   The points pertaining to
> querying issues are certainly better than wading through the release
> notes which I've always felt to be kind of a pain.  What I'm driving
> at is that you should identify actual pain points in the process and
> explain clearly how things would improve.  Also, consider low impact
> solutions first (for example what low tech method makes bug
> identification to release easier?) and move into a big tooling change
> only after discarding them.

Well, if you have suggestions that don't involve an email-driven bug
tracker, please make them.  I don't have any other ideas.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 09/30/2015 01:31 PM, Joshua D. Drake wrote:
> On 09/30/2015 12:02 AM, Jim Nasby wrote:
>
>> If people are hell-bent on every tool being separate then fine, but I
>> get the distinct impression that everyone is discarding GitLab out of
>> hand based on completely bogus information.
>
> Right, we need to stop thinking that every task is not interrelated. 
> They all are. Although I am not a big fan of the gitlab idea but that 
> is more out of ignorance of the software/service than anything else. 
> My core focus on this discussion is to educate the -hackers that don't 
> understand that all of this is related and to have a bug tracker, and 
> a separate commitfest app, and a isolated git server that doesn't 
> interact with any of them except through a commit message is broken.
>
> If we can come to a solution that properly links the processes 
> together (without outright throwing them out the window), that is the 
> best solution. A "bug" tracker doesn't do that. It just adds another 
> piece. An issue tracker (as everything including this discussion is an 
> issue) works because an issue can be classified and tracked for its 
> purpose.
>
>


Frankly, an insistence on moving to some integrated solution is likely 
to result in the adoption of nothing. And your "educating hackers who 
don't understand" is more than a little patronizing. What makes you 
think your experience in software development is better than others'?

cheers

andrew




>
>
>
>

On 09/30/2015 10:45 AM, Andrew Dunstan wrote:

> Frankly, an insistence on moving to some integrated solution is likely
> to result in the adoption of nothing. And your "educating hackers who
> don't understand" is more than a little patronizing. What makes you
> think your experience in software development is better than others'?

I wasn't being patronizing. I was stating a point. Is this discussion 
not educational (whether you agree with the points or not) or are you 
suggesting that somehow you already know everything?

That, was patronizing.

I believe myself (and someone like Nasby) do have a better core 
understanding of development because development is more than some guy 
sitting in a den pushing out code. The community already recognizes 
this, that is why we have the facilities we do now. All I (and others) 
am suggesting is that we make the facilities we have now, work better.

I would also iterate that my entire argument has been to essentially 
update/upgrade our workflow, not replace it. Filling the gaps as it were.

Sincerely,

jD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/30/2015 11:23 AM, Christopher Browne wrote:

> It's well and nice to think that an issue tracker resolves all of this,
> and, if we
> had tiny numbers of issues, we could doubtless construct a repository
> indicating so.  (Seems to me that the bit of "fan service" for GitHub's
> bug tracker fits into that perspective on things...)

CMD has over a 1000 customers. All of those that are active have a 
Redmine tracker. Our current ticket count is over 70k. Without it, we 
would never be able to service them correctly.

What you describe is not a tool problem, it is a people problem. That 
exists regardless of the tool. The tool is designed to (in theory) make 
the people problem, less.

In CMDs considerable experience while not only developing, consulting, 
writing docs, maintain repos, and confidential information... it would 
be impossible to achieve without Redmine at our core.

JD

(I am sure other tools provide the same level of service, it is just 
what we use)



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/30/2015 02:16 PM, Joshua D. Drake wrote:
> On 09/30/2015 10:45 AM, Andrew Dunstan wrote:
>
>> Frankly, an insistence on moving to some integrated solution is likely
>> to result in the adoption of nothing. And your "educating hackers who
>> don't understand" is more than a little patronizing. What makes you
>> think your experience in software development is better than others'?
>
> I wasn't being patronizing. I was stating a point. Is this discussion 
> not educational (whether you agree with the points or not) or are you 
> suggesting that somehow you already know everything?
>
> That, was patronizing.


Nonsense. There is a big difference between "stating a view" and 
"educating people who don't understand." The latter implies that your 
experience is to be preferred to theirs.

>
> I believe myself (and someone like Nasby) do have a better core 
> understanding of development because development is more than some guy 
> sitting in a den pushing out code. The community already recognizes 
> this, that is why we have the facilities we do now. All I (and others) 
> am suggesting is that we make the facilities we have now, work better.


Who exactly is "some guy sitting in a den pushing out code"? And if 
that's not a patronizing put down I don't know what is. If you're 
referring to me you couldn't be more wrong. I have been a development 
director managing a couple of substantial teams, as well as having 
worked in numerous commercial environments.


cheers

andrew

On 09/30/2015 11:33 AM, Andrew Dunstan wrote:

>
> Who exactly is "some guy sitting in a den pushing out code"? And if
> that's not a patronizing put down I don't know what is. If you're
> referring to me you couldn't be more wrong. I have been a development
> director managing a couple of substantial teams, as well as having
> worked in numerous commercial environments.

If I was referencing you, I would say so.

However, all that is off topic to the point of this thread

JD




-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 09/30/2015 12:02 AM, Jim Nasby wrote:
> I wish people would at least consider this as an option because it
> integrates a ton of different features together. It has *the potential*
> to eliminate our need to keep maintaining CommitFest and buildfarm and
> could also replace mediawiki.
> 
> If people are hell-bent on every tool being separate then fine, but I
> get the distinct impression that everyone is discarding GitLab out of
> hand based on completely bogus information.

Well, Gitlab was introduced into this dicussion in the context of being
an OSS version of Github Issues.  If it's more than that, you're going
to have to explain.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Wed, Sep 30, 2015 at 12:43 PM, Josh Berkus <josh@agliodbs.com> wrote:
> On 09/30/2015 07:44 AM, Merlin Moncure wrote:
>> I'm not trolling in any way.  I'm just challenging you to back up your
>> blanket assertions with evidence.  For example, you're assertion that
>> mailing lists are insufficient is simply stated and expected to be
>> taken on faith: *How* is it insufficient and *what* do things like in
>> the new world?  Be specific: glossing over these details doesn't
>> really accomplish anything and avoids the careful examination that may
>> suggest small tweaks to the current processes that could get similar
>> results with a lot less effort.  In this entire massive thread, so far
>> only Josh has come up with what I'd consider to be actionable problem
>> cases.
>
> I don't see any way to make small tweaks to the existing process which
> would fix any of these problems.  I think if that were possible, we'd
> already have done it.  Suggestions welcome, of course.
>
> For example, "just use the wiki for this" has been mentioned as an
> alternative.  But we've tried "just using the wiki" for a number of
> things, and it doesn't really work.  For example, using the wiki as a
> way of breaking down the various multixact issues manifestly didn't
> work.  A big part of the problem there is that there's no good way for
> the wiki to notify people when there's been an update; a smaller part is
> that the formatting gets messed up and impossible to follow.

Agree that wiki are mainly suited for documentation.  But they can
also be used to prototype new processes in a hurry or at least sketch
them out.  This is BTW how the commit fest application spawned up more
or less (which I happen to think works quite well).

But this brings up a couple of points and more questions:
*) Every wiki software I've seen, including mediawiki, allows
subscription to pages for changes.  This is pretty much the same model
most issue trackers use BTW except maybe the automatic subscription
rules are a little different.

*) Given that, I'm not sure that issue trackers are really an
improvement on that point.  In fact, fragmentation of communication is
a central complaint I have with them, including JIRA since people have
to subscribe and/or search for things of interest.  Of course, you can
configure them to just always send an email upon every response, but
then I'm thinking, 'why not just use email?'.  Also, I find the
suggestion that any $tool has a better search facility better than
google's to be laughable, except for very structured searches like
'issue type X by version Y'.

*) I only followed the multi-xact saga very loosely, except to see a
lot of angst for a significant bug (or, set of bugs) slipping out.
Are we sure that this problem didn't in fact stem from insufficient
review and/or testing?  And if not, how did using the wiki and/or not
having individual subtasks run through a tracker contribute to the
problem or its handling?  In other words, what does "manifestly didn't
work" mean?

I guess I'm yammering on here, but between us I'd suck the honey out
of an active beehive to have a day job dev process that more closely
resembles what this project uses, and I frequently exclaim so to
whomever is unfortunate enough to walk by.  But the tool brandishing
zealots have taken over, and I spend a non-trivial of the day filling
out various forms and re-explaining things over and over  (and the
rules are even then much relaxed for me, because I'm, you know,
"special").

My take is that this project is pretty well run and has taught me the
credo, "people and processes first, tools second".  The gripes raised
so far seem awfully vague for the most part, TBH.

merlin

Christopher Browne <cbbrowne@gmail.com> writes:
> It may very well be *worse* than that; it seems quite likely to me that if
> an issue tracker is not being continually curated by substantially ALL of
> its users, then you don't get any of those things.  That *is* a lot more
> pessimistic, and considerably likely, as it's pretty certain that members
> of our email-loving community will decline to get involved in curating
> data in some web app.

I think this is really the issue that's being studiously ignored by a
number of participants in this thread.  Simply installing a tracker
accomplishes diddly-squat.  Getting to a point where the information in
the tracker is actually valid, well-organized, etc. will require a LARGE
amount of real work, both up-front and on a continuing basis, and I do not
see where that effort is going to come from.  Anybody who thinks it's just
going to happen is living in a dream world.  Anybody who thinks they can
tell other people to do it, and then it will happen, is living in an
entire fantasy universe (unless, perhaps, they are paying said people to
do what they want).

I'd be feeling a lot more positive about this whole thread if any people
had stepped up and said "yes, *I* will put in a lot of grunt-work to make
something happen here".  The lack of any volunteers suggests strongly
that this thread is a waste of time, just as the several similar ones
before it have been.
        regards, tom lane

On 09/30/2015 03:10 PM, Tom Lane wrote:
> I'd be feeling a lot more positive about this whole thread if any people
> had stepped up and said "yes, *I* will put in a lot of grunt-work to make
> something happen here".  The lack of any volunteers suggests strongly
> that this thread is a waste of time, just as the several similar ones
> before it have been.

Hmmm?  Frost volunteered to stand up debbugs.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

Josh Berkus <josh@agliodbs.com> writes:
> On 09/30/2015 03:10 PM, Tom Lane wrote:
>> I'd be feeling a lot more positive about this whole thread if any people
>> had stepped up and said "yes, *I* will put in a lot of grunt-work to make
>> something happen here".  The lack of any volunteers suggests strongly
>> that this thread is a waste of time, just as the several similar ones
>> before it have been.

> Hmmm?  Frost volunteered to stand up debbugs.

So he did, and did anyone volunteer to put data into it, or to do ongoing
curation of said data?  If we simply connect it up to the mailing lists,
and then stand back and wait for magic to happen, we will not ever have
anything that's any more useful than the existing mailing list archives.
        regards, tom lane

On 09/30/2015 03:27 PM, Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
>> On 09/30/2015 03:10 PM, Tom Lane wrote:
>>> I'd be feeling a lot more positive about this whole thread if any people
>>> had stepped up and said "yes, *I* will put in a lot of grunt-work to make
>>> something happen here".  The lack of any volunteers suggests strongly
>>> that this thread is a waste of time, just as the several similar ones
>>> before it have been.
> 
>> Hmmm?  Frost volunteered to stand up debbugs.
> 
> So he did, and did anyone volunteer to put data into it, or to do ongoing
> curation of said data?  If we simply connect it up to the mailing lists,
> and then stand back and wait for magic to happen, we will not ever have
> anything that's any more useful than the existing mailing list archives.

Well, it's hard for anyone to volunteer when we don't know what the
actual volunteer tasks are.  I certainly intend to do *something* to
support the bug tracker system, but I don't know yet what that something is.


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 09/30/2015 03:28 PM, Josh Berkus wrote:
> On 09/30/2015 03:27 PM, Tom Lane wrote:
>> Josh Berkus <josh@agliodbs.com> writes:
>>> On 09/30/2015 03:10 PM, Tom Lane wrote:
>>>> I'd be feeling a lot more positive about this whole thread if any people
>>>> had stepped up and said "yes, *I* will put in a lot of grunt-work to make
>>>> something happen here".  The lack of any volunteers suggests strongly
>>>> that this thread is a waste of time, just as the several similar ones
>>>> before it have been.
>>
>>> Hmmm?  Frost volunteered to stand up debbugs.
>>
>> So he did, and did anyone volunteer to put data into it, or to do ongoing
>> curation of said data?  If we simply connect it up to the mailing lists,
>> and then stand back and wait for magic to happen, we will not ever have
>> anything that's any more useful than the existing mailing list archives.
>
> Well, it's hard for anyone to volunteer when we don't know what the
> actual volunteer tasks are.  I certainly intend to do *something* to
> support the bug tracker system, but I don't know yet what that something is.

Yep. Same here. I would be willing to help (as well as assign resources 
to it) if I knew what "it" was. We certainly would be more likely to 
help if it was on Redmine (due to expertise).

JD


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

Josh Berkus wrote:

> Well, it's hard for anyone to volunteer when we don't know what the
> actual volunteer tasks are.  I certainly intend to do *something* to
> support the bug tracker system, but I don't know yet what that something is.

I volunteer to do something too, as long as I don't have to click on
endless web forms.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 09/30/2015 03:49 PM, Alvaro Herrera wrote:
> Josh Berkus wrote:
>
>> Well, it's hard for anyone to volunteer when we don't know what the
>> actual volunteer tasks are.  I certainly intend to do *something* to
>> support the bug tracker system, but I don't know yet what that something is.
>
> I volunteer to do something too, as long as I don't have to click on
> endless web forms.

I think we all agree (YAY!) that whatever solution that is chosen must 
adhere to the idea of issue management via email.

JD

>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Thu, Oct 1, 2015 at 7:10 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Christopher Browne <cbbrowne@gmail.com> writes:
>> It may very well be *worse* than that; it seems quite likely to me that if
>> an issue tracker is not being continually curated by substantially ALL of
>> its users, then you don't get any of those things.  That *is* a lot more
>> pessimistic, and considerably likely, as it's pretty certain that members
>> of our email-loving community will decline to get involved in curating
>> data in some web app.
>
> I think this is really the issue that's being studiously ignored by a
> number of participants in this thread.  Simply installing a tracker
> accomplishes diddly-squat.  Getting to a point where the information in
> the tracker is actually valid, well-organized, etc. will require a LARGE
> amount of real work, both up-front and on a continuing basis, and I do not
> see where that effort is going to come from.  Anybody who thinks it's just
> going to happen is living in a dream world.  Anybody who thinks they can
> tell other people to do it, and then it will happen, is living in an
> entire fantasy universe (unless, perhaps, they are paying said people to
> do what they want).
> I'd be feeling a lot more positive about this whole thread if any people
> had stepped up and said "yes, *I* will put in a lot of grunt-work to make
> something happen here".  The lack of any volunteers suggests strongly
> that this thread is a waste of time, just as the several similar ones
> before it have been.

Amen. +1.
-- 
Michael

On 9/30/15 4:31 PM, Josh Berkus wrote:
> On 09/30/2015 12:02 AM, Jim Nasby wrote:
>> I wish people would at least consider this as an option because it
>> integrates a ton of different features together. It has *the potential*
>> to eliminate our need to keep maintaining CommitFest and buildfarm and
>> could also replace mediawiki.
>>
>> If people are hell-bent on every tool being separate then fine, but I
>> get the distinct impression that everyone is discarding GitLab out of
>> hand based on completely bogus information.
>
> Well, Gitlab was introduced into this dicussion in the context of being
> an OSS version of Github Issues.  If it's more than that, you're going
> to have to explain.

It started as an OSS version of *Github* (not just Github Issues). It 
appears to have all the major features of github (issues, code review, 
wiki) plus a CI framework.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 01.10.2015 00:27, Tom Lane wrote:
> Josh Berkus <josh@agliodbs.com> writes:
>> On 09/30/2015 03:10 PM, Tom Lane wrote:
>>> I'd be feeling a lot more positive about this whole thread if any people
>>> had stepped up and said "yes, *I* will put in a lot of grunt-work to make
>>> something happen here".  The lack of any volunteers suggests strongly
>>> that this thread is a waste of time, just as the several similar ones
>>> before it have been.
>
>> Hmmm?  Frost volunteered to stand up debbugs.
>
> So he did, and did anyone volunteer to put data into it, or to do ongoing
> curation of said data?

Yes, i did.

Greetings,
Torsten

<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Oct 1, 2015 at 9:50 AM, Torsten Zuehlsdorff
<spandir="ltr"><<a href="mailto:mailinglists@toco-domains.de"
target="_blank">mailinglists@toco-domains.de</a>></span>wrote:<br /><blockquote class="gmail_quote"
style="margin:0px0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class=""> On 01.10.2015
00:27,Tom Lane wrote:<br /><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Josh Berkus <<a href="mailto:josh@agliodbs.com"
target="_blank">josh@agliodbs.com</a>>writes:<br /><blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1pxsolid rgb(204,204,204);padding-left:1ex"> On 09/30/2015 03:10 PM, Tom Lane wrote:<br /><blockquote
class="gmail_quote"style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> I'd be
feelinga lot more positive about this whole thread if any people<br /> had stepped up and said "yes, *I* will put in a
lotof grunt-work to make<br /> something happen here". <br /></blockquote></blockquote><br /><blockquote
class="gmail_quote"style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> Hmmm? 
Frostvolunteered to stand up debbugs.<br /></blockquote><br /> So he did, and did anyone volunteer to put data into it,
orto do ongoing<br /> curation of said data?<br /></blockquote><br /></span> Yes, i did.<br /><br
/></blockquote></div><br/>I am also volunteering for this kind of <span class="">grunt-work.<br /><br
/></span></div><divclass="gmail_extra"><span class="">Greetings,<br /></span></div><div class="gmail_extra"><span
class="">Félix<br/></span></div></div> 

> On 09/30/2015 03:27 PM, Tom Lane wrote:
>> Josh Berkus <josh(at)agliodbs(dot)com> writes:
>>> On 09/30/2015 03:10 PM, Tom Lane wrote:
>>>> I'd be feeling a lot more positive about this whole thread if any people
>>>> had stepped up and said "yes, *I* will put in a lot of grunt-work to make
>>>> something happen here".  The lack of any volunteers suggests strongly
>>>> that this thread is a waste of time, just as the several similar ones
>>>> before it have been.
>> 
>>> Hmmm?  Frost volunteered to stand up debbugs.
>> 
>> So he did, and did anyone volunteer to put data into it, or to do ongoing
>> curation of said data?  If we simply connect it up to the mailing lists,
>> and then stand back and wait for magic to happen, we will not ever have
>> anything that's any more useful than the existing mailing list archives.

On 10/01/2015 01:49 AM, Alvaro Herrera wrote:
> Josh Berkus wrote:
> 
>> Well, it's hard for anyone to volunteer when we don't know what the
>> actual volunteer tasks are.  I certainly intend to do *something* to
>> support the bug tracker system, but I don't know yet what that something is.
> 
> I volunteer to do something too, as long as I don't have to click on
> endless web forms.
> 

I volunteer to help populate the new tracker from whatever from the
currently exists in, and will also put into action any reasonable
scraping/augmentation ideas put forth to make it a more productive tool.

Amir

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Josh Berkus <josh@agliodbs.com> writes:
> > On 09/30/2015 03:10 PM, Tom Lane wrote:
> >> I'd be feeling a lot more positive about this whole thread if any people
> >> had stepped up and said "yes, *I* will put in a lot of grunt-work to make
> >> something happen here".  The lack of any volunteers suggests strongly
> >> that this thread is a waste of time, just as the several similar ones
> >> before it have been.
>
> > Hmmm?  Frost volunteered to stand up debbugs.
>
> So he did, and did anyone volunteer to put data into it, or to do ongoing
> curation of said data?  If we simply connect it up to the mailing lists,
> and then stand back and wait for magic to happen, we will not ever have
> anything that's any more useful than the existing mailing list archives.

I'm still planning to stand up debbugs, but as I said earlier on in the
thread, it's lower priority than the current work around getting beta
out the door (and, generally, 9.5 into good shape).  I've been following
the thread and don't see any reason to stray from that plan.

Once it's up, I'll provide information about how it gets populated, how
to interact with it, etc.  Based on the responses on this thread, it
looks like we've got quite a few people who are willing to help manage
it, once it's up and running.  I'll also be involved (either directly or
by bringing in other resources) in the ongoing curation and management.

Thanks!

Stephen

On Wed, Sep 30, 2015 at 10:44 AM, Merlin Moncure <mmoncure@gmail.com> wrote:
> I'm not trolling in any way.  I'm just challenging you to back up your
> blanket assertions with evidence.  For example, you're assertion that
> mailing lists are insufficient is simply stated and expected to be
> taken on faith: *How* is it insufficient and *what* do things like in
> the new world?  Be specific: glossing over these details doesn't
> really accomplish anything and avoids the careful examination that may
> suggest small tweaks to the current processes that could get similar
> results with a lot less effort.  In this entire massive thread, so far
> only Josh has come up with what I'd consider to be actionable problem
> cases.

I think that the mailing list is pretty much just as good as a bug
tracker would be for finding the discussion about some particular bug.
I mean, our web site has all the mails from the email thread, and
that's where the discussion is, and if that discussion were in a bug
tracker it wouldn't have any more information than what is on the
email thread.  The email thread also usually contains a message
indicating whether a fix was committed.

Where the mailing list is less adequate is:

- If you want to see a list of all the bugs by status, you have to
review every thread individually.  It would be useful to have a way to
filter out the bug reports that turn out not to be really bugs vs. the
ones that are real bugs which have been fixed vs. the ones that are
real bugs that have not been fixed.  Associating status with each bug
number would make this easier.

- Bug numbers are sometimes preserved in commit messages, but they
never make it into release notes.  This actually seems like something
we could improve pretty easily and without a lot of extra work (and
also without a bug tracker).  If every committer makes a practice of
putting the bug number into the commit message, and the people who
write the release notes then transcribe the information there, I bet
that would be pretty useful to a whole lotta people.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:
> - Bug numbers are sometimes preserved in commit messages, but they
> never make it into release notes.  This actually seems like something
> we could improve pretty easily and without a lot of extra work (and
> also without a bug tracker).  If every committer makes a practice of
> putting the bug number into the commit message, and the people who
> write the release notes then transcribe the information there, I bet
> that would be pretty useful to a whole lotta people.

The main reason bug numbers don't go into release notes is that only a
fraction of our bugs actually have bug numbers.  Very many problem reports
show up via ordinary email traffic.  If we had a mail-aware tracker and
there were curators making sure that every problem-reporting thread got
into the tracker, then it might become reasonable to cite bug numbers in
the release notes.

Personally I do make a practice of citing bug numbers in commit messages,
but if you go through my commits, you'll soon agree that the coverage is
too spotty to be useful in release notes.  So I have not bothered to
pester other committers to do likewise.  Also, I suspect it will not be
uncommon for tracker entries to appear only after the related commits,
particularly for security bugs; so expecting the commit messages to be the
links may be impractical anyway.

Playing devil's advocate ... would this really do much other than bloat
the release notes?  The entire assumption of this thread is that people
don't, or don't want to, use the release notes to find out what got fixed;
they'd rather search a tracker.
        regards, tom lane

On 10/01/2015 10:35 AM, Robert Haas wrote:
> On Wed, Sep 30, 2015 at 10:44 AM, Merlin Moncure <mmoncure@gmail.com> wrote:
>> I'm not trolling in any way.  I'm just challenging you to back up your
>> blanket assertions with evidence.  For example, you're assertion that
>> mailing lists are insufficient is simply stated and expected to be
>> taken on faith: *How* is it insufficient and *what* do things like in
>> the new world?  Be specific: glossing over these details doesn't
>> really accomplish anything and avoids the careful examination that may
>> suggest small tweaks to the current processes that could get similar
>> results with a lot less effort.  In this entire massive thread, so far
>> only Josh has come up with what I'd consider to be actionable problem
>> cases.
> I think that the mailing list is pretty much just as good as a bug
> tracker would be for finding the discussion about some particular bug.
> I mean, our web site has all the mails from the email thread, and
> that's where the discussion is, and if that discussion were in a bug
> tracker it wouldn't have any more information than what is on the
> email thread.  The email thread also usually contains a message
> indicating whether a fix was committed.
>
> Where the mailing list is less adequate is:
>
> - If you want to see a list of all the bugs by status, you have to
> review every thread individually.  It would be useful to have a way to
> filter out the bug reports that turn out not to be really bugs vs. the
> ones that are real bugs which have been fixed vs. the ones that are
> real bugs that have not been fixed.  Associating status with each bug
> number would make this easier.
>
> - Bug numbers are sometimes preserved in commit messages, but they
> never make it into release notes.  This actually seems like something
> we could improve pretty easily and without a lot of extra work (and
> also without a bug tracker).  If every committer makes a practice of
> putting the bug number into the commit message, and the people who
> write the release notes then transcribe the information there, I bet
> that would be pretty useful to a whole lotta people.
>


A lot of errors get fixed without a bug ever being raised. If we want a 
tracker to represent some sort of historical record, all commits, or all 
non-feature commits if we don't want to track features, should be 
against tracker items. (In my former life I once had to send out a memo 
to developers that said "If you're not working on items in the tracker 
you're not doing your job.")

cheers

andrew

On 2015-10-01 16:48:32 +0200, Magnus Hagander wrote:
> That would require people to actually use the bug form to submit the
> initial thread as well of course - which most developers don't do
> themselves today. But there is in itself nothing that prevents them from
> doing that, of course - other than a Small Amount Of Extra Work.

It'd be cool if there were a newbug@ or similar mail address that
automatically also posted to -bugs or so.

> I think when a patch is directly related to a specific bug as reported
> through the webform, don't most committers already refer to that bug
> number? Maybe not every time, but at least most of the time? It's the many
> discussions that don't actually have a bug number and yet result in a patch
> that don't?

I think it's mentioned somewhere in the commit message most of the time
- but not in an easy to locate way. If we'd agree on putting something like:
Bug: #XXX
Affected-Versions: 9.5-
Fixed-Versions: 9.3-
in commit messages that'd be a fair bit easier to get into the release notes..

Greetings,

Andres Freund

Andres Freund <andres@anarazel.de> writes:
> On 2015-10-01 16:48:32 +0200, Magnus Hagander wrote:
>> That would require people to actually use the bug form to submit the
>> initial thread as well of course - which most developers don't do
>> themselves today. But there is in itself nothing that prevents them from
>> doing that, of course - other than a Small Amount Of Extra Work.

> It'd be cool if there were a newbug@ or similar mail address that
> automatically also posted to -bugs or so.

I believe that's spelled pgsql-bugs@postgresql.org.

> I think it's mentioned somewhere in the commit message most of the time
> - but not in an easy to locate way. If we'd agree on putting something like:
> Bug: #XXX
> Affected-Versions: 9.5-
> Fixed-Versions: 9.3-
> in commit messages that'd be a fair bit easier to get into the release notes..

As one of the people who do most of the gruntwork for release notes,
I can tell you that that sort of fixed-format annotation is useless
and usually annoying.  I can see what branches you fixed the bug in
anyway, from git_changelog's output.  Actually useful information
of that sort would be commentary along the lines of "The bug exists
back to 8.4, but I only fixed it in 9.2 and up because <reason>."
Without the <reason>, you're just adding bloat to what's already
a pretty large file.
        regards, tom lane

On 2015-10-01 11:07:12 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2015-10-01 16:48:32 +0200, Magnus Hagander wrote:
> >> That would require people to actually use the bug form to submit the
> >> initial thread as well of course - which most developers don't do
> >> themselves today. But there is in itself nothing that prevents them from
> >> doing that, of course - other than a Small Amount Of Extra Work.
> 
> > It'd be cool if there were a newbug@ or similar mail address that
> > automatically also posted to -bugs or so.
> 
> I believe that's spelled pgsql-bugs@postgresql.org.

The point is that newbug would automatically assign a bug id, without
going through the form.

> > I think it's mentioned somewhere in the commit message most of the time
> > - but not in an easy to locate way. If we'd agree on putting something like:
> > Bug: #XXX
> > Affected-Versions: 9.5-
> > Fixed-Versions: 9.3-
> > in commit messages that'd be a fair bit easier to get into the release notes..
> 
> As one of the people who do most of the gruntwork for release notes,
> I can tell you that that sort of fixed-format annotation is useless
> and usually annoying.  I can see what branches you fixed the bug in
> anyway, from git_changelog's output.

I know that I very frequently wish that information were in the commit
messages in a easily discernible way.

> Actually useful information of that sort would be commentary along the
> lines of "The bug exists back to 8.4, but I only fixed it in 9.2 and
> up because <reason>."

That should definitely be there as well, agreed.

Greetings,

Andres Freund

Andres Freund <andres@anarazel.de> writes:
> On 2015-10-01 11:07:12 -0400, Tom Lane wrote:
>> As one of the people who do most of the gruntwork for release notes,
>> I can tell you that that sort of fixed-format annotation is useless
>> and usually annoying.  I can see what branches you fixed the bug in
>> anyway, from git_changelog's output.

> I know that I very frequently wish that information were in the commit
> messages in a easily discernible way.

I'm inclined to think that commit messages are not really the right medium
for that at all.  Commit messages ought to be primarily text for
consumption by humans.  If we had a tracker, I think that it would be the
place for fixed-format metadata, such as "fixed in version(s) x,y,z" and
"fixed by commit(s) aaa,bbb,ccc".  Expecting the tracker to link to the
commit rather than vice versa would also solve a bunch of other problems
like assigned-after-the-fact bug numbers.  Not to mention plain old
mistakes: once committed, a log message is immutable, but a tracker could
be updated as needed.

If this process actually works, I could see the tracker becoming the
source material for generating release notes, at least for bug-fix
notes.  We do it off the commit log now because that's all we've got,
but the log isn't really ideal source material.
        regards, tom lane

On 10/01/2015 07:48 AM, Magnus Hagander wrote:

> But using the bugtracker for the discussion itself is usually not a win.

I know you said usually but:

> In fact, I'd say in most cases it's counterproductive because it forces
> a single tool upon everybody, instead of email which allows each person
> to pick their own favourite tool. Using a bugtracker where all
> discussion happens in email removes that downside, and moves it back to
> the state where it doesn't help but also doesn't hinder the communication.

This doesn't seem correct, roundup, debbugs, redmine, and rt all support 
email as the primary form of communication.

>
> That would require people to actually use the bug form to submit the
> initial thread as well of course - which most developers don't do
> themselves today. But there is in itself nothing that prevents them from
> doing that, of course - other than a Small Amount Of Extra Work.

It requires using a tracker somewhere within the email chain which would 
automatically assign an issue number to the email.

Sincerely,

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Thu, Oct 1, 2015 at 10:18 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2015-10-01 11:07:12 -0400, Tom Lane wrote:
>>> As one of the people who do most of the gruntwork for release notes,
>>> I can tell you that that sort of fixed-format annotation is useless
>>> and usually annoying.  I can see what branches you fixed the bug in
>>> anyway, from git_changelog's output.
>
>> I know that I very frequently wish that information were in the commit
>> messages in a easily discernible way.
>
> I'm inclined to think that commit messages are not really the right medium
> for that at all.  Commit messages ought to be primarily text for
> consumption by humans.  If we had a tracker, I think that it would be the
> place for fixed-format metadata, such as "fixed in version(s) x,y,z" and
> "fixed by commit(s) aaa,bbb,ccc".  Expecting the tracker to link to the
> commit rather than vice versa would also solve a bunch of other problems
> like assigned-after-the-fact bug numbers.  Not to mention plain old
> mistakes: once committed, a log message is immutable, but a tracker could
> be updated as needed.
>
> If this process actually works, I could see the tracker becoming the
> source material for generating release notes, at least for bug-fix
> notes.  We do it off the commit log now because that's all we've got,
> but the log isn't really ideal source material.

At least some of that information could be generated by crawling and
parsing commit emails, I think.  The missing link FWICT is reliably
tying the bug resolution to the relevant commit.   Maybe we could
teach the tracker to watch for "fixed by commit ABCDEF"  in the bug
list (and maybe hackers, too), identifying the commit as a bug fix and
associating it to the release branches.  That gets crawled and
rendered to a database for collection and reporting.

merlin

On 10/01/2015 08:18 AM, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2015-10-01 11:07:12 -0400, Tom Lane wrote:

> I'm inclined to think that commit messages are not really the right medium
> for that at all.  Commit messages ought to be primarily text for
> consumption by humans.  If we had a tracker, I think that it would be the
> place for fixed-format metadata, such as "fixed in version(s) x,y,z" and
> "fixed by commit(s) aaa,bbb,ccc".  Expecting the tracker to link to the
> commit rather than vice versa would also solve a bunch of other problems
> like assigned-after-the-fact bug numbers.  Not to mention plain old
> mistakes: once committed, a log message is immutable, but a tracker could
> be updated as needed.

What I imagined was this:

TGL commits foo, part of the commit message says: Status: Committed. 
Then a commit hook is fired from git to the tracker from a fixed 
address, That message would say:

Git commit $hash
Status: Committed

Which would not only link to the specific commit but also automatically 
close the ticket with a status of Committed. Does that make sense for 
-hackers? It seems like it would take a load off but I am not the one in 
it every day.


>
> If this process actually works, I could see the tracker becoming the
> source material for generating release notes, at least for bug-fix
> notes.  We do it off the commit log now because that's all we've got,
> but the log isn't really ideal source material.


Yep.

>
>             regards, tom lane
>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 10/01/2015 07:55 AM, Tom Lane wrote:
> Playing devil's advocate ... would this really do much other than bloat
> the release notes?  The entire assumption of this thread is that people
> don't, or don't want to, use the release notes to find out what got fixed;
> they'd rather search a tracker.

It's not a question of "rather", it's a question of how searchable the
release notes are, which is "not really at all".  Yes, you can scan the
release notes for the latest update, but consider users who have an
issue and are running 9.2.7.  Reasonably enough, they want to know that
their issue is fixed in 9.2.13 (or in 9.4 if it turns out to be a
feature, not a bug) before they ask their boss for a downtime.  Figuring
that out now is really hard.

I tried to tackle this three or four years ago, by writing a tool which
would slurp the release notes and put them into a full-text search
database.  This turned out to be very hard to do; our formatting for the
release notes makes it very difficult for an automated import program to
interpret (SGML doesn't help), especially on point releases to old
versions.  It also turned out that the resulting database was useful
mostly to me, because you had to figure out what terms to search on
based on the bug report in front of you.  As a result, it never went online.

So today, the only time the release notes are useful for a "is this
issue fixed or not" is when a release note message mentions the specific
error message the user is getting, which is a minority of the time.

So in addition to what Haas mentions, I think we want to be able to link
the release notes to the original issues for our hypothetical bug tracker.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 10/01/2015 05:10 PM, Andres Freund wrote:
> On 2015-10-01 11:07:12 -0400, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> On 2015-10-01 16:48:32 +0200, Magnus Hagander wrote:
>>>> That would require people to actually use the bug form to submit the
>>>> initial thread as well of course - which most developers don't do
>>>> themselves today. But there is in itself nothing that prevents them from
>>>> doing that, of course - other than a Small Amount Of Extra Work.
>>
>>> It'd be cool if there were a newbug@ or similar mail address that
>>> automatically also posted to -bugs or so.
>>
>> I believe that's spelled pgsql-bugs@postgresql.org.
> 
> The point is that newbug would automatically assign a bug id, without
> going through the form.

if we only want that - we can trivially implement that on the mailserver
side by asking the backend database sequence for a bugid and rewriting
the subject...
But given debbugs is on the radar not sure we need it...


Stefan

On Thu, Oct 1, 2015 at 12:09 PM, Josh Berkus <josh@agliodbs.com> wrote:
> On 10/01/2015 07:55 AM, Tom Lane wrote:
>> Playing devil's advocate ... would this really do much other than bloat
>> the release notes?  The entire assumption of this thread is that people
>> don't, or don't want to, use the release notes to find out what got fixed;
>> they'd rather search a tracker.
>
> It's not a question of "rather", it's a question of how searchable the
> release notes are, which is "not really at all".  Yes, you can scan the
> release notes for the latest update, but consider users who have an
> issue and are running 9.2.7.  Reasonably enough, they want to know that
> their issue is fixed in 9.2.13 (or in 9.4 if it turns out to be a
> feature, not a bug) before they ask their boss for a downtime.  Figuring
> that out now is really hard.

Yeah -- so maybe it's the wrong path.  The bugs/commits list are very
parse-able for important elements and should be able to be slurped
into a database for tracking and further insertion of metadata.  A
'commit tracker' if you will; it would organize commits and relevant
bug reports (so long as they could be linked by certain conventions).It's a read only system except for what other
humaninputs you'd want
 
to arrange for other processes (such as generating release notes which
might require cleaned up language).

merlin

Hello,

I believe it is pretty obvious we are moving in the direction of having 
a tracker at this point. The problem is exactly which direction. Stephen 
has offered some resources for his ideas and now I am offering some 
resources for mine.

I am proposing to setup a redmine instance on a VM. I am happy to do a 
lot of the legwork and should be able to get most of it done before EU. 
This is what I think I need from my fellows:

1. At least two committers
2. At least three hackers (preferably that are different from the 
committers but not required)
3. At least two users

I think we have reached a point where everyone has ideas of what they do 
and don't want but none of it matters if we don't have a proof of 
concept to determine what we do and don't know or want.

Thoughts? Volunteers?

Sincerely,

JD
-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

> On 2 Oct 2015, at 17:28, Joshua D. Drake <jd@commandprompt.com> wrote:
>
> Hello,
>
> I believe it is pretty obvious we are moving in the direction of having a tracker at this point. The problem is
exactlywhich direction. Stephen has offered some resources for his ideas and now I am offering some resources for mine. 
>
> I am proposing to setup a redmine instance on a VM. I am happy to do a lot of the legwork and should be able to get
mostof it done before EU. This is what I think I need from my fellows: 
>
> 1. At least two committers
> 2. At least three hackers (preferably that are different from the committers but not required)
> 3. At least two users
>
> I think we have reached a point where everyone has ideas of what they do and don't want but none of it matters if we
don'thave a proof of concept to determine what we do and don't know or want. 
>
> Thoughts? Volunteers?

I swore to myself that I'd stay out of this bikeshed, but... we already have a Redmine instance.

On 10/02/2015 09:34 AM, Dave Page wrote:

>> Thoughts? Volunteers?
>
> I swore to myself that I'd stay out of this bikeshed, but... we already have a Redmine instance.

I know but I didn't want to dogfood in an installation that was 
production. I am not going to put up vanilla redmine, I actually planned 
on configuring in a way that could be used by -hackers which could 
(possibly?) cause issues with the install .Org has.

JD


>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

Hi,

On 2015-10-02 09:28:02 -0700, Joshua D. Drake wrote:
> I am proposing to setup a redmine instance on a VM. I am happy to do a lot
> of the legwork and should be able to get most of it done before EU. This is
> what I think I need from my fellows:

-1. This thread has already cost disproportionally much time and
motiviation. Let's not wase more in doing two evaluations at the same
time, when we might already be happy with one of them. We came pretty
close to agreeing on debbugs in a couple past discussions so that seems
the least unlikely choice to actually get adopted.

Greetings,

Andres Freund

On 10/02/2015 09:41 AM, Andres Freund wrote:
> Hi,
>
> On 2015-10-02 09:28:02 -0700, Joshua D. Drake wrote:
>> I am proposing to setup a redmine instance on a VM. I am happy to do a lot
>> of the legwork and should be able to get most of it done before EU. This is
>> what I think I need from my fellows:
>
> -1. This thread has already cost disproportionally much time and
> motiviation. Let's not wase more in doing two evaluations at the same
> time, when we might already be happy with one of them. We came pretty
> close to agreeing on debbugs in a couple past discussions so that seems
> the least unlikely choice to actually get adopted.

I once drove a manual all the time. I swore by the manual. I loved the 
control, the feeling (ridiculously) that I was a race car driver on the 
Urban track.

Then I got an automatic and realized how damn nice it was to not be in 
control or have to worry about whether or not I should shift at 3k RPMS 
or 4k RPMS. (Then I drove a manual on the wrong side of the road in 
Ireland and was even happier that I prefer automatics)

We can't just point at something and say, "Oh that will work" because we 
haven't tried to see how it is going to work with our requirements.

I wouldn't expect any customer to use postgresql because it has an 
Elephant as a mascot. I expect them to use it because it is the best 
choice for their requirements.

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Fri, Oct 2, 2015 at 12:50 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
> I once drove a manual all the time. I swore by the manual. I loved the
> control, the feeling (ridiculously) that I was a race car driver on the
> Urban track.
>
> Then I got an automatic and realized how damn nice it was to not be in
> control or have to worry about whether or not I should shift at 3k RPMS or
> 4k RPMS. (Then I drove a manual on the wrong side of the road in Ireland and
> was even happier that I prefer automatics)
>
> We can't just point at something and say, "Oh that will work" because we
> haven't tried to see how it is going to work with our requirements.
>
> I wouldn't expect any customer to use postgresql because it has an Elephant
> as a mascot. I expect them to use it because it is the best choice for their
> requirements.

I don't know what this has to do with anything Andres said.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 10/02/2015 11:36 AM, Robert Haas wrote:

> I don't know what this has to do with anything Andres said.
>

I am sorry if I wasn't clear.

Put succinctly, I am willing to put resources into testing Redmine for 
our needs. I will need help to do so because I am not a 
committer/hacker. Andres thinks that isn't worth while. I think he is 
wrong. If he doesn't want to help, he doesn't have to, thus the call for 
volunteers.

I am not expecting that redmine will be chosen over debbugs. I am 
expecting that we make an educated decision about which one suits our 
needs. If we only review debbugs, we are making a decision in a vacuum.

JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Fri, Oct 2, 2015 at 2:43 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
> I am sorry if I wasn't clear.
>
> Put succinctly, I am willing to put resources into testing Redmine for our
> needs. I will need help to do so because I am not a committer/hacker. Andres
> thinks that isn't worth while. I think he is wrong. If he doesn't want to
> help, he doesn't have to, thus the call for volunteers.
>
> I am not expecting that redmine will be chosen over debbugs. I am expecting
> that we make an educated decision about which one suits our needs. If we
> only review debbugs, we are making a decision in a vacuum.

OK.  My reading of the thread is that the hackers who expressed an
opinion mostly preferred debbugs and the people less involved in the
project wanted something more like GitHub/GitLab.  Some people also
argued for and against Bugzilla and JIRA.  I didn't really see anybody
agitating for Redmine, so I'm not sure how far you're going to get
with that.

But I'm not really arguing against it.  We use it here at
EnterpriseDB, and it's very helpful, though it is undeniably a
[expletive]-ton of work to maintain.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 10/02/2015 12:52 PM, Robert Haas wrote:

> OK.  My reading of the thread is that the hackers who expressed an
> opinion mostly preferred debbugs and the people less involved in the
> project wanted something more like GitHub/GitLab.  Some people also
> argued for and against Bugzilla and JIRA.  I didn't really see anybody
> agitating for Redmine, so I'm not sure how far you're going to get
> with that.

Right, thus call for Volunteers. If I can get a few to help me, I am 
willing to put my resources behind it. If I can't then I won't. I guess 
what I am really saying here is, "If I am going to bitch about the lack 
of a tracker, I better damn well be willing to put resources into fixing 
the problem".

So I am putting my money where my mouth is.

>
> But I'm not really arguing against it.  We use it here at
> EnterpriseDB, and it's very helpful, though it is undeniably a
> [expletive]-ton of work to maintain.

Yeah, and that is any issue tracker (the ton of work to maintain).

Sincerely,

JD


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

Joshua D. Drake wrote:

> Put succinctly, I am willing to put resources into testing Redmine for our
> needs. I will need help to do so because I am not a committer/hacker. Andres
> thinks that isn't worth while. I think he is wrong. If he doesn't want to
> help, he doesn't have to, thus the call for volunteers.

Nobody asked, but here's my opinion on Redmine.  I worked pretty heavily
with it during my time at Command Prompt.  I have to say that with the
customizations that CMD had at the time, it wasn't that bad -- I was
pretty happy that I could interact with it via email, and most of the
time it wouldn't do anything too stupid.  I could also interact with it
using the web, and it worked pretty well there.  Most other Redmine
installations I've used don't have the email interface at all.

However, the contact surface between these two options wasn't really
well polished.  Formatting would be lost very frequently: I could write
a nice email, and the customer would get a nice email, but if you looked
at it in the web, it was very ugly.  If you used the web form to reply,
the resulting email looked pretty stupid in some cases.  I eventually
learned to use the right {{{ }}} markers in my email replies so that
code would look right in the web.  But if you made a single mistake, you
were fscked and there was no way at all to fix it.

As far as I remember, the main reason for this pain was that it didn't
try to consider an email as an email: instead, what it did was grab the
text and cram it into the comment box.  Same thing in the other
direction, where the text from the comment would be crammed as an email
in output.  All that was needed was for it to store emails in the
rfc/2822 format in the database, and then render them as emails in the
web form, instead of trying to do the conversion in the process.


If you look at the debbugs interface, it is pretty clear that all that
it does is keep track of emails -- which, let it be said, is the soul of
this community's communication, so it seems to me that that is what we
need.  Metadata changes are kept visually separate from actual
commentary, which is convenient; and you can always get the mbox
involving that bug, or look at minute details of it using the web
interface if you need that sort of thing.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Fri, Oct 2, 2015 at 4:26 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> Joshua D. Drake wrote:
>> Put succinctly, I am willing to put resources into testing Redmine for our
>> needs. I will need help to do so because I am not a committer/hacker. Andres
>> thinks that isn't worth while. I think he is wrong. If he doesn't want to
>> help, he doesn't have to, thus the call for volunteers.
>
> Nobody asked, but here's my opinion on Redmine.  I worked pretty heavily
> with it during my time at Command Prompt.  I have to say that with the
> customizations that CMD had at the time, it wasn't that bad -- I was
> pretty happy that I could interact with it via email, and most of the
> time it wouldn't do anything too stupid.  I could also interact with it
> using the web, and it worked pretty well there.  Most other Redmine
> installations I've used don't have the email interface at all.
>
> However, the contact surface between these two options wasn't really
> well polished.  Formatting would be lost very frequently: I could write
> a nice email, and the customer would get a nice email, but if you looked
> at it in the web, it was very ugly.  If you used the web form to reply,
> the resulting email looked pretty stupid in some cases.  I eventually
> learned to use the right {{{ }}} markers in my email replies so that
> code would look right in the web.  But if you made a single mistake, you
> were fscked and there was no way at all to fix it.
>
> As far as I remember, the main reason for this pain was that it didn't
> try to consider an email as an email: instead, what it did was grab the
> text and cram it into the comment box.  Same thing in the other
> direction, where the text from the comment would be crammed as an email
> in output.  All that was needed was for it to store emails in the
> rfc/2822 format in the database, and then render them as emails in the
> web form, instead of trying to do the conversion in the process.
>
>
> If you look at the debbugs interface, it is pretty clear that all that
> it does is keep track of emails -- which, let it be said, is the soul of
> this community's communication, so it seems to me that that is what we
> need.  Metadata changes are kept visually separate from actual
> commentary, which is convenient; and you can always get the mbox
> involving that bug, or look at minute details of it using the web
> interface if you need that sort of thing.

Thanks for this very thoughtful email.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 10/02/2015 01:26 PM, Alvaro Herrera wrote:

> However, the contact surface between these two options wasn't really
> well polished.  Formatting would be lost very frequently: I could write
> a nice email, and the customer would get a nice email, but if you looked
>  at it in the web, it was very ugly.

Newer versions are much better at this from what I can tell.

>  If you used the web form to reply,
> the resulting email looked pretty stupid in some cases.  I eventually
> learned to use the right {{{ }}} markers in my email replies so that
> code would look right in the web.  But if you made a single mistake, you
> were fscked and there was no way at all to fix it.

I don't know anything about the brackets but I do know one thing that is 
unfortunate about redmine is that it assumes plain text unless you tell 
it something different. What does that mean?

If you want to use a preformatted (text based) entry:

<pre>
psql -U postgres
</pre>

Will do exactly the same thing in the web interface. Of course in the 
web interface it gets formatted so it looks great. In email, you get 
HTML code.

I find is that as long as you are working in just text, everything is 
kosher. If you try to do any formatting (so it looks nice on the web), 
you run into problems.

>
> If you look at the debbugs interface, it is pretty clear that all that
> it does is keep track of emails -- which, let it be said, is the soul of
> this community's communication, so it seems to me that that is what we
> need.  Metadata changes are kept visually separate from actual
> commentary, which is convenient; and you can always get the mbox
> involving that bug, or look at minute details of it using the web
> interface if you need that sort of thing.

It is true (I believe roundup doesn't something similar to debbugs) that 
Redmine considers email a business class citizen, not coach but 
certainly not first class.

My main issue with debbugs is that it appears very limited and is yet 
another piece of infrastructure that only provides that infrastructure 
and thus will continue to cause more heartache than is needed. That 
isn't to say it is a bad piece of software but that it is very specific 
in what it does.

We seem to need more than that. As another person (I don't recall who) 
mentioned, Redmine gives us an infrastructure to many things including 
proper mapping between issues and GIT. Perhaps we don't use that now but 
do we want to be in a position a year from now where we want it but now 
we have pitched our tent with a more limited piece of software?

I do appreciate the feedback on it and I am in no way suggesting that 
Redmine is perfect only that it "might" be the solution.

Sincerely,

JD





-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

I don't have the original message for this thread, so I arbitrarily picked a
message to reply to.

Since what has been asked for is a bug *tracker*, and we already have a bugs
mailing list, I put together something.

I downloaded the archives for pgsql-bugs, and fed them into a database.  This
part was easy, since I have already written a pg backed usenet server and had
the code hand for storing and parsing out bits of rfc 2822 messages.

It's dirt simple.  If the system sees a message with 'Bug #(\d+)' in the
subject line, it creates an entry in a bugs table with that bug number (if
needed), and then marks the message as belonging to that bug.  If there seems
to be metadata about the bug in the format of the (unquoted)

Bug reference:
Logged by:          
Email address:
PostgreSQL version:
Operating system:
Description:
Details:

it pulls that out and puts it in the bugs table.  There's also an "open"
boolean in the table, defaulting to true.

The results can be found at https://granicus.if.org/pgbugs/

Ok.  So now we have a bug tracker, but...

Some open questions that I don't think have really been addressed, with my
commentary interspersed:

1: Can a bug be more than "open" or "closed"?

I think yes.  At least we probably want to know why a bug is closed.  Is it not
a bug at all, not our bug, a duplicate submission, a duplicate of another bug,
something we won't fix for some reason (e.g. a bug against version 7)

2: Who can declare a bug closed.

Ugh.  I'm going to close some of them if it seems obvious to me that they
should be closed.  But what if it's not obvious?  I could probably maintain it
to some extent, but I don't know how much time that would actually take.

Related to the next point, it probably makes sense to just close up front
bugs that are marked against unsupported pg versions, or haven't had
any activity for too long, perhaps two years.  Just closing bugs with no
mailing list activity for two years closes 5280 of 6376 bugs.

3: How far back should I actually import data from the bugs list?

I have imported each archived month from December of 1998.  It looks like the
bug sequence was started at 1000 in December of 2003.  Emails with no bug id in
the subject line don't get associated with any bug, they're in the DB bug not
really findable.

4: What should I do with emails that don't reference a bug id but seem to be
talking about a bug?

I suggest we do nothing with them as far as the bug tracker is concerned.  If
people want to mark their message as pertaining to a bug, they can put that in
the subject line.  However, I don't think a bug id can be assigned via email,
that is, I think you have to use a web form to create a bug report with a bug
id.  Presumably that could change if whoever runs the bug counter wants it to.

5: How can we use email to update the status of a bug?

I suggest using email headers to do this.  'X-PGBug-Fixed: <commitid>' and the
like.  I assume here that everyone who might want to do such a thing uses an
MUA that would allow this, and they know how.

6: Does there need to be any security on updating the status?

Probably not.  I don't think it's the sort of thing that would attract
malicious adjustments.  If I'm wrong, I'd need to rethink this.  I realize I'm
making security an afterthought, which makes my teeth itch, but I think layers
of security would make it much less likely to be actually adopted.

Just to be clear, this is both a work in progress and a proof of concept.  It's
slow, it's ugly.  I haven't created any indexes, written any css or javascript,
or implemented any caching.  I may work on that before you read this though.

Comments are welcome, and no, I don't really expect that this will be what gets
adopted, mainly I wanted to show that we can probably just build something
rather effective off our existing infrastructure, and I had Saturday free (as
of this writing, I've got maybe 5 hours into it total, albeit with lots of
code re-use from other projects).  There are some obvious todo items,
filtering and searching being the most salient.

Some data import issues:

March 10, 2003, bad Date header, looked like junk anyway, so I deleted it.

Time zone offsets of --0400 and -0500 (at least), I took these as being -0400
(or whathaveyou).

Date header of Sat, 31 May 2008 12:12:18 +1930, judging by the name on the
email, this is probably posted from Venezuela, which switched to time one -0430
in 2007, which could also be thought of as +1930 if you ignore the implied date
change.

And, by way of some statistics, since I've got the archives in a database:

Emails: 43870
Bugs: 6376
Distinct 'From' headers: 10643

The bugs have 3.5 messages each on average, with 2 being the most common
number, and 113 at the most, for bug 12990.  1284 bugs have only one message
associated with them.

-- 
nw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160


> Comments are welcome, and no, I don't really expect that this will be what gets
> adopted, mainly I wanted to show that we can probably just build something
> rather effective off our existing infrastructure

+1, good job.

> The bugs have 3.5 messages each on average, with 2 being the most common
> number, and 113 at the most, for bug 12990.  1284 bugs have only one message
> associated with them.

For anyone who is dying to know, as I was, what the winning bug report was:

"Missing pg_multixact/members files (appears to have wrapped, then truncated)"


http://www.postgresql.org/message-id/flat/20150406192130.2573.22545@wrigleys.postgresql.org#20150406192130.2573.22545@wrigleys.postgresql.org
or:
http://goo.gl/4lKYOC


- -- 
Greg Sabino Mullane greg@turnstep.com
End Point Corporation http://www.endpoint.com/
PGP Key: 0x14964AC8 201510041854
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAlYRriIACgkQvJuQZxSWSsgJkwCgsROux3esaDxHbitNhHs17Thk
rKIAoNMD6NnKRAvguuvxkg4hiJOfPDH6
=5kJJ
-----END PGP SIGNATURE-----

On 10/04/2015 03:42 PM, Nathan Wagner wrote:
> I downloaded the archives for pgsql-bugs, and fed them into a database.  This
> part was easy, since I have already written a pg backed usenet server and had
> the code hand for storing and parsing out bits of rfc 2822 messages.

That would be the key part, wouldn't it?  Nice that you have that.

So this would be the other option if adopting debbugs doesn't work out.I think it's likely that we'll end up recreating
mostof debbugs in the
 
process (or bugzilla or something else) but whatever.  As long as we
have some kind of bug tracker, I'm happy.

> It's dirt simple.  If the system sees a message with 'Bug #(\d+)' in the
> subject line, it creates an entry in a bugs table with that bug number (if
> needed), and then marks the message as belonging to that bug.  If there seems
> to be metadata about the bug in the format of the (unquoted)
> 
> Bug reference:
> Logged by:          
> Email address:
> PostgreSQL version:
> Operating system:
> Description:
> Details:
> 
> it pulls that out and puts it in the bugs table.  There's also an "open"
> boolean in the table, defaulting to true.
> 
> The results can be found at https://granicus.if.org/pgbugs/
> 
> Ok.  So now we have a bug tracker, but...
> 
> Some open questions that I don't think have really been addressed, with my
> commentary interspersed:
> 
> 1: Can a bug be more than "open" or "closed"?
> 
> I think yes.  At least we probably want to know why a bug is closed.  Is it not
> a bug at all, not our bug, a duplicate submission, a duplicate of another bug,
> something we won't fix for some reason (e.g. a bug against version 7)

We'd want the usual statuses:

* fixed
* duplicate
* unreproduceable
* timed out
* not a bug
* won't fix
* reopened

We'd also want a way to link a bug fix to a commit, and probably a way
to give the bug a list of searchable keywords (and add to that list).

> 2: Who can declare a bug closed.
> 
> Ugh.  I'm going to close some of them if it seems obvious to me that they
> should be closed.  But what if it's not obvious?  I could probably maintain it
> to some extent, but I don't know how much time that would actually take.
> 
> Related to the next point, it probably makes sense to just close up front
> bugs that are marked against unsupported pg versions, or haven't had
> any activity for too long, perhaps two years.  Just closing bugs with no
> mailing list activity for two years closes 5280 of 6376 bugs.

I'm reluctant to close all of those unexamined, since part of the
purpose of this is to find bugs which were never fixed.  Probably we
should organize a posse to comb trhough all of the old bugs and
hand-close them.

> 3: How far back should I actually import data from the bugs list?
> 
> I have imported each archived month from December of 1998.  It looks like the
> bug sequence was started at 1000 in December of 2003.  Emails with no bug id in
> the subject line don't get associated with any bug, they're in the DB bug not
> really findable.
> 
> 4: What should I do with emails that don't reference a bug id but seem to be
> talking about a bug?
> 
> I suggest we do nothing with them as far as the bug tracker is concerned.  If
> people want to mark their message as pertaining to a bug, they can put that in
> the subject line.  However, I don't think a bug id can be assigned via email,
> that is, I think you have to use a web form to create a bug report with a bug
> id.  Presumably that could change if whoever runs the bug counter wants it to.

Yeah, fixing this would probably be tied to the possible change to
mailman.  Unless someone already has a way to get majordomo to append a
bug ID.

> 5: How can we use email to update the status of a bug?
> 
> I suggest using email headers to do this.  'X-PGBug-Fixed: <commitid>' and the
> like.  I assume here that everyone who might want to do such a thing uses an
> MUA that would allow this, and they know how.

I guess that depends on who we expect to use this, at least for closing
stuff.

> 6: Does there need to be any security on updating the status?
> 
> Probably not.  I don't think it's the sort of thing that would attract
> malicious adjustments.  If I'm wrong, I'd need to rethink this.  I realize I'm
> making security an afterthought, which makes my teeth itch, but I think layers
> of security would make it much less likely to be actually adopted.

I think there needs to be some kind of administrative access which
allows, for example, an issue to be closed so that it can't be reopened.

Anyway, I'm not convinced we want to reinvent this particular wheel, but
if we do, you've done a yeoman's job.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Sun, Oct 04, 2015 at 04:30:49PM -0700, Josh Berkus wrote:
> That would be the key part, wouldn't it?  Nice that you have [code to
> store and parse email messages].

Yeah.  It actually made most of the work pretty easy.  It's available
with a bunch of other code at https://pd.if.org/git/ if anyone wants it.
I did find a bug in my header processing though, so I'll need to commit
that fix.

> We'd also want a way to link a bug fix to a commit, and probably a way
> to give the bug a list of searchable keywords (and add to that list).

I've been thinking of hooking it up to the fti machinery and providing
a search box.  I've never really used fti before, so this might be a
good opportunity to learn it for real.

> > it probably makes sense to just close up front bugs that are marked
> > against unsupported pg versions, or haven't had any activity for too
> > long, perhaps two years.
> I'm reluctant to close all of those unexamined, since part of the
> purpose of this is to find bugs which were never fixed.  Probably we
> should organize a posse to comb trhough all of the old bugs and
> hand-close them.

I'm doing some of that as I poke at the bugs pages.  Perhaps it would
make sense to have a closed status of 'stale' or the like (perhaps
that's what you meant by 'timed out') which could be used to get bugs
out of the main list but still be marked as 'not human reviewed'.  These
could then be reviewed by the posse.

> Yeah, fixing this [email's without a bug id] would probably be tied to
> the possible change to mailman.  Unless someone already has a way to
> get majordomo to append a bug ID.

How are bug ids assigned now?  From the evidence, I assume there is a
sequence in a database that the web submission form queries to format a
resulting email to the bugs mailing list.  Do the mailing lists live on
the same server?  If so, perhaps it would be easy to assign a new bug id
to a new thread on -bugs.  But perhaps that's too aggressive in creating
bugs.

> > 5: How can we use email to update the status of a bug?
> > 
> > I suggest using email headers to do this.  'X-PGBug-Fixed:
> > <commitid>' and the like.  I assume here that everyone who might
> > want to do such a thing uses an MUA that would allow this, and they
> > know how.
> 
> I guess that depends on who we expect to use this, at least for
> closing stuff.

I could certainly support more than one mechanism.  A web interface for
those who would prefer such and emails would seem to be the basic
requirements.

> > 6: Does there need to be any security on updating the status?
> > 
> > Probably not.  I don't think it's the sort of thing that would
> > attract malicious adjustments.  If I'm wrong, I'd need to rethink
> > this.  I realize I'm making security an afterthought, which makes my
> > teeth itch, but I think layers of security would make it much less
> > likely to be actually adopted.
> 
> I think there needs to be some kind of administrative access which
> allows, for example, an issue to be closed so that it can't be
> reopened.

I guess technically we have that now since I'm the only one who can
close or open a bug in the db I've set up :)

Seriously though, I think it probably makes the most sense to tie the
system in with the existing pg community accounts if it goes that far.

> Anyway, I'm not convinced we want to reinvent this particular wheel, but
> if we do, you've done a yeoman's job.

Thank you.  (Assuming I've interpreted the phrase correctly, I'm not
familiar with it, and a web search was only semi-enlightening).

-- 
nw

On Mon, Oct 5, 2015 at 5:32 AM, Magnus Hagander <magnus@hagander.net> wrote:
>
>
> On Mon, Oct 5, 2015 at 12:42 AM, Nathan Wagner <nw+pg@hydaspes.if.org>
> wrote:
>>
>> I don't have the original message for this thread, so I arbitrarily picked
>> a
>> message to reply to.
>>
>> Since what has been asked for is a bug *tracker*, and we already have a
>> bugs
>> mailing list, I put together something.
>
> FWIW, I think this is a good approach in general. This makes it a  bug
> *tracker*, rather than a "workflow enforcer". That depends on what we want
> of course, but those are two very different things and many of the other
> tools suggested are more workflow enforcers.

+1

The key points is how people interact with the tool; as long as the
interaction is basically "one way" from email to the tracking tool
(either debbugs or something hand rolled) it should work as long as it
provides value.  The main output of the tool is to do thing like
making qualified searches of bug fixes by version easier and perhaps
supporting release note generation.

Personally I think a hand-rolled tool is a better choice for this
project given that the requirements are so specific; it can be thought
of as an extension of the commit fest framework.

merlin

I don't have the original message for this thread, so I arbitrarily picked a
message to reply to.

Since what has been asked for is a bug *tracker*, and we already have a bugs
mailing list, I put together something.

I downloaded the archives for pgsql-bugs, and fed them into a database.  This
part was easy, since I have already written a pg backed usenet server and had
the code hand for storing and parsing out bits of rfc 2822 messages.

It's dirt simple.  If the system sees a message with 'Bug #(\d+)' in the
subject line, it creates an entry in a bugs table with that bug number (if
needed), and then marks the message as belonging to that bug.  If there seems
to be metadata about the bug in the format of the (unquoted)

Bug reference:
Logged by:          
Email address:
PostgreSQL version:
Operating system:
Description:
Details:

it pulls that out and puts it in the bugs table.  There's also an "open"
boolean in the table, defaulting to true.

The results can be found at https://granicus.if.org/pgbugs/

Ok.  So now we have a bug tracker, but...

Some open questions that I don't think have really been addressed, with my
commentary interspersed:

1: Can a bug be more than "open" or "closed"?

I think yes.  At least we probably want to know why a bug is closed.  Is it not
a bug at all, not our bug, a duplicate submission, a duplicate of another bug,
something we won't fix for some reason (e.g. a bug against version 7)

2: Who can declare a bug closed.

Ugh.  I'm going to close some of them if it seems obvious to me that they
should be closed.  But what if it's not obvious?  I could probably maintain it
to some extent, but I don't know how much time that would actually take.

Related to the next point, it probably makes sense to just close up front
bugs that are marked against unsupported pg versions, or haven't had
any activity for too long, perhaps two years.  Just closing bugs with no
mailing list activity for two years closes 5280 of 6376 bugs.

3: How far back should I actually import data from the bugs list?

I have imported each archived month from December of 1998.  It looks like the
bug sequence was started at 1000 in December of 2003.  Emails with no bug id in
the subject line don't get associated with any bug, they're in the DB bug not
really findable.

4: What should I do with emails that don't reference a bug id but seem to be
talking about a bug?

I suggest we do nothing with them as far as the bug tracker is concerned.  If
people want to mark their message as pertaining to a bug, they can put that in
the subject line.  However, I don't think a bug id can be assigned via email,
that is, I think you have to use a web form to create a bug report with a bug
id.  Presumably that could change if whoever runs the bug counter wants it to.

5: How can we use email to update the status of a bug?

I suggest using email headers to do this.  'X-PGBug-Fixed: <commitid>' and the
like.  I assume here that everyone who might want to do such a thing uses an
MUA that would allow this, and they know how.

6: Does there need to be any security on updating the status?

Probably not.  I don't think it's the sort of thing that would attract
malicious adjustments.  If I'm wrong, I'd need to rethink this.  I realize I'm
making security an afterthought, which makes my teeth itch, but I think layers
of security would make it much less likely to be actually adopted.

Just to be clear, this is both a work in progress and a proof of concept.  It's
slow, it's ugly.  I haven't created any indexes, written any css or javascript,
or implemented any caching.  I may work on that before you read this though.

Comments are welcome, and no, I don't really expect that this will be what gets
adopted, mainly I wanted to show that we can probably just build something
rather effective off our existing infrastructure, and I had Saturday free (as
of this writing, I've got maybe 5 hours into it total, albeit with lots of
code re-use from other projects).  There are some obvious todo items,
filtering and searching being the most salient.

Some data import issues:

March 10, 2003, bad Date header, looked like junk anyway, so I deleted it.

Time zone offsets of --0400 and -0500 (at least), I took these as being -0400
(or whathaveyou).

Date header of Sat, 31 May 2008 12:12:18 +1930, judging by the name on the
email, this is probably posted from Venezuela, which switched to time one -0430
in 2007, which could also be thought of as +1930 if you ignore the implied date
change.

And, by way of some statistics, since I've got the archives in a database:

Emails: 43870
Bugs: 6376
Distinct 'From' headers: 10643

The bugs have 3.5 messages each on average, with 2 being the most common
number, and 113 at the most, for bug 12990.  1284 bugs have only one message
associated with them.

-- 
nw

Nathan Wagner wrote:

> 1: Can a bug be more than "open" or "closed"?
> 
> I think yes.  At least we probably want to know why a bug is closed.  Is it not
> a bug at all, not our bug, a duplicate submission, a duplicate of another bug,
> something we won't fix for some reason (e.g. a bug against version 7)

Not only that -- is the bug closed in all branches or only some of them?
If there's also some data about when a bug appeared (commit ID), then
it's easy to get a report of which minor releases have the bug.  For
instance, see bug #8470 which I fixed by commits in 9.5 and master, but
is yet unfixed in 9.3 and 9.4.  At least one other multixact bug was
fixed in 9.5/master only.


What debbugs allows you to do, is that you write to the bug address and
in the first few lines of the body it looks for commands such as
"close", "merge", "reassign".  I for one am open fo commandeering a bug
tracker in this way, as well as adding fixed-format tags to commit
messages indicating "Fixes: #xyz" so that it can be picked up
automatically.

One of our policies in backpatching fixes is that we use the same commit
in all branches so that they become grouped as a single element in the
output of the src/tools/git_changelog script.  Maybe that can be useful
to the bug tracker as well, in some form.

> 2: Who can declare a bug closed.
> 
> Ugh.  I'm going to close some of them if it seems obvious to me that they
> should be closed.  But what if it's not obvious?  I could probably maintain it
> to some extent, but I don't know how much time that would actually take.

I think at least committers should be able to close bugs.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Wednesday 30 September 2015 14:41:34 you wrote:
> On Tue, Sep 29, 2015 at 12:08:56PM +1300, Gavin Flower wrote:
> > Linux kernel project uses bugzilla (https://bugzilla.kernel.org)
> 
> AIUI this is not mandatory for kernel hackers, and more opt-in from a
> some/many/a few(?) subsystem maintainers.  Some parts use it more, some
> less or not at all.
> 
> > and so does LibreOffice (https://bugs.documentfoundation.org)
> 
> Thas is true, however.
> 
> Same for freedesktop.org and the Gnome project, I believe.
> 
> 
> Michael

What about Trac? http://trac.edgewall.org/wiki/TracUsers


-- 
YUriy Zhuravlev
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Hello,

So I was on #postgresql today. I convinced a newer user that they could 
easily contribute to PostgreSQL even if it was just doc patches. I 
described the basic workflow and the individual was excited.

His first question?

linuxhiker: oooo git interface! Bug tracker for this anywhere?

Joshua D. Drake

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Wed, Jan 6, 2016 at 7:32 AM, Joshua D. Drake <jd@commandprompt.com> wrote:
> Hello,
>
> So I was on #postgresql today. I convinced a newer user that they could
> easily contribute to PostgreSQL even if it was just doc patches. I described
> the basic workflow and the individual was excited.
>
> His first question?
>
> linuxhiker: oooo git interface! Bug tracker for this anywhere?

Potential answer: Yes. As of now, pgsql-docs for doc issues, and
pgsql-bugs for actual bugs :)
-- 
Michael

On 1/5/16 6:53 PM, Michael Paquier wrote:
> On Wed, Jan 6, 2016 at 7:32 AM, Joshua D. Drake <jd@commandprompt.com> wrote:
>> So I was on #postgresql today. I convinced a newer user that they could
>> easily contribute to PostgreSQL even if it was just doc patches. I described
>> the basic workflow and the individual was excited.
>>
>> His first question?
>>
>> linuxhiker: oooo git interface! Bug tracker for this anywhere?
>
> Potential answer: Yes. As of now, pgsql-docs for doc issues, and
> pgsql-bugs for actual bugs :)

Which doesn't help anyone, because neither of those provide a list of 
"hey, here's stuff you could do to contribute". The closest we come to 
that is the TODO, which isn't well known and has almost no items for 
newbies (and the newbie items that are there don't offer much advice).

The reason I this matters is because yesterday I posted a task for a new 
hacker with simple guidelines and 24 hours later it was completed[1]. 
That tells me there's people that would love to contribute but don't 
know how or what to contribute.

I realize a tracker *by itself* won't solve that, but it is the first 
place anyone that wants to contribute code is likely to look. So having 
one makes it more likely that new people will contribute.

On a related note, anyone interested in growing the community should 
take a look at [2]. tl;dr: best way to grow the community is to attract 
some folks that will make growing it their priority.

[1] http://www.postgresql.org/message-id/568ADA20.7090205@BlueTreble.com
[2] http://www.postgresql.org/message-id/568C6E6D.1040306@BlueTreble.com
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 01/05/2016 04:53 PM, Michael Paquier wrote:
> On Wed, Jan 6, 2016 at 7:32 AM, Joshua D. Drake <jd@commandprompt.com> wrote:
>> Hello,
>>
>> So I was on #postgresql today. I convinced a newer user that they could
>> easily contribute to PostgreSQL even if it was just doc patches. I described
>> the basic workflow and the individual was excited.
>>
>> His first question?
>>
>> linuxhiker: oooo git interface! Bug tracker for this anywhere?
>
> Potential answer: Yes. As of now, pgsql-docs for doc issues, and
> pgsql-bugs for actual bugs :)

Yes but that is a horrible answer.

JD


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

Jim, all,

* Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
> Which doesn't help anyone, because neither of those provide a list
> of "hey, here's stuff you could do to contribute". The closest we
> come to that is the TODO, which isn't well known and has almost no
> items for newbies (and the newbie items that are there don't offer
> much advice).

Agreed.  I've not given up on the bugs.p.o project, but I've gotten
wrapped up in migrating the buildfarm server on to our infrastructure.
Hopefully that will be completed as early as this week and I'll be able
to refocus my infra time to finishing bugs.p.o.

Thanks!

Stephen

On 1/5/16 8:19 PM, Stephen Frost wrote:
> * Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
>> Which doesn't help anyone, because neither of those provide a list
>> of "hey, here's stuff you could do to contribute". The closest we
>> come to that is the TODO, which isn't well known and has almost no
>> items for newbies (and the newbie items that are there don't offer
>> much advice).
>
> Agreed.  I've not given up on the bugs.p.o project, but I've gotten
> wrapped up in migrating the buildfarm server on to our infrastructure.
> Hopefully that will be completed as early as this week and I'll be able
> to refocus my infra time to finishing bugs.p.o.

FWIW, I think that's a great example of why we'd be much better off with 
a focus on expanding the community beyond code contributors. There's a 
couple hundred people on the whole planet with your skill at expanding 
Postgres itself and probably millions of people that could work on 
infrastructure. Not a great allocation of resources... ;)
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

Joshua D. Drake wrote:
> On 01/05/2016 04:53 PM, Michael Paquier wrote:
> >On Wed, Jan 6, 2016 at 7:32 AM, Joshua D. Drake <jd@commandprompt.com> wrote:

> >>linuxhiker: oooo git interface! Bug tracker for this anywhere?
> >
> >Potential answer: Yes. As of now, pgsql-docs for doc issues, and
> >pgsql-bugs for actual bugs :)
> 
> Yes but that is a horrible answer.

I wouldn't even call that an answer actually.  To me, that just means
"no, we don't have one", which is kinda sad.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 01/05/2016 06:30 PM, Jim Nasby wrote:
> On 1/5/16 8:19 PM, Stephen Frost wrote:
>> * Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
>>> Which doesn't help anyone, because neither of those provide a list
>>> of "hey, here's stuff you could do to contribute". The closest we
>>> come to that is the TODO, which isn't well known and has almost no
>>> items for newbies (and the newbie items that are there don't offer
>>> much advice).
>>
>> Agreed.  I've not given up on the bugs.p.o project, but I've gotten
>> wrapped up in migrating the buildfarm server on to our infrastructure.
>> Hopefully that will be completed as early as this week and I'll be able
>> to refocus my infra time to finishing bugs.p.o.
>
> FWIW, I think that's a great example of why we'd be much better off with
> a focus on expanding the community beyond code contributors. There's a
> couple hundred people on the whole planet with your skill at expanding
> Postgres itself and probably millions of people that could work on
> infrastructure. Not a great allocation of resources... ;)

I have excellent people right now I could (and would) assign to the task 
if the task had guidelines.

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Wed, Jan 6, 2016 at 3:11 AM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
> Which doesn't help anyone, because neither of those provide a list of "hey,
> here's stuff you could do to contribute". The closest we come to that is the
> TODO, which isn't well known and has almost no items for newbies (and the
> newbie items that are there don't offer much advice).
>
> The reason I this matters is because yesterday I posted a task for a new
> hacker with simple guidelines and 24 hours later it was completed[1]. That
> tells me there's people that would love to contribute but don't know how or
> what to contribute.

Jim,

I want to explicitly thank you for your post about that task, I would
like to see more of that. I share the sentiment that there are more
people wanting to contribute but finding it rather hard to find a
starting point. I was (am?) in that position and so far found two ways
out:

1. I looked at the commit fest patches as a list of things to
contribute to (by doing a review which is currently in progress)
2. Robert at some point mentioned in an email "someone could improve
the docs in this patch" so I did that

But I can see that 1. is intimidating to do for new people that might
think "how can you review without ever having looked at the code
before?". Turns out you can and the wiki mentions it explicitly but
it's probably still intimidating for some. And 2. required noticing
Robert's sentence in the middle of a long email thread.

I also think a list of small things suitable for new contributors
would help attracting them. Not all would stick and go on to larger
items but hopefully at least some would.

On Wed, Jan 6, 2016 at 10:43 AM, Catalin Iacob <iacobcatalin@gmail.com> wrote:
> I also think a list of small things suitable for new contributors
> would help attracting them. Not all would stick and go on to larger
> items but hopefully at least some would.

I agree with this.  Curating such a list is a fair amount of work that
somebody's got to do, though.  The TODO list is full of an awful lot
of things that don't matter and missing an awful lot of things that
do.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 1/6/16 2:18 PM, Robert Haas wrote:
> On Wed, Jan 6, 2016 at 10:43 AM, Catalin Iacob <iacobcatalin@gmail.com> wrote:
>> I also think a list of small things suitable for new contributors
>> would help attracting them. Not all would stick and go on to larger
>> items but hopefully at least some would.
>
> I agree with this.  Curating such a list is a fair amount of work that
> somebody's got to do, though.  The TODO list is full of an awful lot
> of things that don't matter and missing an awful lot of things that
> do.

Right. Personally, I feel the TODO has pretty much outlived it's 
usefulness. An issue tracker would make maintaining items like this a 
lot more reasonable, but it certainly wouldn't be free.

Something else to consider though: I sent one email and the task was 
done in 24 hours. For things that can be well defined and are fairly 
mechanical, I suspect an email to hackers with a big [NEW HACKER] banner 
would do wonders.

Related to that is JD's offer to donate staff time to infrastructure 
work. There's probably a fair amount of things that could be "farmed 
out" that way. Some folks in the community proper would still need to 
keep tabs on things, but they wouldn't have to do the gruntwork. If, 
say, the Ops teams at 2nd Quadrant, CMD, and EDB wanted to work together 
on improving infrastructure, that's pretty much community at that point, 
and not a dependence on a single external entity.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

Robert Haas <robertmhaas@gmail.com> writes:
> On Wed, Jan 6, 2016 at 10:43 AM, Catalin Iacob <iacobcatalin@gmail.com> wrote:
>> I also think a list of small things suitable for new contributors
>> would help attracting them. Not all would stick and go on to larger
>> items but hopefully at least some would.

> I agree with this.  Curating such a list is a fair amount of work that
> somebody's got to do, though.  The TODO list is full of an awful lot
> of things that don't matter and missing an awful lot of things that
> do.

Yeah.  The other problem is that stuff that's actually small doesn't tend
to hang around undone for long, so there's not really a broad array of
stuff just waiting for someone to have a little time.  If we had a more
actively maintained TODO list, it would largely contain (a) stuff that
there's insufficient consensus on, and (b) stuff that's just big mean and
nasty to implement.

Having said that, it occurs to me that one way to contribute without
actually writing C code would be to try to drive those unfinished
discussions to consensus, and come up with specs for features that people
agree are well-thought-out.  Conversely, establishing a consensus that a
proposal is a bad idea and it should go away from the list would be a
useful activity.
        regards, tom lane

On 1/6/16 5:51 PM, Tom Lane wrote:
> Having said that, it occurs to me that one way to contribute without
> actually writing C code would be to try to drive those unfinished
> discussions to consensus, and come up with specs for features that people
> agree are well-thought-out.  Conversely, establishing a consensus that a
> proposal is a bad idea and it should go away from the list would be a
> useful activity.

+1.

Somewhat related to that, I don't believe there's any reason why commit 
fest managers need to be committers; it seems like the job is really 
just about reading through email activity to see where things are at.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 01/06/2016 03:57 PM, Jim Nasby wrote:
> On 1/6/16 5:51 PM, Tom Lane wrote:
>> Having said that, it occurs to me that one way to contribute without
>> actually writing C code would be to try to drive those unfinished
>> discussions to consensus, and come up with specs for features that people
>> agree are well-thought-out.  Conversely, establishing a consensus that a
>> proposal is a bad idea and it should go away from the list would be a
>> useful activity.
>
> +1.
>
> Somewhat related to that, I don't believe there's any reason why commit
> fest managers need to be committers; it seems like the job is really
> just about reading through email activity to see where things are at.

Agreed. That is a great way for people to contribute that aren't "hackers".

JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Wed, Jan 6, 2016 at 11:42 PM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
> Right. Personally, I feel the TODO has pretty much outlived it's usefulness.
> An issue tracker would make maintaining items like this a lot more
> reasonable, but it certainly wouldn't be free.

Eh, a bug tracker that tracks actual bugs would be useful, I don't
think anyone would argue with that. A vague "issue" tracker that just
collects ideas people have had that seemed like a good idea at some
time in history would suffer exactly the same problem the TODO has.

-- 
greg

Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
> Somewhat related to that, I don't believe there's any reason why commit 
> fest managers need to be committers; it seems like the job is really 
> just about reading through email activity to see where things are at.

They don't need to be.  Michael Paquier did a fine job with the last CF.
        regards, tom lane

On Wed, Jan 6, 2016 at 3:51 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Yeah.  The other problem is that stuff that's actually small doesn't tend
> to hang around undone for long, so there's not really a broad array of
> stuff just waiting for someone to have a little time.  If we had a more
> actively maintained TODO list, it would largely contain (a) stuff that
> there's insufficient consensus on, and (b) stuff that's just big mean and
> nasty to implement.

I think that some friendly advice to less experienced contributors
about a good project for them to work on is enormously valuable, which
is why I try to do that whenever I can. Unfortunately, and perversely,
the TODO list is pretty far from that. Things go on the todo list
because they don't have a favorable cost/benefit ratio. I wouldn't
suggest a project to anyone that I would not be willing to work on
myself, which excludes most items on the TODO list.

-- 
Peter Geoghegan

On Thu, Jan 7, 2016 at 9:28 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
>> Somewhat related to that, I don't believe there's any reason why commit
>> fest managers need to be committers; it seems like the job is really
>> just about reading through email activity to see where things are at.
>
> They don't need to be.

More thoughts on that. I would even think the contrary, someone who
has just monitored for 1/2 years -hackers and knowing how things are
handled would be able to do this job as well, the only issue being to
juggle with many threads at the same time, to be sure that each patch
status is correct, and to not lose motivation during the CF. The last
one is the hardest by far.
-- 
Michael

On 1/6/16 6:18 PM, Greg Stark wrote:
> On Wed, Jan 6, 2016 at 11:42 PM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
>> Right. Personally, I feel the TODO has pretty much outlived it's usefulness.
>> An issue tracker would make maintaining items like this a lot more
>> reasonable, but it certainly wouldn't be free.
>
> Eh, a bug tracker that tracks actual bugs would be useful, I don't
> think anyone would argue with that. A vague "issue" tracker that just
> collects ideas people have had that seemed like a good idea at some
> time in history would suffer exactly the same problem the TODO has.

I think one of the biggest hurdles people face is getting the community 
to agree that something is a desired feature. So if there was a list of 
things that the community had agreed would be good I think that itself 
would be useful. Even better if items had a rough outline of the work 
necessary.

Obviously that won't do too much for really big features. But if our 
experienced hackers focused less on coding and more on design and 
creating smaller tasks that people could work on, more people could 
potentially be put to work.

ISTM that the design work needs to be done and documented no matter 
what, so there shouldn't be much overhead there. The overhead would be 
in maintaining the tracker and making sure folks were actively getting 
stuff done. That can be done by a non-coder. That means it shouldn't 
really cost the community much in terms of current resources, as long as 
we attract new people to take on these new tasks.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 1/6/16 9:22 PM, Michael Paquier wrote:
> On Thu, Jan 7, 2016 at 9:28 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
>>> Somewhat related to that, I don't believe there's any reason why commit
>>> fest managers need to be committers; it seems like the job is really
>>> just about reading through email activity to see where things are at.
>>
>> They don't need to be.
>
> More thoughts on that. I would even think the contrary, someone who
> has just monitored for 1/2 years -hackers and knowing how things are
> handled would be able to do this job as well, the only issue being to
> juggle with many threads at the same time, to be sure that each patch
> status is correct, and to not lose motivation during the CF. The last
> one is the hardest by far.

Sorry, I inadvertently used 'committer' when I should have said 'coder'. 
There's nothing code related about CFM, and I think it's a dis-service 
to the community to have coders serving that role.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 01/06/2016 04:18 PM, Greg Stark wrote:
> On Wed, Jan 6, 2016 at 11:42 PM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
>> Right. Personally, I feel the TODO has pretty much outlived it's usefulness.
>> An issue tracker would make maintaining items like this a lot more
>> reasonable, but it certainly wouldn't be free.
>
> Eh, a bug tracker that tracks actual bugs would be useful, I don't
> think anyone would argue with that. A vague "issue" tracker that just
> collects ideas people have had that seemed like a good idea at some
> time in history would suffer exactly the same problem the TODO has.

Not if it was probably integrated it wouldn't. The problem with the TODO 
is that it is in no mans land of the wiki and there is ZERO structure to it.

The wiki is the nosql of the postgresql community. ;)

With an issue tracker you can say:

This is a IN PROGRESS TODO
This is an APPROVED TODO
This is a ISSUE-> BUG CONFIRMED-> USER ERROR

etc....

And if the right software is used, it can all be done via email AND can 
be tracked a hell of a lot easier than the way we track everything 
(literally) now. It is a hell of a lot easier to say:

See #53466

Than every alternative we currently have.

JD


>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Wed, Jan 6, 2016 at 4:18 PM, Greg Stark <stark@mit.edu> wrote:
> On Wed, Jan 6, 2016 at 11:42 PM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
>> Right. Personally, I feel the TODO has pretty much outlived it's usefulness.
>> An issue tracker would make maintaining items like this a lot more
>> reasonable, but it certainly wouldn't be free.
>
> Eh, a bug tracker that tracks actual bugs would be useful, I don't
> think anyone would argue with that. A vague "issue" tracker that just
> collects ideas people have had that seemed like a good idea at some
> time in history would suffer exactly the same problem the TODO has.

I don't completely agree with that.  I have often wanted to know when
a specific item was added to the TODO page, and/or its individual edit
history.  With only a unified history of the entire TODO page, and
with no wiki equivalent of "git blame", figuring this out is extremely
tedious.  A tracker would precisely solve this problem, if nothing
else.  And when I edit the wiki and forget to make a coherent edit
summary, there is no way to fix that, while presumably an issue
tracker would be more tolerant of people's imperfections.

It could also be ameliorated without a tracker by people being more
disciplined about linking to the email archives, but evidently we are
not disciplined enough to do that reliably enough.  I think we are
better about that recently than we were in the past, but without the
ability to readily see when an item was added, it is hard to go back
and find the emails to fix the past mistakes.

But, if we want a list of projects for beginners, I think it has to be
explicitly that.  A list of things an experienced expert could do
trivially, but are consciously refraining from doing so that a
beginner can do them instead.  It would need to be separate from a "we
can't decide if we want this or can't decide how to do it or don't
have the time to do it" list.  There is no reason we have to have an
issue tracker in order to create that separation, but it could help.

Cheers,

Jeff

On 01/07/2016 10:30 AM, Jeff Janes wrote:
> I don't completely agree with that.  I have often wanted to know when
> a specific item was added to the TODO page, and/or its individual edit
> history.  With only a unified history of the entire TODO page, and
> with no wiki equivalent of "git blame", figuring this out is extremely
> tedious.  A tracker would precisely solve this problem, if nothing
> else.  And when I edit the wiki and forget to make a coherent edit
> summary, there is no way to fix that, while presumably an issue
> tracker would be more tolerant of people's imperfections.

Yeah, we could also get rid of this conversation:

"Here's a patch for X, which is on the TODO list"

"Oh, we've obsolesced that, that was added to the TODO before we had Y"

... by auto-closing TODO items at a certain age.

-- 
Josh Berkus
Red Hat OSAS
(opinions are my own)

On 1/7/16 1:49 PM, Josh Berkus wrote:
> Yeah, we could also get rid of this conversation:
>
> "Here's a patch for X, which is on the TODO list"
>
> "Oh, we've obsolesced that, that was added to the TODO before we had Y"
>
> ... by auto-closing TODO items at a certain age.

Even if not auto-closed at least it'd be easy to find old items.

Bonus points if we attract some volunteer project managers that will 
keep tabs of all those kinds of things...
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On 01/07/2016 12:32 PM, Jim Nasby wrote:
> On 1/7/16 1:49 PM, Josh Berkus wrote:
>> Yeah, we could also get rid of this conversation:
>>
>> "Here's a patch for X, which is on the TODO list"
>>
>> "Oh, we've obsolesced that, that was added to the TODO before we had Y"
>>
>> ... by auto-closing TODO items at a certain age.
>
> Even if not auto-closed at least it'd be easy to find old items.
>
> Bonus points if we attract some volunteer project managers that will
> keep tabs of all those kinds of things...

/me waves hand....

There are quite a few contributing companies that likely have people 
that could help out with this in an educated fashion but aren't coders.


JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Thu, Jan 7, 2016 at 6:30 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
> I don't completely agree with that.  I have often wanted to know when
> a specific item was added to the TODO page, and/or its individual edit
> history.

Sure, there might be other things it would be better at. But my point
is that it would have the same problem as the wiki in that it would
accumulate vague ideas that someone thought was a good idea once but
didn't have a good idea how to implement or a compelling argument that
convinced others to work on it.

The wiki is the lowest overhead and highest visibility way of
maintaining communal information. Bug trackers exist to impose extra
structure to match an intended workflow. That's fine for bugs or a
closely managed project but it's the last thing you want if you're
trying to get more people to contribute. The whole selling points of
wikis is that they draw in contributors because anyone can edit
easily.

This really sounds like you're looking for leverage to fix one problem
by finding other problems that you hope to solve with the same hammer.
That's a recipe for a tool that solves neither problem well and gets
ignored by the both sets of users.

-- 
greg

Magnus Hagander <magnus@hagander.net> writes:
> On Thu, Jan 7, 2016 at 10:57 PM, Joshua D. Drake <jd@commandprompt.com>
>> There are quite a few contributing companies that likely have people that
>> could help out with this in an educated fashion but aren't coders.

> Sort of like how they could also have helped out with cf management?

Dunno about that.  While a CF manager doesn't necessarily have to have
a commit bit, I think he/she probably does have to be someone who is known
and respected on the -hackers list.  Otherwise, nagging to finish patches
is likely to be ignored or even counterproductive.
        regards, tom lane

On 1/8/16 9:07 AM, Tom Lane wrote:
> Dunno about that.  While a CF manager doesn't necessarily have to have
> a commit bit, I think he/she probably does have to be someone who is known
> and respected on the -hackers list.  Otherwise, nagging to finish patches
> is likely to be ignored or even counterproductive.

IMHO that touches on the core issue here. JD and anyone else can offer 
up all the resources under the sun, but *it's up to the community to 
decide to use them*. In this specific case, if the community agrees that 
someone that's not a code contributor will be the CFM for a specific CF 
then I think it would work fine. Short of that, your observation might 
be very accurate.

I feel a bit bad about some of the emails I've sent on this and related 
topics because it feels hand-wavy, but I don't know what else to do. I 
can promote these ideas that I think will be very helpful, but without a 
strong consensus in the community that it's desirable any efforts will 
probably fail.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com

On Fri, Jan 8, 2016 at 7:12 AM, Greg Stark <stark@mit.edu> wrote:
> This really sounds like you're looking for leverage to fix one problem
> by finding other problems that you hope to solve with the same hammer.
> That's a recipe for a tool that solves neither problem well and gets
> ignored by the both sets of users.

+1

merlin

On Fri, Jan 8, 2016 at 3:37 AM, Magnus Hagander <magnus@hagander.net> wrote:
>> /me waves hand....
>>
>> There are quite a few contributing companies that likely have people that
>> could help out with this in an educated fashion but aren't coders.
>
> Sort of like how they could also have helped out with cf management?

I agree with this sentiment.  But let me also say this: managing a
CommitFest well is a Hard Job.  It takes a very sizable amount of
time, a fair amount of technical knowledge, and an awful lot of
emotional energy.  It's a completely thankless task: if you do it
well, some people are unhappy because their patches got bumped; if you
do it poorly, other people (or the same ones) are unhappy because the
CommitFest lasted too long.  If you use the wrong tone in writing an
email to somebody, they will flame you as if you were trying to hurt
them rather than, as is actually the case, trying to help the
community.  And you can't make anybody do anything: if not enough
reviewers turn up, or not enough committers turn up, you will fail,
even if you do everything right.  Something under half of the attempts
to do this over the years have succeeded brilliantly (at least, IMHO);
many others have been indifferent successes or else failures, despite
the attempts having been made by community members of long standing.
It's just a really tough problem - you've got to be a combination
motivational speaker, technical whiz, diplomat, and organizational
guru to succeed.

It's unclear to me whether administering the bug tracker would be an
easier job or not.  I think it would probably be somewhat easier.
There's probably not a lot that's real subjective about deciding which
bugs we fixed and which ones we're not gonna fix.  I think the biggest
problem would likely be that we might occasionally have cases where
somebody submits something and somebody from the community replies to
say "we don't really care" and then the bug gets closed and the
submitter gets an email and goes ballistic, and unproductive arguing
ensues.  It will be important to have an understanding that the person
updating the tracker is merely trying to make it reflect the expressed
will of the community, not trying to determine that will.  If somebody
disagrees with the status applied to the bug, they should argue with
the community members who said "we don't care", not the person who set
the status to won't-fix.

If we had an "issue" tracker rather than a bug tracker, I'd expect a
lot more unproductive bickering.  The consensus around which features
are worth having changes frequently, and a lot of features that nobody
desperately objects to are nevertheless awfully low-value and not
worth tracking until somebody comes up with a patch.  Development of
feature X often changes the perceived value of feature Y, sometimes in
a positive way and sometimes in a negative way.  I don't really have
any use for a system that's just a random collection of things
somebody wanted at some point, which is basically what the TODO list
is.  A list of unfixed bugs, though, sounds like it might have real
utility, particularly if we got some volunteers to apply tags to them
so we could search for "multixact" or whatever.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 01/10/2016 06:19 PM, Robert Haas wrote:

[snip]

No arguments with what was written above.

> If we had an "issue" tracker rather than a bug tracker, I'd expect a
> lot more unproductive bickering.

This I disagree with. It wouldn't be any worse than we have now. An 
issue tracker (if it is a good one) becomes the forum, whether you use 
an email or web interface. So expect at least as much banter as there is 
on lists but with a much easier management interface.

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 01/11/2016 03:31 AM, Magnus Hagander wrote:

>
> We can argue about if it's actually an easier management interface ;)

(as long as it is manageable via email as well as web?)

>
> I'd agree with Robert in that it will cause some more such bickering --
> but only because the discussions become visible to a new group of people
> as well. That's not necessarily a bad thing. But thinking that having
> such an issue tracker is actually going to help *get rid of* the
> pointless part of things is a nice dream, but just a dream. The only
> advantage I can see of it is to increase the visibility of them.

Oh, I think you are right there. My point is that we have the ability to 
better manage that inforomation. We can mark something as a bug, feature 
request, approved feature, feature in development etc... Things become 
much more contextually aware.

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Wed, Jan  6, 2016 at 03:18:49PM -0500, Robert Haas wrote:
> On Wed, Jan 6, 2016 at 10:43 AM, Catalin Iacob <iacobcatalin@gmail.com> wrote:
> > I also think a list of small things suitable for new contributors
> > would help attracting them. Not all would stick and go on to larger
> > items but hopefully at least some would.
> 
> I agree with this.  Curating such a list is a fair amount of work that
> somebody's got to do, though.  The TODO list is full of an awful lot
> of things that don't matter and missing an awful lot of things that
> do.

So, if that is true, and people are not improving the TODO list
situation, how likely will a bug tracker be at improving or
deteriorating the situation?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Roman grave inscription                             +