Thread: GSoC 2018

GSoC 2018

From
Stephen Frost
Date:
Greetings -hackers,

Google Summer of Code 2018 was announced back in September and they've
changed up the timeline a bit [1].  Specifically, they moved up the
dates for things like the mentoring organization application deadline,
so it's time to start working on our Idea's page for 2018 in earnest.

The deadline for Mentoring organizations to apply is: January 23.

The list of accepted organization will be published around February 12.

Unsurprisingly, we'll need to have an Ideas page again, so I've gone
ahead and created one (copying last year's):

https://wiki.postgresql.org/wiki/GSoC_2018

Google discusses what makes a good "Ideas" list here:

https://google.github.io/gsocguides/mentor/defining-a-project-ideas-list.html

I marked all the entries with '2017' to indicate they were pulled from
last year.  If the project from last year is still relevant, please
update it to be '2018' and make sure to update all of the information
(in particular, make sure to list yourself as a mentor and remove the
other mentors, as appropriate).

New entries are certainly welcome and encouraged, just be sure to note
them as '2018' when you add it.

Projects from last year which were worked on but have significant
follow-on work to be completed are absolutely welcome as well- simply
update the description appropriately and mark it as being for '2018'.

When we get closer to actually submitting our application, I'll clean
out the '2017' entries that didn't get any updates.

As a reminder, each idea on the page should be in the format that the
other entries are in and should include:

- Project title/one-line description
- Brief, 2-5 sentence, description of the project (remember, these are
  12-week projects)
- Description of programming skills needed and estimation of the
  difficulty level
- List of potential mentors
- Expected Outcomes

As with last year, please consider PostgreSQL to be an "Umbrella"
project and that anything which would be considered "PostgreSQL Family"
per the News/Announce policy [2] is likely to be acceptable as a
PostgreSQL GSoC project.

In other words, if you're a contributor or developer on barman,
pgBackRest, the PostgreSQL website (pgweb), the PgEU/PgUS website code
(pgeu-website), pgAdmin4, PostgresXL, pgbouncer, Citus, pldebugger, the
PG RPMs (pgrpms), the JDBC driver, the ODBC driver, or any of the many
other PG Family projects, please feel free to add a project for
consideration!  If we get quite a few, we can organize the page further
based on which project or maybe what skills are needed or similar.

Let's have another great year of GSoC with PostgreSQL!

Thanks!

Stephen

[1]: https://developers.google.com/open-source/gsoc/timeline
[2]: https://wiki.postgresql.org/wiki/NewsEventsApproval

Attachment

Re: GSoC 2018

From
Aleksander Alekseev
Date:
Hi Stephen,

> New entries are certainly welcome and encouraged, just be sure to note
> them as '2018' when you add it.

I proposed a few ideas:

* High availability / failover based on logical replication
* Thrift datatype support

Hope these ideas are good enough for GSoC.

--
Best regards,
Aleksander Alekseev

Attachment

Re: GSoC 2018

From
Stefan Keller
Date:
Hi,

2017-12-15 4:14 GMT+01:00 Stephen Frost <sfrost@snowman.net>:
> Unsurprisingly, we'll need to have an Ideas page again, so I've gone
> ahead and created one (copying last year's):

What about adding "Learned Index" as project task [*]?
This type of index looks promising for certain properties.

:Stefan

[*] "The Case for Learned Index Structures" Kraska et al. (Dec 2017)
https://arxiv.org/abs/1712.01208


2017-12-15 4:14 GMT+01:00 Stephen Frost <sfrost@snowman.net>:
> Greetings -hackers,
>
> Google Summer of Code 2018 was announced back in September and they've
> changed up the timeline a bit [1].  Specifically, they moved up the
> dates for things like the mentoring organization application deadline,
> so it's time to start working on our Idea's page for 2018 in earnest.
>
> The deadline for Mentoring organizations to apply is: January 23.
>
> The list of accepted organization will be published around February 12.
>
> Unsurprisingly, we'll need to have an Ideas page again, so I've gone
> ahead and created one (copying last year's):
>
> https://wiki.postgresql.org/wiki/GSoC_2018
>
> Google discusses what makes a good "Ideas" list here:
>
> https://google.github.io/gsocguides/mentor/defining-a-project-ideas-list.html
>
> I marked all the entries with '2017' to indicate they were pulled from
> last year.  If the project from last year is still relevant, please
> update it to be '2018' and make sure to update all of the information
> (in particular, make sure to list yourself as a mentor and remove the
> other mentors, as appropriate).
>
> New entries are certainly welcome and encouraged, just be sure to note
> them as '2018' when you add it.
>
> Projects from last year which were worked on but have significant
> follow-on work to be completed are absolutely welcome as well- simply
> update the description appropriately and mark it as being for '2018'.
>
> When we get closer to actually submitting our application, I'll clean
> out the '2017' entries that didn't get any updates.
>
> As a reminder, each idea on the page should be in the format that the
> other entries are in and should include:
>
> - Project title/one-line description
> - Brief, 2-5 sentence, description of the project (remember, these are
>   12-week projects)
> - Description of programming skills needed and estimation of the
>   difficulty level
> - List of potential mentors
> - Expected Outcomes
>
> As with last year, please consider PostgreSQL to be an "Umbrella"
> project and that anything which would be considered "PostgreSQL Family"
> per the News/Announce policy [2] is likely to be acceptable as a
> PostgreSQL GSoC project.
>
> In other words, if you're a contributor or developer on barman,
> pgBackRest, the PostgreSQL website (pgweb), the PgEU/PgUS website code
> (pgeu-website), pgAdmin4, PostgresXL, pgbouncer, Citus, pldebugger, the
> PG RPMs (pgrpms), the JDBC driver, the ODBC driver, or any of the many
> other PG Family projects, please feel free to add a project for
> consideration!  If we get quite a few, we can organize the page further
> based on which project or maybe what skills are needed or similar.
>
> Let's have another great year of GSoC with PostgreSQL!
>
> Thanks!
>
> Stephen
>
> [1]: https://developers.google.com/open-source/gsoc/timeline
> [2]: https://wiki.postgresql.org/wiki/NewsEventsApproval


Re: GSoC 2018

From
Andrey Borodin
Date:
Hi!

> 15 дек. 2017 г., в 15:03, Stefan Keller <sfkeller@gmail.com> написал(а):
> What about adding "Learned Index" as project task [*]?
> This type of index looks promising for certain properties.

I have no high expectation from this idea. But feel that it is necessary at least to validate the idea. I'd sign up as
co-mentorfor this. Moreover this will attract students from huge pool of ML hackers. 

Also, I'm planning to add my previous-year project about sorting. Or add something WAL-G-related under Postgres
umbrella(probably, WAL-scanning for delta-backups). 

Best regards, Andrey Borodin&

Re: GSoC 2018

From
Stephen Frost
Date:
Stefan,

* Stefan Keller (sfkeller@gmail.com) wrote:
> 2017-12-15 4:14 GMT+01:00 Stephen Frost <sfrost@snowman.net>:
> > Unsurprisingly, we'll need to have an Ideas page again, so I've gone
> > ahead and created one (copying last year's):
>
> What about adding "Learned Index" as project task [*]?
> This type of index looks promising for certain properties.

If you can provide a *specific* description with expected outcomes and
we have someone who can mentor such a project, then I think it would be
acceptable.

We would need to get any potential students on the mailing list before
the project is submitted to try and fill out the specifics around what
such an index would actually look like too, I think.  Without that
guideance, the student proposal would be pretty hard to accept I
suspect.

Thanks!

Stephen

Attachment

Re: GSoC 2018

From
Stephen Frost
Date:
Aleksander,

* Aleksander Alekseev (a.alekseev@postgrespro.ru) wrote:
> > New entries are certainly welcome and encouraged, just be sure to note
> > them as '2018' when you add it.
>
> I proposed a few ideas:

Thanks!

> * High availability / failover based on logical replication
> * Thrift datatype support
>
> Hope these ideas are good enough for GSoC.

The main things are to have a detailed enough description that someone
can write a project plan about what they're going to do over the summer
on that project and the project is of a reasonable size.

HA/fail-over is a very broad topic, with a lot of pieces that need to be
done such that I'm not sure it's really viable, but perhaps a precursor
project (synchronous logical replication seems like a prereq, no?) would
make more sense.  Or, perhaps, a different piece of the HA question, but
solving the whole thing in a summer strikes me as unlikely to be
reasonable.

Regarding the thrift data type, that seems like a pretty good GSoC
project, though I'm not sure why you suggest having pl/pgsql functions
for accessing data from .thrift files- plpgsql can't directly access the
filesystem and the input routines for .thrift-style data would certainly
be best in C.

Thanks!

Stephen

Attachment

Re: GSoC 2018

From
Aleksander Alekseev
Date:
Hi Stephen,

> HA/fail-over is a very broad topic, with a lot of pieces that need to be
> done such that I'm not sure it's really viable, but perhaps a precursor
> project (synchronous logical replication seems like a prereq, no?) would
> make more sense.  Or, perhaps, a different piece of the HA question, but
> solving the whole thing in a summer strikes me as unlikely to be
> reasonable.

Frankly I got the impression that logical replication in PostgreSQL 10
is as synchronous as physical replication in terms that it treats
synchronous_commit the same way and gives exactly same guarantees. Is
there a problem with current implementation of logical replication I'm
not aware of? Or by synchronous logical replication you meant something
different?

Regarding the difficulty of the project - in fact it's not that
difficult. Particularly this project can rely on external tools, e.g.
use Consul for service discovery and leader election based on
leader-lease approach (implementation [1]). Having this the only thing
is missing is automatic replica promoting and (optionally)
re-configuring of HAProxy / pgbouncer / whatever. Yes, and lots of
Jepsen-like test of course. I believe it's not such a complicated
project.

> Regarding the thrift data type, that seems like a pretty good GSoC
> project, though I'm not sure why you suggest having pl/pgsql functions
> for accessing data from .thrift files- plpgsql can't directly access the
> filesystem and the input routines for .thrift-style data would certainly
> be best in C.

What I meant was generating PL/pgSQL procedures from .thift files, like
`MyStructure_getFieldX(bytea) returns text`. It took me a few hours to
write pg_protobuf so I decided the project would be way to easy without
implementing such tool. I changed the text on wiki, hopefully it's
better now.

[1]: https://github.com/afiskon/go-elector

--
Best regards,
Aleksander Alekseev

Attachment

Re: GSoC 2018

From
Stephen Frost
Date:
Aleksander,

* Aleksander Alekseev (a.alekseev@postgrespro.ru) wrote:
> > HA/fail-over is a very broad topic, with a lot of pieces that need to be
> > done such that I'm not sure it's really viable, but perhaps a precursor
> > project (synchronous logical replication seems like a prereq, no?) would
> > make more sense.  Or, perhaps, a different piece of the HA question, but
> > solving the whole thing in a summer strikes me as unlikely to be
> > reasonable.
>
> Frankly I got the impression that logical replication in PostgreSQL 10
> is as synchronous as physical replication in terms that it treats
> synchronous_commit the same way and gives exactly same guarantees. Is
> there a problem with current implementation of logical replication I'm
> not aware of? Or by synchronous logical replication you meant something
> different?

synchronous_commit isn't the same as synchronous_standby_names ...

We could have used better names for those GUCs, I suspect.

> Regarding the difficulty of the project - in fact it's not that
> difficult. Particularly this project can rely on external tools, e.g.
> use Consul for service discovery and leader election based on
> leader-lease approach (implementation [1]). Having this the only thing
> is missing is automatic replica promoting and (optionally)
> re-configuring of HAProxy / pgbouncer / whatever. Yes, and lots of
> Jepsen-like test of course. I believe it's not such a complicated
> project.

What you're talking about is rebuilding Patroni, but adding more into it
than even Patroni tries to do, building it on Logical Replication
instead of physical replication, and calling it simple and something
that could get into core PostgreSQL over a 12 week GSoC project.  I've
certainly got doubts about that, even if we decide that it'd be an
external-to-PG project (like Patroni).

What might be interesting is seeing if Logical Replication could be
added to Patroni as an option and then building on that..  Having
someone involved in the Patroni project would be the right way to go
about proposing that though to see what they think of it.  That would
also be much more sensible as a GSoC project, since it'd be an addition
to an existing project and not more-or-less starting a whole new
project.

> > Regarding the thrift data type, that seems like a pretty good GSoC
> > project, though I'm not sure why you suggest having pl/pgsql functions
> > for accessing data from .thrift files- plpgsql can't directly access the
> > filesystem and the input routines for .thrift-style data would certainly
> > be best in C.
>
> What I meant was generating PL/pgSQL procedures from .thift files, like
> `MyStructure_getFieldX(bytea) returns text`. It took me a few hours to
> write pg_protobuf so I decided the project would be way to easy without
> implementing such tool. I changed the text on wiki, hopefully it's
> better now.

Wouldn't it make more sense to have something like what we have for
JSONB where there's operators that are used to pull out specific fields
instead of generated pl/pgsql procedures..?

Thanks!

Stephen

Attachment

Re: GSoC 2018

From
Aleksander Alekseev
Date:
Hi Stephen,

> synchronous_commit isn't the same as synchronous_standby_names ...

What about synchronous_standby_names? Logical replication ignores it and
doesn't wait for ack's from replicas or something?

> What might be interesting is seeing if Logical Replication could be
> added to Patroni as an option and then building on that..  Having
> someone involved in the Patroni project would be the right way to go
> about proposing that though to see what they think of it.  That would
> also be much more sensible as a GSoC project, since it'd be an addition
> to an existing project and not more-or-less starting a whole new
> project.

Completely agree, this project can be an improvement for Stolon (or
Patroni, but I personally never tested or used it, also I got a feeling
that Google guys will prefer a project that is written in Go). This
would make much more sense.

> Wouldn't it make more sense to have something like what we have for
> JSONB where there's operators that are used to pull out specific fields
> instead of generated pl/pgsql procedures..?

I don't see why we can't have both procedures and operators like:

```
x -> 'field1' -- returns int
x ->> 'field2' -- returns string
```

The reason why some users may prefer procedures to operators is
that it's easy to make a typo in a field name. Procedures are safer in
this regard.

--
Best regards,
Aleksander Alekseev

Attachment

Re: GSoC 2018

From
Arseny Sher
Date:
Aleksander Alekseev <a.alekseev@postgrespro.ru> writes:

> Hi Stephen,
>
>> synchronous_commit isn't the same as synchronous_standby_names ...
>
> What about synchronous_standby_names? Logical replication ignores it and
> doesn't wait for ack's from replicas or something?

It actually works, see [1]:

The name of a standby server for this purpose is the application_name
setting of the standby, as set in the standby's connection
information. In case of a physical replication standby, this should be
set in the primary_conninfo setting in recovery.conf; the default is
walreceiver. For logical replication, this can be set in the connection
information of the subscription, and it defaults to the subscription
name.

[1] https://www.postgresql.org/docs/current/static/runtime-config-replication.html

--
Arseny Sher
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company 


Re: GSoC 2018

From
Alex Kliukin
Date:
Hello,

On Fri, Dec 15, 2017, at 14:30, Stephen Frost wrote:
> Aleksander,
> 
> * Aleksander Alekseev (a.alekseev@postgrespro.ru) wrote:
> 
> > Regarding the difficulty of the project - in fact it's not that
> > difficult. Particularly this project can rely on external tools, e.g.
> > use Consul for service discovery and leader election based on
> > leader-lease approach (implementation [1]). Having this the only thing
> > is missing is automatic replica promoting and (optionally)
> > re-configuring of HAProxy / pgbouncer / whatever. Yes, and lots of
> > Jepsen-like test of course. I believe it's not such a complicated
> > project.

Does it make sense to address the  limitations of the logical
replication first, i.e. inability to replicate DDL, sequences and so on?

> 
> What you're talking about is rebuilding Patroni, but adding more into it
> than even Patroni tries to do, building it on Logical Replication
> instead of physical replication, and calling it simple and something
> that could get into core PostgreSQL over a 12 week GSoC project.  I've
> certainly got doubts about that, even if we decide that it'd be an
> external-to-PG project (like Patroni).
> 
> What might be interesting is seeing if Logical Replication could be
> added to Patroni as an option and then building on that..  Having
> someone involved in the Patroni project would be the right way to go
> about proposing that though to see what they think of it.  That would
> also be much more sensible as a GSoC project, since it'd be an addition
> to an existing project and not more-or-less starting a whole new
> project.

Right now logical replication and  physical replication-based HA tools
don't work together nicely, since logical replication position is not
propagated to the promoted replica (I think Craig Ringer has been
tackling this issue for a few releases already, the latest set of
patches I could find is https://commitfest.postgresql.org/15/788/).
Perhaps there is opportunity for a GSoC student to help fixing it. Until
then  we cannot use logical replication for HA, and even doing something
simpler like automating creation of logical replicas in Patroni makes
little sense, as they are doomed to be reinitialized on every failover.

-- 
Sincerely,
Alex


Re: GSoC 2018

From
Stephen Frost
Date:
Alex,

* Alex Kliukin (alexk@hintbits.com) wrote:
> On Fri, Dec 15, 2017, at 14:30, Stephen Frost wrote:
> > * Aleksander Alekseev (a.alekseev@postgrespro.ru) wrote:
> >
> > > Regarding the difficulty of the project - in fact it's not that
> > > difficult. Particularly this project can rely on external tools, e.g.
> > > use Consul for service discovery and leader election based on
> > > leader-lease approach (implementation [1]). Having this the only thing
> > > is missing is automatic replica promoting and (optionally)
> > > re-configuring of HAProxy / pgbouncer / whatever. Yes, and lots of
> > > Jepsen-like test of course. I believe it's not such a complicated
> > > project.
>
> Does it make sense to address the  limitations of the logical
> replication first, i.e. inability to replicate DDL, sequences and so on?

This is what I was trying to get at earlier, which was mostly just
around trying to scope the project down to something a bit more
reasonable since the initial proposal sounded like it could be a whole
new and independent OSS project on its own.

> > What you're talking about is rebuilding Patroni, but adding more into it
> > than even Patroni tries to do, building it on Logical Replication
> > instead of physical replication, and calling it simple and something
> > that could get into core PostgreSQL over a 12 week GSoC project.  I've
> > certainly got doubts about that, even if we decide that it'd be an
> > external-to-PG project (like Patroni).
> >
> > What might be interesting is seeing if Logical Replication could be
> > added to Patroni as an option and then building on that..  Having
> > someone involved in the Patroni project would be the right way to go
> > about proposing that though to see what they think of it.  That would
> > also be much more sensible as a GSoC project, since it'd be an addition
> > to an existing project and not more-or-less starting a whole new
> > project.
>
> Right now logical replication and  physical replication-based HA tools
> don't work together nicely, since logical replication position is not
> propagated to the promoted replica (I think Craig Ringer has been
> tackling this issue for a few releases already, the latest set of
> patches I could find is https://commitfest.postgresql.org/15/788/).
> Perhaps there is opportunity for a GSoC student to help fixing it. Until
> then  we cannot use logical replication for HA, and even doing something
> simpler like automating creation of logical replicas in Patroni makes
> little sense, as they are doomed to be reinitialized on every failover.

This also sounds like a good project for a GSoC student.

To be clear, we can certainly have project ideas on the Ideas page which
build on things that aren't yet committed or to help with existing
projects that are already underway.  The project needs to be the right
level of effort (remember- this will be the student's job for 12 weeks)
and needs to be something that a student would be able to come up to
speed on and make serious and meaningful progress on during their
summer.

When it comes to timeline, student proposals start being accepted in
March.

Thanks!

Stephen

Attachment

Re: GSoC 2018

From
Alex Kliukin
Date:

On Fri, Dec 15, 2017, at 14:52, Aleksander Alekseev wrote:

> Completely agree, this project can be an improvement for Stolon (or
> Patroni, but I personally never tested or used it, also I got a feeling
> that Google guys will prefer a project that is written in Go). This
> would make much more sense.

I don't believe Google will reject a project based on the fact that it
is written in Python (in fact, Python Software Foundation has
successfully participated in GSoC for many years). Based on the github
statistics, Patroni has started earlier and has more contributors than
Stolon (including those contributed more than one patch/pull-request.)
-- 
Sincerely,
Alex


Re: GSoC 2018

From
Stephen Frost
Date:
Alex,

* Alex Kliukin (alexk@hintbits.com) wrote:
> On Fri, Dec 15, 2017, at 14:52, Aleksander Alekseev wrote:
> > Completely agree, this project can be an improvement for Stolon (or
> > Patroni, but I personally never tested or used it, also I got a feeling
> > that Google guys will prefer a project that is written in Go). This
> > would make much more sense.
>
> I don't believe Google will reject a project based on the fact that it
> is written in Python (in fact, Python Software Foundation has
> successfully participated in GSoC for many years). Based on the github
> statistics, Patroni has started earlier and has more contributors than
> Stolon (including those contributed more than one patch/pull-request.)

I am 100% certain that Google will be more than happy to accept a
Python-based project and that they won't favor a Go project over a
Python one.

That said, I don't think we need to get into a discussion about which
project would be best- I'd be happy to have them both listed on the
Ideas page (perhaps independently would be best though) provided there
are mentors for each.

Also, remember, these are just our ideas- students are welcome to pick
which they're interested in or to choose their own.  When we get to the
point of evaluating the student proposals, it will be a discussion among
the GSoC admins and the individuals who have signed up to be mentors.

Thanks!

Stephen

Attachment

Re: GSoC 2018

From
Alvaro Hernandez
Date:

On 15/12/17 13:57, Aleksander Alekseev wrote:
> Hi Stephen,
>
>> HA/fail-over is a very broad topic, with a lot of pieces that need to be
>> done such that I'm not sure it's really viable, but perhaps a precursor
>> project (synchronous logical replication seems like a prereq, no?) would
>> make more sense.  Or, perhaps, a different piece of the HA question, but
>> solving the whole thing in a summer strikes me as unlikely to be
>> reasonable.
> [...]
>


     I agree with most of the comments down-thread that this projects 
seems to undertake much more work than what a GSoC project could be. I'd 
like to chime in with a few other comments regarding other aspects the 
proposal:

- I believe we're mixing what HA and replication is. HA is about 
promoting a secondary node when the primary fails, preventing two 
primaries at the same time, ensuring one master is always up and so 
forth. Replication is just a data replication mechanism, and that 
*helps* with HA, but is not HA nor a necessary pre-requisite (actually 
you could build an HA solution based on shared-disk). Indeed, HA can be 
built on top of SR, logical replication (with many caveats as have been 
already commented), disk replication techniques outside of PG and 
probably others. Topic is way broader.

- If the proejct's goal is to improve on some of the logical 
replication's shortcomings that would be required to make it a useful 
replication technique on which HA solutions could be built on top of, my 
+1 for that.

- If the project goal would be to augment an existing HA solution to 
abstract away data replication techniques (so that SR is not the only 
option, but also logical or disk-based replication or whatever) from the 
core algorithm, I believe this is also a very sensible GSoC project.

- To add a proxy layer to any existing HA project is IMO a project on 
its own. I don't even see this as a GSoC feasible project.


     Cheers,

     Álvaro


-- 

Alvaro Hernandez


-----------
OnGres



Re: GSoC 2018

From
Aleksander Alekseev
Date:
Hello hackers,

Thanks you a lot for your feedback. I modified the project description
to make it more clear that it implies augmenting an existing HA
solution, particularly Stolon, and doesn't imply solving existing
limitations of logical replication like lack of DDL replication.

Removing some of these limitations or adding logical replication support
to other existing HA applications like RepMgr or Patroni or corosync /
Pacemaker or maybe Postgres-XL sounds like good ideas for other GSoC
projects. I strongly encourage to add these ideas to wiki anyone who is
willing to be a mentor.

Unfortunately I don't have any experience of using these applications
and have no idea how they behave in different corner cases like
netsplit, slow network, etc. Also I don't have much motivation to figure
this out because I'm pretty happy with Stolon. For these reasons I
personally can't be a mentor for corresponding projects.

Also today I'm going to contact Simone Gotti, the main developer of
Stolon, to let him know about this thread. I wonder what are his
thoughts on all this.

--
Best regards,
Aleksander Alekseev

Attachment

Re: GSoC 2018

From
Stephen Frost
Date:
Aleksander,

* Aleksander Alekseev (a.alekseev@postgrespro.ru) wrote:
> Thanks you a lot for your feedback. I modified the project description
> to make it more clear that it implies augmenting an existing HA
> solution, particularly Stolon, and doesn't imply solving existing
> limitations of logical replication like lack of DDL replication.

Sounds great, thanks!

> Removing some of these limitations or adding logical replication support
> to other existing HA applications like RepMgr or Patroni or corosync /
> Pacemaker or maybe Postgres-XL sounds like good ideas for other GSoC
> projects. I strongly encourage to add these ideas to wiki anyone who is
> willing to be a mentor.

Yes, it'd be great for anyone who can mentor a project along these lines
to add it to the wiki.

> Also today I'm going to contact Simone Gotti, the main developer of
> Stolon, to let him know about this thread. I wonder what are his
> thoughts on all this.

Great.

Thanks again!

Stephen

Attachment

Re: GSoC 2018

From
Simone Gotti
Date:
On Mon, Dec 18, 2017 at 10:53 AM, Aleksander Alekseev
<a.alekseev@postgrespro.ru> wrote:
> Hello hackers,
>
> Thanks you a lot for your feedback. I modified the project description
> to make it more clear that it implies augmenting an existing HA
> solution, particularly Stolon, and doesn't imply solving existing
> limitations of logical replication like lack of DDL replication.
>
> Removing some of these limitations or adding logical replication support
> to other existing HA applications like RepMgr or Patroni or corosync /
> Pacemaker or maybe Postgres-XL sounds like good ideas for other GSoC
> projects. I strongly encourage to add these ideas to wiki anyone who is
> willing to be a mentor.
>
> Unfortunately I don't have any experience of using these applications
> and have no idea how they behave in different corner cases like
> netsplit, slow network, etc. Also I don't have much motivation to figure
> this out because I'm pretty happy with Stolon. For these reasons I
> personally can't be a mentor for corresponding projects.
>
> Also today I'm going to contact Simone Gotti, the main developer of
> Stolon, to let him know about this thread. I wonder what are his
> thoughts on all this.

Hi,

here at SorintLab we'll be very happy to provide help and mentor this project.

I read and agree with your concerns since some missing features in
logical replication cannot provide, at the current stage, a complete
seamless HA experience like with streaming replication.
But I believe that enhancing stolon to use logical replication will be
a really useful scenario where we can test the features of logical
replication for HA and use it also for future enhancements (like major
upgrades with near zero downtime).
At the same time this could also push me and other contributors to
implement the missing logical replication features needed to achieve
seamless HA.

Cheers,

Simone.

>
> --
> Best regards,
> Aleksander Alekseev


Re: GSoC 2018

From
Andrey Borodin
Date:
Hi, Stefan!
> 15 дек. 2017 г., в 15:03, Stefan Keller <sfkeller@gmail.com> написал(а):
>
> What about adding "Learned Index" as project task [*]?
> This type of index looks promising for certain properties.
>
> [*] "The Case for Learned Index Structures" Kraska et al. (Dec 2017)
> https://arxiv.org/abs/1712.01208

Let's fill in task description here https://wiki.postgresql.org/index.php?title=GSoC_2018

At least we need:
Project title
Project Description
Skills needed
Difficulty Level
Potential Mentors
Expected Outcomes

I propose title: Learned index prototype
Expected outcome: Design for integration of learned indexes into PostgreSQL (possibly, using index-as-extension
technology),prototype implementation and benchmarks. 

Best regards, Andrey Borodin.

Re: GSoC 2018

From
Stephen Frost
Date:
Greetings -hackers,

* Stephen Frost (sfrost@snowman.net) wrote:
> The deadline for Mentoring organizations to apply is: January 23.

We currently only have four (4) projects for 2018 listed on our
projects page here:

https://wiki.postgresql.org/index.php?title=GSoC_2018

Last year we had quite a few more and we had a good GSoC year with four
students making it through the entire process and some really
interesting work getting done.  Let's try to add more to the list, to
ensure that we have a great 2018 GSoC!

The organization application acceptance period for GSoC 2018 has
officially opened and I'm starting to look through the application to
see if anything has changed.  I'm planning to officially submit on or
about January 18th and it would really be great to get those project
ideas up on our projects page before then!

Any questions, feel free to ask, as always.

Thanks!

Stephen

Attachment

Re: GSoC 2018

From
Alexander Korotkov
Date:
Hi!

On Thu, Jan 4, 2018 at 11:36 PM, Stephen Frost <sfrost@snowman.net> wrote:
Greetings -hackers,

* Stephen Frost (sfrost@snowman.net) wrote:
> The deadline for Mentoring organizations to apply is: January 23.

We currently only have four (4) projects for 2018 listed on our
projects page here:

https://wiki.postgresql.org/index.php?title=GSoC_2018

Could 2017 project ideas be reused this year?  IIRC, only 3 of project ideas were used last year.

 * Explicitly support predicate locks in index access methods besides btree (2017)
 * Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions (2017)
 * Foreign keys for Array elements (2017)

And subject of one project wasn't listed in our ideas list (parallel copy with error handling).

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company 

Re: GSoC 2018

From
Stephen Frost
Date:
Alexander,

* Alexander Korotkov (a.korotkov@postgrespro.ru) wrote:
> On Thu, Jan 4, 2018 at 11:36 PM, Stephen Frost <sfrost@snowman.net> wrote:
> > * Stephen Frost (sfrost@snowman.net) wrote:
> > > The deadline for Mentoring organizations to apply is: January 23.
> >
> > We currently only have four (4) projects for 2018 listed on our
> > projects page here:
> >
> > https://wiki.postgresql.org/index.php?title=GSoC_2018
>
> Could 2017 project ideas be reused this year?  IIRC, only 3 of project
> ideas were used last year.

Yes!  As I mentioned in my initial email, they simply need to be updated
and we need to make sure that we have mentors for them.

If you're willing to mentor for one or multiple, please review the
description, remove the previous mentors and put yourself and then
update the '2017' to be '2018' and we'll include it.  Also, feel free to
contact the other former mentors if you believe the'll be interested in
mentoring again this year.

What I don't want to do is assume that people who volunteered to mentor
last year are willing to do so again this year.

Thanks!

Stephen

Attachment

Re: GSoC 2018

From
Alexander Korotkov
Date:
Hi!

On Fri, Jan 5, 2018 at 5:26 AM, Stephen Frost <sfrost@snowman.net> wrote:
* Alexander Korotkov (a.korotkov@postgrespro.ru) wrote:
> On Thu, Jan 4, 2018 at 11:36 PM, Stephen Frost <sfrost@snowman.net> wrote:
> > * Stephen Frost (sfrost@snowman.net) wrote:
> > > The deadline for Mentoring organizations to apply is: January 23.
> >
> > We currently only have four (4) projects for 2018 listed on our
> > projects page here:
> >
> > https://wiki.postgresql.org/index.php?title=GSoC_2018
>
> Could 2017 project ideas be reused this year?  IIRC, only 3 of project
> ideas were used last year.

Yes!  As I mentioned in my initial email, they simply need to be updated
and we need to make sure that we have mentors for them.

If you're willing to mentor for one or multiple, please review the
description, remove the previous mentors and put yourself and then
update the '2017' to be '2018' and we'll include it.  Also, feel free to
contact the other former mentors if you believe the'll be interested in
mentoring again this year.

What I don't want to do is assume that people who volunteered to mentor
last year are willing to do so again this year.

Great!  I've pull following projects for 2018:

 * GiST API advancement
 * TOAST'ing in slices
 * Table density estimation for approximate queries
 * Extract scanning strategy to the separate entity from GiST/GIN/SP-GiST opclasses

Oleg Bartunov, Teodor Sigaev, Andrey Borodin and I confirmed volunteering to mentor these projects this year.  The only thing to clarify: are you volunteering to mentor TOAST'ing in slices this year?  If no, please correct the wiki accordingly.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Re: GSoC 2018

From
Andrey Borodin
Date:
Hello everyone!

> 5 янв. 2018 г., в 1:36, Stephen Frost <sfrost@snowman.net> написал(а):
>
> * Stephen Frost (sfrost@snowman.net) wrote:
>> The deadline for Mentoring organizations to apply is: January 23.
>
> We currently only have four (4) projects for 2018 listed on our
> projects page here:
I've added project for amcheck enhancement to wiki, I hope it's not too late.
Currently, only I'm listed as a mentor (same for WAL-G project too), but, I hope, we will find backup mentor if
necessary.

Best regards, Andrey Borodin.