Thread: Where to put shared designs/images

Where to put shared designs/images

From
Josh Berkus
Date:
All,

PostgreSQL advocacy needs a place to upload and share images, design
files, flyer designs, etc.  Previously we used pgfoundry for this, but
with pgfoundry broken we don't really have a site.

We need something which allows:
* permissions control on who can upload stuff, including easily adding
new contributors
* some kind of change history (and retrieval of older versions) on files
* permanent image links for a minority of images (e.g. Slonik, etc.)
* easy bulk uploading and downloading of files
* ability to eventually handle ~~ 15GB of image files (1.5GB currently)

The PG infrastructure could probably handle this via a combination of
git.postgresql.org and ftp publishing, but this would be a pain for our
infra team, and would require expansion of storage on several hosts.  Or
we could do github + dropbox.  Nothing's completely satisfactory,
though, so suggestions?

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


Re: Where to put shared designs/images

From
"Joshua D. Drake"
Date:
On 02/10/2014 11:44 AM, Josh Berkus wrote:
>
> All,
>
> PostgreSQL advocacy needs a place to upload and share images, design
> files, flyer designs, etc.  Previously we used pgfoundry for this, but
> with pgfoundry broken we don't really have a site.
>
> We need something which allows:
> * permissions control on who can upload stuff, including easily adding
> new contributors
> * some kind of change history (and retrieval of older versions) on files
> * permanent image links for a minority of images (e.g. Slonik, etc.)
> * easy bulk uploading and downloading of files
> * ability to eventually handle ~~ 15GB of image files (1.5GB currently)
>
> The PG infrastructure could probably handle this via a combination of
> git.postgresql.org and ftp publishing, but this would be a pain for our
> infra team, and would require expansion of storage on several hosts.  Or
> we could do github + dropbox.  Nothing's completely satisfactory,
> though, so suggestions?
>

redmine.postgresql.org

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/  509-416-6579
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
For my dreams of your image that blossoms
    a rose in the deeps of my heart. - W.B. Yeats


Re: [pgsql-www] Where to put shared designs/images

From
Robert Treat
Date:
On Mon, Feb 10, 2014 at 2:44 PM, Josh Berkus <josh@agliodbs.com> wrote:
> All,
>
> PostgreSQL advocacy needs a place to upload and share images, design
> files, flyer designs, etc.  Previously we used pgfoundry for this, but
> with pgfoundry broken we don't really have a site.
>
> We need something which allows:
> * permissions control on who can upload stuff, including easily adding
> new contributors
> * some kind of change history (and retrieval of older versions) on files
> * permanent image links for a minority of images (e.g. Slonik, etc.)
> * easy bulk uploading and downloading of files
> * ability to eventually handle ~~ 15GB of image files (1.5GB currently)
>
> The PG infrastructure could probably handle this via a combination of
> git.postgresql.org and ftp publishing, but this would be a pain for our
> infra team, and would require expansion of storage on several hosts.  Or
> we could do github + dropbox.  Nothing's completely satisfactory,
> though, so suggestions?
>

I think github (or maybe bitbucket, does mercurial have the same
issues as git for dealing with images?) is probably the way to go. we
need something where people can do social coding without a lot of
permissions overhead. (As an example, our graphic designer did some
work on the postgres logos for LISA this year, but couldn't get past
the pgfoundry process to give that back publicly; if it had been on
github, it would have been much simpler to do a pull request).


Robert Treat
play: xzilla.net
work: omniti.com


Re: [pgsql-www] Where to put shared designs/images

From
Michael Paquier
Date:
On Tue, Feb 11, 2014 at 8:24 AM, Robert Treat <rob@xzilla.net> wrote:
> On Mon, Feb 10, 2014 at 2:44 PM, Josh Berkus <josh@agliodbs.com> wrote:
>> All,
>>
>> PostgreSQL advocacy needs a place to upload and share images, design
>> files, flyer designs, etc.  Previously we used pgfoundry for this, but
>> with pgfoundry broken we don't really have a site.
>>
>> We need something which allows:
>> * permissions control on who can upload stuff, including easily adding
>> new contributors
>> * some kind of change history (and retrieval of older versions) on files
>> * permanent image links for a minority of images (e.g. Slonik, etc.)
>> * easy bulk uploading and downloading of files
>> * ability to eventually handle ~~ 15GB of image files (1.5GB currently)
>>
>> The PG infrastructure could probably handle this via a combination of
>> git.postgresql.org and ftp publishing, but this would be a pain for our
>> infra team, and would require expansion of storage on several hosts.  Or
>> we could do github + dropbox.  Nothing's completely satisfactory,
>> though, so suggestions?
>>
>
> I think github (or maybe bitbucket, does mercurial have the same
> issues as git for dealing with images?) is probably the way to go. we
> need something where people can do social coding without a lot of
> permissions overhead. (As an example, our graphic designer did some
> work on the postgres logos for LISA this year, but couldn't get past
> the pgfoundry process to give that back publicly; if it had been on
> github, it would have been much simpler to do a pull request).
Definitely. A simple idea would be to create a git repo dedicated to
advocacy in the organization "postgres" of github
(https://github.com/postgres/). Push permissions can be handled easily
from there. My 2c.
Regards,
--
Michael


Re: [pgsql-www] Where to put shared designs/images

From
Josh Berkus
Date:
>> I think github (or maybe bitbucket, does mercurial have the same
>> issues as git for dealing with images?) is probably the way to go. we
>> need something where people can do social coding without a lot of
>> permissions overhead. (As an example, our graphic designer did some
>> work on the postgres logos for LISA this year, but couldn't get past
>> the pgfoundry process to give that back publicly; if it had been on
>> github, it would have been much simpler to do a pull request).
> Definitely. A simple idea would be to create a git repo dedicated to
> advocacy in the organization "postgres" of github
> (https://github.com/postgres/). Push permissions can be handled easily
> from there. My 2c.
> Regards,

So I'm happy with github as a choice.  However, I don't know that
there's any way to do "link to the latest version of raw file X", so
we'd still want to publish stuff to FTP.postgresql.org, no?

Probably not everything, though -- if we were selective, we'd avoid the
space issue.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


Re: [pgsql-www] Where to put shared designs/images

From
Michael Paquier
Date:
On Tue, Feb 11, 2014 at 10:10 AM, Josh Berkus <josh@agliodbs.com> wrote:
>
>>> I think github (or maybe bitbucket, does mercurial have the same
>>> issues as git for dealing with images?) is probably the way to go. we
>>> need something where people can do social coding without a lot of
>>> permissions overhead. (As an example, our graphic designer did some
>>> work on the postgres logos for LISA this year, but couldn't get past
>>> the pgfoundry process to give that back publicly; if it had been on
>>> github, it would have been much simpler to do a pull request).
>> Definitely. A simple idea would be to create a git repo dedicated to
>> advocacy in the organization "postgres" of github
>> (https://github.com/postgres/). Push permissions can be handled easily
>> from there. My 2c.
>> Regards,
>
> So I'm happy with github as a choice.  However, I don't know that
> there's any way to do "link to the latest version of raw file X", so
> we'd still want to publish stuff to FTP.postgresql.org, no?
>
> Probably not everything, though -- if we were selective, we'd avoid the
> space issue.
AFAIK, github supports such URLs, aka linking to the latest version of
a file on a branch:
https://github.com/$ORGANIZATION/$REPO/blob/$BRANCH/$PATH_TO/$FILE
Here is for example the URL pointing to the latest version of Makefile
of root folder in Postgres for branch master:
https://github.com/postgres/postgres/blob/master/Makefile
The trick is not to change the file path and/or the file name.
--
Michael


Re: [pgsql-www] Where to put shared designs/images

From
Stephen Frost
Date:
* Michael Paquier (michael.paquier@gmail.com) wrote:
> On Tue, Feb 11, 2014 at 10:10 AM, Josh Berkus <josh@agliodbs.com> wrote:
> > So I'm happy with github as a choice.  However, I don't know that
> > there's any way to do "link to the latest version of raw file X", so
> > we'd still want to publish stuff to FTP.postgresql.org, no?
> >
> > Probably not everything, though -- if we were selective, we'd avoid the
> > space issue.
> AFAIK, github supports such URLs, aka linking to the latest version of
> a file on a branch:

I thought they *used* to, but dropped it due to issues..?

    Thanks,

        Stephen

Re: [pgsql-www] Where to put shared designs/images

From
Magnus Hagander
Date:
On Tue, Feb 11, 2014 at 3:39 AM, Stephen Frost <sfrost@snowman.net> wrote:
* Michael Paquier (michael.paquier@gmail.com) wrote:
> On Tue, Feb 11, 2014 at 10:10 AM, Josh Berkus <josh@agliodbs.com> wrote:
> > So I'm happy with github as a choice.  However, I don't know that
> > there's any way to do "link to the latest version of raw file X", so
> > we'd still want to publish stuff to FTP.postgresql.org, no?
> >
> > Probably not everything, though -- if we were selective, we'd avoid the
> > space issue.
> AFAIK, github supports such URLs, aka linking to the latest version of
> a file on a branch:

I thought they *used* to, but dropped it due to issues..?

They disabled "uploads", separate from the repositories, was my understanding: https://github.com/blog/1302-goodbye-uploads

Not sure about predictable URLs, but using git itself for very large repos is maybe not the best idea in general. Everybody will have to clone a 10+Gb repository to contribute even something simple, I'm not sure that's very contributor-friendly either...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: [pgsql-www] Where to put shared designs/images

From
"Joshua D. Drake"
Date:

>
> They disabled "uploads", separate from the repositories, was my
> understanding: https://github.com/blog/1302-goodbye-uploads
>
> Not sure about predictable URLs, but using git itself for very large
> repos is maybe not the best idea in general. Everybody will have to
> clone a 10+Gb repository to contribute even something simple, I'm not
> sure that's very contributor-friendly either...

Why aren't we just using the redmine facilities at .org? It has the
ability to have wiki pages, uploads and a repo if you want.

JD


>
> --
>   Magnus Hagander
>   Me: http://www.hagander.net/
>   Work: http://www.redpill-linpro.com/


--
Command Prompt, Inc. - http://www.commandprompt.com/  509-416-6579
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, Postgres-XC, @cmdpromptinc
For my dreams of your image that blossoms
    a rose in the deeps of my heart. - W.B. Yeats


Re: [pgsql-www] Where to put shared designs/images

From
Magnus Hagander
Date:

On Tue, Feb 11, 2014 at 9:33 AM, Joshua D. Drake <jd@commandprompt.com> wrote:



They disabled "uploads", separate from the repositories, was my
understanding: https://github.com/blog/1302-goodbye-uploads

Not sure about predictable URLs, but using git itself for very large
repos is maybe not the best idea in general. Everybody will have to
clone a 10+Gb repository to contribute even something simple, I'm not
sure that's very contributor-friendly either...

Why aren't we just using the redmine facilities at .org? It has the ability to have wiki pages, uploads and a repo if you want.


AFAIK, the redmine file manager doesn't do versioning, which was one of the main requirements.


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: [pgsql-www] Where to put shared designs/images

From
damien clochard
Date:
Le 11/02/2014 02:10, Josh Berkus a écrit :
>
>>> I think github (or maybe bitbucket, does mercurial have the same
>>> issues as git for dealing with images?) is probably the way to go. we
>>> need something where people can do social coding without a lot of
>>> permissions overhead. (As an example, our graphic designer did some
>>> work on the postgres logos for LISA this year, but couldn't get past
>>> the pgfoundry process to give that back publicly; if it had been on
>>> github, it would have been much simpler to do a pull request).
>> Definitely. A simple idea would be to create a git repo dedicated to
>> advocacy in the organization "postgres" of github
>> (https://github.com/postgres/). Push permissions can be handled easily
>> from there. My 2c.
>> Regards,
>
> So I'm happy with github as a choice.  However, I don't know that
> there's any way to do "link to the latest version of raw file X", so
> we'd still want to publish stuff to FTP.postgresql.org, no?
>

First of all, this is a great idea !

We've been hosting the French-related advocacy files at github for a
while. It's a decent way to share things.

https://github.com/postgresqlfr/pgfr_materials

You can have raw link and history link such as

Latest version:
https://raw2.github.com/postgresqlfr/pgfr_materials/master/advocacy/posters/keepcalm/Keep-calm-and-use-postgres.png

an older version:

https://raw2.github.com/postgresqlfr/pgfr_materials/9316144e077fe3c6f084211eb06384d83b37a26d/advocacy/posters/keepcalm/Keep-calm-and-use-postgres.png

However space is an issue because even though Github does not have a
strict limit for the max repo size, they do warn people to avoid repo
larger than 1GB and files larger than 100MB. With a lot of bitmap files
(think Photoshop or Gimp) it may become a problem.

https://help.github.com/articles/what-is-my-disk-quota#rule-of-thumb-1gb-per-repository-100mb-per-file
https://help.github.com/articles/working-with-large-files

With the PostgreSQLFr repo that's ok because we don't do much bitmap
and our activity is pretty low (1 poster and 1 leaflet every year).


The other problem I see is with git (and versioning software in
general). From my experience, using git to create graphics in a
collaborative way is a great idea but it's not really practical. It's
hard to share the sources a graphic file the way you share code. You
need to share the fonts, the brushes, color palettes, etc. (Some of
these materials are not free and connot be pushed to github). So most of
the time, the git repo is seen as a backup solution, and we end up
pushing the final files (PNG or JPEG) instead of the sources (SVG, XCF,
etc.)... The other problem is that you don't really do diff or merge
with graphic files. It doesn't have any sense.

So all in all, Git is good for sharing a few small graphics files but
not for "graphic hacking". And if we just need a place to store/share
files , I would recommend an easier plateform, that can be used by
people would know (and don't want to know) what a versioning tool is...

I think the PostgreSQL wiki may be a good option. There's already a lot
of content there : https://wiki.postgresql.org/wiki/Advocacy and it's
easy to use.

Otherwise If you want it to be outside the PG infra maybe things like
dropbox or google drive would be fine.

--
damien



Re: [pgsql-www] Where to put shared designs/images

From
Magnus Hagander
Date:
On Tue, Feb 11, 2014 at 10:17 AM, damien clochard <damien@dalibo.info> wrote:
Le 11/02/2014 02:10, Josh Berkus a écrit :
>
>>> I think github (or maybe bitbucket, does mercurial have the same
>>> issues as git for dealing with images?) is probably the way to go. we
>>> need something where people can do social coding without a lot of
>>> permissions overhead. (As an example, our graphic designer did some
>>> work on the postgres logos for LISA this year, but couldn't get past
>>> the pgfoundry process to give that back publicly; if it had been on
>>> github, it would have been much simpler to do a pull request).
>> Definitely. A simple idea would be to create a git repo dedicated to
>> advocacy in the organization "postgres" of github
>> (https://github.com/postgres/). Push permissions can be handled easily
>> from there. My 2c.
>> Regards,
>
> So I'm happy with github as a choice.  However, I don't know that
> there's any way to do "link to the latest version of raw file X", so
> we'd still want to publish stuff to FTP.postgresql.org, no?
>

First of all, this is a great idea !

We've been hosting the French-related advocacy files at github for a
while. It's a decent way to share things.

https://github.com/postgresqlfr/pgfr_materials

You can have raw link and history link such as

Latest version:
https://raw2.github.com/postgresqlfr/pgfr_materials/master/advocacy/posters/keepcalm/Keep-calm-and-use-postgres.png

an older version:
https://raw2.github.com/postgresqlfr/pgfr_materials/9316144e077fe3c6f084211eb06384d83b37a26d/advocacy/posters/keepcalm/Keep-calm-and-use-postgres.png

However space is an issue because even though Github does not have a
strict limit for the max repo size, they do warn people to avoid repo
larger than 1GB and files larger than 100MB. With a lot of bitmap files
(think Photoshop or Gimp) it may become a problem.

Repo size at github is not likely to be a problem for us, we have an exception already.

Repo size for end users probably is.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: [pgsql-www] Where to put shared designs/images

From
Stephen Frost
Date:
* Magnus Hagander (magnus@hagander.net) wrote:
> Not sure about predictable URLs, but using git itself for very large repos
> is maybe not the best idea in general. Everybody will have to clone a 10+Gb
> repository to contribute even something simple, I'm not sure that's very
> contributor-friendly either...

Agreed, particularly as it'd have every revision..  Would get large
quick.

    Thanks,

        Stephen