Thread: mailing list offer
Hello, Now that CMD is hosting the archives I thought that I would offer some more resources. I would be willing to purchase a dedicated machine to host the mailing lists at our location. We would use this machine for the archives and lists both. As we already relay a great deal of the archive traffic through two of our machines I do not see our bandwidth (3 DS3s) as being an issue. In terms of reliability the machine would also be placed on a natural gas fired generator and of course the network is switched and firewalled. I would also be willing to dedicate some resources to some other projects that people have brought up, including direct integration of the mail archives and lists into PostgreSQL. What are the thoughts on this? Sincerely, Joshua D. Drake -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
Thanks, but the mailing lists are fine where they currently are ... there are no current plans to move them ... On Mon, 18 Oct 2004, Joshua D. Drake wrote: > Hello, > > Now that CMD is hosting the archives I thought that I would offer some > more resources. I would be willing to purchase a dedicated machine to host > the mailing lists at our location. We would use this machine for the archives > and lists both. > > As we already relay a great deal of the archive traffic through two of > our machines I do not see our bandwidth (3 DS3s) as being an issue. In terms > of reliability the machine would also be placed on a natural gas fired > generator and of course the network is switched and firewalled. > > I would also be willing to dedicate some resources to some other > projects that people have brought up, including direct integration of the > mail archives and lists into PostgreSQL. > > What are the thoughts on this? > > Sincerely, > > Joshua D. Drake > > -- > Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC > Postgresql support, programming shared hosting and dedicated hosting. > +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com > PostgreSQL Replicator -- production quality replication for PostgreSQL > > > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc G. Fournier wrote: > > Thanks, but the mailing lists are fine where they currently are ... > there are no current plans to move them ... Actually, it's an interesting thought and once in place it would free up people's time for other stuff. Joshua, what hardware would you be looking at getting for it, and what kind of software do you guys generally use for mailing lists? The other initial main thought is... what kind of hours of support can we expect? Like, if something fails at 1am, can we ring someone who'll be able to look into it in a decent timeframe and have the ability to get it resolved (within reason of course). :) Regards and best wishes, Justin Clift
On Tue, 2004-10-19 at 11:28, Joshua D. Drake wrote: > Hello, > > Now that CMD is hosting the archives I thought that I would offer some > more resources. I would be willing to purchase a dedicated machine to > host the mailing lists at our location. We would use this machine for > the archives and lists both. > > As we already relay a great deal of the archive traffic through two of > our machines I do not see our bandwidth (3 DS3s) as being an issue. In > terms of reliability the machine would also be placed on a natural gas > fired generator and of course the network is switched and firewalled. > > I would also be willing to dedicate some resources to some other > projects that people have brought up, including direct integration of > the mail archives and lists into PostgreSQL. > I can't see anything that would stop you from doing this now... Marc should be able to provide you with the old mbox archives, and following them, simply subscribe to all of the lists, and you have yourself an up to date archive. I had the forums going, and have been playing with getting the archives integrated, but the importer is way too buggy, and needs a lot of work before they can be brought to life. A fully integrated solution would be sweet. One that combines the features of the archives, the search engine, and the lists themselves. You should be aware tho, that iirc the current tally is closing in on 300.000 emails. Does anyone have any installations with that much data indexed by tsearch2, who would be able to give us some indicative reports on query time? Mnogosearch was scrapped due to it's speed (or lack thereof), and iirc, tsearch2 closely resembles the same type of mechanism for full text searches. Just something to think about. Btw, With regards to the forums, I imagine they'll come back over the weekend... > What are the thoughts on this? > > Sincerely, > > Joshua D. Drake ... John
Justin Clift wrote: > Marc G. Fournier wrote: > >> >> Thanks, but the mailing lists are fine where they currently are ... >> there are no current plans to move them ... > > > Actually, it's an interesting thought and once in place it would free > up people's time for other stuff. > > Joshua, what hardware would you be looking at getting for it, and what > kind of software do you guys generally use for mailing lists? I figured I would put together a small dual opteron with an SATA array... We are fond of mailman but will use anything the project desires. > > The other initial main thought is... what kind of hours of support can > we expect? Like, if something fails at 1am, can we ring someone > who'll be able to look into it in a decent timeframe and have the > ability to get it resolved (within reason of course). I would provide my cell phone to appropriate responsible parties and other on-call numbers when needed. Sincerely, Joshua D. Drake > > :) > > Regards and best wishes, > > Justin Clift -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
John Hansen wrote:
very well. There are some tricks to it though. For example what we had to was have a single
table that has nothing but a product_id, title, title_vectors.
Then on a search, we return the product_ids and grab the titles from the detail table via the product_ids
that were pulled from the search.
Sincerely,
Joshua D. Drake
I have a database now with 1.77 million entries using Tsearch. The entries are book titles. It runsYou should be aware tho, that iirc the current tally is closing in on 300.000 emails. Does anyone have any installations with that much data indexed by tsearch2, who would be able to give us some indicative reports on query time?
very well. There are some tricks to it though. For example what we had to was have a single
table that has nothing but a product_id, title, title_vectors.
Then on a search, we return the product_ids and grab the titles from the detail table via the product_ids
that were pulled from the search.
Sincerely,
Joshua D. Drake
Mnogosearch was scrapped due to it's speed (or lack thereof), and iirc, tsearch2 closely resembles the same type of mechanism for full text searches. Just something to think about. Btw, With regards to the forums, I imagine they'll come back over the weekend...What are the thoughts on this? Sincerely, Joshua D. Drake... John
-- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
On Tue, 19 Oct 2004, John Hansen wrote: > I can't see anything that would stop you from doing this now... Marc > should be able to provide you with the old mbox archives, and following > them, simply subscribe to all of the lists, and you have yourself an up > to date archive. I've also just added in a 'mbox' link to the archives ... so you can download the raw files from the web site now too ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Tue, 19 Oct 2004, John Hansen wrote: > > A fully integrated solution would be sweet. One that combines the > features of the archives, the search engine, and the lists themselves. > > You should be aware tho, that iirc the current tally is closing in on > 300.000 emails. Does anyone have any installations with that much data > indexed by tsearch2, who would be able to give us some indicative > reports on query time? > > Mnogosearch was scrapped due to it's speed (or lack thereof), and iirc, > tsearch2 closely resembles the same type of mechanism for full text > searches. Just something to think about. in principle, www.pgsql.ru/db/mw (fts.postgresql.org in past) was designed to be an integrated solution, but time past and we have no spare time to continue developing (we only reinstalled fts.postgresql.org). As for tsearch2 performance, then it's slow on full archive, but reasonably fast searching last month (default) or last year archive. We have tested our idea using read only archive and online news archive, and it worked very-very fast. Main idea is to have common infrastructure (parsers, dictionaries, ranking) for inverted index (read only part) and tsearch2 index for changing part. Running daemon (tsearchd) accepts text for indexing, all metadata are stored in posgresql, so searching could be constrained using metadata. Tsearchd is just a working prototype and much work is needed. > > Btw, With regards to the forums, I imagine they'll come back over the > weekend... > >> What are the thoughts on this? >> >> Sincerely, >> >> Joshua D. Drake > > ... John > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Tue, 2004-10-19 at 13:24, Joshua D. Drake wrote: > I have a database now with 1.77 million entries using Tsearch. The > entries are book titles. It runs > very well. There are some tricks to it though. For example what we had > to was have a single > table that has nothing but a product_id, title, title_vectors. Ok, i'm guessing none of the titles are as long as an email... what's the speed like? ... John
John Hansen wrote:
So no -- that may not scale that well. We would have to test it out.
Sincerely,
Joshua D. Drake
1.4 seconds for the search, 4.5ms for the pull of the full details..On Tue, 2004-10-19 at 13:24, Joshua D. Drake wrote:I have a database now with 1.77 million entries using Tsearch. The entries are book titles. It runs very well. There are some tricks to it though. For example what we had to was have a single table that has nothing but a product_id, title, title_vectors.Ok, i'm guessing none of the titles are as long as an email... what's the speed like?
So no -- that may not scale that well. We would have to test it out.
Sincerely,
Joshua D. Drake
... John
-- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
>> >> > 1.4 seconds for the search, 4.5ms for the pull of the full details.. > > So no -- that may not scale that well. We would have to test it out. There are also other items that can be done, like keeping the tables per month. And allow searching based on the month. If they want to search the entire archives I think it is pretty much a safe bet that it will take time. Of course, we can always use the Google API as well. Sincerely, Joshua D. Drake > > Sincerely, > > Joshua D. Drake > > >>... John >> >> > > > -- > Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC > Postgresql support, programming shared hosting and dedicated hosting. > +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com > PostgreSQL Replicator -- production quality replication for PostgreSQL > -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
> > I figured I would put together a small dual opteron with an SATA array... > > We are fond of mailman but will use anything the project desires. Would mailman be sufficient? It seems to offer the most versatility. Besides it uses our preferred language (python) :) >> The other initial main thought is... what kind of hours of support can >> we expect? Like, if something fails at 1am, can we ring someone >> who'll be able to look into it in a decent timeframe and have the >> ability to get it resolved (within reason of course). > > > I would provide my cell phone to appropriate responsible parties and > other on-call numbers when needed. To be more clear on this. If there was a problem during business hours you can always call the office. We are PST and a tech is always answering the phone (we don't have an answering service or anything). For after hours stuff, I would provide an after hours call. Basically I would treat this just like I treat our managed services, which means you get 24x7 support. Sincerely, Joshua D. Drake > > Sincerely, > > Joshua D. Drake > > > >> >> :) >> >> Regards and best wishes, >> >> Justin Clift > > > > -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
Justin Clift wrote: > Marc G. Fournier wrote: >> >> Thanks, but the mailing lists are fine where they currently are ... >> there are no current plans to move them ... > > Actually, it's an interesting thought and once in place it would free up > people's time for other stuff. I'm interested in what other people's opinion on this is. Personally, I feel this is worth considering because it gives us more practice at getting existing PG companies further involved, and Command Prompt is a relatively safe bet here. That's a good thing I feel, because we'll likely get further offers as PG gains wider usage. ? Regards and best wishes, Justin Clift
Guys, Hmmm ... from my perspective, what we need is not so much to move the mailing lists, is to have a fallback to another mailing list server in the event that Hub.org becomes unavailable for some reason, such as the trunk line in Costa Rica going down (as happened last fall). The mailing lists are the backbone of our community; we really can't afford for them to go offline for more than a few hours. -- --Josh Josh Berkus Aglio Database Solutions San Francisco
Josh Berkus wrote: > Guys, > > Hmmm ... from my perspective, what we need is not so much to move the mailing > lists, is to have a fallback to another mailing list server in the event that > Hub.org becomes unavailable for some reason, such as the trunk line in Costa > Rica going down (as happened last fall). The mailing lists are the backbone > of our community; we really can't afford for them to go offline for more than > a few hours. Well although that is obviously a possibility (Backhoe comes to mind) our connectivity comes from three different providers: 1. ELI 2. Sprint 3. TimeWarner Also our connectivity is physically accessible my half a dozen people within 30 minutes, even at 3:00 AM. If something goes down, the whole freaking world is notified to make sure it is getting worked on. Did I mention the natural gas fired generator? ;) I just want to help make the parts of the project that we can, as reliable as possible. Sincerely, Joshua D. Drake > -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
Attachment
Hello, I am sure that everyone is doing their best to get the mailing lists back up. However, I would like to re-extend my offer to host the mailing lists. In the time that the lists have been down the last two days I could have built a brand new machine, restored from backup, had the lists back up and running with 16 hours to spare. I am not trying to step on any toes here but the main source of community communication is down right now and has been for 24 hours. This is really not good. Sincerely, Joshua D. Drake Joshua D. Drake wrote: > Josh Berkus wrote: > >> Guys, >> >> Hmmm ... from my perspective, what we need is not so much to move the >> mailing lists, is to have a fallback to another mailing list server >> in the event that Hub.org becomes unavailable for some reason, such >> as the trunk line in Costa Rica going down (as happened last fall). >> The mailing lists are the backbone of our community; we really can't >> afford for them to go offline for more than a few hours. > > > Well although that is obviously a possibility (Backhoe comes to mind) > our connectivity comes from three different providers: > > 1. ELI > 2. Sprint > 3. TimeWarner > > Also our connectivity is physically accessible my half a dozen people > within 30 minutes, even at 3:00 AM. If something goes down, the whole > freaking world is notified to make sure it is getting worked on. Did I > mention the natural gas fired generator? ;) > > I just want to help make the parts of the project that we can, as > reliable as possible. > > Sincerely, > > Joshua D. Drake > > >> > > >------------------------------------------------------------------------ > > >---------------------------(end of broadcast)--------------------------- >TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > > -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
Attachment
Thank you for your offer, but we have taken steps towards setting up a hot failover mirror of that VM to a server in Austria, which we will be working on more next week once the server is ready ... On Wed, 8 Dec 2004, Joshua D. Drake wrote: > Hello, > > I am sure that everyone is doing their best to get the mailing lists back up. > However, I would like to re-extend my offer to host the mailing lists. > In the time that the lists have been down the last two days I could have > built a brand new machine, restored from backup, had the lists back up > and running with 16 hours to spare. > > I am not trying to step on any toes here but the main source of community > communication is down right now and has been for 24 hours. This is really > not good. > > Sincerely, > > Joshua D. Drake > > > Joshua D. Drake wrote: > >> Josh Berkus wrote: >> >>> Guys, >>> >>> Hmmm ... from my perspective, what we need is not so much to move the >>> mailing lists, is to have a fallback to another mailing list server in the >>> event that Hub.org becomes unavailable for some reason, such as the trunk >>> line in Costa Rica going down (as happened last fall). The mailing lists >>> are the backbone of our community; we really can't afford for them to go >>> offline for more than a few hours. >> >> >> Well although that is obviously a possibility (Backhoe comes to mind) our >> connectivity comes from three different providers: >> >> 1. ELI >> 2. Sprint >> 3. TimeWarner >> >> Also our connectivity is physically accessible my half a dozen people >> within 30 minutes, even at 3:00 AM. If something goes down, the whole >> freaking world is notified to make sure it is getting worked on. Did I >> mention the natural gas fired generator? ;) >> >> I just want to help make the parts of the project that we can, as reliable >> as possible. >> >> Sincerely, >> >> Joshua D. Drake >> >> >>> >> >> >> ------------------------------------------------------------------------ >> >> >> ---------------------------(end of broadcast)--------------------------- >> TIP 6: Have you searched our list archives? >> >> http://archives.postgresql.org >> > > > -- > Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC > Postgresql support, programming shared hosting and dedicated hosting. > +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com > PostgreSQL Replicator -- production quality replication for PostgreSQL > > ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Josh, > I am sure that everyone is doing their best to get the mailing lists > back up. However, I would like to re-extend my offer to host the mailing > lists. Apparently the problem is that the network we use to replicate the mailing lists failed. So we can't get at them at the moment, for any purpose: the machine is there and running, but inaccessable, and our backup of the lists is not reliable. Marc and I have been chatting about other places to host/move stuff. I'm sure he'll have more news when it warrants. -- Josh Berkus Aglio Database Solutions San Francisco
>Apparently the problem is that the network we use to replicate the mailing >lists failed. So we can't get at them at the moment, for any purpose: the >machine is there and running, but inaccessable, and our backup of the lists >is not reliable. Marc and I have been chatting about other places to >host/move stuff. I'm sure he'll have more news when it warrants. > > O.k. well we haven't been down for more than 30 minutes (network wise) in 4 years. Let me know if I can help. Sincerely, Joshua D. Drake -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
Attachment
Josh,
the problem is not with mirroring of mailing lists, there is google,
www.pgsql.ru, for example. I think, the problem is with reliable
mailing list gateway.
Oleg
On Wed, 8 Dec 2004, Josh Berkus wrote:
> Josh,
>
>> I am sure that everyone is doing their best to get the mailing lists
>> back up. However, I would like to re-extend my offer to host the mailing
>> lists.
>
> Apparently the problem is that the network we use to replicate the mailing
> lists failed. So we can't get at them at the moment, for any purpose: the
> machine is there and running, but inaccessable, and our backup of the lists
> is not reliable. Marc and I have been chatting about other places to
> host/move stuff. I'm sure he'll have more news when it warrants.
>
>
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
On Sat, 11 Dec 2004, Oleg Bartunov wrote: > Josh, > > the problem is not with mirroring of mailing lists, there is google, > www.pgsql.ru, for example. I think, the problem is with reliable > mailing list gateway. the problem was actually our provider not paying their provider and their provider finally sayin 'enough is enough' ... > > Oleg > On Wed, 8 Dec 2004, Josh Berkus wrote: > >> Josh, >> >>> I am sure that everyone is doing their best to get the mailing lists >>> back up. However, I would like to re-extend my offer to host the mailing >>> lists. >> >> Apparently the problem is that the network we use to replicate the mailing >> lists failed. So we can't get at them at the moment, for any purpose: the >> machine is there and running, but inaccessable, and our backup of the lists >> is not reliable. Marc and I have been chatting about other places to >> host/move stuff. I'm sure he'll have more news when it warrants. >> >> > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 > ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Oleg, > the problem is not with mirroring of mailing lists, there is google, > www.pgsql.ru, for example. I think, the problem is with reliable > mailing list gateway. When I said "mailing lists" I was referring to majordomo and the subscriber lists, not to the archives. -- Josh Berkus Aglio Database Solutions San Francisco