Thread: Beta5 now Available
Check her out and let me know if there are any problems ... I've changed the mk script to pull in the beta3 man pages that I found in the dev/doc directory ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Sun, Nov 21, 2004 at 11:40:29PM -0400, Marc G. Fournier wrote: > > Check her out and let me know if there are any problems ... I've > changed the mk script to pull in the beta3 man pages that I found > in the dev/doc directory ... A much slimmed-down bt.postgresql.org is now serving it. :) Cheers, D -- David Fetter david@fetter.org http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote!
On Mon, 22 Nov 2004, David Fetter wrote: > On Sun, Nov 21, 2004 at 11:40:29PM -0400, Marc G. Fournier wrote: >> >> Check her out and let me know if there are any problems ... I've >> changed the mk script to pull in the beta3 man pages that I found >> in the dev/doc directory ... > > A much slimmed-down bt.postgresql.org is now serving it. :) Sweet, thanks, will include that in the announce later today ... I take it that the 'recursive directory' patch you had mentioned didn't help though? :( ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Mon, Nov 22, 2004 at 12:49:25PM -0400, Marc G. Fournier wrote: > On Mon, 22 Nov 2004, David Fetter wrote: > > >On Sun, Nov 21, 2004 at 11:40:29PM -0400, Marc G. Fournier wrote: > >> > >>Check her out and let me know if there are any problems ... I've > >>changed the mk script to pull in the beta3 man pages that I found > >>in the dev/doc directory ... > > > >A much slimmed-down bt.postgresql.org is now serving it. :) > > Sweet, thanks, will include that in the announce later today ... I > take it that the 'recursive directory' patch you had mentioned > didn't help though? :( It would only help with maintenance, not with memory or CPU. Also, a non-standard patch would mean forking away from the standard distro :P Anyhow, the trimmed-down version doesn't appear to be affecting system load much. Cheers, D -- David Fetter david@fetter.org http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote!
On Mon, 22 Nov 2004, David Fetter wrote: > On Mon, Nov 22, 2004 at 12:49:25PM -0400, Marc G. Fournier wrote: >> On Mon, 22 Nov 2004, David Fetter wrote: >> >>> On Sun, Nov 21, 2004 at 11:40:29PM -0400, Marc G. Fournier wrote: >>>> >>>> Check her out and let me know if there are any problems ... I've >>>> changed the mk script to pull in the beta3 man pages that I found >>>> in the dev/doc directory ... >>> >>> A much slimmed-down bt.postgresql.org is now serving it. :) >> >> Sweet, thanks, will include that in the announce later today ... I >> take it that the 'recursive directory' patch you had mentioned >> didn't help though? :( > > It would only help with maintenance, not with memory or CPU. Also, a > non-standard patch would mean forking away from the standard distro :P > > Anyhow, the trimmed-down version doesn't appear to be affecting system > load much. Nope, definitely better then a loadavg of 54 :) She's running <1 right now ... What about the Java version that Gavin had mentioned? Aegus or something like that? Would moving away from the Python version help any? Something to look into? jdk1.4.2 is available on that VM right now, not sure what else, if anything, would need to be installed though ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc G. Fournier wrote: > What about the Java version that Gavin had mentioned? Aegus or > something like that? > http://azureus.sourceforge.net/ Regards, Thomas Hallgren
On Mon, 22 Nov 2004, Thomas Hallgren wrote: > Marc G. Fournier wrote: > >> What about the Java version that Gavin had mentioned? Aegus or something >> like that? >> > http://azureus.sourceforge.net/ There is a FreeBSD port of it also but it says "A BitTorrent client written in Java" ... does it work as server too, or, by its nature, are servers == clients in Bittorrent? :) ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc G. Fournier wrote: > There is a FreeBSD port of it also but it says "A BitTorrent client > written in Java" ... does it work as server too, or, by its nature, are > servers == clients in Bittorrent? :) > Yes. While you're downloading, others might pick bits and pieces from the segmetns that you've obtained so far and once you're finished, you may "seed" a file, i.e. make it available for others to download. Regards, Thomas Hallgren
The problem is it requires a box with X on it. (ie it's not console Java, it's gui java) I don't have a server to run it on right now, but will be readdressing server allocations shortly and may be able to set something up with x/vnc and would be happy to use that as a primary bt seeding site. Gavin Thomas Hallgren wrote: > Marc G. Fournier wrote: > >> What about the Java version that Gavin had mentioned? Aegus or >> something like that? >> > http://azureus.sourceforge.net/ > > Regards, > Thomas Hallgren > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster
It's all peer to peer client type stuff with the exception of the tracker server. Gavin Marc G. Fournier wrote: > On Mon, 22 Nov 2004, Thomas Hallgren wrote: > >> Marc G. Fournier wrote: >> >>> What about the Java version that Gavin had mentioned? Aegus or >>> something like that? >>> >> http://azureus.sourceforge.net/ > > > There is a FreeBSD port of it also but it says "A BitTorrent client > written in Java" ... does it work as server too, or, by its nature, > are servers == clients in Bittorrent? :) > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) > Email: scrappy@hub.org Yahoo!: yscrappy ICQ: > 7615664 > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly
Am Montag, 22. November 2004 17:40 schrieb David Fetter: > A much slimmed-down bt.postgresql.org is now serving it. :) Out of curiosity, what purpose does a bittorrent source serve in this case? The download servers have enough bandwidth to serve any client faster than the client can take. The traffic on the download servers is not reduced, only distributed differently. I don't see any advantage. -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Tue, Nov 23, 2004 at 05:33:15PM +0100, Peter Eisentraut wrote: > Am Montag, 22. November 2004 17:40 schrieb David Fetter: > > A much slimmed-down bt.postgresql.org is now serving it. :) > > Out of curiosity, what purpose does a bittorrent source serve in > this case? BitTorrent was designed to take bandwidth load off servers that would otherwise need to be on very large and expensive pipes. It does this by serving mostly information about where other servers are, rather than serving the same (much larger) chunks of data over and over again to clients. You can find more information on what BitTorrent does and how it does it at http://bittorrent.com/introduction.html HTH :) Cheers, D -- David Fetter david@fetter.org http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote!
On Tue, 23 Nov 2004, Peter Eisentraut wrote: > Am Montag, 22. November 2004 17:40 schrieb David Fetter: >> A much slimmed-down bt.postgresql.org is now serving it. :) > > Out of curiosity, what purpose does a bittorrent source serve in this case? I've always just seen it as an alternative option for downloading *shrug* just like ftp:// or http:// ... > The download servers have enough bandwidth to serve any client faster than > the client can take. The traffic on the download servers is not reduced, > only distributed differently. I don't see any advantage. Actually, and here is where I exhibit my total lack of knowledge of BT internals ... my understanding was that each 'client' becomes a 'server' by the fact that they have it on their machine and running ... so, over time, the amount of load on the central server would decrease, since new downloads would come from closer "client machines" ... essentially, a whole new set of "unofficial mirror sites" for the source code ... Is this a wrong understanding? This is David's baby though, not mind :) I don't know much about it, and based on what little I've read about it (and original discussions), believe its a more open source 'kazaa/napster', and, as such, works similar ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc G. Fournier wrote: >> The download servers have enough bandwidth to serve any client faster >> than >> the client can take. The traffic on the download servers is not reduced, >> only distributed differently. I don't see any advantage. > > > Actually, and here is where I exhibit my total lack of knowledge of BT > internals ... my understanding was that each 'client' becomes a 'server' > by the fact that they have it on their machine and running ... so, over > time, the amount of load on the central server would decrease, since new > downloads would come from closer "client machines" ... essentially, a > whole new set of "unofficial mirror sites" for the source code ... This is essentially true, although it makes a lot more sense for things that are a lot larger (full ISO's like Linux distributions) and have a higher desirability than "official" avenues to get to them. That's not to say that it shouldn't be offered, it's just a niche thing & is generally time-sensitive (i.e., it does the best when there a lot of people using it & the time most people use it is when something is "hot off the presses"). PostgreSQL is sufficiently small and has high enough availibility that either you won't have to think twice about downloading through standard avenues or BT won't help you. -- Jeff Hoffmann jeff@propertykey.com
* Marc G. Fournier (scrappy@postgresql.org) wrote: > On Tue, 23 Nov 2004, Peter Eisentraut wrote: > >The download servers have enough bandwidth to serve any client faster than > >the client can take. The traffic on the download servers is not reduced, > >only distributed differently. I don't see any advantage. > > Actually, and here is where I exhibit my total lack of knowledge of BT > internals ... my understanding was that each 'client' becomes a 'server' > by the fact that they have it on their machine and running ... so, over > time, the amount of load on the central server would decrease, since new > downloads would come from closer "client machines" ... essentially, a > whole new set of "unofficial mirror sites" for the source code ... > > Is this a wrong understanding? Nope, that's about right, from what I understand. Not only that, but for far-flung people (from the server) it's possible that there are links between the server and the client that are too slow, bt could reduce the bandwidth demands on those links too if other people on the far side are also grabbing the stream. Stephen
On Tue, Nov 23, 2004 at 11:06:40AM -0600, Jeff Hoffmann wrote: > Marc G. Fournier wrote: > >>The download servers have enough bandwidth to serve any client > >>faster than the client can take. The traffic on the download > >>servers is not reduced, only distributed differently. I don't see > >>any advantage. > > > > > >Actually, and here is where I exhibit my total lack of knowledge of > >BT internals ... my understanding was that each 'client' becomes a > >'server' by the fact that they have it on their machine and running > >... so, over time, the amount of load on the central server would > >decrease, since new downloads would come from closer "client > >machines" ... essentially, a whole new set of "unofficial mirror > >sites" for the source code ... > > That's not to say that it shouldn't be offered, it's just a niche > thing & is generally time-sensitive (i.e., it does the best when > there a lot of people using it & the time most people use it is when > something is "hot off the presses"). ^^^^^^^^^^^^^^^^^^^ The above is precisely the use case I set the thing up for. :) Cheers, D -- David Fetter david@fetter.org http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote!
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Marc G. Fournier wrote: | On Mon, 22 Nov 2004, Thomas Hallgren wrote: | |> Marc G. Fournier wrote: |> |>> What about the Java version that Gavin had mentioned? Aegus or |>> something like that? |>> |> http://azureus.sourceforge.net/ | | | There is a FreeBSD port of it also but it says "A BitTorrent client | written in Java" ... does it work as server too, or, by its nature, are | servers == clients in Bittorrent? :) Bittorrent is based on a tracker, the tracker is embedded in the metafile (.torrent file ) and also is based on the "first client" that is launched pointing to the complete file; so the very first client is the real server that must be run 24/24. What do you have against the python implementation ? Regards Gaetano Mendola -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBo8I97UpzwH2SGd4RAiXcAJ4oa5EAN2QpUnM2ajxXVrkpzWCZlwCgpVyT hG8UO4kGUZnYBfJRt+SchTs= =RaCu -----END PGP SIGNATURE-----
Gaetano Mendola wrote: > ...so the very first client is the real server > that must be run 24/24. > I don't think this is correct. You need a tracker for downloaders to be able to find each other but no client is more important than the others. Regards, Thomas Hallgren
On Wed, 24 Nov 2004, Thomas Hallgren wrote: > Gaetano Mendola wrote: >> ...so the very first client is the real server >> that must be run 24/24. >> > I don't think this is correct. You need a tracker for downloaders to be able > to find each other but no client is more important than the others. can there be multiple trackers? for instance, if we ran bt.postgresql.org on two different servers, could they both run trackers at the same time? ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Tue, Nov 23, 2004 at 08:43:56PM -0400, Marc G. Fournier wrote: > On Wed, 24 Nov 2004, Thomas Hallgren wrote: > > >Gaetano Mendola wrote: > >>...so the very first client is the real server that must be run > >>24/24. > >> > >I don't think this is correct. You need a tracker for downloaders > >to be able to find each other but no client is more important than > >the others. > > can there be multiple trackers? for instance, if we ran > bt.postgresql.org on two different servers, could they both run > trackers at the same time? I suspect the best thing would be to run the tracker on one server (bt) and seeders elsewhere. Cheers, D -- David Fetter david@fetter.org http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote!
No you can not, but the tracker isn't very resource intesive from my past experience. I can host it if needed. Gavin Marc G. Fournier wrote: > On Wed, 24 Nov 2004, Thomas Hallgren wrote: > >> Gaetano Mendola wrote: >> >>> ...so the very first client is the real server >>> that must be run 24/24. >>> >> I don't think this is correct. You need a tracker for downloaders to >> be able to find each other but no client is more important than the >> others. > > > can there be multiple trackers? for instance, if we ran > bt.postgresql.org on two different servers, could they both run > trackers at the same time? > > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) > Email: scrappy@hub.org Yahoo!: yscrappy ICQ: > 7615664 > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly
On Wed, 24 Nov 2004, Gavin M. Roy wrote: > No you can not, but the tracker isn't very resource intesive from my past > experience. I can host it if needed. It wasn't that that I was thinking of ... just wondering if there was some way of having it redundant, instead of centralized ... nice thing about ftp mirrors, we have about 60 of them, so if one goes down, it doesn't really affect anything ... from what everyone is saying, if the tracker goes down, it affects everything ... seems odd to have "new technology" still having single points of failure :( > > Gavin > > Marc G. Fournier wrote: > >> On Wed, 24 Nov 2004, Thomas Hallgren wrote: >> >>> Gaetano Mendola wrote: >>> >>>> ...so the very first client is the real server >>>> that must be run 24/24. >>>> >>> I don't think this is correct. You need a tracker for downloaders to be >>> able to find each other but no client is more important than the others. >> >> >> can there be multiple trackers? for instance, if we ran bt.postgresql.org >> on two different servers, could they both run trackers at the same time? >> >> >> ---- >> Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) >> Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664 >> >> ---------------------------(end of broadcast)--------------------------- >> TIP 3: if posting/reading through Usenet, please send an appropriate >> subscribe-nomail command to majordomo@postgresql.org so that your >> message can get through to the mailing list cleanly > > > ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc G. Fournier wrote: > On Wed, 24 Nov 2004, Gavin M. Roy wrote: > >> No you can not, but the tracker isn't very resource intesive from my >> past experience. I can host it if needed. > > > It wasn't that that I was thinking of ... just wondering if there was > some way of having it redundant, instead of centralized ... nice thing > about ftp mirrors, we have about 60 of them, so if one goes down, it > doesn't really affect anything ... from what everyone is saying, if > the tracker goes down, it affects everything ... seems odd to have > "new technology" still having single points of failure :( O.k. I know nothing of bittorrent but couldn't we just have to machines that are identically configured that have a round robin DNS thing going on? That would help with load. If the machines were on the same network we could even heartbeat or service check between them. J > > > > > Gavin > > >> Marc G. Fournier wrote: >> >>> On Wed, 24 Nov 2004, Thomas Hallgren wrote: >>> >>>> Gaetano Mendola wrote: >>>> >>>>> ...so the very first client is the real server >>>>> that must be run 24/24. >>>>> >>>> I don't think this is correct. You need a tracker for downloaders >>>> to be able to find each other but no client is more important than >>>> the others. >>> >>> >>> >>> can there be multiple trackers? for instance, if we ran >>> bt.postgresql.org on two different servers, could they both run >>> trackers at the same time? >>> >>> >>> ---- >>> Marc G. Fournier Hub.Org Networking Services >>> (http://www.hub.org) >>> Email: scrappy@hub.org Yahoo!: yscrappy ICQ: >>> 7615664 >>> >>> ---------------------------(end of >>> broadcast)--------------------------- >>> TIP 3: if posting/reading through Usenet, please send an appropriate >>> subscribe-nomail command to majordomo@postgresql.org so that your >>> message can get through to the mailing list cleanly >> >> >> >> > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) > Email: scrappy@hub.org Yahoo!: yscrappy ICQ: > 7615664 > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
Attachment
On Wed, 24 Nov 2004, Joshua D. Drake wrote: > Marc G. Fournier wrote: > >> On Wed, 24 Nov 2004, Gavin M. Roy wrote: >> >>> No you can not, but the tracker isn't very resource intesive from my past >>> experience. I can host it if needed. >> >> >> It wasn't that that I was thinking of ... just wondering if there was some >> way of having it redundant, instead of centralized ... nice thing about ftp >> mirrors, we have about 60 of them, so if one goes down, it doesn't really >> affect anything ... from what everyone is saying, if the tracker goes down, >> it affects everything ... seems odd to have "new technology" still having >> single points of failure :( > > > O.k. I know nothing of bittorrent but couldn't we just have to machines that > are identically configured that have a round robin DNS thing going on? we're not talking load issues this time ... the way I understand it, bittorrent has a 'tracker' process that only one can be running on the "BT Distributed Network" at once ... so, if the bt "central server" goes down, the whole bt network goes down with it ... At least, this is my understanding, someone please correct me if I'm wrong ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
To a degree you are correct. AFAIK new downloads could not start if the tracker crashed. The tracker is the traffic cop that tells peer nodes about each other. I dont believe the tracker that comes from the main bit torrent author allows for multiple trackers with a common data repository, but if we're really interested, maybe we could hack up the code to talk to a central pgsql database allowing multiple trackers on a dns rr. Gavin Marc G. Fournier wrote: > On Wed, 24 Nov 2004, Joshua D. Drake wrote: > >> Marc G. Fournier wrote: >> >>> On Wed, 24 Nov 2004, Gavin M. Roy wrote: >>> >>>> No you can not, but the tracker isn't very resource intesive from >>>> my past experience. I can host it if needed. >>> >>> >>> >>> It wasn't that that I was thinking of ... just wondering if there >>> was some way of having it redundant, instead of centralized ... nice >>> thing about ftp mirrors, we have about 60 of them, so if one goes >>> down, it doesn't really affect anything ... from what everyone is >>> saying, if the tracker goes down, it affects everything ... seems >>> odd to have "new technology" still having single points of failure :( >> >> >> >> O.k. I know nothing of bittorrent but couldn't we just have to >> machines that are identically configured that have a round robin DNS >> thing going on? > > > we're not talking load issues this time ... the way I understand it, > bittorrent has a 'tracker' process that only one can be running on the > "BT Distributed Network" at once ... so, if the bt "central server" > goes down, the whole bt network goes down with it ... > > At least, this is my understanding, someone please correct me if I'm > wrong ... > > ---- > Marc G. Fournier Hub.Org Networking Services > (http://www.hub.org) > Email: scrappy@hub.org Yahoo!: yscrappy ICQ: > 7615664
On Wed, Nov 24, 2004 at 11:28:31AM -0600, Gavin M. Roy wrote: > To a degree you are correct. AFAIK new downloads could not start if the > tracker crashed. The tracker is the traffic cop that tells peer nodes > about each other. I dont believe the tracker that comes from the main > bit torrent author allows for multiple trackers with a common data > repository, but if we're really interested, maybe we could hack up the > code to talk to a central pgsql database allowing multiple trackers on a > dns rr. I think I've seen some torrents with a "multi-host" definition of tracker. Not sure how that works, or how clients react to it. But before you hack that up, make sure to check for previous attempts, mainly for client compatibility. -- Alvaro Herrera (<alvherre[@]dcc.uchile.cl>) "Tiene valor aquel que admite que es un cobarde" (Fernandel)
Thomas Hallgren wrote: > Gaetano Mendola wrote: > >> ...so the very first client is the real server >> that must be run 24/24. >> > I don't think this is correct. You need a tracker for downloaders to be > able to find each other but no client is more important than the others. > I'm sorry to say that you're wrong. A tracker without a client running on a complete file is completelly useless. The tracker even doesn't know what you are "sharing". If you want publish your files you have to start a tracker and a client on each content to distribute. The client that you run on the complete content will tell you: "Downloaded" and will stay there waiting for other client connections. Regards Gaetano Mendola
* Marc G. Fournier (scrappy@postgresql.org) wrote: > we're not talking load issues this time ... the way I understand it, > bittorrent has a 'tracker' process that only one can be running on the "BT > Distributed Network" at once ... so, if the bt "central server" goes down, > the whole bt network goes down with it ... > > At least, this is my understanding, someone please correct me if I'm wrong > ... I *think*, not 100% sure, but I believe you could have multiple trackers, hosted by different systems, but the problem is that you'll split your clients between them and they won't talk to each other even though they potentially could. Also, if one of the trackers went down the clients using it would also go down (or at least, the stream from that tracker would). Stephen
Stephen Frost wrote: >* Marc G. Fournier (scrappy@postgresql.org) wrote: > > >>we're not talking load issues this time ... the way I understand it, >>bittorrent has a 'tracker' process that only one can be running on the "BT >>Distributed Network" at once ... so, if the bt "central server" goes down, >>the whole bt network goes down with it ... >> >>At least, this is my understanding, someone please correct me if I'm wrong >>... >> >> > >I *think*, not 100% sure, but I believe you could have multiple >trackers, hosted by different systems, but the problem is that you'll >split your clients between them and they won't talk to each other even >though they potentially could. Also, if one of the trackers went down >the clients using it would also go down (or at least, the stream from >that tracker would). > > A well organized Bittorrent FAQ that answers most questions regarding trackers etc. can be found at: http://www.filesoup.com/faq/index.php?sid=281125&aktion=anzeigen Regards, Thomas Hallgren