Re: programmatic way to fetch latest release for a given major.minor version - Mailing list pgsql-general

    Here you go.

    Fetches versions and prints most recent minor for each major
    Tests all mirrors for speed and prints out the 4 fastest (takes some time)
    http://www.crummy.com/software/BeautifulSoup/

    Have a nice day !

#! /bin/env python
# -*- coding: utf-8  -*-

import urllib, BeautifulSoup, re, time, sys


def get_all_versions():
    soup =
BeautifulSoup.BeautifulSoup( urllib.urlopen( "http://ftp3.fr.postgresql.org/pub/postgresql/source/"
).read() )
    for a in soup( 'a', {'href': re.compile( r"v\d+.\d+.\d+" ) } ):
        yield map( int, re.search( r"v(\d+)\.(\d+)\.(\d+)*", a['href']
).groups() )

def get_latest_versions():
    lastversions = {}
    for a,b,c in sorted( get_all_versions() ):
        lastversions[ (a,b) ] = c
    return sorted( lastversions.items() )

def parse_query_string( url ):
    return dict( map( urllib.unquote_plus, pair.split('=',1) ) for pair in
re.split( "&(?:amp;|)", urllib.splitquery( url )[1] ) )

def get_mirrors():
    soup =
BeautifulSoup.BeautifulSoup( urllib.urlopen( "http://wwwmaster.postgresql.org/download/mirrors-ftp"
).read() )
    for a in soup( 'a', {'href': re.compile( r"\?setmir=.*url=" ) } ):
        yield parse_query_string( a['href'] )['url']

def get_fastest_mirrors( urls, filename ):
    for url in urls:
        sys.stdout.write( "        %s\r" % url )
        t = time.time()
        try:
            urllib.urlopen( url + filename )
        except:
            pass
        d = time.time()-t
        print "%.02f s" % d
        yield d, url

for major, minor in get_latest_versions():
    print "%d.%d.%d" % (major[0], major[1], minor)

mirrors = get_mirrors()
fastest = sorted( get_fastest_mirrors( mirrors, "sync_timestamp" ))[:4]
for d, mirror in fastest:
    print "%.02f s    %s" % (d,mirror)









On Tue, 10 Apr 2007 00:34:02 +0200, Andrew Hammond
<andrew.george.hammond@gmail.com> wrote:

> On 4/9/07, CAJ CAJ <pguser@gmail.com> wrote:
>> On 9 Apr 2007 14:47:20 -0700, Andrew Hammond
>> <andrew.george.hammond@gmail.com> wrote:
>> > I'm writing a script that wants to know the latest release for a given
>> > major.minor version. Is there some better way than parsing
>> > http://www.postgresql.org/ftp/source/ or trying to
>> connect to ftp
>> > (which is invariably timing out on me today. Is that box getting
>> > hammered or something?) and doing the parsing that? Both approaches
>> > feel quite awkward to me.
>>
>> Use wget to download via  HTTP (added recently).  Probably wise to add a
>> couple mirrors in your script.
>
> I'm not asking how to download stuff. I'm asking how to figure out the
> current release number for a given major.minor. I thought that was
> clear in my original post, but I guess not. For example, how do I
> determine (programmatically) the lastest version of 8.1.
>
> I'm also interested in a clever way to select one of the "close"
> mirrors at random for downloading via http. However I had planned to
> hold that question until I'd solved the first issue.
>
> Andrew
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq