Dear all,
Thank you for taking the time to subscribe to this mailing list, and I'm sorry it's taken me so long to write this first email. This probably indicates why I'm looking for help, though :)
First, a bit of background. I am one of the founders of the npemap.org.uk web site which provides postcode collection facilities as well as display of out of copyright Ordnance Survey maps (but I'm assuming that most people on this list will have seen that site already), and I've also been looking after freethepostcode.org for the last three or so years, after Steve Coast indicated that he was no longer interested in keeping it going. I thought (and still think) that it was a good idea, and it seemed a shame to throw away the existing infrastructure as well as risk the postcode database becoming stale.
Since then, both sites have continued to collect postcodes, but (certainly on the FTP.org side) I've had to turn down many good suggestions due to a lack of time to implement changes, as well as my own relative unfamiliarity of Ruby (and lack of desire to rewrite it) and lack of clarity of how to grow the project.
Many of the suggestions and offers I've had relate to data validation and quality checking, and lots of people have been very helpful in providing lists of dubious postcodes to remove from the database, and so on. However I've been very slow to act on those requests too.
A long time ago, even before I took over freethepostcode.org, I saw that freethepostcode.org, npemap.org.uk and (later) the dracos.co.uk postcode locator could use a single collection point for users who just wanted to get hold of high quality, freely available postcode data. On a whim, I registered freepostcodes.org.uk for this purpose.
What I wanted (want) to do (even in the light of the open data initiatives from the UK Government) is provide a common methodology for data collection, processing (validation, quality control and collation), the publication of high quality postcodes data with several data sets with varying degrees of accuracy and coverage, data curation (being able to track where a given data point came from over time) and also provide useful APIs for people to query, submit, quality control, in realtime. Essentially this would mean improving a lot of the existing work that's already been done on npemap.org.uk and freethepostcodes.org, and making it as generic as possible.
What's clear to me is that in order to realise my vision, I need to allow many people to contribute, in the same way that OSM works. (by the way, I would be interested if people say "but OSM already does everything you need" - I haven't paid all that much attention to OSM recently, but I believe that the global mapping goals and the UK postcode collection goals are fairly separate, although obviously OSM can benefit from the postcodes). There are many areas from expertise in current methodologies in postcode data processing, to web/application design and more that I need help in.
I have at my disposal a Linux web/database hosting platform (I'm primarily a sysadmin by trade), a subversion server, mailing lists, trac, etc etc and I'm willing to be a kind of project coordinator and hosting agent for what could be a highly collaborative project, but I would really love to not be a bottleneck (which is the current situation) so having a number of people on board who can contribute would be wonderful.
Maybe there is a better way than all of that old-fashioned infrastructure out there in "the cloud", which could allow even easier collaboration. I'm open to all ideas.
At the end of the day I want to play a small part in creating useful data for people, and (bearing in mind how many thousands of people have contributed to the sites so far) ensure that the results are made available in the most useful forms possible, for as long as we can imagine (and if one data the raw postcode files straight from the Royal Mail become completely Free, I'd be delighted and we could all go to the pub instead!)
So, is this ringing any bells? Does this make people immediately want to sit down and start coding, or suggest the latest and greatest way of running a project like this? I'm hoping that at least some discussion will emerge from this email.
All the best, Dominic.
On Wed, 10 Mar 2010, Dominic Hargreaves wrote:
What I wanted (want) to do (even in the light of the open data initiatives from the UK Government) is provide a common methodology for data collection, processing (validation, quality control and collation), the publication of high quality postcodes data with several data sets with varying degrees of accuracy and coverage, data curation (being able to track where a given data point came from over time) and also provide useful APIs for people to query, submit, quality control, in realtime. Essentially this would mean improving a lot of the existing work that's already been done on npemap.org.uk and freethepostcodes.org, and making it as generic as possible.
In case people haven't seen, the OS+RM today released the full Code Point database (postcode to easting, northing plus council/nhs details). The license it's under is similar to a creative commons attribution v3 license.
To a certain extent, that make make the aim of collecting a public domain set of postcodes (as freethepostcode and npemap do) slightly less important, but at least we do now have a free-ish dataset to check ourselves against :)
Nick
On Thu, Apr 01, 2010 at 01:59:56PM +0100, Nick Burch wrote:
In case people haven't seen, the OS+RM today released the full Code Point database (postcode to easting, northing plus council/nhs details). The license it's under is similar to a creative commons attribution v3 license.
To a certain extent, that make make the aim of collecting a public domain set of postcodes (as freethepostcode and npemap do) slightly less important, but at least we do now have a free-ish dataset to check ourselves against :)
Good news! A page about this data set is
http://data.gov.uk/dataset/os-code-point-open
but the link to the data, as well as the licence, currently appears to be password protected. I guess they haven't quite released it yet :)
News story at
http://news.bbc.co.uk/1/hi/technology/8597779.stm
Dominic.
On 1 April 2010 14:17, Dominic Hargreaves dom@earth.li wrote:
Good news! A page about this data set is
http://data.gov.uk/dataset/os-code-point-open
but the link to the data, as well as the licence, currently appears to be password protected. I guess they haven't quite released it yet :)
The data has been released, but the OS servers are being hammered - I suspect the data.gov.uk server is too. MySociety have a copy of the data: http://parlvid.mysociety.org:81/os/
That page also has a copy of the postcode database with latitude and longitude instead of National Grid, which is likely to be more useful to some people, at least.
Russ
freepostcodes@lists.urchin.earth.li