Under the hood of Sorell Council

More of the gory details of how information gets into PlanningAlerts. This is here so you can see and learn how it's done but also so you are able to fix things and see in detail what isn't working.

The steps

  1. Applications are published on the Sorell Council website in a human readable, non-structured form.
  2. The latest scraper code on Github is loaded by our scraping platform morph.io.
  3. The scraper planningalerts-scrapers/sorell on our scraping platform morph.io collects the information daily and converts it into a machine-readable format.
  4. If the scraper errors anyone who is watching the scraper on morph.io gets informed via a daily email alert.
  5. The machine readable data is validated, imported and geocoded by PlanningAlerts daily. If there are any validation errors on an application, the application is skipped and the error is logged. See below for the most recent logs.
  6. The information is published on PlanningAlerts, made available via the API, and people are informed of new applications via email alerts.

Most recent import logs

Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.228.1 Development Application 8 Susans Bay Road, Primrose Sands for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.234.1 Development Application 205B Carlton Beach Road, Carlton for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.275.1 Development Application 1 Stanford Court, Sorell for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.312.1 Development Application 36 Tamarix Road, Primrose Sands for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.353.1 Development Application 93-95 Linden Road, Primrose Sands for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.357.1 Development Application 812 Marchwiel Road, Bream Creek for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.361.1 Development Application 29 Valleyfield Road, Sorell Representation Close Monday 22nd November 2021 for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.365.1 Development Application 5 Abruzzi Court, Orielton for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.114.1 Development Application Lot 1000 Old Forcett Road, Dodges Ferry for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.231.1 Development Application 15 Tamarix Road Primrose Sands Representations Close Monday 6th December 2021 for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.277.1 Development Application 10 Fresne Way, Sorell for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.278.1 Development Application 40 Forcett Street Sorell Representations Close Monday 6th December 2021 for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.279.1 Development Application 68 Ridge Road, Dodges Ferry for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.305.1 Development Application 174 Fulham Road, Dunalley for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.314.1 Development Application 11A Gordon Street, Sorell for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.317.1 Development Application 13 Ollie Drive, Sorell for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.331.1 Development Application 13 Mary Street, Lewisham for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.334.1 Development Application 1375 Kellevie Road, Nugent for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.336.1 Development Application 2 Erica Road, Primrose Sands for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.337.1 Development Application 664 Shark Point Road Penna Representations Close Monday 6th December 2021 for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.339.1 Development Application 33 Linden Road, Primrose Sands for Sorell Council, TAS. So, skipping
Error Mysql2::Error: Data too long for column 'council_reference' at row 1 while trying to save application 5.2021.342.1 Ddevelopment Application 22 Midden Road, Primrose Sands for Sorell Council, TAS. So, skipping
0 applications found for Sorell Council, TAS with date from 2021-11-18
33 applications errored for Sorell Council, TAS with date from 2021-11-18

What you can do next

If something isn't right with the scraper or the data coming in then you could do one or several of the following

If everything is working fine right now but you want to help if something goes wrong

  • Watch the scraper on morph.io, so you will get an email if it errors. Press the "Watch" button in the top right while you're logged in to morph.io.
  • View any scraper issues. Help fix them or comment on them if they are out of date

This week

Find PlanningAlerts useful?

This independent project is run by a local charity, the OpenAustralia Foundation. PlanningAlerts is powered by small, tax-deductible donations from the people who use it to stay informed about changes to their local area. If you find it useful, chip in to support PlanningAlerts.

Back PlanningAlerts