Under the hood of Noosa Shire Council

More of the gory details of how information gets into PlanningAlerts. This is here so you can see and learn how it's done but also so you are able to fix things and see in detail what isn't working.

The steps

  1. Applications are published on the Noosa Shire Council website in a human readable, non-structured form.
  2. The latest scraper code on Github is loaded by our scraping platform morph.io.
  3. The scraper planningalerts-scrapers/multiple_technology_one on our scraping platform morph.io collects the information daily and converts it into a machine-readable format.
  4. If the scraper errors anyone who is watching the scraper on morph.io gets informed via a daily email alert.
  5. The machine readable data is validated, imported and geocoded by PlanningAlerts daily. If there are any validation errors on an application, the application is skipped and the error is logged. See below for the most recent logs.
  6. The information is published on PlanningAlerts, made available via the API, and people are informed of new applications via email alerts.

Most recent import logs

Error Validation failed: Description can't be blank while trying to save application EXE24/0015 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application EXE24/0016 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application EXE24/0017 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0188 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0190 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0191 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0192 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0193 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0194 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0195 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0196 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0197 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0198 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0199 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0200 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0201 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0202 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0203 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0204 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0205 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0206 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0207 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0208 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0209 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0210 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0211 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0212 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0213 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0214 for Noosa Shire Council, QLD. So, skipping
Error Validation failed: Description can't be blank while trying to save application SSL24/0215 for Noosa Shire Council, QLD. So, skipping
57 applications found for Noosa Shire Council, QLD with date from 2024-05-12
30 applications errored for Noosa Shire Council, QLD with date from 2024-05-12
Took 24 s to import applications from Noosa Shire Council, QLD

What you can do next

If something isn't right with the scraper or the data coming in then you could do one or several of the following

If everything is working fine right now but you want to help if something goes wrong

  • Watch the scraper on morph.io, so you will get an email if it errors. Press the "Watch" button in the top right while you're logged in to morph.io.
  • View any scraper issues. Help fix them or comment on them if they are out of date

This week

Find PlanningAlerts useful?

This independent project is part of the digital library from the local charity, the OpenAustralia Foundation. PlanningAlerts is powered by small donations from the people who use it to stay informed about changes to their local area. If you find it useful, chip in to support PlanningAlerts.

Back PlanningAlerts