Climactic Project Video On Youtube
Problem
Climate change is here [1]! How do we make this personal to the citizens of Australia through data? How do we show them the impact of rising temperatures, erratic weather patterns, increasing drought & floods, rising sea levels?
In our hack, we are focussing on insights that matter today! How can I be more informed about the areas where I am buying a new home using existing Gov data? How will rising temperature, changing rainfall patterns and flooding my risk factors as a potential buyer?
We also provide a vision of the data platform as we see it evolving though continuous feedback via events like GovHack! We ask: as systems capture data about the health of our macro and micro environment, how do we bring this data into the risk analysis and decision making processes we do as citizens, community and government on a real-time basis?
Solution
Our solution consists of an immersive data experience through AR / VR to view the area of interest with data from Government (climate and environment data) and private sector (realestate prices) combined
Our application combines land, building height, flood, wetland, urban heat etc data in a 3-dimensional view allowing the user to walk around structures and really see the impact of Climate change
The viewer can then move a slider to change their risk profile, for example, to change the projected environment information and view this to make more informed decision
Citizen Data Platform: Data Query portal
Query for information and get data mashup across Public, Private and SEED datasets. Using the latest in GraphQL over existing SEED data assests for Maps and Point
Citizen Data Platform: Data Upload Portal
Along with the SEED and Private datasets, we all benefit from contributions by Citizen scientists! As IoT devices become more ubiquitous, our platform can support the ingestion of data and create rich mashups with SEED datasets for better information
Use Case
Scenario: Citizen Jane wants to buy a property around parramatta is looking for a data-mashup to make better decision for now and the future using the existing datasets. Jane is able to use the Climactic Augmented reality application to search the local area and overlay interesting data such as realestate price, flood plain, wetland, heatmap during summer etc to make a more informed decision about her risks.
Jane is especially concerned about the impact of climate change and wants projections for the future because she wants this to be the family home for the next generation. She is able to use the sliders on the Climactic application to increase the projected values to what she-thinks will be the reality
Jane is amazed to see a mashup of government data from SEED and realestate data from public domain visible in 3-dimensions using the augmented reality (AR) application
Data Mashup
1. Property Layer: What properties are for sale in Parramatta?
2. Planning Layer: What land and building (height) data is available from SEED?
3. Environment Data Layer: What flood plains, wetlands are there and where?
4. Environment Projection: How close is the projected flooding area to my potential home?
How about the impact to Transit around my area due to flooding? Can I overlay Transport NSW data over SEED flooding data to view impact to my commute as I live here?
5. Urban heat: We have had pretty bad heatwaves in the summer & with the new building coming up in Parramatta - I wonder what the current projections look like?
6. Better projection Using Machine Learning (ML): Government and Citizen data produced is analysed to do better projections for the future
7. Mashup: Final result is a mashup of the SEED data, AI ML projection and public dataset
Architecture
Microservices supporting SEED services: GraphQL API over existing datasets reduce hurdle rate in searching and consuming backend services
- ArcGIS Map REST services: A GraphQL front end over domain data collection and ArcGIS Map accepts requests as GraphQL query, fires ArcGIS or other map server specific REST query and returns mash-up data as a GeoJson file
- Excel or other tables: Simple adapters ( short lines of serverless code) transform complex Spreadsheet layouts to standard GeoJson or Json formats
Platform: Cloud based platform for ingesting high volume data, supporting smart queries and making smart decisions using machine learning. A conceptual AWS architecture is presented below
Standardised Data Format: Contextual data with GeoJson SEED data for better information
Datasets
- IPCC AR4 SEA-LEVEL PROJECTIONS: The Sea-level rise projection from 1990 - 2100 is used to calculate the water rising in and around sydney wet lands
- Temperature Data Description of Use: The temperature data set for Parramatta will be used to feed into the visualisation we have for the climate change. We want to use the information and also combine with a data set similar to https://www.climatechangeinaustralia.gov.au/en/climate-projections/explore-data/data-download/gridded-data-download/
- Planning/EPI_Protection_Layers: Coastal wetlands are most at risk due to climate change. Understanding the impact of climate change and more specifically of sea-level rise on coastal wetlands must take into account factors that affect the ecological balance of wetland ecosystems https://mapprod1.environment.nsw.gov.au/arcgis/rest/services/Planning/EPI_Protection_Layers/MapServer/1
- Planning/EPI_Protection_Layers: Coastal wetlands are most at risk due to climate change. Understanding the impact of climate change and more specifically of sea-level rise on coastal wetlands must take into account factors that affect the ecological balance of wetland ecosystems https://mapprod1.environment.nsw.gov.au/arcgis/rest/services/Planning/EPI_Protection_Layers/MapServer/3
- Open Street Maps Description of Use: Provides base maps on which we provide our solutions. Allows our solution to be used globally
- NarCLIM Data: Climate change data with Rainfall, Temperature, Flood projections https://mapprod.environment.nsw.gov.au/arcgis/rest/services/NarCLIM/ClimateRegion/MapServer/1
- DPI - Department of Primary Industries Water
- Ground Water Sharing plan
- Real time monitoring sites - Ground water
- Real time monitoring sites - Major storage
- Real time monitoring sites - Surface water
- Regulated Rivers
- Surface water sharing plan
- Major Catchment, PORT JACKSON / GEORGES RIVER Name of the Plan: Water Sharing Plan for the Greater Metropolitan Region Unregulated River Water Sources 2011. Water Source; Northern Sydney Rivers Water Source Start Date; 1/07/2011 End Date; 28/06/2021
- Local Environment Plan (LEP): Height of Building https://mapprod1.environment.nsw.gov.au/arcgis/rest/services/Planning/EPI_Primary_Planning_Layers/MapServer/5
Suggestions for SEED Data Improvement
Existing services provide a great potential for government agencies, citizen scientists and weekened hackers to consume and mashup data from various sources. However we made a few key observations as we consumed this data and have suggestions to make this even more accessible
Observations & Suggestions for improvement
- A Common platform for hosting, cleansing, analysing and serving data. While the SEED portal is amazing it draws from various data sources hosted and collected in different formats - a Common platform will help data providers & consumers a single location to share, process, search and analyse gov and citizen data
- Streaming Real Time Data: While static data is fun, streaming data is the future. As more connected devices and individuals provide this data there needs to be a way to ingest this real time for processing & serving later. Existing technologies like AWS Kinesis, Confluent Kafka Cloud services, Google Cloud PubSub would be used to ingest data into _streams_ for cleansing, processing and storage. There could be multiple consumers to the streams, allowing end consumers to subscribe to real time events
This also opens up the potential to obtain real-time data from new technologies - smart phone, smart homes, smart cars to make more real-time decisions about our local environment & move the dial the other way faster!
- Microservices: As we were consuming the data for govhack we found most of the map information is served by a single monolithic product (ArcGIS REST service) which required subject matter knowledge around the product to query the data. As an open data system, we this as a _hurdle_ to _self-service_ and a not scale. Instead we propose using light-weight services by domains and surfacing map data through more standardised, product agnostic RESTful APIs
Summary
A streaming platform with GraphQL API for querying data and a Citizen portal for uploading batch/real time data. The platform would integrate with existing SEED service, for example microservices layer over the existing ArcGIS RESTful services. Lightweight event-driven microservices and wrap the SME knowledge around ArcGIC Rest services & have optimised queries, queries by popular regions, relations to other data sets. These domain services would also accept inbound data, batch / real time ( upload streaming NSW costal wetland data for a geometry ) and push it to the data pipelines
Credits:
Vinod Ralh