FOSS4GNA Day 2

Keynote

• Andreas: NC HB2 Legislation -> conference venue has been great

Jerry Johnston: US Dept Interior

NSDI

Been a great ‘success’

• unified catalog
• shared costs
• Massive # of datasets

so WHAT?

• metadata not machine readable
• only small fraction of datasets available via web services

Moving forward:

• Scalable web services
• 170 Core datasets
• All datasets available with web services
• Allow 3rd party management of products on provided infrastrucure
• Better search knowledge graph
• Easy sharing/map collaboration

Beyond Mapping Population Density: Cartography for Big Data

Speaker: Mamata Akella Room: 301B Abstract Slides

Big data display

10k points per map tile, tens of millions of points

• display all points
• glow maps: better to highlight higher density areas
  • Can add categories
• Aggregation: bin to a grid and apply counts per variable

Torque

• typically for time series/animcations
• Can use static SQL aggregate with statistics to bins

Deep insights

• multi dimensional pattern analysis (categories, histogram)
• real-time filtering
• seems really fast… displaying tiles generated from the vector information

Map Demo

• graduated symbols not weighted: just set width from calculated breakpoints

Thoughts

• sample dataset contained 30m points… how much would that cost?
  • didn’t know
• how do you normalize non locational information to a point?
  • You don’t, all points must contain geospatial info

Awaken the QGIS Within

Speaker: Gretchen Peterson Room: 301A Abstract

Beginner project setup with QGIS

Set up a basic map with labeled roads/coffee shops.

1. CRS setup
2. background color
3. add datasets
4. Style lines
5. rule based renderer: SQL where clause
6. Line labels
7. show labels for layer, Museo Sans font
8. placement = curved
9. Style coffee shops maker
10. draw effects = drop shadow
11. Coffee shops lables
12. text buffer
13. new shapfile layer
14. define attributes
15. add point tool
16. Add style -> draw effects
17. Add label

Print composer

1. Set size
2. Add map to page
3. Update extent to match map view
4. Lock layer extent/styles
5. Overview map
6. Need to adjust original map document to appropriate layer/extent
7. Any way to maintain both sources?
8. Title
9. White to transparent fill, can adjust orientation
10. Re-order in items list

Tips

1. Inverted polygon to highlight an area
2. Inverted starburst symbology

Climbing the data mountain: Scaling and optimizing the delivery of vector tiles

Speaker: Jake Guthmiller [mySidewalk] Room: 302C Abstract Slides

Visualize census data across US (800 attributes, 14 layers)

Vector tile generation:

• clients,

• requests

• tile complexity (features/tile + feature complex)

• attributes

Architecture

• client: leaflet.mapboxvectortile
• tileserver: tilestache (mapzen)
• db: pgpool
• cache: redis
• load balance: nginx
• microservices

Client

• domain sharding (a., b., c.*)
• CORS: preflight requests -> Add auth in req

Load balancer

• use cpu, network
• nginx
  • connection pooling to tile servers

Tile server

• set cache control headers
• db connection pool, cache connection pool
• Mapzen version
  • uses PostGIS to cut tiles rather than shapely/python
  • Proper Mapbox vector tiles

Microservices

• Tilestache: wants file for layer config -> Add REST endpoint to requst config from

Redis Cache

• redis-py: hiredis parser (C bindings)

Database

• pgpool II
  • connection pooling
  • streaming replication
  • load balancing

Client rendering

• Leaflet.MapboxVectorTile - canvas
• MapboxGL - webGL

Deployment

• Google cloud platform
  • compute
  • container registry
  • monitoring (stackdriver)

Autoscaling

• Swarm, Mesos, Kubernetes, OpenShift

Marine Protected Area Mapping - Promoting Awareness, Compliance, and Enforcement via Open Data and Tools

Speaker: Virgil Zetterlind [Anthropocene Institute] Room: 301A Abstract

Create a central map of all marine protected areas

Challenge

• multiple administrative orgs
• not mapped, only verbal description -> draft regions sent to org for review
• Regulations change
• Coastlines change w/ contradictory shapes for multiple orgs

Date upkeep

• Sounds like epic amounts of work…
  • Change detection on websites -> notified -> check it out
  • External URLs checked with monthly script for existence -> check it out

All very manual

• Massive struggles with data management

GeoWave: Visualizing Massive Datasets

Speaker: Rich Fecher [RadiantBlue Technologies] Room: 301B Abstract

Demo

• Fast distributed scalable storage / retrieval of geospatial data
• key:value store (Accumulo)
  • Data must undergo dimensionality reduction (2D -> 1D)
• 1.19 Billion rows
• Query returns from 20 node cluster in a couple seconds

Architecture

• Amazon EMR 20 node cluster 4 core large
• Accumulo

Visualizing GPS Tracks with RikiTraki

Speaker: Ricardo Morin Room: 304 Abstract

Motivation: Share GPS tracks from hiking trips

• Photos matched to GPS track by time if lat/lon not avail (with time adjustment)
• Nice UI with elevation profile, geolocated pictures
• Cesium layer for 3D viz

Architecture

• Leaflet and Cesium
• Bootstrap/JQuery front end
• Backend REST API in nodejs/express
• MongoDB database

Tips?

• Globalsign: free certs for open source
• Javascript finite state machine

Github

In-browser, scale-independent geo data analysis using vector tiles

Speaker: Anand Thakker [Development Seed] Room: 302C Abstract Slides

In browser interactive analysis

• geoJSON (turf.js?)
  • Size limited, scale issues
• PostGIS
  • Complexity, round tripping

Same problems as rendering tiles… piggyback?

How

• Building vector tiles
  • Simplify/drop features when rendering (tippy canoe?) OK, not for statistics
  • solution: running total for arbitrary grid cell -> aggregate
• Data driven styling
  • mapbox-gl-datadriven (polyfill - separated to layers, style by bins)

vt-grid

• tilereduce (aggregates raw data)
• tippecanoe (takes gridded data, makes tile)