Category Archives: Empire Statewide GIS

SPEED 2.0: Authoritative Environmental Remediation Mapping in New York City

Posted on by
Reply

Application Includes the use of both Open Source Software and Open Data Content

A lot of great geospatial projects and content are coming out of the NYC OpenData ecosystem.  In the same space  throughout the city is the deployment of applications and viewers using open source software.  One such app is the Searchable Property Environmental E-Database SPEED 2.0, built on top of CARTO and published by the Mayor’s Office of Environmental Remediation (OER).  I was introduced to the application via an online presentation organized by GISMO in March of this year.

SPEED 2.0 is an impressive collection of local/city, state, and federal geospatial datasets wrapped into one application for the purpose of helping individuals identify environmental issues – both current and past – on and/or adjacent to specific properties in New York City.  It is a sister application to the NYC Office of Environmental Remediation’s Environmental Project Information Center (EPIC) that provides information about the cleanup of brownfield sites across the city.

Individual parcels can be buffered by either 250’ or 500’ to show the proximity of adjacent parcels with current or past environmental issues, permitting, or contamination issues. Access to pertinent metadata is readily available.

Background

According to Lee Ilan, Chief of Planning in the Mayor’s Office of Environmental Remediation, the first version of SPEED was launched in 2009 as a web map with limited functionality and developed with PostGIS.  It was launched in support of the newly created office’s focus on the cleanup of brownfields across the city.  However, support for the initial application waned over the next several years with minimal new content added.  Post – SuperStorm Sandy provided new funding through the U.S. Housing and Urban Development (HUD) Community Development Block Grant Disaster Recovery Program which OER secured and offered the opportunity for a major rewrite and update of the original application. SPEED 2.0 was designed by their vendor Applied Geographics (AppGeo) to be a cloud-based application.  Originally the application was managed by the vendor but since December 2020, OER has assumed managing the app in the Google Cloud on their own.

The application also includes advanced search functions. For example, in the left-hand column using the filter options, I was able to identify only those OER projects in FEMA 100-year floodplains. Query is rendered in the map viewer.

“Carto software is helpful by providing a very modern user interface that generates layers which are compatible with Leaflet”, notes OER’s IT Director Maksim Kleban.  “It makes the transition from uploading our layers, and turning them into fully functional, interactive maps seamless.”  AppGeo proposed the use of CARTO to OER which has since found the software to be user friendly and simple to use with standalone online applications. Carto is licensed annually for the amount of space and resources needed for the SPEED application and works very similar to any other cloud solution, like Amazon Web Services, Microsoft Azure (AWS), or Google Cloud.

Currently there are about 50-55 datasets included in the SPEED viewer right now. The large majority are OER datasets which are updated automatically by syncing with data from external agencies’ datasets on Open Data, or from OER’s internal data sources.  Generally, they each have an independent update schedule which is also automated.   The data is managed mostly by automatic updates on OER’s server which communicates directly to Carto through an API. For layers which are not on an automatic update schedule, OER uses either a custom-designed interface or manually uploads data into Carto’s online platform.

User can search the SPEED database using a standardized address, common place names such as Bryant Park or Madison Square Garden (btw – even “MSG”!) or borough, block and lot (BBL) numbers.  The application also includes mark-up, feature transparency, and sharing tools,  great HELP documentation and easy access to metadata (as illustrated in the first image above) which is very helpful given the bevy of similar datasets from local, state and federal datasets accessible in the app.  Historical aerial photography from 1996, 1951, and 1924 enables users to identify previous land cover which can be an indicator of the presence of historic fill.  A “Sensitive Receptors“ layer includes the locations of facilities (schools, parks, libraries, health care, etc) where occupants are more susceptible to the effects of environmental contamination.

It continues to be a work in progress” says Ilan, “in the future we would like to also have functionalities for registered users. We also would like to add more analysis capabilities where new layers can be easily integrated with advanced search features”. 

SPEED 2.0 Featured on NYC Open Data Week

For the first time ever, OER participated in NYC Open Data Week in early March.  For those looking for a deeper dive into SPEED 2.0, use the link below to listen to Lee’s presentation.

Contact:

Ms. Lee Ilan
Chief of Planning
NYC Mayor’s Office of Environmental Remediation
lilan@cityhall.nyc.gov

Taking a Peek Under the Hood: New York State GIS Association

Posted on by
Reply

How’s Your Regional GIS Association Group Doing? Biden’s Infrastructure Plan. And Dude, What’s Up with Senate Bill S1466?

Its mud season in the Adirondacks so outdoor activities are kinda slow in the North Country.  Not much on the trails and the black flies loom.  Similarly is the blog content, scrounging around for new material and the like.  Couple articles in development hoping to pull together over the next couple months.  Some cool stuff coming out of NYC.  Seemed like a good time to pull together some topics and issues I’ve been staring at for a while.  Much of it related, for the most part, to the NYS GIS Association.

New York State GIS Association Regional Groups

The end of last fall I started a conversation with a GIS colleague who was responsible coordinating meetings for a local/regional GIS user group. An honorable task given the amount of time and effort which goes into doing so – often solo or with only one or two other folks.  Our conversation focused on a combination of declining attendance and participation throughout the region albeit at the time this may have been more of factor of COVID.  The play book almost the same everywhere:  virtually no in-person meetings, Zoom fatigue, and/or in general everyone completely pre-occupied with the pandemic.  All of this adding to the challenge of working with others to develop and propose meeting themes which would/could focus on something other than COVID.  And as we all know, much easier said than done over the past 16 months.

Curbing my own personal thoughts and opinions about where the profession is headed in this space across the Empire State – fodder for another article and another day – I put together a short 10-question Google forms questionnaire which was sent to the 15 regional coordinators listed on the Association’s Regional Coordination Committee web page.

Response rate was actually very good as I received responses from 14 of the 15 individuals I sent surveys to.  One of the 14 opted not to complete the survey leaving only one coordinator not responding.   Selected questions are listed below for which I’ve included summary results.   A spreadsheet containing all questions and and responses by all respondents can be downloaded here.  For the purposes of this article I removed the name of the Regional User Group so responses could not be linked to a specific person or group.

Taking a look at some of the questions and grouped responses:

1. When was the last time your Regional Group met?

2.  Was the last meeting in-person or online?

3.  Whether in-person or online, how many people participated in the last meeting?

4.  How many people are involved in helping coordinate and administer your user group activities (i.e., scheduling meetings and speakers, establishing agendas, maintaining email lists, maintaining any kind of user group website, etc) as of March 2021?

5.  Any thoughts as to whether or not your Regional User Group will go back to in-person meetings once the COVID pandemic has generally passed?

6.  Are any future Regional Group Meetings scheduled?

(Note:  During preparation of this article one Regional Group announced a Fall 2021 in-person meeting

7.   Is the membership of your User Group mainly composed of government workers or from private sector/industry?

8.  When User Group Meetings are held, do the topics and content of discussion focus on local/regional geospatial issues or more on statewide GIS issues?

Synopsis

Given the high percentage of the number of regional coordinators who responded to the survey (14 of 15; one responding to me but not completing the survey) the information gathered should be considered reflective of the current activities and engagement of the regional groups.  Admittedly the survey was really only 10 questions and completed during our time of  COVID, thus one could argue that some of the organizational issues which the Association is facing is similar to other organizations due to the pandemic.

That said, couple immediate takeaways on the pulse of the Association’s regional groups:

While the pie chart shows nearly 54% (7 of 13) of the groups haven’t met in over 12 months, overall its probably closer to 60% as I suspect the two groups which did not respond to the survey have not met during the same time period as well.  One of those for sure.  Sixty percent is significant.

Closely related is the question about whether or not coordinators anticipate getting back to “in-person” meetings after the pandemic has passed.  Seemingly more relevant, however, was the issue of “re-energizing” the group first (61% or 8 of 13 responding).  My guess it’s safe to assume the other two survey non-respondents are in the same space or mindset – and the percentage goes to a whopping 66% – 10 of 15.  75% of those responding indicated the next meeting wasn’t even planned and again, if you throw in the non-respondents to this questions it goes to over 80%.

A noticeable feeling of malaise?  Seems like enough to give the Regional Coordination Committee Co-Chairs something to think about.

Findings to other questions which included, for the most part, anticipated results such as:

  • Meetings taking place (independent of how long ago) were about half in-person and half-online)
  • Numbers of people participating in meetings (in-person or online)
  • As we all have painfully come to know, normally a handful of people (in many cases only 1-2) individuals coordinating the group and meetings
  • Probably didn’t ask the question properly, but nonetheless it appears the groups are split about 50-50 on private/industry sector vs. government individuals; and
  • Meeting agendas/discussion tends to focus about 50-50 on local/regional issues vs. statewide issues

Biden’s Infrastructure Bill

I included the question about meeting content (local level vs. state level) to get a sense of the discussion as to who/what may be directing the narrative.  Why, you say?  How about Biden’s $2.3B Infrastructure Plan.  Rarely has so much federal money been allocated which links itself to local and regional GIS programs.  Roads, buildings and utilities, bridges, public transit, water and sewer systems, disaster resiliency, public schools, and much much more.  Large amounts of geographic features and systems best managed at the local level.  And with local resources.  If not local and regional governments themselves then with trusted consultants and business partners.  Engineering consultants with broad and capable geospatial capacity.

Yes, the time for local and regional geospatial programs to proactively and reach out to state organizations such as the NYS Association of Towns, NYS Association of Counties, ot NYS Conference of Mayors to work towards making sure this new funding sources are secured for local use and application in the geospatial space..  As well as enhancing collaborations and efforts with professional organizations such as the New York State Society of Professional Engineers and New York State Association of Professional Land Surveyors which have an established presence in Albany.

Of course state assets are in play with regard to this funding, but local and regional geospatial programs, in May 2021, cannot afford to sit on the sidelines to wait and later find out distribution of this new federal funding is going to state programs first and being left to fight for the remaining scraps. And/or left to some funding distribution formula which included no local and regional geospatial programs input.

Time now for local and regional GIS/geospatial programs and organizations to mobilize on this funding opportunity.

Senate Bill S1466

Hello, McFly? Anybody home?  Anybody watching and monitoring anything with regard to the state legislature and legislation which is connected  to the Empire State geospatial space?  In the big scheme of things, S1466 probably doesn’t amount to much but it does seem something is amiss in the broader context when the words Geographic Information Systems Mapping Technology are used in describing any proposed New York statewide legislation and it goes completely under the radar screen without review and input from the statewide geospatial community. Can’t say absolutely no discussion because something, on some level, some people, somewhere, were talking and exchanging information to frame the legislation.  Was the Association part of that discussion?  Or maybe your Geospatial Advisory Committee?

And a bigger head scratcher is connecting the state GIS office with “strategic planning and municipal study assistance”.   Specifically the proposed legislation reads, in part:

“The usage of available floor space within a given political subdivision
is a critical detail. Evaluation of floor space usage allows the local
area to understand how buildings are being used, and evaluate whether
new zoning or construction plans are needed to stimulate activity in
certain sectors. However, conducting a study through questionnaires and
or geographic information mapping (GIS) technology is difficult and
costly for local entities. On the other hand, the Office of Information
Technology Services already maintains statewide GIS information and can
readily adapt that material for more specific purposes.”

This legislation would allow towns, villages, cities, and counties to
request GIS and study assistance from OITS to undertake a review of
floor space usage. Such assistance could include specific GIS maps,
online questionnaires, and other technological methods that would assist
the conducting of such studies.

Maybe I need to get out more.  Granted, if passed, the whole process of ITS of getting involved presumably wouldn’t happen unless requested locally, though it is interesting to see this office ready to provide these professional services.  Also, interesting it hasn’t ruffled a few feathers in the professional planning and GIS consultant communities as well.  NYS Association of Regional Councils?

Almost two years ago I made reference to a similar geospatial community asleep at the wheel legislative moment with regard to Senate Bill 9061 involving Google Maps.  Have there been similar pieces of legislation?  In absence of full-time staff – including an Executive Director who can operate in the Albany space – its almost an impossible task on staying on top of pending legislation proposed by others.  To say the least of identifying sponsors and proposing legislation for the benefit of the industry and membership.

One way or another, the legislative space is where the Association needs to expand and build capacity.   Thus far it hasn’t been easy and will continue to be difficult to do so.

 

Celebrating – and Visualizing – New York State Scenic Byways

Posted on by
Reply

Recent national designation highlights impressive Scenic Byways inventory across the Empire State

With little fanfare in January, the Federal Highway Administration, through the National Scenic Byways Program (NSBP), added the Palisades Parkway Scenic Byway to its list of New York State “national” designated Scenic Byways.  The Palisades designation – northern and southern most boundaries highlighted in the image below –  brings the Empire State’s National Scenic Byways total to four joining the Great Lakes Seaway Trail, Lakes to Locks Passage, and the Mohawk Towpath Byway.  These four combine for nearly 700 miles of nationally – designated scenic byways across the state.

The new nationally designated Palisades Parkway Scenic Byway, part of the State’s Scenic Byways program, originates in the south at the New York/New Jersey boundary heading north approximately 26.5 miles to just west of the Bear Mountain Bridge.

The designation augments the established New York State Scenic Byways program which includes both the federal and state designated scenic byways.  Administered through NYS Department of Transportation, the State’s Scenic Byway program was created in Highway Law (HAY 349-aa-dd) in 1995 authorizing the Commissioner of the NYSDOT to establish the scenic byways program and to coordinate the activities of state agencies, local governments and not-for-profit organizations.  Prior to the establishment of the state Scenic Byway program, the NYS Department of Environmental Conservation had a scenic roads program under Article 49 of the Environmental Conservation Law.  Those roads, along with all state parkways (listed in the regulations of the Commissioner of the Office of Parks, Recreation and Historic Preservation pursuant to Section 13.03 of Parks, Recreation and Historic Preservation Law) were incorporated into the Scenic Byway program under HAY 349-dd (k) and (l).

HAY 349-bb defines a “Scenic Byway” as a transportation route and adjacent area of particular scenic, historic, recreational, cultural or archeological characteristics which is managed to protect such characteristics and to encourage economic development through tourism and recreation.  Today, the State’s program includes the four national and 26 state designations.  A list of all designations the program is available here which includes a link to a thumbnail graphic of each byway.

State Scenic Byways Map

The January designation gave me reason to revisit a favorite piece of cartography I’ve been eyeing and wanting to write about for quite some time.  It’s a map on the Scenic Byways web page which speaks to the art of a well and purposely designed hardcopy map.  Increasingly lost as part of our insatiable appetite for interactive online mapping as the primary “cartographic” end product. Substituting the web viewer for what once was the primary GIS desktop objective:  The map.

Descriptive information and an individual map of each byway is found on the Scenic Byways homepage.

The map was created by staff in the Landscape Architecture Bureau (LAB) in the Department of Transportation. According to Christine Colley, Senior Landscape Architect and Scenic Byway Coordinator, its original creator was Marcy Sammons – who now works for another state agency.  Map revisions – to update routes and add new routes not previously included in the original map – is now the responsibility of Alexandra von Bieberstein.  Both staff are landscape architects whose background clearly contributed to the “feel” and overall simplistic cartographic design of the map. Symbology – colors, fonts, labels and shields – are all appropriately selected.   The state boundary clearly defined with green/gray green underlying base map colors which accentuates the colors representing the statewide byways. Even the limited use of selected hydrologic features – providing more context to the map than anything – as well as the limited amount of place names adds to the visual appeal of the map.  All map components combined to perfectly convey the statewide scenic byways story.

Colley notes “The map is created using ArcMap (LAB currently using version 10.7.1  although the original map may have been created using an earlier different version) and then then exported into Adobe Illustrator for editing and providing more detailed graphic enhancements.”

In absence of online mapping applications, Colley also adds that Scenic Byways staff respond to “map a specific Scenic Byway or to key locations on the Scenic Byway.  Staff also assists the public – using Google Maps to create a route – in navigating through the state to enable travelers to” requests by sending the public Google Maps “routes” from their location to:

  • Avoid parkways (which are limited access – no trucks, trailers or RVs on the parkways)
  • Encounter the most scenic route
  • Encounter the most scenic byways

For a myriad of reasons, hardcopy maps will continue to be an important product of the geospatial profession and a very special skillset to maintain.  The utility and importance of the traditional map has even been noted in online geospatial blogs and media publications in recent months.  Just last month there was an article in USA Today entitled ““Move aside, Google Maps, Apple Maps and GPS:  Why people still love their paper maps”

If you haven’t had a chance, download the map and take a look.  Pan around, zoom in to larger scales and take a spin around the state.

Enjoy the views.

Contact:

Christine Colley, RLA
Scenic Byway Coordinator
Senior Landscape Architect
New York State Department of Transportation
Office of the Environment
50 Wolf Rd, POD 4-1, Albany, NY  12232
518-485-9313 | christine.colley@dot.ny.gov
www.DOT.NY.gov

Cruising Empire State Data Portals for Geospatial Content

Posted on by
Reply

Expanding Suite of Online Portals Offer Data For All Types of GIS Users

Ever since the federal government launched Data.gov in May 2009 and President Barack Obama issuing an executive order in May 2013 establishing the Open Data Policy, as well as   Governor Andrew M. Cuomo signing Executive Order 95 (“Using Technology to Promote Transparency, Improve Government Performance and Enhance Citizen Engagement”) in March of the same year, governments at all levels across the country have responded in creating an array of online data portals with the intent of making data free and easily accessible.  Available and open to everyone including nonprofits, application developers, researchers and academia, business and industry, service industries, the geospatial community and everything in-between. Such portals purpose and intent is to promote government transparency, reduce data request and distribution costs, fuel business development and job growth, and drive innovation.  Open data portals are particularly useful in times of crisis or major incident when multiple agencies require access to the same data and/or may be limited in context of  data sharing agreements either ad hoc or formal.

Current Empire State Data Portal Offerings

Today, geospatial data portals across the Empire State have a variety of aliases including Open, Gateway, Hub, even the old school Clearinghouse title.  For the purpose of this article we’ll take a look at a range of the current publically available sites focusing on current content and some of the general functionality as it relates to the GIS/geospatial user. Sites included are those currently available in New York City, state government, and at the county and municipal.  It is by no means inclusive of all the sites across the state though for the ones itemized below, take a drive and see what you find.

 Map Services vs. Files

Accessing, or consuming, geospatial content has changed significantly since the early years of data sharing.  The term “open data” was barely in the geospatial vernacular in old school GIS with many sites providing access to data which was governed by formal data sharing agreements and/or requiring some kind of organizational membership.  Offering file-based downloads to support, which at the time, was a heavy-client software-based world of GIS computing.

While file-based downloads from the myriad of portals across the Empire State are more than adequate for most users of the sites, GIS/geospatial users often have software enhanced capabilities to consume large volumes of data content in both client and browser applications via map services.   Consuming web services is now even an option within certain versions of AutoCAD which greatly expands the use of government authored and published geospatial content.

Given the option of having to take the time to download numerous individual files vs. consuming one map service with many data layers and picking only the data layers needed – the choice is often pretty obvious.  As such, data portals which offer geospatial content as a service (or by extension developer tools to provide similar functionality) increasingly have added value to the GIS/geospatial user.

New York State Government

HEALTH.DATA.NY.GOV & DATA.NY.GOV

Launched approximately around the same time in 2013,  these two portals provide lots of file based content for the GIS/geospatial community.  Built on top of Socrata software (Tyler Technologies) the two sites differ only in content. DATA.NY.GOV serving more as a generalized repository for state agencies (and selected local governments) and HEALTH.DATA.NY.GOV focusing specifically on public health related datasets.  Creating an user account (if needed) provides developers and publishers to identical tools and documentation to each site including customization options, database connections and API endpoints, metadata documentation, visualization tools, and more..  Both portals as part of the broader Open NY initiative.

While centralized data access and data visualization (charts, dashboards, etc) have always been a primary Socrata focus, the software also includes a basic toolkit  to map and render datasets which include coordinate geometry.  Both sites also include the ability to download datasets in a variety of formats including KML, KMZ, Shapefile, and GeoJSON which can be used by a wide range of GIS software platforms.

Users of DATA.NY.GOV site need to be aware this site ALSO includes content from other statewide Socrata sites (HEALTH.DATA.NY.GOV, NYC Open Data, and  Open Data Buffalo) so it is important to use the filtering function in the lower left corner of the page after the initial data theme/category has been selected.  For example, in the image below, after selecting the “Transportation” category, the application returns 521 records – which includes those meeting the same criteria from the other three statewide Socrata sites. (And in this category many from the NYC Socrata library.)   After filtering for “This site only” (DATA.NY.GOV) the application returns just 251 records.  A small nuance but filtering does help if the search is specific to certain jurisdictional or geographic footprints.  Also, when searching for state agency data across the available statewide portals, users will begin to see that some agency data is published and available on more than one portal.  For example, some state agency data can be found on  DATA.NY.GOV, CUGIR, and the state Clearinghouse.

The filtering function on DATA.NY.GOV limits the search to specific domains

Plenty of geospatial data in both of these state portals albeit as the GIS community has come to know during the COVID-19 pandemic, address or large scale data is hard come by in the public health space.  Respectfully a whole other discussion and until there are fundamental changes in how public health data is published, it will be difficult for non-public health agencies, advocacy groups, or the general public to access such data at units of geography smaller than the county level.  Or sometimes if one is lucky, finding some at the zip code level.

For additional information on how to export Socrata data to other formats and/or platforms see this help document.  Also, Koop software is an Open Geospatial ETL Engine which enables users to leave geospatial data where it lives and transform it into a variety of formats including GeoJSON, CSV, KML, a Shapefile, or Feature Service dynamically.

There is apparently ongoing work/research within the state GIS offices with regard to both mapping NY Open data using the Socrata JavaScript API as well as enabling the Socrata platform to use the NYS GIS Program Office geocoder.  If successful, such efforts will greatly expand the Socrata data portals for the geospatial community.

NY Department of State Geographic Information Gateway

Launched in 2015, the Gateway is administered out of the NY Department of State, Office of Planning, Development and Community Infrastructure and is now home to over 700 datasets. The Gateway is an online mapping application that makes it possible to view geographic information in its native viewer or Google Earth and users can also download information in multiple file formats or even pull into other online mapping applications.  While the original geographic footprint of the Gateway focused on New York’s offshore planning area in the ocean and State’s portion of the Great Lakes, the repository now contains a wide range of geospatial datasets covering the entire State.

The Gateway has datasets  grouped  thematically and made available as map services. All content includes accompanying metadata enabling users to identify which data layers to select and easily consume from within the composite map service.    Perfect for desktop or web client mash-ups and analysis.  The site also includes links, integration, and content with crowdsourced apps.

The Gateway is a must see site having itself apart from other NYS departments in context of building a geospatial data portal with so much functionality and content.  There is a good chance you’ll find content for your GIS project here.

The Gateway data browser has icons for actions that can be taken with desired datasets; download, view in native map viewer, export to Google Earth, information for ingesting as a map service and metadata.

 NYS GIS Clearinghouse

The Clearinghouse was originally implemented in the late 1990s to facilitate statewide data sharing.   For many years it was the defacto #1 portal for all things/files GIS.  The focus being based on formal/written data sharing agreements between government and non-profit organizations. Amazingly in 2021 these agreements are still part of the Clearinghouse administrative structure though it does not get in the way of non-data sharing members – even those from industry and business – from downloading many datasets.  Give credit to those individuals who helped create the original Clearinghouse as it was one of the first of its kind at the time.

Much has changed since the Clearinghouse was initially launched which has impacted the data component of the site.  Time itself, and in this case decades, has proven to be a major challenge.  Publishers have come and gone, numerous retirements and point of contacts no longer exist, and limited metadata and datasets being out-of-date.  It’s a next to impossible to manage and keep most of the data files up-to-date and uniquely different in portals like the Clearinghouse.  Given the new generation of geospatial data portals that  we should expect to continue to emerge (particularly at the local level), the relevancy gap of the Clearinghouse’s data file inventory will likely continue to widen.  To say the least of the role of other similar data and web services available online via other free government and business sites such as The National Map, EPA Envirofacts, and ESRI’s Living Atlas.

The upside is the Clearinghouse is emphasizing and pushing data content  as  web services having placed this “option” at the top of the data catalog list. Seemingly to encourage users to “try this first” before sitting down and mining through the catalog on a file-by-file basis.  Much of the web service content are from the go-to State GIS office program areas (imagery, elevation, addresses, geocoding services, and to a limited degree tax parcels) albeit other state agencies are contributing services as well.   Kudos to the state agencies are keeping their individual file catalogs up-to-date and current albeit as earlier mentioned some of these state agencies are placing data on multiple portals requiring users to search in more than one location.

Instead of performing file based searches, users can access content via map services from a variety of state agencies

Hopefully, the legal shackles of the Clearinghouse data sharing agreement – instituted in the previous century – alone do not continue to influence the long term focus of the Clearinghouse with regard to managing and publishing individual datasets. One day, the statewide geospatial community would probably be better served when all of the remaining relevant  Clearinghouse datasets are consolidated into DATA.NY.US or CUGIR.     And perhaps leave the Clearinghouse to focus just on publishing data as a service with the current capacity it has built.

Municipal Government

NYC Open Data

Another Socrata site, NYC Open Data is the big kid in the statewide open data sandbox with over 3,000 datasets grouped into just five categories available from nearly 100 reporting NYC departments.  In addition to just datasets, the site also includes files, documents, and external links.  So, if you are looking for geospatial data for the NYC geographic footprint this is the place to go.  The site has been around for several years already and is the creation of the  Mayor’s Office of Data Analytics (MODA) and the Department of Information Technology and Telecommunications (DoITT). As a hub of analytics in the City, MODA advocates for the use of Open Data in citywide data analytics and in the community.

The NYC Open Data Portal also provides excellent narratives and examples of how geospatial data is used and is critical to the operations of city government.

NYC Open Data exemplifies the use of open data giving rise to a broad user community including organizations such as BetaNYC which hosts events such as the NYC School of Data, Open Data Journeys, various Open Data Classes, and in concert with the Mayor’s Office, the signature NYC Open Data Week – this year March 6 – 14. And data from the portal is routinely used in the NYC BigApps competition.

To fully appreciate how far open data has come in NYC, take a look at the list of publications and reports on this web page which chronicles its growth over the past several years.  And if you need further convincing, take a look at the NYC Open Data Project Gallery.

Go to the site and drive around.   Its worth the ride.

City of Rochester (DataROC)

Cities are great units of geography to build data portals in that their governments often oversee management of all the geographic features and data within the municipal boundary footprint.  Albeit this does vary on occasion with regard to utilities and some infrastructure  Geospatial data is consistent at the city level as government business applications are built and deployed to capture data to the same geographic extent.  As city applications continue to be geo-enabled, and open data initiatives are initiated,  this geofencing of the data provides a consistent geographic extent and a great framework to support a geospatial data portal.

DataROC was launched in 2020 and due to the city’s long time relationship with ESRI and staff who are statisticians and/or business intelligence analysts.   “ESRI’s open data framework was a good fit for the city” according to Kate Day, Chief Performance Officer, Mayor’s Office of Data Analytics & Performance.   Prior to the rollout of DataROC, the city already had both a police data portal and a map gallery of different ESRI tools, so the first major part of DataROC was to bring previously decentralized content together and make it easy to search. GIS staff in the city’s IT department assisted as change champions to make sure that the wider GIS community spread out across the City were aware of what the focus of the portal was to be and drive more traffic to the  work analysts in various department had done over the years.  Currently there are 203 items in the city’s open data catalog containing data files, apps, maps, documents.  If you need geospatial data for a project inside the Rochester footprint, this is the portal to visit and search for data.

DataROC offers easy access to data, applications that feature maps curated for a specific purpose, featuring up-to-date data, and a gallery of web maps to get around the city

With regard to an “example” as how specific governments vet data to their portals, the City of Rochester “publishes anything that is classified as public data / documents / applications that have been requested to be published by an internal or external stakeholder, but only after they have been approved by a Data Governance Committee”.  This committee ensures each dataset is truly for public consumption (essentially that it poses no individual or group harm for being published, that it would not violate any law/regulation to publish, and that the dataset meets quality standards to accurately represent whatever subject is being depicted or summarized).

Other illustrative statewide local government data portals include the City of Buffalo (Open Data Buffalo), City of Albany (openAlbany),  Westchester County (GeoHub), and City of Syracuse (DataCuse).

Academic

Cornell University Geospatial Information Repository (CUGIR)

I’ve always been a huge fan of the CUGIR portal and related program offerings.  Housed in academia and on the fringe of the statewide GIS mainstream, its original homegrown portal was built  and developed in Java in the 1990s.   The current CUGIR portal was launched in January 2018 and  is is built on several modern open-source components.  The front-end interface uses GeoBlacklight — https://geoblacklight.org/  which is a collaborative project being developed by several universities, including NYU, Minnesota, Stanford, Princeton, UC Berkeley, Cornell, and others.  CUGIR has made various customizations, some of which have been incorporated back to the project.

Subjects such as landforms and topography, soils, hydrology, environmental hazards, agricultural activities, wildlife and natural resource management are included  in the CUGIR catalog. All data files are cataloged in accordance with FGDC standards and made available in widely used geospatial data formats.

Behind the scenes, CUGIR uses Solr for searching and faceting (also known as “data filters”).  The site  use GeoServer to access copies of the data in a PostGIS database to generate map previews via publically-accessible WMS/WFS web services for each dataset.  Amazon S3 is used to host all the downloadable data as pre-generated zipfiles, so downloads are quick and reliable. The site uses  Leaflet to display the map on top of a CARTO basemap (which is built from OpenStreetMap data the web viewer.   Maintaining and supporting the site is long-time CUGIR GIS front man, Keith Jenkins.

Today, CUGIR’s datasets total about 15GB.  The total number of “datasets” listed in the catalog is currently 458, which appears much smaller than the nearly 8000 it used to be because in the old system CUGIR had content split up into many smaller downloads to accommodate the bandwidth limitations of the late 1990s and early 2000s.  So to leverage newer technologies and internet speeds, several data series were merged into statewide datasets.  As part of the process, CUGIR also converted old formats like ArcInfo Interchange (.e00) files to shapefiles wherever possible.  As part of the site upgrade, old links which had bookmarked or published were updated as well.  Today, NYS DEC and USDA NRCS have the most CUGIR records with other large catalogs from NYS Department of Agriculture and Markets and Cornell Institute for Resource Information Sciences (IRIS).   Data is also available from nearby Tompkins County GIS.   Excellent filtering tools enable users to search data by  several different categories  and download datasets in Shapefile, PDF, or KML format and export to either KMZ or GeoJSON.

CUGIR focuses exclusively on geospatial data which sets it apart from other general sites like DATA.NY.GOV.  The NYS GIS Clearinghouse and the NYS Geographic Information Gateway are more similar to CUGIR with each having their own unique datasets which in some instances can be confusing for someone looking for data and not sure where to look.  Additional CUGIR functionality includes pointing to and maintaining an index map to external datasets such as the National Elevation Dataset (NED).    Jenkins and his CUGIR team hope to build out similar functionality and integration with other Empire State geospatial portals in the future.

 

MGRS Mapper: Software for Military Operations Mapping

Posted on by
Reply

Program built and designed by Rochester area New York Army National Guard member

While serving in a Joint Multinational Training Group in Ukraine (JMTG-U) in 2018 along with other NATO nation soldiers, New York Army National Guard CPT James Pistell, a Rochester resident, was using a website to make operations graphics as they planned training operations while training Ukrainian soldiers and battle staffs. A self-taught software web developer in civilian life, Pistell felt the website was outdated, had limited functionality, and believed he could build a much better version based on his military in-field service and programming skills.

Pistell makes available both free and subscription-based versions of MGRS-Mapper. The web site landing page provides access to the free version enabling site visitors to see samples of the military symbology and easy-to-use drag and drop functionality

Upon his return to the states,  leaving his full-time job in marketing and dedicating over a year of full-time development work,  Pistell released MGRS-Mapper (MGRS for Military Grid Reference System) in late 2018.  (The MGRS overlay is based on standard Grid Zone Designator intervals commonly found on military installation maps using the NAD 83/WGS 84 industry standard).  

Developed entirely with JavaScript the application enables users to easily overlay the MGRS grid atop of a Google base map (other base maps are available as well) and provides Premium users access to the latest standard in military symbols as part of  ADP 1-02 (Army Doctrine Publication) which contains over 737 unique symbols used by many branches of the military. This set includes all Land Forces, Equipment, Activities, Graphic Control Measures, Tactical Mission Tasks, Installations, Supplies, Command Posts, Civilian Authorities, Sea Units, Air Units and Sustainment.  Definitely not the same symbol sets the GIS community is accustomed to using in making land use and environmental maps albeit interesting nonetheless!

Premium Version

With access to the premium version I was able to build, if only an introductory amateur version at best, a CONOP (Concept of Operation) map.  Using symbols compliant with ADP 1-02, I’ve mapped out a National Guard training program in the area of Ft. Drum showing the locations of friendly (blue symbology) platoons and infantry, armored tank units, and field artillery. Red colored symbols, or hostile resources, identifying the locations of infantry and artillery units. Also included in the map show the locations target reference points (black crosses), check points on the western edge of Ft. Drum along Route 11 and a larger blue symbol in the southwestern part of the map to highlight a National Guard Command Post.  The symbol circled in green in the middle of the map is an Amnesty Point. Hundreds of other military symbols are available for the user.

The horizontal toolbar on the left side of the application provides the user with the capabilities of taking a screen snapshot (camera icon) and saving as a .png to be downloaded and emailed to another user, printed out as a handout, or embedded in slides.

Users can click on any symbol on the map and drag it to your preferred location. From there you can rotate, resize, invert, scale up or scale the symbol down.

Pistell is currently working on several enhancements to the application including allowing Premium mobile users to cache maps on their device for use in places with limited internet connection, porting the application for use on both Apple iOS and Android devices, and the ability to share live links of user maps for live map scenario collaboration.  MGRS Mapper also offers certain organizations a large discount on team accounts including Volunteer Search and Rescue Teams, Volunteer Fire Departments, ROTC Battalions, U.S. Military elements at section level or above, as well as any  organization that relies primarily on volunteers.

MGRS Mapper allows users to find locations by either typing in an address or inputting a valid MGRS grid coordinate as well offering several easily accessible and open source base maps including OpenStreetMaps (shown below), National Geographic, ArcGIS, Stamen (black & white) and OpenStreetMap.

Pistell notes he has about 225 active subscriptions with many in military (Army).  He also includes on the list the Panamanian National Guard, Californai Search and Rescue Team, and numerous civilians who are military trainers.  “It’s incredibly rewarding to see individuals and organizations using a software program that you’ve written all the code for and built from scratch”, he says.  Continuing he notes “ this all started with my experience overseas and wanting military leaders and colleagues to make training maps and apply the same principals to organizations stateside.”

It would appear with the release of MGRS Mapper that Pistell is fulfilling that vision.

He currently serves as Cyber Network Defense Manager for the 173rd Cyber Protection Team in the New York Army National Guard.  

Contact:

CPT James Pistell
MGRS-Mapper
admin@mgrs-mapper.com

D.I.Y MS4: Erie County

Posted on by
Reply

The County’s Homegrown Application Provides Mapping and Data Collection Support to the Multi-Agency Western New York Stormwater Coalition

While the  Municipal Stormwater Sewer System (MS4) regulatory program may be taking a back seat in focus to COVID-19 across Empire State governments in 2020, compliance efforts nonetheless continue in the background.  The DEC program has seen governments responding to meet annual submission requirements in a variety of ways ranging from full vendor supported solutions, hybrid approaches of using in-house resources and commercial-off-the-shelf (COTS) products, while others have built the solution totally in-house.  Aka Do-it-Yourself (DIY).

One such MS4 DIY organization is Erie County which provides administrative and technical support to the Western New York Stormwater Coalition (WNYSC) which has a broad geographic footprint.  Erie County took the stormwater management regional lead in western New York in 1999 when it became apparent both Erie and Niagara Counties would be required to come in compliance with the United States Environmental Protection Agency (U.S. EPA) Phase II Stormwater requirements. Recognizing much of the MS4 General Permit was general in context of most permit requirements being the same for all governments, the Coalition was created in 2003 as a means to share work such as public education and outreach, development of policies and procedures, employee training – and mapping.  While the Coalition is not an official Erie County government agency, it is housed administratively inside County offices.

Erie County MS4 Support 

Initial development of the program started in 2012 by the current Director of Erie County’s Office of GIS, Lisa Matthies-Wiza, who was part of a team that designed databases, field applications, and quality assurance plans to manage data collection.  This was facilitated through a series of grants through NYSDEC and partnerships with Erie County and Buffalo State College provided to the Coalition.

More recently, support to the Coalition’s MS4 compliance efforts has been Michael Ruffino who came to the County in March of 2019.  Prior to Erie County, he had experience in the use of ArcGIS Online (AGOL) Survey123 and Collector in Chautauqua County where he was able to incorporate similar apps for the environmental health department septic systems and water supply inspections.  Currently Erie County hosts all of the GIS data and hardware/software infrastructure for the entire WNYSC program.

Continue reading →

Focus on Hudson River Flooding Impacts

Posted on by
Reply

Online Application Provides Tools and Data for Communities at Risk Along the Hudson River

Background

The Hudson River Flood Impact Decision Support System (HRFIDSS) is an online mapping application as well as a collection of geospatial products and data which illustrate the scale of potential flooding for tidally-affected shorelines of the Hudson River Valley and Westchester County coastal communities. In all, the viewer covers the shoreline communities in 11 counties in the region.  Developed by researchers from Center for International Earth Science Information Network (CIESIN), Stevens Institute of Technology, and the Columbia Water Center, the application enables individual residents, community groups as well as municipal and regional planners to evaluate risk based on potential outcomes, and to determine the impact on and effect of social vulnerability, natural resilience, and critical infrastructure.

The tool also provides downloadable estimates of flooding impacts, including affected population and critical infrastructure, and cost estimates of building damage.  It is further enhanced by including hydrological models which expands the flooding impacts to include storm tides/storm intensity and the contributions of river tributaries and streams.

The Hudson River Flood Impact Decision Support System helps address the growing interest across all levels of government in the New York region towards developing better forecasting capabilities and a better understanding of flood probabilities under a variety of sea level rise and storm frequency scenarios.

The application was developed using OpenLayers.  Critical infrastructure data is housed in a relational PostGIS database which are served as a Web Feature Service (WFS) using Geoserver.  Flood layers are raster data published from an ArcGIS Server at CIESIN while supporting imagery layers are consumed from New York State.

Impact Assessments

In addition to a variety of unique mapping tools, one of key features of the application is the generation of “Impact Assessments”.  These are generated by application flood maps using a combination of historical and synthetic flood events which include water depth, surge, rainfall, and  sea level rise scenarios of 0″, 6″, 12″, 18″, 24″, 30″, 36″, 48″, 60″, 72″ as well as 5-year through 1000-year flood zones.  Flood scenarios presented in the mapping tool are accompanied by a set of impact estimates at the local level. These are divided into three themes: critical infrastructure, social vulnerability, and natural resilience features.
Continue reading →

Green Map 2020: Evolving with Technology and Staying Community Focused

Posted on by
Reply

For nearly 30 years, Green Map has been an anchor in the Metro NYC nonprofit mapping community.  Led by its original founder, Wendy Brawer, Green Map’s portfolio of work and advocacy in locally-led mapping and sustainability efforts is quite impressive.  Since the launch of the Green Map System by Wendy Brawer in 1995 and publication of the original Green Map of NYC through her eco-design company, the program’s engagement tools and mapping resources have been adapted by locally-led projects in 65 countries.   Many joined the Green Map network when Greenhouse – their first content-managed website – was activated in 2007.   The site grew to become a rich resource highlighting community developed mapping work in all parts of the world.

While “green” mapping was one of its early defining products and focus,  the organization has expanded its services to include interactive web mapping, education, multimedia, event coordination, workshops, tours, planning and consulting, and creating software development tools.

 

Community Engagement Maps and Tools

Green Map’s award-winning icons, tools, platform(s) and tutorials help create engaging and visually powerful mapping content used as guides for local nature, culture, community living, social justice and resources for sustainable living. In addition to providing tools and tutorials on making maps, Green Map promotes their mapping efforts as part of Project Stories.  Project Stories are locally written and use tags to categorize the story type, theme, and purpose. Users can  discover how Green Map projects were created, people and organizations involved and about the project’s impact on their community.   Green Map projects are tagged and can be searched as part of the following 11 categories:

Besides its own mapping platform (with version 2 of the Open Green Map now in beta), the program also offers suggestions and tutorials on how to promote the use of the Green Map icons in other interactive mapping platforms such as ArcGIS Online (Story Maps), CARTO, Google Maps, Google Earth, and the Map of Tomorrow.

Continue reading →

COVID Era Geospatial Applications to Support Empire State Communities

Posted on by
Reply

Here in southeastern New York State, it’s amazing how fast the COVID wave came and went. At first, mid-March to mid-May seemed like an eternity, but now, not so much.  I likened the preparation for COVID to preparing for Super Storm Sandy.  Albeit phenomenally different, the buildup and the anxiety of waiting for it to show up.  Peering out the windows to see the outcome of a completely different kind of damage and suffering  altogether.  Physical and psychological.  Now July, it seems and feels like we can see the first wave of the pandemic in the rear view mirror.  But then as we refocus on the road in front of us, we know there are speed bumps ahead of us.  Not if, but when.

There were a plethora of interesting and meaningful geospatial viewers created across the state during the mid-March to mid-May time frame.  Most included highlighting COVID case numbers by larger units of geography (municipal or zip code boundaries),  generalized heat or pattern maps, impacts to businesses and institutions, as well as  information on hotlines and sources of assistance.  However, its pretty well documented the geospatial community across the state has struggled with being part of providing detailed spatial analysis or mapping support functions at the address level.   Or for that matter, even being asked to participate in this regard.

Personal health related data is uber protected with its use regulated and governed by the Health Insurance Portability and Accountability Act (HIPPA).  HIPPA  regulations reign supreme and contributed to a lesser involvement of GIS analysts in  providing geospatial content, metrics and analytics during the pandemic.  There is also the unspoken reluctance on the part of the public health community to release data sets  which might be difficult to understand or easily misinterpreted by the general public.  Department of Health  professionals often cautious to release data – particularly in new instances such as the COVID pandemic – and leaving analysis/interpretation  to internal professional staff only.  Ultimately though,  releasing some flavor of the data but in a more generalized context.

NYS Department of Health (DOH)  COVID-19 reporting has improved dramatically since the pandemic appeared in March.  Though while numbers  are made available only on a county basis, users can access and visualize the data in a variety of ways.  Reporting and posting of the data has become much more timely.  DOH is also now publishing a regional “Early Warning Monitoring Dashboard“.

Continue reading →

Google Earth in the Classrooms

Posted on by
Reply

Chromebooks, Google Classroom and Google Earth Together Provide a Framework for Teaching Geography and many other subjects in K-12

I actually started drafting an article on Google Earth in the classroom last fall but since then it’s been one thing or another pushing things to the back burner.  Mostly work priorities but then COVID-19 sent everything sideways.  Seemingly pointing us all in a new and uncharted direction.  A new and greater reliance on the internet as many of us found ourselves working remotely.

The remote work force in many of our extended families includes teachers of whom I now have even greater respect after watching what is involved in teaching remotely and online.  The lesson plans, prep work, surgically attached to the computer, and the seemingly endless hours of the encompassing interaction with students – and parents – well beyond the normal eight-hour work day.

In a roundabout way watching this teaching workflow unfold during the COVID lockdown brought me back to the Google Earth article.  Google’s presence in the classroom is ubiquitous built around Google Classroom which I can only imagine will grow immensely as remote K-12 education expands in the future.  Wrapped in with kabillons of lesson plans, templates, covering every education topic, discussion groups, forums, and blogs from around the world – everything built on top of Google technology.  Easily delivered and made available to the masses at all levels of the educational spectrum with the uber cheap and incredibly functional Google Chomebook.  (btw – using one as I type).   And easily accessible within this framework – and bringing me back to the beginning of this article – is the Google Earth product for use in the classroom.  It’s an incredible product to augment K-12 classroom teaching and not just for geography.

Continue reading →