Jersey City Zoning in 3D

Thanks to the latest by Mapbox I was able to add 3D buildings to my Jersey City Zoning Map. The buildings rendered below come from my shapefile of Jersey City buildings, which you can download from the city's open data portal, as opposed to OpenStreetMap. My data has more attributes (including zoning and assessment information) from when I merged buildings and parcel data. While the footprints are all the same as those in OSM (I first prepared that dataset specifically for OSM), you should use my dataset if you're interested in building ages, sale information, and zoning information. 

I created the building heights property from the "zoning description" field in the parcel dataset using an admittedly flawed method. The number of stories is usually listed first in a string of codes, followed by an "S". I used a regular expression to extract the number of stories and then I created a "bldgHeight" property (in meters) by multiplying the number of stories per building by 3. Lots of buildings are multi-level (e.g., a building might be one story across the entire lot but have two-stories on a portion of the lot). I grabbed the maximum number of stories in these cases. Some buildings are missing parcel information and so I don't know the number of stories. Some buildings that do merge with parcel information are missing the "zoning description" field. Unfortunately, all I could do for buildings with missing data is pipe in a "3" for one-story. 

I then did a few passes of clean-up based on personal knowledge. I flew around my map and corrected a few dozen buildings I know of that otherwise would have been a "3". For example, the buildings around Journal Square Station and Grove Station, the local high schools, and a few spots downtown. In some of these cases I was able to add additional layers to create the appearance of a 3D rendering (as opposed to just an extrusion). The Goldman Sacks Tower (the tallest building in view, below) is an example of this. All of the buildings have a "bldgHeight" of at least 3 (see my example code below for how I use this field), and all buildings have a minHeight of 0 (this field can be used to create the appearance of raised structures, like bridges) with one exception - the house I grew up in. Good luck trying to find it!

Right now, I've added different layers for each zone category by repeating code blocks. A much more elegant way of doing this would simply be to pass a different "filter" and "fill-color" property to the add.Layer function, but this works for now. I also added a link to a Google form so that anyone can submit updates to the building information. I'd like to make the map overlay collapsable at some point (it's a bit clunky right now). Finally, under advisement from Brian Platt in the Office of Innovation, I'm going to add a toggleable layer for recent development (2013+). That one might take a few more weeks to realize. Enjoy for now!

Go to the map. Go to the code.

    function loadBuildings() {
        map.addSource('Special', {
            type: 'vector',
            url: 'mapbox://sarahmlevine.sarahmlevine.1rz41on6'
        });
        map.addLayer({
            'id': 'Residential',
            'source': 'composite',
            'filter': ['==', 'zone', ('R-1' || 'R-1A' || 'R-1F' || 'R-2' || 'R-3' || 
                                      'R-4' || 'OR' || 'Caven Point')],
            'source-layer': 'buildings-1909vz',
            'type': 'fill',
            'minzoom': 14,
            'paint': {
                'fill-color': '#42e5f4',
                'fill-extrude-height': {
                    'type': 'identity',
                    'property': 'bldgHeight'
                },
                'fill-extrude-base': {
                    'type': 'identity',
                    'property': 'minHeight'
                },  
                'fill-opacity': .5
            }
        });
      };
  map.on('load', function() {
    loadBuildings();
  }

Playing with Carto: Jersey City Building Heights & Years

It's embarrassing to admit, but I finally got around to playing with Carto for the first time.  I loaded in my Jersey City buildings dataset and whipped up this map with two toggle-able layers: buildings by number of stories, and buildings by year built (the default display). You can click on buildings for other parcel information. The year built value comes straight from county assessor data, so it's a bit faulty (although I did a little manual clean up, e.g. replacing bizarre values like "9999" with NULL). The number of stories field was extracted from the "zoning description field" (I grabbed the numeric characters before the "S") from the parcel data. See my last post for information about where to download the Jersey City building footprints data, and see my last, last post for information on how I created that data.

Mapping Jersey City II: Every Building

SEE IT LIVE. SEE THE CODE. LEARN WHY.

If no map appears below (if there's a white background) it's probably because you need to enable WebGL in your browser.

It's been a long-term dream of mine to map every building in Jersey City. See my last post for more about why.  I reached out to the Office of Innovation to see how to go about doing it and they gave me the green light, so I had to pull the trigger.

Once I created the building footprints using the process documented in my last post (these polygons now live in OpenStreetMap) I used QGIS to merge on several other publicly available datasets from the city including zoning, wards, and parcel information. I did my best to make the data comply with the Project Open Data Metadata Schema v1.1 as per their request. 

The zoning and wards merges were extremely clean and easy. The parcel merge was not, to say the least. There are often many buildings inside one parcel (Mun-Bloc-Lot-QCode) or many parcels inside one building. In the first case, I allowed buildings to inherit all parcel information. In the second case, I populate the Lot and Bloc fields with "MANY" as necessary. QCodes, which identify the smallest parcel boundary, were almost never uniquely identifying, so I exclude them. Mun-Bloc-Lot is sufficient to join with county assessment and tax information. 

I completed the parcel merge by calculating building centroids and spatially merging those with parcel polygons (preserving a unique building identifier). I experimented with other methods (like parcel centroids inside building polygons), but I found this method to be the cleanest and require the least manual clean-up. I found the realcentroid plugin extremely helpful, considering some geometry irregularities. I also found the QuickMultiAttributeEdit plugin to be extremely useful for updating the fields on a few objects that merged sloppily.

I used one of my favorite plugins, qgis2web, to produce a quick and sloppy Open Layers 3 web map to immediately send to my favorite people. Unfortunately, I don't think the plugin is equipped for a dataset this large, so I wasn't able to use it to produce a Leaflet map (my preference). A little formatting with the Table Manager plugin and I sent the data off to the city with a data dictionary. They're in the process of putting it up on the Open Data Portal now.

Finally, I uploaded the geojson with footprints and all of the merged fields into Mapbox studio as a tileset. I added it to two styles: one dark basemap and one satellite imagery layer with semi-transparent road information. Finally, I used Mapbox GL JS to code the map shown above. I added an overlay and a legend, and functionality to click on buildings for their information, and toggle between my two basemap styles. I then agonized over colors, added Google fonts, and promptly went to bed.


Things to do and problems to solve:

(1) To solve: It would be great to see the whole city at once, but the dataset is so large that Mapbox enforces that it only be viewable from zoom >=14. I don't love having to pick one part of the city to focus on (at least for the default view), especially because I'm not interested in promoting a downtown-centric image of Jersey City. For now I've settled on what I think is a readily recognizable part of the city.  I would love advice on how to manage this.

(2) To do: Add more data, starting with addresses. This shouldn't be too tricky with some geocoding. This will also make dealing with messy parcel data (and recovering QCODEs) much easier. If I can get that done,  then I can merge on parcel information from the county including owner information, building codes, year built, and building/land assessments. This is definitely feasible (and is just a matter of time). I'd also like to add links to specific, relevant sections of the zoning code for each building. That's another no-brainer.

One Size Does Not Fit All Data Science

As I mentioned a while back, Alex Albright (of The Little Dataset That Could) and I had the chance to present some of our thoughts at Bloomberg's first annual Data for Good Exchange. We decided to talk about what we view as the shortcomings in popular data science education programs and bootcamps. Specifically, we wanted to shine a light on the ways that data scientists are (and are not) adequately trained to contribute to social good projects and work with foreign data. I've included the abstract (below) and introduction (after the jump). You can also read the full text and check out the poster. Thanks to Alex for working on this while on vacation in Portland and thanks to SLS for letting us write things we believe and not firing us for it.

One Size Does Not Fit All: The Shortcomings of the Mainstream Data Scientist Working for Social Good

Data scientists are increasingly called on to contribute their analytical skills outside of the corporate sector in pursuit of meaningful insights for nonprofit organizations and social good projects. We challenge the assumption that the skills and methods necessary for successful data analysis come in a “one size fits all” package for both the nonprofit and for-profit sectors. By comparing and contrasting the key elements of data science in both domains, we identify the skills critical for the successful application of data science to social good projects. We then analyze five well-known data science programs and bootcamps in order to evaluate their success in providing training that transfers smoothly to social impact projects. After surveying these programs, we make a number of recommendations with respect to data science training curricula, non-profit hiring systems, and the data science for social good community’s practices. 

table-1
table-2

While the overwhelming majority of data scientists are employed in the for-profit sector, there is a growing movement taking advantage of their technological savvy and unique toolkit for the benefit of social good projects and programs. Conventionally trained data-scientists are encouraged more and more to play a pivotal role in data-driven social good projects as team members, consultants, or volunteers. However, this phenomenon assumes that the data scientists’ standard toolkit in the for-profit sector translates seamlessly to the realm of social good. We challenge this assumption and argue that while the term “data scientist” has become an amorphous catch-all for programmers, statisticians, bloggers, and other empirically inclined individuals, the skills and methodological knowledge required of a data scientist can and should differ across the for-profit and non-profit sectors. We use this paper as an opportunity to highlight the shortcomings of mainstream data science education and practice when it comes to the non-profit sector and social impact endeavors. 

We begin by comparing and contrasting the roles of data scientists in the for-profit and non-profit environments, and identify three key differences. First, while for-profit data scientists often work with in-house data, non-profit data science often involves working with foreign data that merits greater scrutiny and sensitivity in its treatment. Second, while the corporate environment provides control over the quality of “insights” in the form of management, the non-profit environment can lack effective checks and balances on data and analysis quality. Third, in experimental design, for- profit data scientists often have near-omniscient control over the environment containing study variables, whereas real-world data and studies are seldom so fortunate. We conclude that whereas for-profit data science can often afford to be “insights”-driven and results-oriented, non-profit data science must be less content- driven and more process oriented to avoid results, conclusions, and even policies that are built on poor quality data and inappropriate methods. 

Next, we survey popular data science curricula across bootcamps, online courses, and master’s degree programs in order to generalize the baseline knowledge of emerging data scientists. We then compare and contrast the skills delivered by contemporary data science education with those required for meaningful contribution to social impact projects, and find that the former caters strikingly to a for-profit position. For example, we find that there is little to no focus in current data science education on investigating the quality of data or the identification and integrity of experimental variables. The curricula of these courses illustrate that data scientists are molded to be corporate workers as the default, necessitating a further mechanism to help empirical researchers transition across sectors, even if they bear the same title: “data scientist.” 

Ultimately, we make several recommendations as to (1) how data science training programs can better prepare their students for roles in organizations doing social good, (2) how non-profit organizations can and must be more targeted in their hiring practices to find data scientists who are adequately suited for their projects, and (3) how the data science for social good community can and must develop best practices and ethical codes akin to those in the academic community. 

Mines and Mapbox GL JS

The new Mapbox GL JS is unreal. I made a quick and sloppy map to try out some of the new features. I grabbed a dataset of mines in the US from MSHA's website. I loaded it up into Mapbox and created two styles: satellite imagery and streets. I took some code from their GL JS how-to's and threw it together. My Frankencode is below. Side note: JavaScript is hard. I put in some code to toggle between the two base layers, and to add two more optional layers for hillshade and contours. 

Issues to fix and things to improve in the next version: The dataset was HUGE. I had to limit to zoom levels to keep the size reasonable. You'll notice that if you zoom out too much, the data disappears. I might need to pay for Mapbox to make this work... Also there are two attributions at the bottom of the map for OSM and Mapbox. I do love OSM enough to thank it twice, but I think this has to do with two styles being loaded in. Another thing to investigate with more time.

Cool features: Hold shift and use the arrow keys to fly around. Note how place labels stay horizontal while you spin the Earth as if you stuck a pin in it first. 

Thanks, Mapbox. Here's the code I embedded above.

<html>
<head>
    <meta charset='utf-8' />
    <meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />
    <script src='https://api.tiles.mapbox.com/mapbox-gl-js/v0.12.0/mapbox-gl.js'></script>
    <link href='https://api.tiles.mapbox.com/mapbox-gl-js/v0.12.0/mapbox-gl.css' rel='stylesheet' />
    <style>
        body { margin:0; padding:0; }
        #map { position:absolute; top:0; bottom:0; width:100%; }
    </style>
</head>
<body>

<style>
    #menu {
        background: #fff;
        position: absolute;
        z-index: 1;
        top: 10px;
        right: 10px;
        border-radius: 3px;
        width: 120px;
        border: 1px solid rgba(0,0,0,0.4);
        font-family: 'Open Sans', sans-serif;
    }
    #menu a {
        font-size: 13px;
        color: #404040;
        display: block;
        margin: 0;
        padding: 0;
        padding: 10px;
        text-decoration: none;
        border-bottom: 1px solid rgba(0,0,0,0.25);
        text-align: center;
    }
    #menu a:last-child {
        border: none;
    }
    #menu a:hover {
        background-color: #f8f8f8;
        color: #404040;
    }
    #menu a.active {
        background-color: #3887be;
        color: #ffffff;
    }
    #menu a.active:hover {
        background: #3074a4;
    }
</style>

<div id='map'></div>
<div id='menu'>
    <input id='sarahmlevine/cihs248is00aa95lylu5a7x1d' type='radio' name='rtoggle' value='satellite' checked='checked'>
    <label for='satellite'>satellite layer</label>
    <input id='sarahmlevine/cihvjue4p00h095lyyyyv2347' type='radio' name='rtoggle' value='basic'>
    <label for='basic'>streets</label>
</div>
<script>
mapboxgl.accessToken = 'pk.eyJ1Ijoic2FyYWhtbGV2aW5lIiwiYSI6IlAweXNYVEUifQ._dz0522LtBABUYyfqP503Q';
var map = new mapboxgl.Map({
    container: 'map',
    style: 'mapbox://styles/sarahmlevine/cihs248is00aa95lylu5a7x1d',
    zoom: 8.1,
    center: [-121.41, 37.91]
});
var layerList = document.getElementById('menu');
var inputs = layerList.getElementsByTagName('input');
function switchLayer(layer) {
    var layerId = layer.target.id;
    map.setStyle('mapbox://styles/' + layerId);
}
for (var i = 0; i < inputs.length; i++) {
    inputs[i].onclick = switchLayer;
}
map.on('style.load', function () {
    map.addSource('contours', {
        type: 'vector',
        url: 'mapbox://mapbox.mapbox-terrain-v2'
    });
    map.addLayer({
        'id': 'contours',
        'type': 'line',
        'source': 'contours',
        'source-layer': 'contour',
        'layout': {
            'line-join': 'round',
            'line-cap': 'round'
        },
        'paint': {
            'line-color': '#877b59',
            'line-width': 1
        }
    });
    map.addSource('hillshade', {
        type: 'vector',
        url: 'mapbox://mapbox.mapbox-terrain-v2'
    });
    map.addLayer({
        'id': 'hillshade',
        'type': 'line',
        'source': 'hillshade',
        'source-layer': 'hillshade',
        'layout': {
            'line-join': 'round',
            'line-cap': 'round'
        },
        'paint': {
            'line-color:': '#877b59',
            'line-width': 1,
            'line-opacity': 0.6,
        }
    });
});
addLayer('Contours', 'contours');
addLayer('Hillshade', 'hillshade');
function addLayer(name, id) {
    var link = document.createElement('a');
    link.href = '#';
    link.className = 'active';
    link.textContent = name;
    link.onclick = function (e) {
        e.preventDefault();
        e.stopPropagation();
        var visibility = map.getLayoutProperty(id, 'visibility');
        if (visibility === 'visible') {
            map.setLayoutProperty(id, 'visibility', 'none');
            this.className = '';
        } else {
            this.className = 'active';
            map.setLayoutProperty(id, 'visibility', 'visible');
        }
    };
    var layers = document.getElementById('menu');
    layers.appendChild(link);
}
// Add zoom and rotation controls to the map.
map.addControl(new mapboxgl.Navigation());
</script>
</body>
</html>

Common Prophets of Abrahamic Religions

Aaron 4AaronAbraham 4AbrahamAdam 4AdamDaniel 4DanielDavid 4Daviddelete 82Elijah 4ElijahEnoch 4EnochHud 2HudIsaac 3IsaacIsaiah2 3IsaiahIshmael 2IshmaelJacob 4JacobJeremiah2 3JeremiahJesus of Nazareth 3Jesus of NazarethJob 4JobJoel3 3JoelJohn (the Baptist) 3John (the Baptist)Joseph 4JosephMoses 4MosesMuhammad 2MuhammadNoah 4NoahSalih 2SalihShu'aib 2Shu'aibSolomon 4SolomonZechariah (priest) 3Zechariah (priest)Aaron 4AaronAbraham 4AbrahamAdam 4AdamAhijah HaShiloni 2Ahijah HaShiloniAhiyah 2AhiyahAmos 2AmosAzariah 2AzariahBaruch ben Neriah 2Baruch ben NeriahDeborah 2Deborahdelete 29Eliezer 2EliezerElijah 4ElijahElisha 3ElishaEsther 2EstherEzekiel 3EzekielGad 2GadHabakkuk 2HabakkukHaggai 2HaggaiHanani 2HananiHosea 2HoseaHuldah 2HuldahIddo 2IddoIsaac 3IsaacIsaiah 3IsaiahJacob 4JacobJahaziel 2JahazielJehu 2JehuJeremiah 3JeremiahJob 4JobJoel 3JoelJonah 3JonahJoseph 4JosephJoshua 2JoshuaKing David 4King DavidKing Solomon 4King SolomonMalachi 2MalachiMicah 2MicahMicaiah 2MicaiahMiriam 2MiriamMoses 4MosesNahum 2NahumNathan 2NathanNeriah 2NeriahNoah 4NoahObadiah 2ObadiahOded 2OdedPhinehas 2PhinehasSamuel 2SamuelSeraiah 2SeraiahShemaiah 2ShemaiahUriah 2UriahZechariah (the Priest) 3Zechariah (the Priest)Zechariah/Zacharias 2Zechariah/ZachariasZephaniah 2ZephaniahAaron (Harun) 4Aaron (Harun)Abraham 4AbrahamAdam 4Adam Danial 4DanialDavid (Daud) 4David (Daud)delete 77Elijah (Ilyas) 4Elijah (Ilyas)Elisha (Al-Yasa) 3Elisha (Al-Yasa)Enoch (Idris) 4Enoch (Idris)Ezekiel (Dhul-Kifl) 3Ezekiel (Dhul-Kifl)Ezra (Uzair) 3Ezra (Uzair)Hud2 2HudIsa (Jesus of Nazareth) 3Isa (Jesus of Nazareth)Isaac 3IsaacIshmael (Ismā'īl) 2Ishmael (Ismã'íl)Jacob (Yaqub) 4Jacob (Yaqub)Job (Ayub) 4Job (Ayub)John (the Baptist - Yahya) 3John (the Baptist - Yahya)Jonah (Yunus) 3Jonah (Yunus)Joseph (Yusuf) 4Joseph (Yusuf)Moses (Musa) 4Moses (Musa)Muhammad2 2MuhammadNoah (Nuh) 4Noah (Nuh)Saleh 2SalehSeth2 2SethShoaib 2ShoaibSolomon (Sulayman) 4Solomon (Sulayman)Zechariah/Zakariya (priest) 3Zechariah/Zakariya (priest)Aaron 4AaronAbraham 4AbrahamAdam 4AdamAhijah HaShiloni2 2Ahijah HaShiloniAhiyah2 2AhiyahAmos2 2AmosAzariah2 2AzariahBaruch ben Neriah2 2Baruch ben NeriahDaniel2 4DanielDavid2 4DavidDeborah2 2Deborahdelete 10Eliezer2 2EliezerElijah 4ElijahElisha 3ElishaEnoch2 4EnochEsther2 2EstherEzekiel/Ezechiel 3Ezekiel/EzechielEzra/Esdras 3Ezra/EsdrasGad2 2GadHabakkuk/Habacuc 2Habakkuk/HabacucHaggai/Aggeus 2Haggai/AggeusHanani2 2HananiHosea/Osee 2Hosea/OseeHuldah2 2HuldahIddo2 2IddoIsaac 3IsaacIsaiah/Isaias 3Isaiah/IsaiasJacob 4JacobJahaziel/Chaziel 2Jahaziel/ChazielJehu2 2JehuJeremiah/Jeremias 3Jeremiah/JeremiasJesus of Nazareth2 3Jesus of NazarethJob 4JobJoel2 3JoelJohn (the Baptist)2 3John (the Baptist)Jonah/Jonas 3Jonah/JonasJoseph 4JosephJoshua/Josue 2Joshua/JosueMalachi/Malachias 2Malachi/MalachiasMicah/Micheas 2Micah/MicheasMicaiah2 2MicaiahMiriam2 2MiriamMoses 4MosesNahum2 2NahumNathan2 2NathanNeriah2 2NeriahNoah 4NoahObadiah/Abdias 2Obadiah/AbdiasOded2 2OdedPhinehas2 2PhinehasSamuel2 2SamuelSeraiah2 2SeraiahSeth 2SethShemaiah2 2ShemaiahSolomon 4SolomonUriah2 2UriahZ-placeholder 3Zechariah 2ZechariahZephaniah/Sophonias 2Zephaniah/Sophonias

Many thanks to Ben Discoe for pointing out some errors in my earlier viz (Moses and Noah were floating listlessly). I took the opportunity to add prophets from Bahá'í as well. The columns now read: Judaism, Christianity, Islam, and Bahá'í. New lesson learned: Zechariah/Zacharias (prophet of the kingdom of Judah) and Zechariah the priest (the father of John the Baptist) are two different people. The next step might have to be a character/family tree. Source.

#D4GX

I've been on a blog vacation (which is very different from a real vacation). More maps coming soon, but in the meantime, check out (and add) my curated collection of mappy blogs on Feedly.

http://feedly.com/sarahmakesmaps/maps

Stick around, and if you're heading to the first annual Bloomberg Data for Good Exchange, make sure to come say hi to me and Alex Albright of SLS and The Little Dataset that Could, as we'll be presenting some of our thoughts on how data scientists are (and are not - spoiler) trained to contribute to social good projects and programs.

http://www.bloomberg.com/company/d4gx/

#D4GX

 

San Francisco Tree Census

All street trees in San Francisco as of April 26, 2015 recorded by Department of Public Works. Shades of green represent trees grouped by genus (legend below).

(Street trees set excludes parks, including Golden Gate and Presidio.) Data from SF Open Data. Map created with Tilemill and Mapbox Studio. Preview in iframe above, teasers in gallery below. See here for full-page map, and sarahmakesmaps.com/trees for isolated page. See the code.


LEGEND

Acacia
Maple (acer)
Chestnut (Aesculus)
Willow (Agonis)
Pine (Araucaria)
Birch (Betula)
Palm (Brahea)
Bottlebrush (Callistemon)
Hornbeam (Carpinus)
Beefwood (Casurina)
Cedar (Cedrus)
Hackberry (Celtis)
Gum (Corymbia)
Hawthorn (Crataegus)
Carrotwood (Cupaniopsis)
Cypress (Cupressus)
Bush (Dodonaea)
Dragon Tree (Dracaena)
Loquat (Eriobotrya)
Eucalyptus
Beech (Fagus)
Fig (Ficus)
Ash (Fraxinus)
Australian Willow (Geijera)
Gingko
Locust (Gleditsia)
Silk Oak (Grevillea)
Urchin/Hakea Tree (Hakea)
Sweet Shade (Hymenosporum)
Holly (Ilex)
Jacaranda
Juniper (Juniperus)
Laurel (Laurus)
Tea Tree (Leptospermum)
Sweet Gum (Liquidambar)
Magnolia
Crabapple (Malus)
Maleleuca
Olive (Olea europaea)
Cherry (Prunus)
Pear (Pyrus)
Oak (Quercus)
Elm (Ulmus)
Hawthorn (Rhaphiolepis)
Fan Palm (Washingtonia)
Yew (Taxus)