This package was made to help with data handling.
Included
VitalSigns uses functions found in our Dataplay Module and vice-versa.
These scripts will download and clean ACS data for Baltimore and then construct indicators from the data.
AcsDownload.ipynb Mount notebook to google drive Read in ACS Meta Data from XLSX and the crosswalk data from a csv Add path to python script that performs download function Enter a year and Run the download function for every record in XLSX sheet For each dataset, remove columnID’s then save it as Raw Then, Append Community Names using a crosswalk and save again in as Clean
AcsIndicators.ipynb Mount notebook to google drive Read in ACS Meta Data from XLSX Prepare the Compare Historic Data
The code is on PyPI so you can install the scripts as a python library using the command:
!pip install VitalSigns geopandas
Important: Contributers should follow the maintanance instructions and will not need to run this step.
Their modules will be retrieved from the VitalSigns-GDrive repo they have mounted into their Colabs Enviornment.
Then...
from VitalSigns.acsDownload import retrieve_acs_data
retrieve_acs_data(state, county, tract, tableId, year, saveAcs)
Now you could do something like merge it to another dataset!
from dataplay.merge import mergeDatasets
mergeDatasets(left_ds=False, right_ds=False, crosswalk_ds=False, use_crosswalk = True, left_col=False, right_col=False, crosswalk_left_col = False, crosswalk_right_col = False, merge_how=False, interactive=True)
You can get information on the package by using the help command.
Here we look at the package's modules:
import VitalSigns help(VitalSigns)Help on package VitalSigns: NAME VitalSigns PACKAGE CONTENTS BCPSS HUD _nbdev acsDownload bidbaltimore bpd citistat cityfinance closecrawl create dhr enoch fares fdic indicators infousa liquor mdprop rbintel tidyaddr treebaltimore utils VERSION 0.0.5 FILE /usr/local/lib/python3.7/dist-packages/VitalSigns/__init__.pyLets take a look at what functions the geoms module provides:
import VitalSigns.acsDownload help(VitalSigns.acsDownload)Help on module VitalSigns.acsDownload in VitalSigns: NAME VitalSigns.acsDownload - # AUTOGENERATED! DO NOT EDIT! File to edit: notebooks/90_ACS_Explore_and_Download.ipynb (unless otherwise specified). FUNCTIONS retrieve_acs_data(state, county, tract, tableId, year, save) DATA __all__ = ['retrieve_acs_data'] __warningregistry__ = {'version': 268, ('Passing a negative integer is... FILE /usr/local/lib/python3.7/dist-packages/VitalSigns/acsDownload.pyAnd here we can look at an individual function and what it expects:
import VitalSigns.acsDownload help(VitalSigns.acsDownload.retrieve_acs_data)Help on function retrieve_acs_data in module VitalSigns.acsDownload: retrieve_acs_data(state, county, tract, tableId, year, save)So heres an example:
Import your modules
from VitalSigns.acsDownload import retrieve_acs_data#hide import pandas as pd pd.set_option('display.max_rows', 10) pd.set_option('display.max_columns', 6) pd.set_option('display.width', 10) pd.set_option('max_colwidth', 20)Read in some data
Define our download parameters.
More information on these parameters can be found in the tutorials!
# Our download function will use Baltimore City's tract, county and state as internal paramters # Change these values in the cell below using different geographic reference codes will change those parameters tract = '*' county = '510' state = '24' # Specify the download parameters the function will receieve here tableId = 'B19001' year = '17' saveAcs = FalseAnd download the Baltimore City ACS data using the imported VitalSigns library.
df = retrieve_acs_data(state, county, tract, tableId, year, saveAcs) df.head(1)Number of Columns 17B19001_001E_Total | B19001_002E_Total_Less_than_$10,000 | B19001_003E_Total_$10,000_to_$14,999 | ... | state | county | tract | |
---|---|---|---|---|---|---|---|
NAME | |||||||
Census Tract 2710.02 | 1510 | 209 | 73 | ... | 24 | 510 | 271002 |
1 rows × 20 columns
From there, you can go on to do even greater things using our dataplay library. Like these visuals:
This section is not definite but provides a good idea of how our scripts are made.
Basic Indicator Creation Outline
Miscellaneous things I should have for every notebook
From a local copy of the git repo: 0. Clone the repo local onto GDrive
git clone https://github.com/BNIA/VitalSigns.git
!nbdev_build_lib
to build .py modules.%load_ext autoreload %autoreload 2
!nbdev_update_lib
and !relimport2name
!nbdev_build_docs --force_all True --mk_readme True
!git commit -m ...
%%capture ! pip install twine
!nbdev_bump_version
! make pypi