CityIQ API

This module and command line tool provides access to the CityIQ API, with particular focus on getting metadata and parking events. The interface includes a basic access object for getting metadata and events, a scraper object for mass downloading events, and a command line tool for downloading and processing events.

Installation

Use pip:

pip install cityiq

Configuration

The CityIq module and programs require a configuration file that hold credentials and urls. You can generate a default configuration with

ciq_config  -w

The generated file is configured for the San Diego system. To you this system , you will just need to add your client id and secret to the file.

The Config object can be constructed can constructed on a path where the config file is location. If none is specified it will look for this file in several places, in this order:

  • The path specified in the constructor
  • The path specified by the CITYIQ_CONFIG env var
  • .city-iq.yaml in the current dir
  • city-iq.yaml in the current dir
  • .city-iq.yaml in the user’s home dir

Each of the configuration files can be overridden with a keywork in the Config object constructor, and each value can be accessed as an attribute or an index:

# Load from well-known file and override cache_dir
c = Config(cache_dir='/tmp')

print(c.cache_dir)
print(c['cache_dir'])

Basic Use

API objects are the primary way to get access to assets and events. CityIq is the top level access object. The API offers access to Locations, Events and Events.

Typically, you will construct CityIq from a Config. If a configuration is not specific, the system will look for the file in default locations. You can also override individual configuration parameters with keyword arguments to the constructor.

Metadata Access

Metadata, for both locations and assets, can be fetched with property accessors. The bounding box for the queries can be set in the configuration, or on the CityIq constructor.

The asset metadata properties are:

The location metadata properties are:

Events can be fetched with cityiq.api.CityIq.events()

Each of these acessor properties or functions returns a generator that generates objects of a specific type, one base class for each of Locations, Assets or Events:

bbox = '32.718987:-117.174244,32.707356:-117.154850'

c = CityIq(bbox=bbox) # Use default config, override bbox

# Get Locations
locations = list(c.locations)

# Get the assets at this location:
for location in locations:
    do_something_with(location.assets

Scraping

The ciq_events program is used to scrape and process events. To properly run the program, two directories should be specified in the configuration:

  • events_cache: The directory where hourly event fiels will be specified
  • cache_dir: The directory where processed event files will be written.

To scrape events from the API, run ciq_events --scrape --start_time. If <isotime> is omitted, the program will start from the start_time specified in the config. This will download events from the start time in hourly batches, and save one file per hour.

To process events, first break up the hourly event files by location with ciq_events --split command. This will create one CSV file per location per month. Then, re-combine and renormalize the data with ciq_events --normalize, which will write a CSV file to the local directory.

The final output file will have columns for delta, which is the number of cars that went into or out of a parking zone per 15 minute interval. However, there are a lot of suprious events, so the delta_norm has a normalized value that tries to remove the spurious events.

These programs can produce a lot of data. For the San Diego system, the extracted PKIN and PKOUT events for September 2018 through Feb 2019 is 21GB, and the download process takes several days. The final processed CSV file, with records at 15 minute intervals, is about 81MB and akes about an hour to process.

For instance:

$ ciq_events -s -e PKIN -e PKOUT -t 20190901
$ ciq_events -S
$ ciq_events -n

Indices and tables