cityiq package

Submodules

cityiq.api module

API objects are the primary way to get access to assets and events. CityIq is the top level access object. The API offers access to Locations, Events and Events.

Typically, you will construct CityIq from a Config. If a configuration is not specific, the system will look for the file in default locations. You can also override individual configuration parameters with keyword arguments to the constructor.

Metadata Access

Metadata, for both locations and assets, can be fetched with property accessors. The bounding box for the queries can be set in the configuration, or on the CityIq constructor.

The asset metadata properties are:

The location metadata properties are:

Events can be fetched with cityiq.api.CityIq.events()

Each of these acessor properties or functions returns a generator that generates objects of a specific type, one base class for each of Locations, Assets or Events:

bbox = '32.718987:-117.174244,32.707356:-117.154850'

c = CityIq(bbox=bbox) # Use default config, override bbox

# Get Locations
locations = list(c.locations)

# Get the assets at this location:
for location in locations:
    do_something_with(location.assets
class cityiq.api.Asset(client, data)[source]

Bases: cityiq.api.CityIqObject

children

Sub assets of this asset

children_url_suffix = '/v2/metadata/assets/{}/subAssets'
dclass_map = {'CAMERA': 'CameraAsset', 'EM_SENSOR': 'EmSensorAsset', 'MIC': 'MicSensorAsset', 'NODE': 'NodeAsset', 'XENV_SENSOR': 'EnvSensorAsset'}
detail

Asset details

detail_url_suffix = '/v2/metadata/assets/{}'
lat
locations

Assets at this location

locations_url_suffix = '/v2/metadata/assets/{}/locations'
lon
parent
row

Return most important fields in a row format

row_header = ['assetUid', 'assetType', 'parentAssetUid', 'mediaType', 'events', 'geometry']
types = ['NODE', 'EM_SENSOR', 'MIC', 'ENV_SENSOR', 'CAMERA']
class cityiq.api.CameraAsset(client, data)[source]

Bases: cityiq.api.Asset

class cityiq.api.CityIq(config=None, **kwargs)[source]

Bases: object

asset_dataframe

Return assets in row form in a pandas Dataframe

assets

Return all system assets

assets_search_suffix = '/v2/metadata/assets/search'
cameras

Return camera assets

em_sensors

Return some other kind of sensor. Electro-magnetic?

env_sensors

Return environmental sensors

events(start_time=None, end_time=None, age=None, span=None, bbox=None, event_type=' ')[source]
Parameters:
  • start_time
  • end_time
  • age – if start_time is not specified, the start time in terms of seconds before the end time
Returns:

events_async(events=['PKIN', 'PKOUT'])[source]

Use the websocket to get events. The websocket is run in a thread, and this function is a generator that returns results.

events_url = '/v2/locations/events?bbox={bbox}&locationType={location_type}&eventType={event_type}&startTime={start_time}&endTime={end_time}&pageSize=100'
get_assets(device_type=None, zone=None, bbox=None)[source]
get_locations(location_type=None, zone=None, bbox=None)[source]
get_pages(url, query=None, zone=None, bbox=None)[source]
http_get(url, zone=None, params=None, *args, **kwargs)[source]
locations
locations_dataframe
locations_search_suffix = '/v2/metadata/locations/search'
mics

Return microphone assets

nodes

Return all nodes

parking_zones
token
total_bounds

Return a bounding box for the system from all of the assets. This will be affected by the bbox set in the config, so it should usually be smaller than the one in the config

traffic_lanes
walkways
class cityiq.api.CityIqObject(client, data)[source]

Bases: object

geometry

Return a Shapely polygon for the coordinates

class cityiq.api.EmSensorAsset(client, data)[source]

Bases: cityiq.api.Asset

class cityiq.api.EnvSensorAsset(client, data)[source]

Bases: cityiq.api.Asset

class cityiq.api.Event(client, data)[source]

Bases: cityiq.api.CityIqObject

types = ['PKIN', 'PKOUT', 'PEDEVT', 'TFEVT', 'TEMPERATURE', 'PRESSURE', 'ORIENTATION', 'METROLOGY', 'HUMIDITY', 'ENERGY_TIMESERIES', 'ENERGY_ALERT']
class cityiq.api.EventWorker(client, events, queue) → None[source]

Bases: threading.Thread

Thread worker for websociet events

run() → None[source]

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

class cityiq.api.Location(client, data)[source]

Bases: cityiq.api.CityIqObject

assets

Assets at this location

assets_url_suffix = '/v2/metadata/locations/{}/assets'
dclass_map = {'PARKING_ZONE': 'ParkingZoneLocation', 'TRAFFIC_LANE': 'TrafficLaneLocation', 'WALKWAY': 'WalkwayLocation'}
detail
detail_url_suffix = '/v2/metadata/locations/{}'
events(eventType, start_time, end_time=None, span=None, ago=None)[source]
events_url_suffix = '/v2/locations/{locationUid}/events?eventType={eventType}&startTime={startts}&endTime={endts}'
row

Return most important fields in a row format

row_header = ['locationUid', 'locationType', 'parentLocationUid', 'geometry']
types = ['WALKWAY', 'TRAFFIC_LANE', 'PARKING_ZONE']
class cityiq.api.MicSensorAsset(client, data)[source]

Bases: cityiq.api.Asset

class cityiq.api.NodeAsset(client, data)[source]

Bases: cityiq.api.Asset

class cityiq.api.ParkingZoneLocation(client, data)[source]

Bases: cityiq.api.Location

class cityiq.api.TrafficLaneLocation(client, data)[source]

Bases: cityiq.api.Location

class cityiq.api.WalkwayLocation(client, data)[source]

Bases: cityiq.api.Location

cityiq.api.current_time()[source]

Return the epoch time in miliseconds

cityiq.api.se_time(start_time=None, end_time=None, ago=None, span=None)[source]
cityiq.api.time_ago(days=None, hours=None, minutes=None, seconds=None)[source]

cityiq.clean_events module

Clean events, removing duplicates

cityiq.clean_events.augment_df(df)[source]
cityiq.clean_events.clean_events(s, use_tqdm=True, locations=None)[source]

Given an event scraper that has has

cityiq.clean_events.dedup_1s(df)[source]

Deduplicate by dropping events of sametype that occur in 1s window.

cityiq.clean_events.dedup_rolling(df)[source]

Entry point for rolling count debouncing

cityiq.clean_events.plot_loc_norming(t)[source]
cityiq.clean_events.re_norm_location(df)[source]

cityiq.config module

The CityIq module and programs require a configuration file that hold credentials and urls. You can generate a default configuration with

ciq_config  -w

The generated file is configured for the San Diego system. To you this system , you will just need to add your client id and secret to the file.

The Config object can be constructed can constructed on a path where the config file is location. If none is specified it will look for this file in several places, in this order:

  • The path specified in the constructor
  • The path specified by the CITYIQ_CONFIG env var
  • .city-iq.yaml in the current dir
  • city-iq.yaml in the current dir
  • .city-iq.yaml in the user’s home dir

Each of the configuration files can be overridden with a keywork in the Config object constructor, and each value can be accessed as an attribute or an index:

# Load from well-known file and override cache_dir
c = Config(cache_dir='/tmp')

print(c.cache_dir)
print(c['cache_dir'])
class cityiq.config.Config(path=None, **kwargs)[source]

Bases: object

dict

cityiq.exceptions module

exception cityiq.exceptions.CityIqError[source]

Bases: Exception

exception cityiq.exceptions.ConfigurationError[source]

Bases: cityiq.exceptions.CityIqError

cityiq.scrape module

class cityiq.scrape.EventScraper(config, start_time, event_types)[source]

Bases: object

event_locations_dir = 'event-locations'
get_events(start_time, span)[source]
get_type_events(start_time, span, event_type)[source]

Get the events of one type :param start_time: :param span: time span in seconds :param event_type: :param tz_name: :return:

iterate_records(records=None)[source]

For a set of file name records ( from yield_file_names), yield event objects

iterate_splits(use_tqdm=False, locations=None)[source]

Iterate over the splits produced by split_locations

scrape_events()[source]
split_locations(use_tqdm=False)[source]

Split the scraped event files into sperate files per month and location, which are required for later stages of processing.

try_get_events(start_time, span)[source]

Run get_events and try to be resilient to some errors

yield_file_names(start=None, end=None)[source]
yield_months()[source]

Yield the results of yield_file_names

cityiq.token module

cityiq.token.get_cached_token(cache_path, uaa, client, secret)[source]

Return a cached access token from the CityIQ service. Returns just the token. Use _get_token() to get the full response

Parameters:
  • cache_path – Directory where cached token will be stored, if a directory or token name if a file
  • uaa – Url to the user authentication service
  • client
  • secret
Returns:

A token string

If a directory is specified in cache_dir, the file name will be ‘cityiq-token-<crc>.json’, with the CRC32 of the source url. The token will be expired after 8 hours.

cityiq.token.get_token(uaa, client, secret)[source]

Get an access token from the CityIQ service. Returns just the token. Use _get_token() to get the full response

Parameters:
  • uaa – Url to the user authentication service
  • client
  • secret
Returns:

Module contents