Пропустить навигацию

Boston Safety Data

311 Boston Case Management City Services CRM Public Safety

311 calls reported to the city of Boston.

Refer to this link to learn more about BOS:311.

Volume and Retention

This dataset is stored in Parquet format. It is updated daily, and contains about 100K rows (10MB) in total as of 2019.

This dataset contains historical records accumulated from 2011 to the present. You can use parameter settings in our SDK to fetch data within a specific time range.

Storage Location

This dataset is stored in the East US Azure region. Allocating compute resources in East US is recommended for affinity.

Additional Information

This dataset is sourced from city of Boston government. More details can be found from here. Reference Open Data Commons Public Domain Dedication and License (ODC PDDL) for the license of using this dataset.

Notices

MICROSOFT PROVIDES AZURE OPEN DATASETS ON AN “AS IS” BASIS. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, GUARANTEES OR CONDITIONS WITH RESPECT TO YOUR USE OF THE DATASETS. TO THE EXTENT PERMITTED UNDER YOUR LOCAL LAW, MICROSOFT DISCLAIMS ALL LIABILITY FOR ANY DAMAGES OR LOSSES, INCLUDING DIRECT, CONSEQUENTIAL, SPECIAL, INDIRECT, INCIDENTAL OR PUNITIVE, RESULTING FROM YOUR USE OF THE DATASETS.

This dataset is provided under the original terms that Microsoft received source data. The dataset may include data sourced from Microsoft.

Доступ

Доступно вСценарии использования
Azure Notebooks

Quickly explore the dataset with Jupyter notebooks hosted on Azure or your local machine.

Azure Databricks

Use this when you need the scale of an Azure managed Spark cluster to process the dataset.

Предварительная версия

dataType dataSubtype dateTime category subcategory status address latitude longitude source
Safety 311_All 6/25/2019 1:48:00 PM Trees Tree Maintenance Requests Open 74 Chelsea St East Boston MA 02128 42.3719 -71.0368 Employee Generated
Safety 311_All 6/25/2019 1:43:00 PM Housing Lead Open 63 Davison St Hyde Park MA 02136 42.2574 -71.1179 Constituent Call
Safety 311_All 6/17/2019 9:02:00 AM Housing Cross Metering - Sub-Metering Open 47 Stimson St West Roxbury MA 02132 42.2606 -71.1588 Constituent Call
Safety 311_All 6/16/2019 11:41:00 AM Animal Issues Animal Generic Request Open 134 Poplar St Roslindale MA 02131 42.2836 -71.128 Constituent Call
Safety 311_All 6/14/2019 4:35:46 PM Housing Unsatisfactory Living Conditions Open 91 Ames St Dorchester MA 02124 42.2895 -71.0839 Constituent Call
Safety 311_All 6/14/2019 4:31:00 PM Animal Issues Animal Generic Request Open 93 Whitfield St Dorchester MA 02124 42.2928 -71.0747 Constituent Call
Safety 311_All 6/12/2019 10:58:00 PM Massport Aircraft Noise Disturbance Open 25 Parmenter St Boston MA 02113 42.3641 -71.0549 Constituent Call
Safety 311_All 6/12/2019 7:37:00 PM Building Working Beyond Hours Open 1465 Commonwealth Ave Brighton MA 02135 42.348 -71.1411 Constituent Call
Safety 311_All 6/12/2019 12:47:00 PM Building Building Inspection Request Open 40 Jamaicaway Jamaica Plain MA 02130 42.3307 -71.1122 Constituent Call
Safety 311_All 6/10/2019 11:38:00 AM Building Building Inspection Request Open 48 Hazleton St Mattapan MA 02126 42.2798 -71.0956 Constituent Call
Имя Тип данных Уникальные Значения (пример) Описание
address string 60,450 \"\"
1 City Hall Plz Boston MA 02108

Location.

category string 48 Sanitation
Highway Maintenance

Reason of the service request.

dataSubtype string 1 311_All

“311_All”

dataType string 1 Safety

“Safety”

dateTime timestamp 124,591 2016-11-10 11:20:00
2019-02-25 12:43:00

Open date and time of the service request.

extendedProperties string 12,630 "CASE_ENQUIRY_ID:101002783595,CASE_TITLE:Unshoveled Sidewalk,CLOSURE_REASON:Case Closed. Closed date : 2019-01-25 07:46:30.157 Case Noted No violation found at this time ,ClosedPhoto:https://cityworker.cityofboston.gov:8443/attachments/report/5c485764d052e8a7269e3c61/closed_photo/Report.jpg,Department:PWDx,LOCATION_STREET_NAME:821 E Fourth St,LOCATION_ZIPCODE:02127,OnTime:OVERDUE,QUEUE:PWDx_Code Enforcement,SUBJECT:Public Works Department,TARGET_DT:2019-01-25 06:59:11,closed_dt:2019-01-25 07:46:30,city_council_district:2,neighborhood:South Boston / South Boston Waterfront,neighborhood_services_district:5,police_district:C6,precinct:0608,pwd_district:05,ward:Ward 6"
"CASE_ENQUIRY_ID:101002761351,CASE_TITLE:Requests for Street Cleaning,CLOSURE_REASON:Case Closed. Closed date : 2018-12-19 10:36:21.473 Case Resolved Trash cleared from around tree completed ,ClosedPhoto:https://cityworker.cityofboston.gov:8443/attachments/report/5c1a5871d052e8a726989107/closed_photo/Report.jpg,Department:PWDx,LOCATION_STREET_NAME:589 Park St,LOCATION_ZIPCODE:02124,OnTime:ONTIME,QUEUE:PWDx_District 03: North Dorchester,SUBJECT:Public Works Department,TARGET_DT:2018-12-20 09:39:45,closed_dt:2018-12-19 10:36:21,city_council_district:4,neighborhood:Dorchester,neighborhood_services_district:9,police_district:B3,precinct:1701,pwd_district:03,ward:Ward 17"

Additional fields with “key:value” pair format for each 311 service request.

latitude double 1,606 42.3594
42.3603

This is the latitude value. Lines of latitude are parallel to the equator.

longitude double 1,777 -71.0587
-71.0583

This is the longitude value. Lines of longitude run perpendicular to lines of latitude, and all pass through both poles.

source string 7 Constituent Call
Citizens Connect App

Original source of the case.

status string 2 Closed
Open

Case status.

subcategory string 189 Schedule a Bulk Item Pickup
Parking Enforcement

Type of the service request.

Выберите предпочитаемую службу:

Azure Notebooks

Azure Databricks

Azure Notebooks

Пакет: Язык: Python Python
In [1]:
# This is a package in preview.
from azureml.opendatasets import BostonSafety

from datetime import datetime
from dateutil import parser


end_date = parser.parse('2016-01-01')
start_date = parser.parse('2015-05-01')
safety = BostonSafety(start_date=start_date, end_date=end_date)
safety = safety.to_pandas_dataframe()
ActivityStarted, to_pandas_dataframe ActivityStarted, to_pandas_dataframe_in_worker Looking for parquet files... Reading them into Pandas dataframe... Reading Safety/Release/city=Boston/part-00196-tid-845600952581210110-a4f62588-4996-42d1-bc79-23a9b4635c63-447039.c000.snappy.parquet under container citydatacontainer Done. ActivityCompleted: Activity=to_pandas_dataframe_in_worker, HowEnded=Success, Duration=2213.69 [ms] ActivityCompleted: Activity=to_pandas_dataframe, HowEnded=Success, Duration=2216.01 [ms]
In [2]:
safety.info()
<class 'pandas.core.frame.DataFrame'> Int64Index: 1 entries, 56262 to 56262 Data columns (total 11 columns): dataType 1 non-null object dataSubtype 1 non-null object dateTime 1 non-null datetime64[ns] category 1 non-null object subcategory 1 non-null object status 1 non-null object address 1 non-null object latitude 1 non-null float64 longitude 1 non-null float64 source 1 non-null object extendedProperties 0 non-null object dtypes: datetime64[ns](1), float64(2), object(8) memory usage: 96.0+ bytes
# Pip install packages
import os, sys

!{sys.executable} -m pip install azure-storage
!{sys.executable} -m pip install pyarrow
!{sys.executable} -m pip install pandas

# COMMAND ----------

# Azure storage access info
azure_storage_account_name = "azureopendatastorage"
azure_storage_sas_token = r""
container_name = "citydatacontainer"
folder_name = "Safety/Release/city=Boston"

# COMMAND ----------

from azure.storage.blob import BlockBlobService

if azure_storage_account_name is None or azure_storage_sas_token is None:
    raise Exception("Provide your specific name and key for your Azure Storage account--see the Prerequisites section earlier.")

print('Looking for the first parquet under the folder ' + folder_name + ' in container "' + container_name + '"...')
blob_service = BlockBlobService(account_name = azure_storage_account_name, sas_token = azure_storage_sas_token,)
blobs = blob_service.list_blobs(container_name)
sorted_blobs = sorted(list(blobs), key=lambda e: e.name, reverse=True)
targetBlobName=''
for blob in sorted_blobs:
    if blob.name.startswith(folder_name) and blob.name.endswith('.parquet'):
        targetBlobName = blob.name
        break

print('Target blob to download: ' + targetBlobName)
_, filename = os.path.split(targetBlobName)
parquet_file=blob_service.get_blob_to_path(container_name, targetBlobName, filename)

# COMMAND ----------

# Read the local parquet file into Pandas data frame
import pyarrow.parquet as pq
import pandas as pd

appended_df = []
print('Reading the local parquet file into Pandas data frame')
df = pq.read_table(filename).to_pandas()

# COMMAND ----------

# you can add your filter at below
print('Loaded as a Pandas data frame: ')
df

# COMMAND ----------


Azure Databricks

Пакет: Язык: Python Python
In [1]:
# This is a package in preview.
# You need to pip install azureml-opendatasets in Databricks cluster. https://docs.microsoft.com/en-us/azure/data-explorer/connect-from-databricks#install-the-python-library-on-your-azure-databricks-cluster
from azureml.opendatasets import BostonSafety

from datetime import datetime
from dateutil import parser


end_date = parser.parse('2016-01-01')
start_date = parser.parse('2015-05-01')
safety = BostonSafety(start_date=start_date, end_date=end_date)
safety = safety.to_spark_dataframe()
ActivityStarted, to_spark_dataframe ActivityStarted, to_spark_dataframe_in_worker ActivityCompleted: Activity=to_spark_dataframe_in_worker, HowEnded=Success, Duration=2380.02 [ms] ActivityCompleted: Activity=to_spark_dataframe, HowEnded=Success, Duration=2381.75 [ms]
In [2]:
display(safety)
dataTypedataSubtypedateTimecategorysubcategorystatusaddresslatitudelongitudesourceextendedProperties
Safety311_All2015-07-24T12:48:24.000+0000Call InquiryOCR Front Desk InteractionsClosed 42.3594-71.0587Constituent Callnull
# Databricks notebook source
# Azure storage access info
blob_account_name = "azureopendatastorage"
blob_container_name = "citydatacontainer"
blob_relative_path = "Safety/Release/city=Boston"
blob_sas_token = r""

# COMMAND ----------

# Allow SPARK to read from Blob remotely
wasbs_path = 'wasbs://%s@%s.blob.core.windows.net/%s' % (blob_container_name, blob_account_name, blob_relative_path)
spark.conf.set(
  'fs.azure.sas.%s.%s.blob.core.windows.net' % (blob_container_name, blob_account_name),
  blob_sas_token)
print('Remote blob path: ' + wasbs_path)

# COMMAND ----------

# SPARK read parquet, note that it won't load any data yet by now
df = spark.read.parquet(wasbs_path)
print('Register the DataFrame as a SQL temporary view: source')
df.createOrReplaceTempView('source')

# COMMAND ----------

# Display top 10 rows
print('Displaying top 10 rows: ')
display(spark.sql('SELECT * FROM source LIMIT 10'))

City Safety

From the Urban Innovation Initiative at Microsoft Research, databricks notebook for analytics with safety data (311 and 911 call data) from major U.S. cities. Analyses show frequency distributions and geographic clustering of safety issues within cities.