Skip Navigation

Boston Safety Data

311 Boston Case Management City Services CRM Public Safety

311 calls reported to the city of Boston.

Refer to this link to learn more about BOS:311.

Volume and Retention

This dataset is stored in Parquet format. It is updated daily, and contains about 100K rows (10MB) in total as of 2019.

This dataset contains historical records accumulated from 2011 to the present. You can use parameter settings in our SDK to fetch data within a specific time range.

Storage Location

This dataset is stored in the East US Azure region. Allocating compute resources in East US is recommended for affinity.

Additional Information

This dataset is sourced from city of Boston government. More details can be found from here. Reference Open Data Commons Public Domain Dedication and License (ODC PDDL) for the license of using this dataset.

Notices

MICROSOFT PROVIDES AZURE OPEN DATASETS ON AN “AS IS” BASIS. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, GUARANTEES OR CONDITIONS WITH RESPECT TO YOUR USE OF THE DATASETS. TO THE EXTENT PERMITTED UNDER YOUR LOCAL LAW, MICROSOFT DISCLAIMS ALL LIABILITY FOR ANY DAMAGES OR LOSSES, INCLUDING DIRECT, CONSEQUENTIAL, SPECIAL, INDIRECT, INCIDENTAL OR PUNITIVE, RESULTING FROM YOUR USE OF THE DATASETS.

This dataset is provided under the original terms that Microsoft received source data. The dataset may include data sourced from Microsoft.

Access

Available inWhen to use
Azure Notebooks

Quickly explore the dataset with Jupyter notebooks hosted on Azure or your local machine.

Azure Databricks

Use this when you need the scale of an Azure managed Spark cluster to process the dataset.

Preview

dataType dataSubtype dateTime category subcategory status address latitude longitude source
Safety 311_All 12/4/2019 11:40:00 PM Code Enforcement Unshoveled Sidewalk Open 292 Foster St Brighton MA 02135 42.3405 -71.1592 Constituent Call
Safety 311_All 12/4/2019 11:33:00 PM Highway Maintenance Work Hours-Loud Noise Complaints Open 70 Lincoln St Boston MA 02111 42.352 -71.0579 Constituent Call
Safety 311_All 12/4/2019 11:31:38 PM Enforcement & Abandoned Vehicles Parking Enforcement Open 101 Farragut Rd South Boston MA 02127 42.3347 -71.0253 Citizens Connect App
Safety 311_All 12/4/2019 11:18:00 PM Sanitation Missed Trash/Recycling/Yard Waste/Bulk Item Open 36 Worcester Sq Roxbury MA 02118 42.3364 -71.074 Constituent Call
Safety 311_All 12/4/2019 11:17:31 PM Street Lights Street Light Outages Open 606 Harvard St Mattapan MA 02126 42.2843 -71.0974 Citizens Connect App
Safety 311_All 12/4/2019 11:17:00 PM Enforcement & Abandoned Vehicles Parking Enforcement Closed INTERSECTION of Chandler St & Cazenove St Boston MA 42.3594 -71.0587 Citizens Connect App
Safety 311_All 12/4/2019 11:12:59 PM Enforcement & Abandoned Vehicles Abandoned Vehicles Open 514 Atlantic Ave Boston MA 02210 42.354 -71.0525 Citizens Connect App
Safety 311_All 12/4/2019 11:03:59 PM Code Enforcement Unshoveled Sidewalk Closed INTERSECTION of Shawmut Ave & Waltham St Roxbury MA 42.3594 -71.0587 Citizens Connect App
Safety 311_All 12/4/2019 11:03:36 PM Administrative & General Requests Misc. Snow Complaint Open 5 Saint James St Roxbury MA 02119 42.3261 -71.0838 Citizens Connect App
Safety 311_All 12/4/2019 11:00:03 PM Code Enforcement Unshoveled Sidewalk Open INTERSECTION of Milford St & Bradford St Roxbury MA 42.3594 -71.0587 Citizens Connect App
Name Data type Unique Values (sample) Description
address string 139,317 \"\"
1 City Hall Plz Boston MA 02108

Location.

category string 54 Sanitation
Street Cleaning

Reason of the service request.

dataSubtype string 1 311_All

“311_All”

dataType string 1 Safety

“Safety”

dateTime timestamp 1,513,546 2015-07-23 10:51:00
2015-07-23 10:47:00

Open date and time of the service request.

extendedProperties string 194,565 "CASE_ENQUIRY_ID:101002920759,CASE_TITLE:Missed Bulk Item Pickup: District 05,CLOSURE_REASON:Case Closed. Closed date : 2019-06-03 06:49:37.403 Case Noted No materials out upon inspection at location at 6:47 a.m. on 6/3/19. ,ClosedPhoto:https://cityworker.cityofboston.gov:8443/attachments/report/5cf1a94ad052a96fa7c9376e/closed_photo/Report.jpg,Department:PWDx,LOCATION_STREET_NAME:140 W Sixth St,LOCATION_ZIPCODE:02127,OnTime:ONTIME,QUEUE:PWDx_Missed Trash\\Recycling\\Yard Waste\\Bulk Item,SUBJECT:Public Works Department,TARGET_DT:2019-06-04 08:30:00,closed_dt:2019-06-03 06:49:37,city_council_district:2,neighborhood:South Boston / South Boston Waterfront,neighborhood_services_district:5,police_district:C6,precinct:0602,pwd_district:05,ward:Ward 6"
"CASE_ENQUIRY_ID:101001888162,CASE_TITLE:Missed Trash: District 06,CLOSURE_REASON:Case Closed. Closed date : 2016-08-23 08:36:50.107 Case Noted All wood. No household. Ticketed ,ClosedPhoto:https://cityworker.cityofboston.gov:8443/attachments/report/57bb356370b47b799fdb9fbd/closed_photo/Report.jpg,Department:PWDx,LOCATION_STREET_NAME:78 Park St,LOCATION_ZIPCODE:02132,OnTime:ONTIME,QUEUE:PWDx_Missed Trash\\Recycling\\Yard Waste\\Bulk Item,SUBJECT:Public Works Department,TARGET_DT:2016-08-23 13:23:47,closed_dt:2016-08-23 08:36:50,city_council_district:6,neighborhood:West Roxbury,neighborhood_services_district:12,police_district:E5,precinct:2012,pwd_district:06,ward:Ward 20"

Additional fields with “key:value” pair format for each 311 service request.

latitude double 1,622 42.3594
42.3603

This is the latitude value. Lines of latitude are parallel to the equator.

longitude double 1,806 -71.0587
-71.0583

This is the longitude value. Lines of longitude run perpendicular to lines of latitude, and all pass through both poles.

source string 7 Constituent Call
Citizens Connect App

Original source of the case.

status string 2 Closed
Open

Case status.

subcategory string 208 Parking Enforcement
Schedule a Bulk Item Pickup

Type of the service request.

Select your preferred service:

Azure Notebooks

Azure Databricks

Azure Notebooks

Package: Language: Python Python
In [1]:
# This is a package in preview.
from azureml.opendatasets import BostonSafety

from datetime import datetime
from dateutil import parser


end_date = parser.parse('2016-01-01')
start_date = parser.parse('2015-05-01')
safety = BostonSafety(start_date=start_date, end_date=end_date)
safety = safety.to_pandas_dataframe()
ActivityStarted, to_pandas_dataframe ActivityStarted, to_pandas_dataframe_in_worker Looking for parquet files... Reading them into Pandas dataframe... Reading Safety/Release/city=Boston/part-00196-tid-845600952581210110-a4f62588-4996-42d1-bc79-23a9b4635c63-447039.c000.snappy.parquet under container citydatacontainer Done. ActivityCompleted: Activity=to_pandas_dataframe_in_worker, HowEnded=Success, Duration=2213.69 [ms] ActivityCompleted: Activity=to_pandas_dataframe, HowEnded=Success, Duration=2216.01 [ms]
In [2]:
safety.info()
<class 'pandas.core.frame.DataFrame'> Int64Index: 1 entries, 56262 to 56262 Data columns (total 11 columns): dataType 1 non-null object dataSubtype 1 non-null object dateTime 1 non-null datetime64[ns] category 1 non-null object subcategory 1 non-null object status 1 non-null object address 1 non-null object latitude 1 non-null float64 longitude 1 non-null float64 source 1 non-null object extendedProperties 0 non-null object dtypes: datetime64[ns](1), float64(2), object(8) memory usage: 96.0+ bytes
# Pip install packages
import os, sys

!{sys.executable} -m pip install azure-storage
!{sys.executable} -m pip install pyarrow
!{sys.executable} -m pip install pandas

# COMMAND ----------

# Azure storage access info
azure_storage_account_name = "azureopendatastorage"
azure_storage_sas_token = r""
container_name = "citydatacontainer"
folder_name = "Safety/Release/city=Boston"

# COMMAND ----------

from azure.storage.blob import BlockBlobService

if azure_storage_account_name is None or azure_storage_sas_token is None:
    raise Exception("Provide your specific name and key for your Azure Storage account--see the Prerequisites section earlier.")

print('Looking for the first parquet under the folder ' + folder_name + ' in container "' + container_name + '"...')
blob_service = BlockBlobService(account_name = azure_storage_account_name, sas_token = azure_storage_sas_token,)
blobs = blob_service.list_blobs(container_name)
sorted_blobs = sorted(list(blobs), key=lambda e: e.name, reverse=True)
targetBlobName=''
for blob in sorted_blobs:
    if blob.name.startswith(folder_name) and blob.name.endswith('.parquet'):
        targetBlobName = blob.name
        break

print('Target blob to download: ' + targetBlobName)
_, filename = os.path.split(targetBlobName)
parquet_file=blob_service.get_blob_to_path(container_name, targetBlobName, filename)

# COMMAND ----------

# Read the local parquet file into Pandas data frame
import pyarrow.parquet as pq
import pandas as pd

appended_df = []
print('Reading the local parquet file into Pandas data frame')
df = pq.read_table(filename).to_pandas()

# COMMAND ----------

# you can add your filter at below
print('Loaded as a Pandas data frame: ')
df

# COMMAND ----------


Azure Databricks

Package: Language: Python Python
In [1]:
# This is a package in preview.
# You need to pip install azureml-opendatasets in Databricks cluster. https://docs.microsoft.com/en-us/azure/data-explorer/connect-from-databricks#install-the-python-library-on-your-azure-databricks-cluster
from azureml.opendatasets import BostonSafety

from datetime import datetime
from dateutil import parser


end_date = parser.parse('2016-01-01')
start_date = parser.parse('2015-05-01')
safety = BostonSafety(start_date=start_date, end_date=end_date)
safety = safety.to_spark_dataframe()
ActivityStarted, to_spark_dataframe ActivityStarted, to_spark_dataframe_in_worker ActivityCompleted: Activity=to_spark_dataframe_in_worker, HowEnded=Success, Duration=2380.02 [ms] ActivityCompleted: Activity=to_spark_dataframe, HowEnded=Success, Duration=2381.75 [ms]
In [2]:
display(safety)
dataTypedataSubtypedateTimecategorysubcategorystatusaddresslatitudelongitudesourceextendedProperties
Safety311_All2015-07-24T12:48:24.000+0000Call InquiryOCR Front Desk InteractionsClosed 42.3594-71.0587Constituent Callnull
# Databricks notebook source
# Azure storage access info
blob_account_name = "azureopendatastorage"
blob_container_name = "citydatacontainer"
blob_relative_path = "Safety/Release/city=Boston"
blob_sas_token = r""

# COMMAND ----------

# Allow SPARK to read from Blob remotely
wasbs_path = 'wasbs://%s@%s.blob.core.windows.net/%s' % (blob_container_name, blob_account_name, blob_relative_path)
spark.conf.set(
  'fs.azure.sas.%s.%s.blob.core.windows.net' % (blob_container_name, blob_account_name),
  blob_sas_token)
print('Remote blob path: ' + wasbs_path)

# COMMAND ----------

# SPARK read parquet, note that it won't load any data yet by now
df = spark.read.parquet(wasbs_path)
print('Register the DataFrame as a SQL temporary view: source')
df.createOrReplaceTempView('source')

# COMMAND ----------

# Display top 10 rows
print('Displaying top 10 rows: ')
display(spark.sql('SELECT * FROM source LIMIT 10'))

City Safety

From the Urban Innovation Initiative at Microsoft Research, databricks notebook for analytics with safety data (311 and 911 call data) from major U.S. cities. Analyses show frequency distributions and geographic clustering of safety issues within cities.