Ignorar Navegação

New York City Safety Data

New York City Social Services 311 Service Requests City Government Public Safety

Todos os pedidos de emergência de 311 de Nova Iorque desde 2010 até hoje.

Volume e Retenção

Este conjunto de dados é armazenado no formato Parquet. É atualizado diariamente e contém cerca de 12 milhões de linhas (500 MB) no total à data de 2019.

Este conjunto de dados contém registos históricos acumulados desde 2010 até ao presente. Pode utilizar as definições de parâmetros no nosso SDK para obter os dados num intervalo de tempo específico.

Localização do Armazenamento

Este conjunto de dados é armazenado na região do Azure E.U.A. Leste. A alocação de recursos de computação nos E.U.A. Leste é recomendada por questões de afinidade.

Informações adicionais

Este conjunto de dados foi obtido junto da câmara de Nova Iorque. Pode encontrar mais detalhes aqui. Veja os termos de utilização deste conjunto de dados aqui.

Avisos

A MICROSOFT DISPONIBILIZA OS CONJUNTOS DE DADOS ABERTOS DO AZURE TAL COMO ESTÃO. A MICROSOFT NÃO FAZ GARANTIAS, EXPRESSAS OU IMPLÍCITAS, NEM CONDIÇÕES RELATIVAMENTE À SUA UTILIZAÇÃO DOS CONJUNTOS DE DADOS. ATÉ AO LIMITE MÁXIMO PERMITIDO PELA LEGISLAÇÃO LOCAL, A MICROSOFT REJEITA QUALQUER RESPONSABILIDADE POR DANOS OU PERDAS, INCLUINDO DIRETOS, CONSEQUENCIAIS, ESPECIAIS, INDIRETOS, INCIDENTAIS OU PUNITIVOS, QUE RESULTEM DA SUA UTILIZAÇÃO DOS CONJUNTOS DE DADOS.

Este conjunto de dados é disponibilizado de acordo com os termos originais em que a Microsoft recebeu os dados de origem. O conjunto de dados pode incluir dados obtidos junto da Microsoft.

Access

Available inWhen to use
Azure Notebooks

Quickly explore the dataset with Jupyter notebooks hosted on Azure or your local machine.

Azure Databricks

Use this when you need the scale of an Azure managed Spark cluster to process the dataset.

Azure Synapse

Use this when you need the scale of an Azure managed Spark cluster to process the dataset.

Preview

dataType dataSubtype dateTime category subcategory status address latitude longitude source extendedProperties
Safety 311_All 5/5/2021 2:00:24 AM Noise - Commercial Banging/Pounding In Progress MYRTLE AVENUE 40.6931332361995 -73.9698315292218 null
Safety 311_All 5/5/2021 2:00:13 AM Noise - Street/Sidewalk Loud Music/Party In Progress 424 WEST 205 STREET 40.8626343241443 -73.9186166119019 null
Safety 311_All 5/5/2021 2:00:06 AM Noise - Residential Loud Music/Party In Progress 818 8 AVENUE 40.6661952992583 -73.9788660878437 null
Safety 311_All 5/5/2021 1:59:16 AM Non-Emergency Police Matter Other (complaint details) In Progress 180 ATKINS AVENUE 40.6747687507063 -73.8790672972994 null
Safety 311_All 5/5/2021 1:59:16 AM Noise - Residential Loud Music/Party In Progress 816 59 STREET 40.6357161250239 -74.0083769235818 null
Safety 311_All 5/5/2021 1:59:15 AM Taxi Complaint Driver Complaint - Passenger In Progress JFK AIRPORT 40.6483204862013 -73.7882812513018 null
Safety 311_All 5/5/2021 1:58:45 AM Noise - Residential Banging/Pounding In Progress 605 WEST 42 STREET 40.7610424556956 -73.9988123898813 null
Safety 311_All 5/5/2021 1:58:29 AM Illegal Parking Blocked Hydrant In Progress 56-07 62 AVENUE 40.7136053591523 -73.9064980580313 null
Safety 311_All 5/5/2021 1:55:02 AM Abandoned Vehicle With License Plate In Progress 1695 GRAND CONCOURSE 40.8454247060112 -73.9107511813732 null
Safety 311_All 5/5/2021 1:53:18 AM Noise - Residential Loud Music/Party In Progress 31 WEST 26 STREET 40.7441316249004 -73.9897869801139 null
Name Data type Unique Values (sample) Description
address string 1,642,784 655 EAST 230 STREET
78-15 PARSONS BOULEVARD

Número de porta da morada do incidente indicado pelo autor.

category string 446 Noise - Residential
HEAT/HOT WATER

Este é o primeiro nível de uma hierarquia que identifica o tópico do incidente ou do estado (Tipo de Queixa). Pode ter uma subcategoria (Descritor) correspondente ou ser único.

dataSubtype string 1 311_All

“311_All”

dataType string 1 Safety

“Segurança”

dateTime timestamp 17,359,228 2013-01-24 00:00:00
2015-01-08 00:00:00

A data em que o Pedido de Emergência foi criado.

latitude double 1,518,912 40.89187241649303
40.72195913199264

Latitude baseada na geografia da localização do incidente.

longitude double 1,518,935 -73.86016845296459
-73.80969682426189

Longitude baseada na geografia da localização do incidente.

status string 13 Closed
Pending

Estado do Pedido de Emergência submetido.

subcategory string 1,717 Loud Music/Party
ENTIRE BUILDING

Está associada à categoria (Complaint Type [Tipo de Queixa]) e mostra mais detalhes sobre o incidente ou o estado. Os respetivos valores dependem de Complaint Type e nem sempre são necessários no Pedido de Reparação.

Select your preferred service:

Azure Notebooks

Azure Databricks

Azure Synapse

Azure Notebooks

Package: Language: Python Python
In [1]:
# This is a package in preview.
from azureml.opendatasets import NycSafety

from datetime import datetime
from dateutil import parser


end_date = parser.parse('2016-01-01')
start_date = parser.parse('2015-05-01')
safety = NycSafety(start_date=start_date, end_date=end_date)
safety = safety.to_pandas_dataframe()
ActivityStarted, to_pandas_dataframe ActivityStarted, to_pandas_dataframe_in_worker Looking for parquet files... Reading them into Pandas dataframe... Reading Safety/Release/city=NewYorkCity/part-00026-tid-845600952581210110-a4f62588-4996-42d1-bc79-23a9b4635c63-446869.c000.snappy.parquet under container citydatacontainer Done. ActivityCompleted: Activity=to_pandas_dataframe_in_worker, HowEnded=Success, Duration=106593.46 [ms] ActivityCompleted: Activity=to_pandas_dataframe, HowEnded=Success, Duration=106687.96 [ms]
In [2]:
safety.info()
<class 'pandas.core.frame.DataFrame'> Int64Index: 1204035 entries, 7 to 12307252 Data columns (total 11 columns): dataType 1204035 non-null object dataSubtype 1204035 non-null object dateTime 1204035 non-null datetime64[ns] category 1204035 non-null object subcategory 1203974 non-null object status 1204035 non-null object address 1010833 non-null object latitude 1169358 non-null float64 longitude 1169358 non-null float64 source 0 non-null object extendedProperties 0 non-null object dtypes: datetime64[ns](1), float64(2), object(8) memory usage: 110.2+ MB
In [1]:
# Pip install packages
import os, sys

!{sys.executable} -m pip install azure-storage-blob
!{sys.executable} -m pip install pyarrow
!{sys.executable} -m pip install pandas
In [2]:
# Azure storage access info
azure_storage_account_name = "azureopendatastorage"
azure_storage_sas_token = r""
container_name = "citydatacontainer"
folder_name = "Safety/Release/city=NewYorkCity"
In [3]:
from azure.storage.blob import BlockBlobServicefrom azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient

if azure_storage_account_name is None or azure_storage_sas_token is None:
    raise Exception(
        "Provide your specific name and key for your Azure Storage account--see the Prerequisites section earlier.")

print('Looking for the first parquet under the folder ' +
      folder_name + ' in container "' + container_name + '"...')
container_url = f"https://{azure_storage_account_name}.blob.core.windows.net/"
blob_service_client = BlobServiceClient(
    container_url, azure_storage_sas_token if azure_storage_sas_token else None)

container_client = blob_service_client.get_container_client(container_name)
blobs = container_client.list_blobs(folder_name)
sorted_blobs = sorted(list(blobs), key=lambda e: e.name, reverse=True)
targetBlobName = ''
for blob in sorted_blobs:
    if blob.name.startswith(folder_name) and blob.name.endswith('.parquet'):
        targetBlobName = blob.name
        break

print('Target blob to download: ' + targetBlobName)
_, filename = os.path.split(targetBlobName)
blob_client = container_client.get_blob_client(targetBlobName)
with open(filename, 'wb') as local_file:
    blob_client.download_blob().download_to_stream(local_file)
In [4]:
# Read the parquet file into Pandas data frame
import pandas as pd

print('Reading the parquet file into Pandas data frame')
df = pd.read_parquet(filename)
In [5]:
# you can add your filter at below
print('Loaded as a Pandas data frame: ')
df
In [6]:
 

Azure Databricks

Package: Language: Python Python
In [1]:
# This is a package in preview.
# You need to pip install azureml-opendatasets in Databricks cluster. https://docs.microsoft.com/en-us/azure/data-explorer/connect-from-databricks#install-the-python-library-on-your-azure-databricks-cluster
from azureml.opendatasets import NycSafety

from datetime import datetime
from dateutil import parser


end_date = parser.parse('2016-01-01')
start_date = parser.parse('2015-05-01')
safety = NycSafety(start_date=start_date, end_date=end_date)
safety = safety.to_spark_dataframe()
ActivityStarted, to_spark_dataframe ActivityStarted, to_spark_dataframe_in_worker ActivityCompleted: Activity=to_spark_dataframe_in_worker, HowEnded=Success, Duration=4392.11 [ms] ActivityCompleted: Activity=to_spark_dataframe, HowEnded=Success, Duration=4395.98 [ms]
In [2]:
display(safety.limit(5))
dataTypedataSubtypedateTimecategorysubcategorystatusaddresslatitudelongitudesourceextendedProperties
Safety311_All2015-12-28T13:58:58.000+0000HEAT/HOT WATERENTIRE BUILDINGClosed548 11 STREET40.664924841709606-73.98101480555805nullnull
Safety311_All2015-06-14T01:11:08.000+0000Noise - ResidentialLoud Music/PartyClosednull40.86969422534882-73.86620623861982nullnull
Safety311_All2015-06-14T04:47:37.000+0000Noise - ResidentialLoud TalkingClosednull40.858744389082254-73.93011726711445nullnull
Safety311_All2015-06-16T16:56:00.000+0000SewerCatch Basin Clogged/Flooding (Use Comments) (SC)Closed82 JEWETT AVENUE40.63510898432114-74.12886658384302nullnull
Safety311_All2015-06-22T14:03:05.000+0000ELECTRICLIGHTINGClosed2170 BATHGATE AVENUE40.852335329676464-73.89389734164266nullnull
In [1]:
# Azure storage access info
blob_account_name = "azureopendatastorage"
blob_container_name = "citydatacontainer"
blob_relative_path = "Safety/Release/city=NewYorkCity"
blob_sas_token = r""
In [2]:
# Allow SPARK to read from Blob remotely
wasbs_path = 'wasbs://%s@%s.blob.core.windows.net/%s' % (blob_container_name, blob_account_name, blob_relative_path)
spark.conf.set(
  'fs.azure.sas.%s.%s.blob.core.windows.net' % (blob_container_name, blob_account_name),
  blob_sas_token)
print('Remote blob path: ' + wasbs_path)
In [3]:
# SPARK read parquet, note that it won't load any data yet by now
df = spark.read.parquet(wasbs_path)
print('Register the DataFrame as a SQL temporary view: source')
df.createOrReplaceTempView('source')
In [4]:
# Display top 10 rows
print('Displaying top 10 rows: ')
display(spark.sql('SELECT * FROM source LIMIT 10'))

Azure Synapse

Package: Language: Python Python
In [15]:
# This is a package in preview.
from azureml.opendatasets import NycSafety

from datetime import datetime
from dateutil import parser


end_date = parser.parse('2016-01-01')
start_date = parser.parse('2015-05-01')
safety = NycSafety(start_date=start_date, end_date=end_date)
safety = safety.to_spark_dataframe()
In [16]:
# Display top 5 rows
display(safety.limit(5))
Out[16]:
In [1]:
# Azure storage access info
blob_account_name = "azureopendatastorage"
blob_container_name = "citydatacontainer"
blob_relative_path = "Safety/Release/city=NewYorkCity"
blob_sas_token = r""
In [2]:
# Allow SPARK to read from Blob remotely
wasbs_path = 'wasbs://%s@%s.blob.core.windows.net/%s' % (blob_container_name, blob_account_name, blob_relative_path)
spark.conf.set(
  'fs.azure.sas.%s.%s.blob.core.windows.net' % (blob_container_name, blob_account_name),
  blob_sas_token)
print('Remote blob path: ' + wasbs_path)
In [3]:
# SPARK read parquet, note that it won't load any data yet by now
df = spark.read.parquet(wasbs_path)
print('Register the DataFrame as a SQL temporary view: source')
df.createOrReplaceTempView('source')
In [4]:
# Display top 10 rows
print('Displaying top 10 rows: ')
display(spark.sql('SELECT * FROM source LIMIT 10'))

City Safety

From the Urban Innovation Initiative at Microsoft Research, databricks notebook for analytics with safety data (311 and 911 call data) from major U.S. cities. Analyses show frequency distributions and geographic clustering of safety issues within cities.