{"id":1560,"date":"2019-03-18T00:00:00","date_gmt":"2019-03-18T00:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview"},"modified":"2025-06-18T05:16:00","modified_gmt":"2025-06-18T12:16:00","slug":"onnx-runtime-integration-with-nvidia-tensorrt-in-preview","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/","title":{"rendered":"ONNX Runtime integration with NVIDIA TensorRT in preview"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Today we are excited to open source the preview of the NVIDIA TensorRT execution provider in <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-is-now-open-source\/\">ONNX Runtime<\/a>. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework. Developers can now tap into the power of <a href=\"https:\/\/developer.nvidia.com\/tensorrt\">TensorRT<\/a> through ONNX Runtime to accelerate inferencing of ONNX models, which can be exported or converted from PyTorch, TensorFlow, and many other popular frameworks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Microsoft and NVIDIA worked closely to integrate the TensorRT execution provider with ONNX Runtime and have validated support for all the ONNX Models in the <a href=\"https:\/\/github.com\/onnx\/models\">model zoo<\/a>. With the TensorRT execution provider, ONNX Runtime delivers better inferencing performance on the same hardware compared to generic GPU acceleration. We have seen up to 2X improved performance using the TensorRT execution provider on internal workloads from Bing MultiMedia services.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"how-it-works\">How it works<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ONNX Runtime together with its TensorRT execution provider accelerates the inferencing of deep learning models by parsing the graph and allocating specific nodes for execution by the TensorRT stack in supported hardware. The TensorRT execution provider interfaces with the TensorRT libraries that are preinstalled in the platform to process the ONNX sub-graph and execute it on NVIDIA hardware. This enables developers to run ONNX models across different flavors of hardware and build applications with the flexibility to target different hardware configurations. This architecture abstracts out the details of the hardware specific libraries that are essential to optimizing the execution of deep neural networks.<\/p>\n\n\n\n<figure class=\"wp-block-image has-custom-border\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp\" alt=\"Infographic showing input data and output result using the ONNX model\" style=\"border-radius:0px\" title=\"Infographic showing input data and output result using the ONNX model\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"how-to-use-the-tensorrt-execution-provider\">How to use the TensorRT execution provider<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ONNX Runtime together with the TensorRT execution provider supports the ONNX Spec v1.2 or higher, with version 9 of the Opset. TensorRT optimized models can be deployed to all N-series VMs powered by NVIDIA GPUs on Azure.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To use TensorRT, you must first <a href=\"https:\/\/github.com\/Microsoft\/onnxruntime\/blob\/master\/BUILD.md\">build<\/a> ONNX Runtime with the TensorRT execution provider (use<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; auto-links: false; gutter: false; title: ; quick-code: false; notranslate\" title=\"\">\n\u00a0--use_tensorrt --tensorrt_home <path to location for TensorRT libraries in your local machine>\n<\/pre><\/div>\n\n\n<p class=\"wp-block-paragraph\">\u00a0flags in the build.sh tool). You can then take advantage of TensorRT by initiating the inference session through the ONNX Runtime APIs. ONNX Runtime will automatically prioritize the appropriate sub-graphs for execution by TensorRT to maximize performance.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; auto-links: false; gutter: false; title: ; quick-code: false; notranslate\" title=\"\">\nInferenceSession session_object{so};\nsession_object.RegisterExecutionProvider(std::make_unique<::onnxruntime::TensorrtExecutionProvider>());\nstatus = session_object.Load(model_file_name);\u200b\n<\/pre><\/div>\n\n\n<p class=\"wp-block-paragraph\">Detailed instructions are available on <a href=\"https:\/\/aka.ms\/trt-onnxrt\">GitHub<\/a>. In addition, a collection of standard tests are available through the onnx_test_runner utility in the repo to help verify the ONNX Runtime build with TensorRT execution provider.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"what-is-onnx-and-onnx-runtime\">What is ONNX and ONNX Runtime<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/onnx.ai\/\">ONNX<\/a> is an open format for deep learning and traditional machine learning models that Microsoft co-developed with Facebook and AWS. ONNX allows models to be represented in a common format that can be executed across different hardware platforms using ONNX Runtime. This gives developers the freedom to choose the right framework for their task, as well as the confidence to run their models efficiently on a variety of platforms with the hardware of their choice.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/github.com\/microsoft\/onnxruntime\">ONNX Runtime<\/a> is the first publicly available inference engine with full support for ONNX 1.2 and higher including the ONNX-ML profile. ONNX Runtime is lightweight and modular with an <a href=\"https:\/\/github.com\/Microsoft\/onnxruntime\/blob\/master\/docs\/HighLevelDesign.md\">extensible architecture<\/a> that allows hardware accelerators such as TensorRT to plug in as \u201cexecution providers.\u201d These execution providers unlock low latency and high efficiency neural network computations. Today, ONNX Runtime powers core scenarios that serve billions of users in Bing, Office, and more.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"another-step-towards-open-and-interoperable-ai\">Another step towards open and interoperable AI<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The preview of the TensorRT execution provider for ONNX Runtime marks another milestone in our venture to create an open and interoperable ecosystem for AI. We hope this makes it easier to drive AI innovation in a world with ever-increasing latency requirements for production models. We are continuously evolving and improving ONNX Runtime, and look forward to your feedback and contributions!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To learn more about using ONNX for accelerated inferencing on the cloud and edge, check out the <a href=\"https:\/\/gputechconf2019.smarteventscloud.com\/connect\/search.ww#loadSearch-searchPhrase=ONNX&amp;searchType=session&amp;tc=0&amp;sortBy=dayTime&amp;p=\">ONNX session<\/a> at NVIDIA GTC. Have feedback or questions about ONNX Runtime? <a href=\"https:\/\/github.com\/Microsoft\/onnxruntime\/issues\">File an issue<\/a> on GitHub, and follow us on <a href=\"https:\/\/twitter.com\/onnxruntime\">Twitter<\/a>.<\/p>\n\n\n<p><code><code><\/code><\/code><\/p>\n<pre>\u00a0<\/pre>\n<p><code><code><\/code><\/code><\/p>\n<p>\u00a0<\/p>\n<p><code><code><\/code><\/code><\/p>","protected":false},"excerpt":{"rendered":"<p>Today we are excited to open source the preview of the NVIDIA TensorRT execution provider in ONNX Runtime. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ms_queue_id":[],"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","_alt_title":"","footnotes":"","msx_community_cta_settings":[]},"categories":[1454],"tags":[],"audience":[3057,3055,3056],"content-type":[1511],"product":[],"tech-community":[],"topic":[],"coauthors":[602],"class_list":["post-1560","post","type-post","status-publish","format-standard","hentry","category-ai-machine-learning","audience-data-professionals","audience-developers","audience-it-implementors","content-type-best-practices","review-flag-1680286581-56","review-flag-1-1680286581-825","review-flag-2-1680286581-601","review-flag-9-1680286581-259","review-flag-integ-1680286579-214","review-flag-lever-1680286579-649","review-flag-machi-1680286585-314","review-flag-ml-1680286585-776"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>ONNX Runtime integration with NVIDIA TensorRT in preview | Microsoft Azure Blog<\/title>\n<meta name=\"description\" content=\"Today we are excited to open source the preview of the NVIDIA TensorRT execution provider in ONNX Runtime. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ONNX Runtime integration with NVIDIA TensorRT in preview | Microsoft Azure Blog\" \/>\n<meta property=\"og:description\" content=\"Today we are excited to open source the preview of the NVIDIA TensorRT execution provider in ONNX Runtime. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Azure Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/microsoftazure\" \/>\n<meta property=\"article:published_time\" content=\"2019-03-18T00:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-18T12:16:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp\" \/>\n<meta name=\"author\" content=\"Manash Goswami\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@azure\" \/>\n<meta name=\"twitter:site\" content=\"@azure\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Manash Goswami\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/\"},\"author\":[{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/manash-goswami\/\",\"@type\":\"Person\",\"@name\":\"Manash Goswami\"}],\"headline\":\"ONNX Runtime integration with NVIDIA TensorRT in preview\",\"datePublished\":\"2019-03-18T00:00:00+00:00\",\"dateModified\":\"2025-06-18T12:16:00+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/\"},\"wordCount\":649,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp\",\"articleSection\":[\"AI + machine learning\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/\",\"name\":\"ONNX Runtime integration with NVIDIA TensorRT in preview | Microsoft Azure Blog\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp\",\"datePublished\":\"2019-03-18T00:00:00+00:00\",\"dateModified\":\"2025-06-18T12:16:00+00:00\",\"description\":\"Today we are excited to open source the preview of the NVIDIA TensorRT execution provider in ONNX Runtime. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework.\",\"breadcrumb\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#primaryimage\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Blog home\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI + machine learning\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/ai-machine-learning\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"ONNX Runtime integration with NVIDIA TensorRT in preview\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"name\":\"Microsoft Azure Blog\",\"description\":\"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.\",\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\",\"name\":\"Microsoft Azure Blog\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Azure Blog\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/microsoftazure\",\"https:\/\/x.com\/azure\",\"https:\/\/www.instagram.com\/microsoftdeveloper\/\",\"https:\/\/www.linkedin.com\/company\/16188386\",\"https:\/\/www.youtube.com\/user\/windowsazure\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117\",\"name\":\"shakir\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"caption\":\"shakir\"},\"sameAs\":[\"https:\/\/azure.microsoft.com\"],\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"ONNX Runtime integration with NVIDIA TensorRT in preview | Microsoft Azure Blog","description":"Today we are excited to open source the preview of the NVIDIA TensorRT execution provider in ONNX Runtime. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/","og_locale":"en_US","og_type":"article","og_title":"ONNX Runtime integration with NVIDIA TensorRT in preview | Microsoft Azure Blog","og_description":"Today we are excited to open source the preview of the NVIDIA TensorRT execution provider in ONNX Runtime. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework.","og_url":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/","og_site_name":"Microsoft Azure Blog","article_publisher":"https:\/\/www.facebook.com\/microsoftazure","article_published_time":"2019-03-18T00:00:00+00:00","article_modified_time":"2025-06-18T12:16:00+00:00","og_image":[{"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp","type":"","width":"","height":""}],"author":"Manash Goswami","twitter_card":"summary_large_image","twitter_creator":"@azure","twitter_site":"@azure","twitter_misc":{"Written by":"Manash Goswami","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#article","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/"},"author":[{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/manash-goswami\/","@type":"Person","@name":"Manash Goswami"}],"headline":"ONNX Runtime integration with NVIDIA TensorRT in preview","datePublished":"2019-03-18T00:00:00+00:00","dateModified":"2025-06-18T12:16:00+00:00","mainEntityOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/"},"wordCount":649,"commentCount":0,"publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp","articleSection":["AI + machine learning"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/","name":"ONNX Runtime integration with NVIDIA TensorRT in preview | Microsoft Azure Blog","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#primaryimage"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp","datePublished":"2019-03-18T00:00:00+00:00","dateModified":"2025-06-18T12:16:00+00:00","description":"Today we are excited to open source the preview of the NVIDIA TensorRT execution provider in ONNX Runtime. With this release, we are taking another step towards open and interoperable AI by enabling developers to easily leverage industry-leading GPU acceleration regardless of their choice of framework.","breadcrumb":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#primaryimage","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/03\/67b71a1a-0165-453c-bc2b-9fe997c603c3.webp"},{"@type":"BreadcrumbList","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/onnx-runtime-integration-with-nvidia-tensorrt-in-preview\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog home","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/"},{"@type":"ListItem","position":2,"name":"AI + machine learning","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/ai-machine-learning\/"},{"@type":"ListItem","position":3,"name":"ONNX Runtime integration with NVIDIA TensorRT in preview"}]},{"@type":"WebSite","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","name":"Microsoft Azure Blog","description":"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.","publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization","name":"Microsoft Azure Blog","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","width":512,"height":512,"caption":"Microsoft Azure Blog"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/microsoftazure","https:\/\/x.com\/azure","https:\/\/www.instagram.com\/microsoftdeveloper\/","https:\/\/www.linkedin.com\/company\/16188386","https:\/\/www.youtube.com\/user\/windowsazure"]},{"@type":"Person","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117","name":"shakir","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4","url":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","caption":"shakir"},"sameAs":["https:\/\/azure.microsoft.com"],"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/"}]}},"msxcm_display_generated_audio":false,"msxcm_animated_featured_image":null,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Azure Blog","distributor_original_site_url":"https:\/\/azure.microsoft.com\/en-us\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/1560","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/comments?post=1560"}],"version-history":[{"count":1,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/1560\/revisions"}],"predecessor-version":[{"id":42332,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/1560\/revisions\/42332"}],"wp:attachment":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/media?parent=1560"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/categories?post=1560"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tags?post=1560"},{"taxonomy":"audience","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/audience?post=1560"},{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/content-type?post=1560"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/product?post=1560"},{"taxonomy":"tech-community","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tech-community?post=1560"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/topic?post=1560"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/coauthors?post=1560"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}