Speed and Scale AI Inference Operations Across Multiple Architectures

When executing inference operations, developers need an efficient way to integrate components that deliver great performance at scale while providing a simple interface between the application and execution engine.

Thus far, TensorFlow Serving has been the serving system of choice. But with it come challenges including its lack of cross-architecture inference execution on GPUs, VPUs, and FPGAs.

The 2021.1 release of the Intel® Distribution of OpenVINO™ toolkit solves these challenges with its improved Model Server, a Docker container capable of hosting machine-learning models for high-performance inference.

Join Principal Engineer and AI Solution Architect Adam Marek and AI Developer Tools Product Manager Zoe Cayetano to learn about this serving system for production environments, including how to:

  • More easily deploy new algorithms and AI experiments for your AI models
  • Take advantage of a write-once, deploy-anywhere programming paradigm, from edge to cloud
  • Leverage Docker containers to simplify the integration of AI inference with a wide range of platforms and solutions

Get the software

  • Download the OpenVINO™ toolkit—includes nearly 20 dev tools and libraries for creating cross-architecture applications.
  • Sign up for an Intel® DevCloud for oneAPI account—a free development sandbox with access to the latest Intel® hardware and oneAPI software.
Adam Marek, Principal Engineer and AI Solutions Architect, Intel Corporation

Adam is a Principal Engineer and AI Solutions Architect whose focus includes distributed platforms and components for deep learning platforms. Joining Intel in 2003, his expertise in software architecture and design has been honed since 1998, working for companies including Softmatic and Oke Software and Communication on projects for telcos, embedded speech processing solutions, and large-scale backends for secure online services. Adam holds a Master of Science in Computer Science from Gdańsk University of Technology and a number of patents in various software fields.

Zoe Cayetano, AI Developer Tools Product Manager, Intel Corporation

Passionate about democratizing technology access for everyone and working on projects with outsized impact on the world, Zoe is a Product Manager for AI and IoT working on a variety of interdisciplinary business and engineering problems. Prior to Intel, she was a data science researcher for a particle accelerator at Arizona State University, where she analyzed electron beam dynamics of novel x-ray lasers that were used for crystallography, quantum materials and bioimaging. She holds bachelor’s degrees in Applied Physics and Business.

Dariusz Trawinski, Deep Learning Software Engineer, Intel Corporation

Dariusz is a Senior Software Engineer who specializes in Deep Learning (DL) applications and solutions. Joining Intel in 2000, he has honed his software expertise through web application development, system administration, information security, and data center and cloud management. Most recently, his focus has been on optimizing AI application performance and improving user experiences in DL and inference platforms. Dariusz has a MSc in Telecommunications from Technical University in Gdansk, Poland.

For more complete information about compiler optimizations, see our Optimization Notice.