Red Hat powers architectural efficiencies for DNEG

DNEG, a world-leading visual entertainment services company, is using Red Hat OpenShift as a consistent global platform to speed up software innovation and improve artist productivity for competitive differentiation.

DNEG is a visual effects (VFX) and animation studio for feature film and television with seven Academy AwardⓇ wins for 'Best VFX' in recent years, including Inception, Interstellar and Dune: Part One. What makes DNEG unique is its software developed by hundreds of engineers around the world, and its artists accessing those creations to deliver DNEG’s renowned work. A fundamental part of any visual effects workflow is the ability to track and manage a large volume of digital assets and their metadata to assemble everything from cityscapes to jostling crowds to flowing water.

From its roots 25 years ago as a small company based in London, DNEG has been steadily growing and now has more than 9,000 employees in ten locations around the world. DNEG needs to move fast to be the best in its field, yet aspects of its asset tracking system, a large monolithic application, had become a bottleneck. The steady increase in artists and workload brought a corresponding increase in the amount of metadata, all competing for access to the same database services. It was clear that the asset tracking system needed a significant architectural overhaul in order to scale to support this expanding, globally-distributed team. DNEG also continually faces increasing compute demands as the industry’s technology evolves, such as with the introduction of the latest industry-standard collaboration-based solution for constructing disparate elements into animated 3D scenes – Universal Scene Description (USD).

DNEG decided to break up key parts of the asset tracking system into microservices and manage these in containers using Red Hat OpenShift, the industry’s leading hybrid cloud application platform powered by Kubernetes. The first, and most acutely critical, component of this was a high-volume metadata interrogation API, Pipe Query. DNEG had a clear vision, and leaned on Red Hat consultants and training to enhance its expertise in DevOps, infrastructure-as-code, API management and other relevant areas, developing first a proof of concept, then a global infrastructure. Now in production in six datacentres around the globe, Red Hat OpenShift provides an abstraction and orchestration layer between artists and the database, enabling developers to deliver Pipe Query in a standardised, cloud-native way to artists, wherever they are in the world.

Red Hat OpenShift automates the application lifecycle to reduce manual processes for DNEG’s developers and enable workloads to be managed dynamically. DNEG has taken a progressive DevOps approach to delivering Pipe Query from development and build, through quality assurance (QA) and all the way to production. Red Hat OpenShift Pipelines, based on the upstream open source Tekton project, and Red Hat OpenShift GitOps, which uses Argo CD, are helping DNEG to create advanced workflows for continuous integration and continuous deployment (CI/CD) using industry best practices. This helps developers deliver Pipe Query updates in a faster, more stable and flexible way – deploying features when they are available rather than waiting for rigid release cycles, and with minimal disruption: releasing a change now takes minutes where before it took hours. Red Hat OpenShift gives DNEG visibility into application usage so that it can streamline the creative process of asset design for artists, and help them be productive instead of waiting for the system to fetch data.

With its deployment of Red Hat OpenShift Platform Plus, DNEG can also take advantage of:

Red Hat OpenShift’s in-built security features, enabling DNEG to control its trusted sources of content, testing and deployments, to defend applications from attacks and vulnerabilities in all layers of the platform and to extend secure services through standard interfaces and APIs.

Red Hat Advanced Cluster Management for Kubernetes to control clusters and applications from a single console, with built-in governance and compliance policies.

Red Hat Advanced Cluster Security for Kubernetes to help protect containerized Kubernetes workloads in all major clouds and hybrid platforms.

Red Hat OpenShift Data Foundation for persistent software-defined storage and essential data services.

Red Hat Quay, a scalable, central registry of available software to help distribute it efficiently to multiple clusters.

DNEG is planning to onboard more of its software to Red Hat OpenShift for ease of distribution and orchestration, prioritising services that deliver the most value. This includes its asset provisioning system and its file transfer system, which carries 40 petabytes of data. As industry technology evolves, DNEG anticipates that this could also include workloads that use machine learning (ML), as these have high compute demands and can save hours of laborious manual work by helping artists to apply animation principles from one digital entity to another.

DNEG is also in the process of implementing Red Hat 3scale API Management to help manage workload, such as balancing requests from artists with batch requests from a server to reduce congestion. Red Hat 3Scale API Management can help DNEG to prioritise artist’s computing consumption based on the importance or delivery date of the work. This helps artists learn the most efficient way to use resources to deliver their work, and enables DNEG to better understand how much resource each production is using, so it can more accurately quote for future jobs.