Video Editing

Video Editing

Challenges

Video pipelines use ingest, transcode, edit, render, distribute workflow. This requires high-speed capture from a few points, then low-latency reads from many clients in parallel. In addition to performance, flexible capacity at low cost is required for scalability as projects come and go. Finally, low administrative overhead is desired.

Solution

WARP 38002-H hybrid storage system with a mix of SSDs and HDDs. SSDs are configured to accelerate reads, while high capacity drives yield a large pool for high capacity storage at low cost.

Results

The hybrid storage pool seamlessly combines disk with capacity- and read-optimized devices to achieve smooth capture. It can deliver up to 48G Bytes per second (~384 Gbps) of low-latency reads on the active dataset in a single shelf. Built-in dual controllers provide advanced features like compression, thin provisioning, replication, and more. At up to 720TB in 4u, the dense chassis allows for growth in capacity & throughput without high footprint.


Video production crews know that local storage may work for single users, but an efficient central datastore is essential to keep a team’s pipeline moving efficiently. Pipeline environments typically ingest data from capture sources such as cameras, then transcode into different formats to support editing and effects. Many crews work against the same data sets for a given project, adding new elements and metadata in layers, before final rendering “flattens” the layers into a deliverable format.

MediaPipelineEach dataset per project is typically hundreds of GBs, and a single feature length project can be 10s of TBs or more. Lost frames or excessive retransmissions can result in corruption in the final product, making low-latency storage and data integrity important factors at each phase of production.

Media production teams face a tough decision between affordable “generic” storage, versus high-performance purpose-built systems. Often, they resort to buying legacy OEM hardware that is neither affordable nor flexible enough for long-term use, nor fast enough to keep pace with simultaneous capture and playback of multiple projects.

The WARP 38002-H series of unified storage appliances optimizes both performance and cost. Backed by powerful, commercially-supported open software, enterprise-class hardware, and enhanced management tools, they offer unmatched performance and scale at an industry-leading price. Because it provides the flexibility of custom storage and the ease of use of an appliance, but without the legacy OEM markup, this is ideally suited for high definition media storage, processing, and distribution.

Capacity & Performance

The WARP 38002 can hold petabytes, but for most teams, 10s of terabytes is often sufficient. This is not “large” by modern standards, but starting with capacity that can grow as needed in a simple way is important.

38001-HGST_600To address this, WARP offers dense storage that is scalable, fast, and affordable. A single 4U chassis can grow from 48TB to 720TB just by adding hard drives. Expanding further is a simple matter of racking JBODs. WARPware controllers manage the system, and a few simple commands fold new drives into the existing system in real time. This is faster, less costly, and less complex than scale-out approaches.

In addition to scalability, most legacy systems struggle to handle mixed workloads without complex performance tuning or exorbitant costs in hardware. Legacy systems have limited cache, making it impossible to cache a complete working data set. To work around this, OEMs can support SSD options, but these solutions inevitably carry exorbitant markups and license fees. WARP Mechanics never charges extra to activate the hardware customers already paid for, which finally makes hybrid solutions affordable.

For latency-sensitive IO like capture and playback, the WARP 38002-H supports hybrid pools integrated with high-performance flash and disk modules. When coupled with other WARPware components, throughput scales from gigabits per second, to terabytes per second in a single namespace.

Integrity

Of course, WARP Mechanics solutions use enterprise, RAID protected storage with single-, double-, or triple-parity. However, WARPware goes beyond traditional RAID. There are failure cases that corrupt data with legacy RAID, like write holes, silent data corruption, bit rot, and multiple simultaneous disk failures. Therefore, WARPware includes multi-layered protection for every block of data at every phase. For instance, it is protected by ECC memory, network checksums, RAID parity, and ZFS block-level checksums. If a bit flip occurs at any stage for any reason, it will be detected and corrected through background integrity checks.

Solution

The WARP 38002-H HybridMatrix appliances combine the best open software with enterprise-grade hardware. WDS-9460_3q_1024This allows it to supply the highest levels of throughput, scale, and protection in a hybrid storage solution, without breaking the bank. Configured as a WARPnas filer, the 38002-H makes accessibility easy through standard protocols such as NFS, SMB/CIFS, and FTP. Each platform comes with two embedded 10Gb Ethernet ports as a baseline, and supports additional interfaces such as SAS, 40/100GbE, Infiniband, and Omni-Path.

Value

  • Purpose-built appliance architecture – Guarantees compatibility, reliability, and ongoing manageability with all components pre-configured and tested.
  • High-performance design – high throughput at both network and filesystem levels.
  • Multiple data protection mechanisms – Data is protected with integrity features to eliminate “write holes,” “silent corruption,” “bit rot,” and more.
  • Virtually unlimited scalability – With ZFS inside each appliance, capacity can scale to astronomical levels.
DOWNLOAD DATA SHEET