Portable Video Storage

Portable Video Storage

Challenges

Video pipelines use ingest-transcode- edit-render-distribute workflows requiring high-speed capture from a few points, then low-latency reads from many clients in parallel. They must support capture and edit in remote locations without datacenters. Finally, low administrative overhead is desired.

Solution

WARP 26200-H HybridCube desktop storage system with a mix of SSDs and HDDs. SSDs accelerate reads while high capacity drives yield a large pool for fairly large data sets. The compact chassis is both low cost and portable.

Results

The hybrid storage pool seamlessly combines disk with flash and RAM to achieve high performance. Depending on configuration, it can deliver more than 4 Gigabytes per second (~40 Gbps) of low-latency reads on the active dataset in a single shelf. Software includes advanced features such as compression, deduplication, replication, and more. At up to 200TB, it allows high capacity & throughput without high cost.


Video crews know that a fast and reliable central datastore is essential to keep a pipeline moving efficiently. And WARP Mechanics has been producting storage for this market for many years, like the WARP 38000 and 44000 series of storage appliances.

MediaPipeline-Cube

But it also isn’t practical to rely entirely on centralized storeage.

Each dataset per project is typically hundreds of GBs, and a single feature can be 10s of TBs or more. With multiple projects active at once, a single storage solution can become unmanagably large and expensive, and outages have a massive business impact.

Lost frames can corrupt the final product, making low-latency storage and data integrity important factors at every phase of production. Local storage systems without complex scale-out filesystems have better latency and fewer networking issues than some centralized systems.

Furthermore, crews often travel to remote areas for shooting, ADR, and post-production editing. It simply isn’t practical to transport 10s, 100s, or 1000s of TBs to many locations.

To exacerbate the problem, shared storage that’s fast enough to keep up with the pipeline is usually in heavy rack mount enclosures designed for datacenters. These are difficult to transport into the field, and too noisy to be in proximity to people. On the other hand, old school portable storage like a USB/Firewire/Thunderbolt array, is often difficult to share and prone to reliability problems, especially in capacities between 40TB to 200TB.

The WARP 26200-H portable unified storage appliance optimizes both performance and cost in a compact form factor. Backed by powerful, commercially-supported software and enhanced management tools, they offer unmatched performance and scale in a package that fits under a desk. It provides the flexibility of custom storage with the ease of use of an appliance, and is ideally suited for HD media storage, processing, and distribution.

Capacity & Performance

The WARP 26200 can hold up to 60TB inside the cube enclosure itself. This is not “large” by modern datacenter standards, but is sufficient for the majority of field work and departmental storage use cases. Growing beyond the internal capacity is a simple matter of attaching an expansion JBOD to the SAS ports. In this way, a single appliance can seamlessly grow to over 200TB. Using WARPware management tools, a few simple commands fold new drives into the existing system in real time. This is faster, less costly, and less complex than other approaches.

JMJ_4255In addition to scalability, most legacy systems struggle to handle mixed workloads without complex tuning or exorbitant costs in hardware. Legacy systems have only a few gigabytes of cache, making it impossible to cache a complete working data set. To work around this, OEMs can support SSD options, but these solutions inevitably carry exorbitant markups and license fees. WARP Mechanics never charges extra to activate the hardware customers already paid for, which finally makes hybrid solutions affordable.

For latency-sensitive IO like capture and playback, the WARP 26200-H supports hybrid pools integrated with up to 4TB of high-performance SSD, or can use NVRAM modules to accelerate synchronous writes. These components add the benefit of memory-speed storage without the hassle of administering hierarchical pools: WARPware manages data automatically, caching an entire working set if needed.

Integrity

WARP Mechanics solutions use enterprise, RAID protected storage with single-, double-, or triple-parity. However, WARPware goes beyond traditional RAID. There are failure cases that corrupt data with legacy RAID, like write holes, silent data corruption, bit rot, and multiple simultaneous disk failures. Therefore, WARPware includes multi-layered protection for every block of data at every phase. The system uses ECC memory, network checksums, RAID parity, and block-level checksums, so if a “bit flip” occurs at any stage for any reason, it will be detected and corrected through background integrity checks.

Solution

The WARP 26200-H HybridCube appliances combine the best software with enterprise-grade hardware in a small form factor. This allows it to supply the highest levels of throughput and protection in a portable package, without breaking the bank – or your back. Configured as a WARPnas “unified storage” system, the 26200-H makes shared access easy through standard protocols such as NFS, SMB/CIFS, iSCSI, and FTP. Each platform comes with two embedded 1Gb Ethernet ports, one IPMI port for out-of-band management, external SAS ports, and additional interface options such as 10GbE, 40GbE, and 56Gb FDR Infiniband.

Value

  • Purpose-built appliance architecture – Guarantees compatibility, reliability, and ongoing manageability with all components pre-configured and tested.
  • Light Weight & Small Footprint – optimized for smaller locations without full scale data centers
  • Designed for performance – high throughput at both network and filesystem levels.
  • Multiple data protection mechanisms – Data is protected with integrity features to eliminate “write holes,” “silent corruption,” “bit rot,” and more.