The conventional wisdom in video streaming has been straightforward: better quality requires better codecs. AV1 promised 30-40% bandwidth savings over H.264, and the industry has invested billions in making it a reality. GPU manufacturers added hardware support, browsers implemented decoders, and platforms began migrations. Yet in 2026, a different approach is reshaping video optimization — one that challenges the codec-first mentality.
AI-powered upscaling has evolved from a post-production tool into a real-time optimization technique, changing the economics of live video. Instead of pushing for incremental codec improvements that take years to standardize and deploy, platforms can achieve better results by transmitting lower-resolution video and reconstructing quality on the receiving end.
The Codec Adoption Problem
AV1 hardware encoding arrived in 2022 with Intel’s Arc GPUs, followed by NVIDIA and AMD. Four years later, adoption remains incomplete. Apple’s M-series processors still lack hardware AV1 encoding. Many mobile devices decode AV1 but cannot encode efficiently enough for live applications.
The quality improvements are real but come with significant tradeoffs. CPU-based AV1 encoding delivers excellent compression but runs too slowly for real-time streaming. GPU-based AV1 encoding achieves real-time performance but sacrifices quality, reducing the compression advantage. Development complexity multiplies as platforms must support H.264 for compatibility, HEVC where available, and AV1 for cutting-edge implementations.
The standardization cycle spans years. H.266/VVC was finalized in 2020 but remains virtually absent from production platforms in 2026 due to hardware and licensing challenges. By the time a new codec achieves broad support, the industry has invested billions over 5-7 years for typically 30-40% bandwidth reduction.
AI Upscaling Changes the Equation
Modern AI upscaling operates in real-time at 30-60 frames per second on consumer hardware. NVIDIA’s RTX Video technology upscales to 4K in real-time while sharpening edges and removing compression artifacts. AMD’s REAPPEAR delivers similar capabilities on Ryzen AI platforms, making real-time upscaling available across major GPU ecosystems.
The economics are compelling. Transmitting 720p instead of 1080p reduces bandwidth by approximately 55%. The processing cost of AI upscaling on the receiving device is significantly lower than encoding higher-resolution video on the sending side. This asymmetry fundamentally changes the optimization equation.
Production deployments demonstrate measurable results. Testing with SimaBit’s preprocessing engine showed 22% average bitrate reduction while achieving a 4.2-point increase in VMAF quality scores. Platforms serving one petabyte monthly could save approximately 220 terabytes through AI preprocessing alone.
Perhaps most importantly, AI upscaling deploys incrementally without infrastructure replacement. Platforms can enable upscaling for some users or streams without migrating entire video pipelines — a stark contrast to codec migrations requiring coordinated updates across encoders, media servers, CDNs, and clients.
The Asymmetric Advantage
The fundamental advantage becomes clear when examining encoding versus decoding costs. In a live streaming scenario with one broadcaster and thousands of viewers, encoding happens once but decoding happens thousands of times simultaneously. Investing computational resources in sophisticated encoding creates a bottleneck at the single point where latency is most critical.
AI upscaling inverts this cost structure. The sender encodes at lower resolution using fast, efficient encoding, typically H.264 at 720p or 540p. This encoding is computationally cheap and introduces minimal latency. Receivers independently upscale using local GPU resources. Because upscaling is frame-independent, GPU performance excels. Modern GPUs can upscale video far exceeding real-time playback speed.
This architectural shift distributes computational load from the single encoding point to thousands of decoding endpoints. Each viewer contributes their own GPU resources to enhance their experience, removing the bottleneck at the encoder.
| Optimization Approach | Encode Complexity | Decode Complexity | Latency Impact | Bandwidth Savings | Quality Result |
| H.264 baseline | Low | Low | Minimal | Baseline | Baseline |
| AV1 GPU encode | Medium | Medium | Low (+20-40ms) | 30-35% | Good |
| AV1 CPU encode | Very High | Medium | High (+100-300ms) | 35-40% | Excellent |
| AI upscale 720p→1080p | Low | Medium | Minimal (+5-20ms) | 45-55% | Excellent |
| AI upscale 540p→1080p | Very Low | Medium-High | Low (+15-30ms) | 60-70% | Very Good |
GPU Usage: Untapped Capacity
Modern GPUs sit largely idle during video playback. Decoding a 1080p H.264 stream typically consumes 5-15% of available GPU capacity. Even 4K HEVC decoding rarely exceeds 30% utilization on GPUs from the past three years. This represents massive untapped computational capacity.
AI upscaling leverages this idle capacity productively. Real-time upscaling from 720p to 1080p consumes 20-40% GPU capacity on mid-range GPUs — well below levels that would cause thermal throttling or impact other applications.
The efficiency gains are architectural. GPUs excel at matrix operations required for neural network inference. Tensor cores, now standard across modern GPUs, accelerate these operations dramatically. These specialized compute units sit unused during traditional video decode but become highly effective for AI upscaling.
Hardware manufacturers are accelerating this trend. NVIDIA’s RTX Video integrates upscaling directly into the video decode pipeline, allowing applications to request upscaled output without implementing custom ML inference. This keeps video data in GPU memory throughout the decode-upscale-display pipeline.
The contrast with encoding is stark. Encoding requires sequential processing — each frame depends on previous frames for motion estimation. This limits parallelization and makes encoding CPU-bound unless using specialized GPU hardware. Upscaling processes frames independently, making it perfectly suited to GPU acceleration.
Latency: Where Milliseconds Matter
For live applications, latency determines usability. Video conferencing becomes frustrating above 300-400ms. Live auctions fail with delays exceeding 500ms. Any optimization increasing latency must justify that cost with substantial benefits.
GPU-based hardware encoding adds minimal latency — typically 20-40ms for AV1 versus H.264. However, achieving full quality potential often requires CPU encoding with slower presets, adding 100-300ms of encoding latency. This often exceeds acceptable thresholds for live streaming.
AI upscaling’s latency impact depends on implementation. Real-time upscaling typically adds 15-30ms on modern GPUs. When integrated into the display pipeline — processing frames as they’re rendered — latency impact becomes negligible. NVIDIA’s RTX Video adds only 5-10ms because upscaling happens in parallel with frame composition.
The calculation shifts further when considering total system latency. Lower-resolution transmission reduces network transmission time. A 720p stream has 55% less data than 1080p, proportionally reducing transmission time. On bandwidth-constrained connections, this can save 50-150ms of network latency — more than offsetting upscaling processing time.
Real-World Latency Comparison
Three implementations of a 1080p live video call:
- Traditional H.264: Capture at 1080p (20ms encode) → transmit (120ms) → decode (15ms) → display (10ms) = 165ms total
- AV1 Quality-Focused: Capture at 1080p (180ms CPU encode) → transmit with 35% reduction (78ms) → decode (25ms) → display (10ms) = 293ms total
- AI Upscaling: Capture at 720p (18ms encode) → transmit (54ms) → decode (12ms) → upscale (20ms) → display (10ms) = 114ms total
The AI upscaling approach delivers the lowest latency while achieving bandwidth savings comparable to or better than AV1.
The Quality Paradox
Extensive testing reveals surprising results: 720p streams upscaled using modern AI models often appear sharper and cleaner than native 1080p streams compressed at the same total bitrate.
The explanation lies in compression artifact interaction. When encoding 1080p at constrained bitrates, encoders make tradeoffs — reducing frame rate, introducing blockiness, or compromising color accuracy. These artifacts become more visible on larger displays.
A 720p stream encoded at the same bitrate maintains higher quality per pixel because there are fewer pixels to compress. AI upscaling reconstructs missing detail based on learned patterns from high-quality video training. Modern upscaling networks recognize and eliminate compression artifacts while enhancing detail. The result preserves the clean 720p encoding while adding plausible high-frequency detail.
Production data confirms these findings. The 22% bandwidth reduction with simultaneous 4.2-point VMAF quality increase demonstrates that optimized lower-resolution encoding combined with AI upscaling outperforms higher-resolution encoding at matched bandwidth. Subjective testing with over 3,700 participants validated these metrics — viewers consistently preferred AI-upscaled streams.
Implementation Considerations
Deploying AI upscaling in production requires attention to practical details:
Model Selection: Production implementations use lighter models optimized for speed rather than research-focused models. SimaUpscale delivers 2x to 4x upscaling in real-time and integrates with all major codecs. NVIDIA RTX Video provides hardware-accelerated upscaling integrated into GPU drivers. AMD REAPPEAR offers edge-optimized upscaling for Ryzen AI platforms.
Fallback Strategies: Not all clients support AI upscaling. Older devices or browsers without WebGPU support may lack computational resources. Production implementations require:
- Client capability detection during session initialization
- Adaptive quality ladders with different encodes for AI-capable versus traditional clients
- Hybrid approaches with partial upscaling for intermediate devices
Quality Monitoring: AI upscaling quality varies by content type. Sports content upscales differently from video conferences. Continuous quality monitoring should include:
- VMAF or similar perceptual quality metrics in client feedback loops
- User satisfaction data correlated with upscaling parameters
- GPU utilization and thermal throttling monitoring
- Tracking of upscaling-related playback failures
Partnering for Implementation
Successfully implementing AI upscaling requires expertise across real-time video streaming architectures, machine learning model optimization, GPU acceleration and hardware integration, and quality measurement. These capabilities rarely exist within a single organization.
Trembit brings comprehensive expertise in building advanced video streaming platforms leveraging cutting-edge optimization techniques. The team has deep experience in WebRTC implementations, codec optimization, and AI-enhanced video processing. Trembit understands both the theoretical advantages and practical challenges of deploying these technologies at scale.
The approach emphasizes practical results, delivering measurable improvements in bandwidth costs, video quality, and user experience rather than chasing theoretical benchmarks. Whether building new platforms or optimizing existing deployments, Trembit helps navigate the rapidly evolving landscape of video optimization technologies.
What Comes Next
The question is no longer whether AI upscaling will become standard — it’s how quickly adoption will occur. The technology has moved from experimental to production-ready. Hardware support is widespread. The economics are compelling.
Looking forward, the distinction between codec optimization and AI enhancement will blur. Future video pipelines will likely employ both efficient modern codecs for transmission, combined with AI-based enhancement on the receiving end. This hybrid approach captures the benefits of both paradigms.
The industry is also seeing the emergence of AI-aware codecs. Research into neural video compression using AI models for both encoding and decoding promises even greater efficiency. However, these technologies remain years from standardization. In the meantime, AI upscaling provides immediate benefits using existing infrastructure.
For organizations operating live video platforms in 2026, the strategic question is not whether to adopt AI upscaling but how to integrate it effectively. Platforms that successfully deploy AI enhancement will enjoy significant competitive advantages: lower bandwidth costs, better video quality, improved experience on constrained connections, and reduced latency compared to codec-heavy approaches.
The codec wars aren’t over — AV1 will continue gaining adoption, and H.266/VVC will eventually arrive. But the industry’s center of gravity is shifting. The future of video optimization is not just about compressing more efficiently — it’s about transmitting smarter and reconstructing better. AI upscaling represents this shift from pure codec optimization to intelligent, hybrid approaches that leverage both compression and enhancement.
Building a next-generation video platform? Trembit’s team has extensive experience implementing AI upscaling, advanced codecs, and hybrid optimization strategies for live video applications. Reach out to discuss video optimization challenges and learn how to deliver better quality at lower cost while maintaining the low latency that live applications demand.
