Edge Compute Silicon — TOPS-Watt Economics

Executive summary

Physical AI—robotics, autonomous machines, automated warehouses, field robots, drones, and embodied AI—moves inference and increasingly training/fine-tuning to the edge. The constraint is not just peak INT8/INT4 TOPS; it is the energy per unit of useful compute, typically expressed as TOPS/Watt but modulated by memory bandwidth, utilization, quantization, sparsity, and power-management headroom.

Public silicon vendors monetize this as modules/SoCs (NVIDIA Jetson, Qualcomm Robotics/Snapdragon, AMD Versal AI Edge, Intel OpenVINO/Movidius, TI Sitara, Apple Neural Engine, Alphabet Coral Edge TPU) rather than selling TOPS/W as a standalone subscription. The economics are classic semiconductor hardware economics: high non-recurring engineering costs recovered across die sales, with design ecosystem lock-in and software tooling as a competitive moat.

The best public evidence is not a marketing TOPS number. Peer-reviewed work such as the MIT Eyeriss row-stationary dataflow chip showed years ago that data movement energy, not arithmetic, often dominates inference energy; compressed-inference work such as EIE showed that pruning and quantization can shift the energy-per-useful-inference curve. Those arguments remain central to edge physical AI economics.

Technology primer: What “TOPS/Watt” does and does not capture

For edge AI silicon, vendors quote peak INT8 TOPS (tera-operations per second, \(10^{12}\) operations/s). Power is usually expressed as module TDP or typical board power. The derived efficiency metric is:

\[

\eta = \frac{\text{effective INT8 throughput}}{\text{wall power}}

\]

A typical advertised example: NVIDIA’s Jetson AGX Orin is rated at up to 275 TOPS in the Jetson Orin family, with the AGX Orin module power envelope commonly cited around 60 W depending on configuration NVIDIA Jetson Orin product page. That gives a nominal vendor-derived peak ratio of \(275/60 \approx 4.6\) TOPS/W. This is a derived figure from vendor self-reported inputs; it is not an independently measured sustained workload figure.

Hailo-8, a dedicated edge inference processor, is advertised at 26 TOPS at 2.5 W typical, giving \(26/2.5 = 10.4\) TOPS/W Hailo-8 product page. This illustrates why purpose-built ASICs often beat GPU-style SoCs on raw efficiency for narrow workloads.

But the metric is incomplete. TOPS/W hides:

• DRAM/off-chip energy — moving weights and activations can dominate. The Eyeriss paper specifically argued that a row-stationary dataflow reduces off-chip DRAM accesses and improves measured energy efficiency Eyeriss JSSC 2017, DOI 10.1109/JSSC.2016.2616357.
• Utilization — peak TOPS assumes all multiply-accumulate units are busy. Real workloads rarely achieve this.
• Quantization — INT4 can double or quadruple advertised TOPS, but not every workload holds accuracy at low precision.
• Precision and sparsity — unstructured sparsity often requires metadata and custom dataflow; it is not free.
• Thermal throttling — peak TOPS/W at burst is different from sustained TOPS/W under fanless or industrial temperature constraints.

The EIE work, specifically, demonstrated that deep compression plus a specialized inference engine could reduce energy per inference by exploiting weight sharing, pruning, and Huffman coding EIE arXiv:1602.01528. This is a preprint source; it is useful for architecture context but should be labeled as such and not treated as a journal peer-reviewed result.

Physical AI workload map

Physical AI differs from phone/cloud inference because the machine must perceive, plan, and act in real time. Typical edge workloads include:

• Perception: camera/LiDAR/radar fusion, object detection, occupancy networks, segmentation.
• Localization and mapping: visual-inertial odometry, SLAM, scene representation.
• Manipulation and control: grasp detection, inverse kinematics, whole-body MPC, learned policies.
• Embodied reasoning: vision-language-action models, small instruction-tuned LLMs/VLMs, world models.
• Safety isolation: deterministic control loops and fail-safe logic adjacent to the neural accelerator.

NVIDIA’s Jetson platform positions itself for “autonomous machines” with software stacks including Isaac, ROS, and DeepStream NVIDIA Jetson product page. Qualcomm’s Robotics RB5/RB6 platform targets industrial automation, AMRs, and drones with integrated AI DSP/NPU and connectivity Qualcomm Robotics RB5 product page.

The key economic insight: the buyer is not buying TOPS/W in isolation. They buy deployable throughput per dollar per watt under their workload and safety envelope. That explains why ecosystems, safety certifications, sensor I/O, and model toolchains command pricing power.

Architecture landscape

4.1 GPU-style SoCs with large software moats

NVIDIA Jetson Orin uses an Ampere-generation GPU with tensor cores, Cortex-A78AE CPUs, DLA, and vision accelerators NVIDIA Jetson Orin product page. The efficiency is lower than dedicated ASIC NPUs at peak, but CUDA/TensorRT/Isaac ecosystem support is the economic hook.

4.2 Heterogeneous SoCs with NPU/DSP

Qualcomm integrates Hexagon DSP/NPU, Spectra ISP, compute CPU, and GPU into robotics/automotive SoCs. Its value proposition is integration: compute, wireless, vision, and power management on one design Qualcomm Robotics RB5.

4.3 Adaptive silicon

AMD Versal AI Edge combines programmable logic with AI engines for adaptable sensor processing and low-latency control; it targets industrial vision, autonomous vehicles, and aerospace/defense AMD Versal AI Edge product page. This is a Tier 2 self-reported product claim.

4.4 Dedicated edge ASICs

Hailo-8 is an example of a dedicated inference ASIC optimized for TOPS/W: 26 TOPS at 2.5 W typical Hailo-8 product page. Google Coral Edge TPU advertises 4 TOPS at 2 W for a module, aimed at prototype and small-device inference Google Coral Edge TPU product page.

4.5 CPU/IP licensing layer

Arm’s Cortex CPUs and Ethos NPUs are licensed into many edge SoCs. This is a royalty/design-IP model rather than a chip-sale model. Arm Holdings PLC is listed as ticker ARM on Nasdaq; its Design IP underpins many edge AI designs Arm Ethos product page. This is an important structural point: the TOPS/W economics of an OEM’s final silicon depends on third-party IP licensing costs.

Quantified worked calculation: marketing TOPS/W vs derated sustained TOPS/W

Use the formula:

\[

\eta = \frac{T_{\text{INT8}}}{P_{\text{typ}}}

\]

Where \(T_{\text{INT8}}\) is vendor peak INT8 TOPS and \(P_{\text{typ}}\) is typical/configured power.

Example A: NVIDIA Jetson AGX Orin vendor nominal

\[

T = 275 \text{ TOPS},\quad P = 60 \text{ W}

\]

\[

\eta = 275 / 60 = 4.58 \text{ TOPS/W}

\]

This is derived from NVIDIA’s self-reported 275 TOPS and a 60 W nominal envelope; the source does not provide an independent sustained workload confirmation in the cited page NVIDIA Jetson Orin.

Example B: Dedicated ASIC vendor nominal

\[

T = 26 \text{ TOPS},\quad P = 2.5 \text{ W}

\]

\[

\eta = 26 / 2.5 = 10.4 \text{ TOPS/W}

\]

Source: Hailo-8 product page Hailo-8.

Sensitivity

Sensitivity analysis is essential because both numerator and denominator are vendor-rated. If Hailo-8 runs at 3.0 W under thermal load with DRAM and I/O, efficiency falls to \(26/3.0 = 8.7\) TOPS/W. If actual average throughput is 70% of peak due to memory bandwidth and scheduling, useful efficiency becomes \((26 \times 0.70)/2.5 = 7.3\) TOPS/W. If both degradation effects occur, useful efficiency is \((26 \times 0.70)/3.0 = 6.1\) TOPS/W.

This quantitatively shows why physical AI economics cannot be reduced to a peak datasheet number. In power-limited robots, the efficient frontier shifts toward dataflow-specific ASICs and quantized models only when the actual activation/weight data movement can be managed.

Parameter table

The table intentionally marks vendor TOPS and power as Tier 2, single-source, and self-reported. Independent workload-level cross-verification would require MLPerf/MLCommons results or a named lab measurement.

Patent/IP review

Granted patent requirement status: not satisfied without risking fabrication.

The user’s hard requirement asks for at least one granted patent with a verified grant number. No patent number is cited here because the identifier could not be verified against the USPTO/EPO/WIPO record in this research pass. A patent number should be retrieved via:

• USPTO Patent Public Search: [ppubs.uspto.gov
• Google Patents: [patents.google.com

The relevant patent search would focus on:

• Edge-inference accelerator dataflow: search classes such as G06N 3/063, G06N 3/084, G06F 9/50.
• Low-power neural network scheduling.
• Row-stationary or spatial-array dataflow associated with the Eyeriss lineage.

The Eyeriss paper provides a peer-reviewed Tier 1 reference, but a peer-reviewed paper is not a patent and does not establish patent moat. The distinction is material: a granted patent is evidence of recorded claims, not proof of performance or commercial viability. This note should be treated as non-compliant on the patent leg until a verified grant is inserted with exact URL, document number, assignee, grant date, and access date.

Peer-reviewed evidence: What the academic record actually says

The strongest Tier 1 technical source located is:

• Yu-Hsin Chen, Tushar Krishna, Joel S. Emer, Vivienne Sze, “Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,” IEEE Journal of Solid-State Circuits, vol. 52, no. 1, pp. 127–138, Jan. 2017. DOI: 10.1109/JSSC.2016.2616357. Accessed 2026-05-09.

The Eyeriss paper is directly relevant to edge TOPS/W because it evaluates a spatial array architecture and proposes a row-stationary dataflow to reduce energy consumed in data movement. Its central engineering claim is that dataflow design can improve energy efficiency at fixed MAC throughput.

The follow-on EIE work is a preprint and should be labeled as such:

• EIE: Efficient Inference Engine on Compressed Deep Neural Network, arXiv:1602.01528. [arxiv.org . Accessed 2026-05-09.

EIE is useful because it links compression, sparsity, and quantization to lower energy per inference, but it is not treated here as journal peer-reviewed.

Contradiction between marketing and measurement

There is a structural contradiction in this market. Datasheet TOPS/W is a clean multiplicative peak; measured inference efficiency is workload-dependent. MLCommons runs standardized inference benchmarks, but those results still depend on system implementation and power measurement methodology. The MLCommons inference benchmark exists as a structured cross-vendor measurement attempt; no single anonymized result is relied on here because specific result IDs were not independently retrieved MLCommons inference benchmarks, accessed 2026-05-09.

Company filings cross-reference: Marketing claims vs 10-K disclosures

NVIDIA’s FY2024 10-K is the core filing cross-reference. The filing’s AI/accelerated-computing narrative is bullish, but its risk factors contain the necessary caveats. Product pages cite 275 TOPS, but the 10-K does not guarantee that marketed TOPS/W will be achieved in all customer operating conditions. NVIDIA’s risk factors include reliance on third-party manufacturing and the risk of product defects or performance shortfalls. The company’s fiscal 2024 GAAP gross margin was 72.7%, which is one reason Nvidia can reinvest heavily in software and new silicon NVIDIA 10-K, CIK 0001045810, accessed 2026-05-09.

The discrepancy category to track:

• Marketing claim: “up to 275 TOPS” on Jetson AGX Orin.
• Filing reality: NVIDIA’s 10-K does not independently provide a workload-level sustained TOPS/W attestation. The metric is vendor-derived from a product page, not an audited filed financial or performance disclosure.

This does not imply the marketing claim is false; it means the high-confidence legal/financial filing and the high-confidence performance claim are not yet reconciled in a single primary source.

Qualcomm and AMD have similar structural exposure but monetize through different architectures:

• Qualcomm sells integrated Snapdragon/Robotics platforms with AI processing, connectivity, and multimedia IP Qualcomm Robotics RB5 product page, accessed 2026-05-09.
• AMD sells Versal AI Edge adaptive compute for edge AI and real-time signal processing AMD Versal AI Edge, accessed 2026-05-09.

Public filings for Qualcomm and AMD can be retrieved via SEC EDGAR company search; the individual document identifiers were not manually pulled into this note but the company-level query is source-stable:

• Qualcomm 10-K: CIK 0000804328
• AMD 10-K: CIK 0000002488

Unit economics / business model

Who monetizes the technology

• Silicon/module vendors: NVIDIA sells Jetson modules and dev kits; Qualcomm sells SoCs and reference-design platforms; AMD sells adaptive SoCs; Hailo sells dedicated ASIC modules/chips; Google Coral sells Edge TPU modules.
• IP licensors: Arm captures per-core/per-design licensing and royalty revenue for CPU/NPU IP; this is present in many edge SoC BOMs.
• Software/toolchain vendors: NVIDIA monetizes the Jetson hardware through CUDA/TensorRT/JetPack/Isaac lock-in; the disclosed economics remain primarily hardware sales, not subscription income.

Margin structure

NVIDIA’s FY2024 10-K reports GAAP gross margin of 72.7%. This is the clearest audited data point. The hardware-sale pricing model means gross margin is captured at chip/module sale; deep software moats support premium pricing over raw component cost.

Value capture

The value chain captures value in this order:

1. The silicon vendor that owns the software stack has the strongest pricing power.
2. The industrial OEM/integrator that takes a generic edge module and adds sensing, safety, fleet software, and customer workflow often captures the end-customer margin.
3. The physical AI application buyer—retailer, warehouse operator, manufacturer, farmer—captures labor productivity, uptime, or throughput value.

The buyer’s decision is rarely “who has the most TOPS/W.” It is more often “which vendor’s platform reaches deployable performance, certified reliability, and software integration at the lowest total cost of ownership.”

Disconfirming Evidence

The thesis that edge compute silicon TOPS/W economics will create durable physical-AI silicon franchises would be wrong, or at least materially weakened, if:

1. The TOPS/W frontier stalls. If independent benchmark or peer-reviewed data show measured sustained TOPS/W no longer improves despite lithography and dataflow gains, the hardware differentiation story collapses below a threshold needed for robot payback.
2. Memory/DRAM power dominates irreducibly. If sparsity, activation compression, and on-chip data reuse cannot prevent off-chip access from consuming most inference energy, then raw TOPS/W improvements will not translate into deployable runtime efficiency. The Eyeriss data-movement concern is the early-warning evidence Eyeriss DOI.
3. Physical AI demand shifts to cloud/remote connectivity. If robots become thin clients and safety-critical inference moves to datacenter GPUs over low-latency networks, edge silicon revenue would decouple from robot unit growth.
4. ASPs commoditize faster than product cost declines. If enough vendor-neutral NPU IP and reference designs enter the market, the high gross margins currently visible in NVIDIA’s 10-K would compress, breaking the unit-economics case for differentiated edge silicon NVIDIA 10-K/SEC.
5. Adoption fails in named buyer classes. If warehouse robotics, industrial automation, and field autonomy rollouts stall, the customer-class demand cited from product pages remains only positioning, not revenue.