GUILTY
OPTIMIZATION_

Pong begins at the other end.

The first surprise is that Atari’s 1972 Pong was already a hardware game. Al Alcorn built the arcade machine from logic circuitry rather than a CPU executing a game program. Counters and gates generated the moving objects, scoring and television timing. In his oral history, Alcorn describes combining digital TTL logic with the analog world of video. There was no software renderer waiting to become a graphics chip. [1]

That is more than a historical footnote. A circuit designed for one game can be wonderfully efficient, but its rules are difficult to replace. The next great step was partly away from fixed hardware: move game behavior into software, then retain a small collection of hardware functions that many games can reuse. The trade is flexibility against the cost of generality.

The Atari 2600: hardware objects, software choreography

The 1977 Atari VCS/2600 combined a programmable CPU with the Television Interface Adaptor. The TIA offered a playfield, two player graphics, two missiles and a ball. It did not offer a conventional full-frame bitmap that a game could leisurely draw and hand over. The program supplied data in time with the television raster; Atari’s guide describes 76 CPU cycles per scanline. Changing registers as the beam progressed let programmers reuse the limited display machinery. [2]

The hardware primitives were simple. The ingenuity lay in scheduling them. A programmer could trade time, vertical position and carefully timed updates for the appearance of more elaborate scenes. When later display systems stored more of the picture and handled more composition themselves, much of that cycle counting stopped being the central problem.

Pixel pushing becomes a service.

Imagine animating an object in a bitmap entirely with the CPU: restore its old background, combine its visible pixels with the new background, write the result, then repeat. Masking, shifting and copying consume instructions that cannot be spent on the game itself. It is a natural place for specialization because the operations are repetitive and their meaning is stable.

The Amiga’s blitter made this concrete. Commodore’s manual describes a coprocessor in Agnus that moves rectangular memory regions, combines up to three sources through logical operations, shifts data, fills areas and draws lines. Software specifies the job; the blitter performs the repeated work. This is a direct example of a software loop becoming a hardware facility. [3]

The optimization did not vanish so much as change shape. Once a fast copying engine exists, the programmer cares about its setup costs, memory arrangement and contention with other users of memory. The old question was “How cheaply can I copy a pixel?” The new one is “How do I organize the frame so this engine stays productive?” That second question resembles GPU programming decades later.

Carmack’s scrolling demo: native somewhere else is not native here

John Romero remembers a September 1990 morning when he ran a demo made overnight by John Carmack and Tom Hall. It recreated a Mario-style scrolling scene using Dangerous Dave on an EGA PC. Carmack had been working with EGA panning registers; Romero saw an immediate opening for a different kind of PC game business. His account places the moment at the beginning of the group’s path toward id Software. [4]

This is a crucial qualification to the “just wait” argument. Smooth scrolling was a familiar console experience, yet making it work on the PC market they could reach was valuable. Availability on a technically superior machine does not eliminate a constraint on your customers’ machines. A bridge can matter enormously even if another shore already exists.

Star Fox: buy the missing capability in the cartridge

Nintendo and Argonaut’s 1993 Star Fox used a Super FX chip inside the Super NES cartridge to support its polygon graphics. Nintendo’s developers later described shaping ships so their direction remained readable with very few polygons. The accelerator enlarged the possible design space, but the art still had to be built around its limits. [5]

Here the response to an inadequate platform was to ship additional hardware with the software. It is a useful intermediate case: a game can reveal sufficient demand for an operation before that operation belongs in every machine. The later mainstreaming of 3D hardware should not erase the value of making a compelling 3D game earlier.

Before accelerating the answer,
simplify the question.

Early first-person engines did not begin with an unrestricted world and simply draw it faster. They chose worlds that were cheap to draw. Wolfenstein 3D’s grid-based walls made it possible to cast through a regular map and draw wall columns efficiently; its enemies and objects were scaled sprites. Those choices are visible in id’s released source. [6]

A grid is an algorithmic advantage and a design restriction at the same time. It reduces the renderer’s problem but also influences the rooms the designer can make. One of hardware progress’s largest payoffs is therefore easy to miss: it removes restrictions on the game world, not merely instructions from the renderer.

Doom made a richer bargain. Its BSP tree organized space so the engine could visit relevant geometry in a useful order and clip covered wall ranges. The released code makes the division explicit: determine what can contribute to the image before spending time drawing it. This is work avoidance, a fundamentally different strategy from doing all the work on a faster chip. [8]

Doom’s release notes also make clear that the renderer was not a general answer to every possible 3D scene: Carmack listed slopes and looking up or down among the features future programmers might add. The famous restrictions were part of an effective solution for that game and that machine budget. [7]

“you probably wouldn’t even need to be clever”

In that same note, Carmack suggested a fast Pentium plus 3dfx could draw the whole old level at reasonable speed. It is almost a laboratory statement of McNeill’s thesis: the carefully minimized workload of one generation could become small enough for a later system to attack more directly. It is also explicitly a proposal about Doom-sized content, not a promise that visibility algorithms would become irrelevant forever. [7]

Hold the scene constant and hardware makes the old engineering look extravagant. Let the scene grow and the reason for avoiding work returns. This distinction explains much of the next thirty years. The point of a new generation is rarely to draw exactly the same corridor with less effort. Developers spend the headroom on denser geometry, more lights, more objects, wider views or higher resolution.

Quake’s beautiful inner loops meet 3dfx.

Texture mapping asks which point on a surface image corresponds to each screen pixel. Perspective makes the answer nonlinear: a simple linear walk through texture coordinates distorts a receding surface. Chris Hecker’s 1995 tutorial laid out the mathematics for programmers who were implementing this machinery themselves. Today, that very requirement often arrives as an assumed property of a rendering pipeline. [9]

Abrash’s Quake postmortem describes calculating perspective-correct texture coordinates at intervals of up to 16 pixels, interpolating between them and overlapping the expensive floating-point divide with integer drawing work. It also describes caching surfaces with their lighting already combined, so repeated drawing could reuse the result. These were carefully chosen approximations and reuse strategies, rather than gratuitous complexity. [10]

Now compare 3dfx’s period programming guide. Voodoo Graphics supplied texture-mapping hardware with perspective correction and bilinear filtering; the guide said those features did not impose an extra performance penalty relative to the corresponding simpler sampling modes. That is a statement about that hardware pipeline, not a claim that texture access or rendering was free. [11]

For an engineer who had spent weeks scheduling divides, that difference is profound. The CPU approximation exists to make an expensive operation affordable. The dedicated unit changes the relative prices. Preserving the clever approximation can become less useful than feeding the new unit well.

GLQuake changed the architecture, not just the processor

Abrash describes GLQuake dispensing with Quake’s edge-list machinery, drawing polygons from the potentially visible set and using texture and lightmap passes instead of the software surface-cache approach. He reports that an initial working GLQuake took a weekend. Carmack’s January 22, 1997 log announces the preliminary public release and accompanying 3dfx driver. A working port and a production-ready ecosystem are different milestones. [10] [12]

This is the strongest kind of “guilty optimization”: not a 10% improvement to the old loop, but the disappearance of the reason for that loop. Yet the engine still needs a scene representation, visibility decisions, asset processing, gameplay and tools. The accelerator commoditizes one layer while making investment above it more productive.

There is also an important scheduling lesson in Abrash’s retrospective. He thought his 1996 recommendation to target hardware-only rendering had been about a year early. Hardware arrival and installed-base readiness did not coincide. The history contains both rapid technical obsolescence and slower commercial adoption. [10]

A filtering trick with an expiry date.

Tim Sweeney offered a wonderfully specific example in a 2000 technical answer about Unreal’s software renderer. It perturbed texture coordinates using an ordered 2×2 pattern before selecting a texel. At sufficient resolution, he explained, the result could resemble bilinear filtering. It spread sampling choices spatially without calculating a full blend of neighboring texels for every output pixel. [13]

The effect illustrates a general bargain: tolerate a carefully structured error to avoid an expensive operation. Once a hardware texture sampler makes the better operation routine, this particular reason for the approximation weakens. The idea of exploiting perception does not disappear; it will return in level of detail, temporal reconstruction and neural upscaling.

1999: move the front of the pipeline too

Early PC 3D accelerators chiefly took on the work of turning prepared primitives into pixels. NVIDIA’s GeForce 256 made hardware transform and lighting a major consumer-PC milestone in 1999: converting vertices between coordinate systems and evaluating a prescribed lighting model could move off the host CPU. NVIDIA’s “first GPU” branding should not be confused with the invention of graphics acceleration, which plainly predates it. [14]

But a hardwired lighting equation can become a new constraint. What if the material needs a different equation, the surface deforms in an unusual way, or the artist wants a stylized effect? The next step could not simply be to put every proposed effect into a new circuit.

The 2001 generation, including GeForce3, introduced programmable vertex processing, with more constrained early pixel-stage capabilities. The fuller floating-point shader generation followed. NVIDIA’s own period tutorial distinguishes these stages rather than treating early “programmable” GPUs as equivalent to modern general compute. [15]

Doom 3, Far Cry and the next stack of tricks.

Once shaders become practical, the goal moves from drawing textured surfaces to controlling how those surfaces respond to light. Hardware capability and software invention become tightly coupled. A new primitive can unlock an effect while leaving most of its implementation to the engine.

Doom 3: a stencil buffer is not a shadow engine

The shadow-volume approach associated with Doom 3 uses depth and stencil operations to determine which visible pixels are shadowed. The renderer constructs and processes shadow volumes and then applies lighting accordingly. McGuire’s contemporary explanation shows the engineering surrounding the hardware operations. A GPU’s ability to increment or decrement a stencil value is useful machinery; it does not automatically determine the shadowing strategy for a game. [16]

This distinction matters because “hardware shadows” can sound like the hardware understands lights, occluders and artistic intent. It actually offers operations that software assembles into a solution. The visible effect belongs to the composition.

Far Cry: a feature arrives, but its first implementation is awkward

Carsten Wenzel’s GDC 2005 talk explains why Crytek avoided dynamic loops and branching for its Far Cry lighting path: practical restrictions and costs favored static branches and unrolled loops. The resulting shader combinations needed caching. His slide gives about 430 MB for the Far Cry 1.3 shader cache, compressed to approximately 23 MB in the patch executable. [17]

The same talk describes hardware-supported instancing for vegetation and an HDR path using an RGBA render target with 16-bit floating-point channels. That path needed floating-point blending and filtering, and still performed bloom, luminance adaptation and tone mapping. The talk notes no full-scene anti-aliasing on that path and explains that HDR lightmaps and skyboxes were prototyped but excluded from the patch because of bandwidth, compression, package-size and testing constraints. [17]

GeForce 6’s architecture documentation helps separate two often-conflated changes: Shader Model 3.0 expanded programming capabilities, while floating-point render-target, filtering and blending support enabled particular HDR implementations. “SM3 gives you HDR” is too imprecise; the individual capabilities and chosen rendering path matter. This was high-dynamic-range scene computation, tone-mapped for display, not a requirement for today’s HDR monitors. [18]

The larger lesson is that a nominal feature can precede an economical implementation. You can support a branch instruction without making every branching workload fast. You can support a floating-point buffer without solving bandwidth or fitting revised art into a downloadable patch. The workarounds die at different times because they answer different constraints.

Crysis: the approximation comes back at a higher level

Mittring’s 2007 CryEngine 2 notes describe screen-space ambient occlusion: estimate nearby occlusion from the depth image, rather than solve all light transport through the full scene. It adds local contact and crevice cues efficiently, but the screen-space representation has incomplete information. This is a new shortcut made practical by programmable graphics hardware, not a dedicated Crysis circuit. [19]

Far Cry’s more capable hardware did not end the need for approximation. It helped move the frontier to a richer problem. At fixed visual ambition, the engineering gets easier. At the frontier, developers usually spend the newly available budget.

Not every good algorithm deserves its own chip.

The next decade is best understood as a collection of boundary decisions. Some tightly specified work becomes fixed hardware. Some moves to general GPU execution. Some becomes an engine service. The distinctions are more informative than a single story of inevitable hardwiring.

Tessellation: hardware manufactures samples, software gives them meaning

Direct3D 11’s tessellation pipeline is unusually explicit. A programmable hull shader specifies patch information and tessellation factors. A fixed-function tessellator generates a subdivision pattern. A programmable domain shader evaluates the surface at the generated locations. The hardware accelerates a stable combinatorial operation while leaving the surface definition and level-of-detail policy programmable. [20]

This is not a universal button for “more detail.” If the engine selects too much subdivision, spends too much bandwidth on displacement data or creates tiny triangles inefficiently, it can still lose. A fast primitive can amplify a poor policy as readily as a good one. The lasting skill is knowing which samples are worth producing.

Virtual texturing: address translation is not asset management

Consider a world whose texture data exceeds graphics memory. A software system can split the data into pages, retain a working set and translate a requested virtual location into a resident physical one. Khronos’s 2013 ARB_sparse_texture extension exposes a limited form of this virtualization by separating address-space reservation from physical backing. [21]

That facility cannot decide what the player will look at next, how aggressively to prefetch, which texture deserves higher resolution or how to balance storage against memory. Those decisions remain the engine’s responsibility. Hardware assistance can simplify the mechanism without replacing the product-level policy.

Physics: an accelerator can itself be absorbed by a broader processor

The AGEIA/PhysX story offers a different trajectory. PhysX was associated with dedicated physics acceleration; NVIDIA’s 2008 technical presentation describes GPU acceleration and a software SDK spanning processors. The useful counterexample is the direction of travel: physics work can become software on a more general parallel processor rather than a permanently distinct physics device. [22]

Specialization has a price: silicon area, engineering effort, toolchains, compatibility and an installed base. A programmable processor can win if it is already present and can share its resources across many jobs. Conversely, a stable operation repeated often enough can justify its own block. Neither “everything becomes fixed-function” nor “everything becomes general compute” is a reliable law.

Ray tracing makes the pattern literal again.

A ray tracer asks what a ray hits. Testing every ray against every triangle would be wasteful, so it typically searches a hierarchy of bounding volumes and performs detailed intersection tests only where needed. Those repeated searches and tests form an attractive target for specialization.

NVIDIA’s 2018 Turing architecture introduced RT cores that accelerate bounding-volume-hierarchy traversal and ray intersection work. Its own description places these units alongside programmable shading and denoising in a hybrid pipeline. This is a strong example of narrow, recurring software work acquiring dedicated hardware support. [24]

Notice what has not been hardwired: the choice of which rays to fire; how many samples a surface deserves; what a material does at an intersection; how to reuse results over time; and what level of error is acceptable. Accelerating intersection tests lowers the price of asking a question. The renderer must still decide which questions matter.

Lumen makes the remaining complexity visible

Epic’s Lumen documentation describes screen traces, a surface cache, software ray tracing using distance-field representations and hardware ray tracing against triangles. These are complementary parts of a lighting system with different coverage and costs. The existence of hardware tracing does not eliminate all approximations or all other representations. [25]

There is no contradiction between owning ray-tracing hardware and using a shortcut. The question is whether a particular ray supplies enough information to justify its cost. A cache or screen-space result can still be the better trade for part of the image. Hybrid rendering is an economic choice within a frame budget.

The historical comparison to Doom is especially useful. Its renderer spent engineering effort avoiding unnecessary pixels. A modern renderer spends engineering effort avoiding unnecessary rays, shading or temporal recomputation. The operations and representations have changed dramatically. The obligation to spend a finite budget on visible value has not.

The next chip also erases a loading loop.

Games must deliver assets to the renderer. When storage is slow, decompression may be a tolerable background task. Increase storage throughput substantially and decoding can become a significant consumer of CPU time. The bottleneck moves because one part of the system improves.

Charles Bloom’s account of PS5 provides a particularly clean software-to-hardware lineage. RAD originally designed Kraken as a fast software decompressor for modern CPUs. PS5 incorporated hardware decoding of that format. Bloom explains how reducing serial dependencies was helpful for both implementations. The same post distinguishes the decoder from Oodle Texture, an offline tool that prepares texture data to compress more effectively. [26]

The optimization did not simply disappear; part became circuitry, while part became an input-generation advantage. Once decoding is standardized, a better encoder can produce streams that make the existing decoder more useful. The hardware changes which side of the boundary deserves engineering attention.

Xbox Series X similarly combined an SSD, dedicated decompression blocks, DirectStorage and sampler feedback streaming in its 2020 architecture. Microsoft presented the combination as a coordinated I/O system. Buying a faster storage device alone is not the same intervention as rebuilding the path from stored bytes to useful game assets. [27]

On PC, Microsoft’s 2022 DirectStorage 1.1 release added GPU decompression. That is a different category of handoff: work can execute through the GPU path without implying that every compatible GPU has a PS5-like dedicated decoder for the same format. [28]

In all these cases the game still needs to determine what to request and when, construct usable runtime objects and avoid memory exhaustion. Faster delivery can make larger worlds feasible; larger worlds create more delivery work. The same cycle that followed texture acceleration reappears in storage.

Sweeney’s reversal: software returns inside the GPU.

In a 2008 interview, Sweeney argued that increasingly programmable processors would loosen the grip of the traditional fixed raster pipeline. The interview also revisited his earlier prediction of a return to CPU rendering. That specific destination and timetable were not what the subsequent mainstream GPU path delivered. His deeper concern—developers needing freedom to choose a rendering architecture—proved more durable. [23]

Nanite provides a concrete way to separate the useful insight from the overextended forecast. In Epic’s 2021 presentation, small triangles can be rasterized by software running on GPU compute, while larger triangles use the hardware rasterizer. Both paths contribute to a visibility buffer, with atomic operations helping resolve the visible result. Epic’s implementation addresses workloads for which the conventional triangle pipeline is an awkward fit. [29]

This is not a return to drawing a whole game on a 1990s-style host CPU. It is software using a powerful parallel GPU to bypass part of that GPU’s fixed-function machinery. The old boundary is redrawn inside the chip.

The apparent paradox disappears once you remember that dedicated hardware embodies assumptions. A rasterizer is excellent for the workload it was designed to process. Change the distribution of triangle sizes, the representation of visibility or the overhead per primitive, and an unconventional software path can become competitive. The hardware is not slow in the abstract; its organization may be mismatched to the new job.

That also explains why Nanite is a valuable engine capability rather than proof that engine engineering is finished. Artists can experience a simpler workflow precisely because a complex system manages geometry, streaming, culling and rasterization on their behalf. “Native to Unreal” describes an abstraction available to the customer. It does not mean the graphics card contains a Nanite circuit.

From hand-tuned sampling to neural reconstruction.

The newest turn is especially relevant to McNeill’s AI analogy. Instead of explicitly drawing every final output sample in the most expensive way, a system can reconstruct an image from partial rendering information and history. The goal is still to allocate computation where it produces the most perceptual value.

NVIDIA’s 2025 DLSS 4 account describes transformer-based super resolution and ray reconstruction, alongside frame-generation improvements. Its developer documentation, checked in October 2026, lists the DLSS 4.5 plugin package and the distinct feature families. These are related technologies with different inputs and purposes, not one interchangeable “AI graphics” switch. [30] [31]

This is co-design across several layers. Specialized arithmetic makes neural computation efficient; trained models implement reconstruction behavior; engine integration supplies the information and timing the models need. A trained model is still software, even when it relies on accelerator hardware. The whole effect has not become immutable circuitry.

Sweeney’s texture-coordinate dither and modern reconstruction share an abstract goal—avoid expensive work while preserving a convincing image—but differ enormously in inputs, computation and scope. The useful continuity is the trade between work performed and error perceived, not a claim that one technique directly evolved into the other.

The same caution applies to generated frames. Increasing the number of displayed images is not automatically equivalent to simulating game state or sampling player input at that higher frequency. Presentation smoothness, response latency and simulation rate are separate requirements. A new technique can improve one while leaving another bottleneck intact.

What died. What moved. What stayed.

This is a summary of the mechanisms developed above, not a claim that each transition happened everywhere in a single product cycle.

Temporary can still be the right investment.

The historical error would be to conclude that the disappearing work was wasted. Quake needed to ship before its hardware-only future was ready. Far Cry needed to run on actual driver and GPU combinations. A game released years earlier can establish a team, a toolchain, a community and a market. The useful life of the implementation is only one part of the return.

Here is a more practical decision framework, inferred from the cases rather than asserted as a universal law:

1. Name the expiring constraint.Is the workaround for missing hardware functionality, an immature API, insufficient memory, expensive content production or a genuinely hard inference problem? Only some of those disappear when a chip gets faster.
2. Use the customer’s machine as the clock.A capability demonstrated at a conference is not yet a safe minimum requirement. Installed base, price, power, drivers and console generations determine when you can remove the fallback.
3. Count the value of arriving early.What can the bridge enable before its replacement is usable? A temporary implementation can earn lasting customers, learning or content. Those assets need not expire with the code.
4. Keep the replaceable part replaceable.Separate the product contract from the trick that currently fulfills it. If a sampler, model or API changes, preserve the interface and evaluation process while replacing the implementation.
5. Check whether the target is moving.Will the team keep the same workload and enjoy the savings, or consume the headroom with a larger world? Optimize against the intended product, not an imaginary frozen benchmark.

Nothing in that comparison requires a permanent technical moat. The relevant issue is whether the work earns its keep during its life and leaves behind something useful. An approximation is a problem when its cost and complexity outlive the constraint that justified it.

What has looked evergreen?

Across these transitions, several responsibilities recur: represent the world well; avoid work that does not affect the result; organize memory and movement; make content creation efficient; measure the player-visible outcome; and choose the right compromises. The exact implementations change. These responsibilities are durable because they connect the machine’s finite resources to an expanding human objective.

By contrast, manually scheduling around one processor’s divide latency, emulating one missing filter or maintaining permutations around one generation’s shader limitations is more obviously perishable. It can be excellent engineering and still be a poor place to expect permanent exclusivity.

The implication for engines and platforms

For an engine business, losing exclusivity in a rendering technique need not imply losing relevance. If a new chip makes a feature cheaper, more developers may want to use it. The engine’s opportunity is to make it reliable, integrate it with the rest of production and keep customers from having to redo the migration themselves.

That is a conditional business inference, not evidence that every engine vendor wins. A platform that cannot translate technical transitions into a better developer experience can lose its position. The defensible asset may be the ability to absorb change, rather than any particular implementation preserved in the source tree.