When I started writing the SD-8516 I wanted it to have 256k of memory, because I had done a cursory review of various 80's consoles and computers and decided 256k was a charming homage to a “more advanced C128”. I now realize that even though I capped performance at 4mhz, it actually operates much faster due to some of the newer CISC-like opcodes. “Four times the MHz, four times the ram” was a nice idea, but it doesn't work out in practice due to a few factors.

1. This isn't an actual computer.
What I mean is, if you compare it to something like a SNES, the SNES had 128k of RAM, plus 64k for video and 64k for sound. Thats 256k, sure, but cartridges like “The Legend of Zelda” had 128k of ROM. It's true that Zelda famously used less than half the ROM on the cart, but most games actually came with far more on-cart memory than that. Even early games like MegaMan started to use over 100kb, and while Super Mario Bros. was 40k, Super Mario Bros. III used over 350k on-cart. These games were not small.

We don't have extra RAM/ROM space on cartridges.

By the SNES era, games like Super Mario World were 500k, and A Link to the Past was 1MB in size. The implication here is that it would be very difficult if not impossible to port A Link to the Past on our 256k system. We don't have as much RAM left over as a SNES after ROM, Video and Audio.

There just wouldn't be enough space in memory to hold the assets, and pulling it in from OPFS would be very expensive. Not like reading data from a cartridge.

2. Twice the speed, twice the RAM
The truth is, we're not just twice as fast as a C128, or four times faster than a C64; we're about ten to twenty times faster. The reason is that there is almost no register pressure here, and we have a PPU like 80s era consoles did. Therefore to make use of this speed, as restrained as it is, we are going to need more ram. Here's a chart explaining the issue by comparing our memory map to various retro-era consoles: