Solana Scaling Strategy: Parallelize Across Multiple Machines?

Random thought; what if the demand on the Solana network exceeds that which is possible to be provided by a single (even very powerful) machine? How could Solana scale across multiple hardware machines?

Solana’s Sealevel runtime was designed for massive parallelization; each transaction has a header that specifies the accounts that will be read from and written to within that transaction. That allows transactions to be sorted into groups that will not affect each other. Each of these groups can be processed concurrently on multiple CPU cores, and in the future they may be run on GPUs as well.

But what if this is still not enough? Theoretically, Solana could add a ‘load balancer’ validator, which accepts transactions, groups them, and then forwards those transaction-groups to a set of leaders (leaders rotate every 4 slots as currently). These leaders process their-transaction group within the time allotted, and then broadcast them to the rest of the network to produce the finished block. The partial-blocks are disseminated via a gossip network to the rest of the validators on the network, who vote on each block as a whole.

The problem with this is:

1. The load-balancer step will add latency to the network, mostly because of the time it takes to do the network transmission (it would probably add 150ms per block). But the benefits are worth the sacrifice.
2. Each validator will still need to carry the entire blockchain head in their RAM or hard-disk, because they will not know which accounts they will be called upon to process or validate ahead of time. This would limit scalability in terms of memory; we cannot scale to having a head larger than what the smallest validator can hold in its disc drive.* If we partitioned Solana by dividing all Solana accounts into Shard-A and Shard-B, and assign validators to each of these shards (meaning that each validator only needs to worry about storying its own half of the network in memory), then this would work fine until there was a transaction involving accounts from both Shard-A and Shard-B at the same time; in that case it would be impossible for any one validator to process this transaction because it does not have the relevant data to do this. Solana’s programming model also lacks the ability to do asynchronous transactions (transactions that settle across multiple blocks), meaning partitioning Solana is impossible unless we rewrite all code on the blockchain.
3. The usefulness of this parallelization would be limited, in that each validator would be left with mostly sequential groups of transactions to work on. This means it wouldn’t be able to use its own parallelization strategy (multiple CPU or GPU threads).
4. Parallelization in general (across a single machine or multiple machines) will be frustrated by very popular contracts. For example, a single liquidity-pool that gets lots of arbitrage-bot activity, or a single minting-contract for a large collection. These transactions depend on each other and hence must be done sequentially. Remember the ’80 / 20' rule; most likely 80% of your transactions will come from 20% of your accounts.

Overall, I’m going to make this idea as a failure; parallelizing Solana across multiple machines will not work without fundamentally changing Solana’s programming model.

*Theoretically Solana can ‘run out of memory’; SOL has a 342.6 million circulating supply right now. At 7 SOL per 1 MB, that means that the Solana head could grow to a size of 48.94 TBs, which far exceeds most validator’s ability to store. (Validators currently only have 1 TB of storage plus 256 GB of RAM.) Although storing this much data would, in today’s price of $38 per SOL, cost about $13 billion (lol lol lol).