Better hardware means OpenAI, Anthropic, etc. are doomed in the future?

Question

This is something I don't understand, how will all these AI-as-a-service companies survive in the future when hardware gets better and people are able to run LLMs locally? Of course right now the rent vs. buy equation is heavily tilted towards rent, but eventually I could see people buying a desktop they keep at home, and having all their personal inference running on that one machine. Or even having inference pools to distribute load among many people.do you think what this is possible, and what are these companies plans in that event?

farseer · Accepted Answer

The frontier of how good models are also shifts and will remain ahead of local models unless we hit some dead end limitation in the algorithms themselves. A ceiling so to speak on how good LLM can get before the law of diminishing returns starts to apply.

freakynit · Answer

I do believe this is gonna get commodatized like the internet has. Hardware obviously keeps getting better and cheaper as the time goes by. Sofware in this case is already free/open-weights.
The moats these companies might end up having in near future:
1. Government and enterprise contracts;
2. Even better private models not released to public and only accessible through long-term/exclusive contracts;
3. Gatekeeping the access to millions of their users, especially the non-technical ones, and charging premium for the same;
4. Becoming more and more as the full-stack OS'es to build on top of them.. By proving ready-made foundational layers like knowledge, memory, search/research, sandboxes, deployments, etc...
5. Data/network effects from large-scale usage and feedback loops.
...

verdverm · Answer

1. Is it cheaper for me to buy hardware and electricity than to call an API? (doesn't seem like it right now)
2. The best models are still worth it, unclear when this changes
3. Average person doesn't have the skill to do this. They are afraid to run even simpler things

throwaway5465 · Answer

Young people have had even the concepts of filesystems conditioned out for files to live in a 'folder' of an APP.Local sovereignty isn't a pressing need for most users.

Better hardware means OpenAI, Anthropic, etc. are doomed in the future?

The frontier of how good models are also shifts and will remain ahead of local models unless we hit some dead end limitation in the algorithms themselves. A ceiling so to speak on how good LLM can get before the law of diminishing returns starts to apply.

1. Is it cheaper for me to buy hardware and electricity than to call an API? (doesn't seem like it right now)2. The best models are still worth it, unclear when this changes3. Average person doesn't have the skill to do this. They are afraid to run even simpler things

Young people have had even the concepts of filesystems conditioned out for files to live in a 'folder' of an APP.Local sovereignty isn't a pressing need for most users.

1. Is it cheaper for me to buy hardware and electricity than to call an API? (doesn't seem like it right now)
2. The best models are still worth it, unclear when this changes
3. Average person doesn't have the skill to do this. They are afraid to run even simpler things

Young people have had even the concepts of filesystems conditioned out for files to live in a 'folder' of an APP.
Local sovereignty isn't a pressing need for most users.