2024 Presidential Election
9h
629
Trump may consider shaking up the federal reserve if elected.
World Conflicts
Yesterday
827
Is Genocide Sometimes Understandable?
Personal Finance
Yesterday
869
Low Net worth @35
Tech Industry
3h
1320
Got terminated after disputing a PIP
Tech Industry
4h
250
A lot of people here serve money and not God
We still don't seem to have a great handle on what product experiences we can reliably build using large language models. And yet lot of energy is being spent trying to optimize their performance. I get that both directions are necessary and that one could argue that if LLMs can do inference faster then more people will adopt them. That said I feel like the limiting factor isn't performance but the right product experience. I could be wrong. It feels like all this energy is better spent trying to find better uses of language models to build compelling product experiences. What do you think?
Agree. A big problem with LLM is that it is unbounded. So, the outputs are just probability. You start to see the value in simpler methodologies as it isn’t as cumbersome or expensive to enable them effectively. How useful is an LLM that can tell you a bunch of historical facts if it is being implemented for a fast food restaurant or something? What is the waste in that? You see, the issue is the issue we always have had in tech. Brand new ideas and tech being lead by dinosaurs that haven’t kept up with the times and are trying to just implement and say it’s done. This technology needs much more time to mature.
Good point. I hadn't considered all the parameter waste related to these models being used for much simpler tasks. So if at all you want to optimize them then what I'm hearing is that much smaller models can do what the specific use case demands eh?
Smaller or just simpler imo. An LLM is just an unbounded beast and tons of efforts goes into aligning it, imo.
It’s just money. Less resources, less time it takes - more money left in your pocket.
This is a good point. I want us to keep optimizing these LLMs so I can get industry-leading performance on my $700 used RTX 3090, instead of paying $20K for an A100 80GB.
No chance ;) You’ll still need A100s or H100s, but will be able to use better bigger models, for better profit. The more you buy Nvidia, the more you save.
As someone who has worked in ML and DL for over 10 years, I have often seen people blindly jump into whatever the newest ML /DL method is , without first understanding what problem they are trying to solve. The approach should always be, I) understand the problem, ii) find the best solution which can solve that problem, III) if that solution happens to be ML/DL, then use it. As opposed to going around finding problems which you can solve using a particular ML/DL method you like. You don't buy a tool kit and then go around finding doors and windows that you can fix with it. In case of LLMs, question answer problems, chatbots, data summarization etc where some of the problems for which the best solution was LLMs. And hence the LLMs trained for those problems have done an amazing job.
LLM are the DBMS of the 2020's.
How can you say something that is so controversial yet so brave! 😎
It’s never premature to optimize for cost.
Not if the costs are being offset with revenue. At least in the near term.
The limiting factor is accuracy. As long as they keep hallucinating they're gonna stay in the dumpster
Isn't the hallucination property what allows them to converse and talk like humans and say sentences that aren't exactly what they have seen in the training data?
I think every company is exploring product experiences and trying different things out. But it’s very difficult to run a large enough scale experiment when it’s so expensive to run LLMs in the first place. That’s why improving performance at the same time is important. So that any company will not be limited by resources to experiment and iterate on product ideas.
I agree that the product experience isn't there yet, but I disagree that it's premature. I think having an optimal/low-cost/low latency way of building and operating an LLM app is the way to help people "unlock" the right product experience by making it accessible enough for companies & startups to build on. Lowering the "minimum bet" makes it possible to experiment more.
You’re absolutely right that we still don’t have the right product experience. I think the basic problem is that *people don’t want to interact with chatbots.* People want to ask a natural language question, and then get a menu of relevant results. People want the Google Search experience, but for other use cases than searching web pages. Product and engineering teams have completely missed that that’s what users really want.