In 2023, Industry spent 60B in just on the GPUs. This year, it’s at least 100B as per forecast, Just on the GPU ( not counting full system costs, energy and engineers/researchers cost) Now, with this much capex across companies, What happens if companies don’t deliver on ML? And if they do deliver ( Not saying about AGI but specialized cases) , it’s a lot of white collar jobs ( not saying programming jobs) that will be eliminated.
Not my field, but I have heard some voices comparing the results of very expensive large scale training and much less expensive (smaller scale) methods and finding that the results are not super different. I have always wondered the cost benefit in terms of training results for this huge GPU clusters/models.
Someone has to bear the expenses of our business class flights, right ?