Your units don’t make sense. Watts shouldn’t be used for a fixed energy usage it’s like saying a car drove across the U.S. and it did it at 4 gallons per hour.
The more useful metric to use is Gwh so chatgpt3 used 1.3 Gwh which isn’t bad but gpt4 used 62.3 Gwh in training plus an extra 1 Gwh per day
That said, I still get the analogy by saying a car used 4 gallons per hour - it still indicates how fuel efficient something is. Especially if compared to something else
But the problem is you don’t have anything to tie that to. if you have a car that gets 4 gph but goes 100 mph then it’s more efficient than a car that gets 3 gph but only goes 50mph but even with those you miss out on the actual efficiency which for a car is usually transporting people.
So if car A gets 4 gph at 100 mph and transports 2 people it gets 50 passenger miles per gallon of gas which is finally an actually useful metric
For LLMs that becomes much harder to quantify but a useful metric might be wh per minute of time saved or mL of water per minute of time saved. Unfortunately to quantify those you would need to do much more in depth analysis and probably also factor in false readings and time lost from that
Your units don’t make sense. Watts shouldn’t be used for a fixed energy usage it’s like saying a car drove across the U.S. and it did it at 4 gallons per hour.
The more useful metric to use is Gwh so chatgpt3 used 1.3 Gwh which isn’t bad but gpt4 used 62.3 Gwh in training plus an extra 1 Gwh per day
I get what you’re saying now.
That said, I still get the analogy by saying a car used 4 gallons per hour - it still indicates how fuel efficient something is. Especially if compared to something else
But the problem is you don’t have anything to tie that to. if you have a car that gets 4 gph but goes 100 mph then it’s more efficient than a car that gets 3 gph but only goes 50mph but even with those you miss out on the actual efficiency which for a car is usually transporting people.
So if car A gets 4 gph at 100 mph and transports 2 people it gets 50 passenger miles per gallon of gas which is finally an actually useful metric
For LLMs that becomes much harder to quantify but a useful metric might be wh per minute of time saved or mL of water per minute of time saved. Unfortunately to quantify those you would need to do much more in depth analysis and probably also factor in false readings and time lost from that