I’m using it locally, with models my existing computer can handle. I think, but I’m not sure, that it’s better for the environment than buying into some massive model hosted in a data centre. And I don’t want to lose my skills so I still try to do as much as I can without calling on it.
I will say that using a super efficient cloud model (deepseek for me, v4 flash is very cheap) is probably more efficient than using a local model in most cases, especially if you need strong capabilities. The question is whether your own grid or the provider’s grid is more environmentally friendly (hint: china’s energy mix is about 50% fossil fuels while the US’s mix is around 75% fossil fuels overall)
Training is of course expensive still, but Chinese labs have all but caught up to US labs despite much lower compute resources being available to them.
No, and I do understand that training is the most energy-intensive part of the process. I don’t know how exactly it divides up: how much of the work of massive data centres is training and how much is running people’s queries on finished models.
I’m using it locally, with models my existing computer can handle. I think, but I’m not sure, that it’s better for the environment than buying into some massive model hosted in a data centre. And I don’t want to lose my skills so I still try to do as much as I can without calling on it.
I will say that using a super efficient cloud model (deepseek for me, v4 flash is very cheap) is probably more efficient than using a local model in most cases, especially if you need strong capabilities. The question is whether your own grid or the provider’s grid is more environmentally friendly (hint: china’s energy mix is about 50% fossil fuels while the US’s mix is around 75% fossil fuels overall)
Training is of course expensive still, but Chinese labs have all but caught up to US labs despite much lower compute resources being available to them.
That’s a fair point. Where I am is about 28% fossil fuels I believe, so it’s a slightly different calculation.
did you generate your model locally?
No, and I do understand that training is the most energy-intensive part of the process. I don’t know how exactly it divides up: how much of the work of massive data centres is training and how much is running people’s queries on finished models.
maybe ask the vendor to let him pay the actual cost plus a proper profit margin to figure that out?