How does a query to GPT-4 compare to driving a car?
A single query to GPT-4 with a 10,000-word answer size produces 450 gCO2e (*). An average UK car produces 132 gCO2e/km. So this query is equivalent to driving a car 3.4 km.
[#]FrugalComputing
(*) https://wimvanderbauwhede.codeberg.page/articles/google-search-vs-chatgpt-emissions/ and assuming 100 words for the queries in that article; in reality it is probably closer to 30 words, but I prefer to be conservative.
Car stats from https://www.nimblefins.co.uk/average-co2-emissions-car-uk
=> More informations about this toot | More toots from wim_v12e@scholar.social
@wim_v12e
How did you arrive at the number 450 gCO2e? The post you linked cites five estimates for energy use per ChatGPT query, all of them much lower than 0.01 kWh/query. CO2 intensity of energy generation varies a lot, but even with the dirtiest coal power it's around 1000 g/kWh. So a ChatGPT query powered using coal would emit much less than 10g of CO2. It's still a lot compared to traditional Google-style search, but more like driving a hundred meters by car.
@Stoori
=> More informations about this toot | More toots from osma@sigmoid.social
@osma
1.5 gCO2e for GPT-3 to generate 100 words, and energy use is linear with word count, so 150 gCO2e for 10,000 words; GPT-4 cost is 3x higher than GPT-3, so 450 gCO2e.
@Stoori
=> More informations about this toot | More toots from wim_v12e@scholar.social
@wim_v12e
I see. How typical is a 10,000 word GPT-4 answer though? Does OpenAI even allow it? For such a query, it would take a lot of time to generate (minutes probably) and the energy cost alone must be at least a large fraction of a US dollar, so if people did these often on a free or monthly subscription, OpenAI would soon be bankrupt.
Inference is being optimized all the time. 3x over GPT-3 may have been valid when GPT-4 was released but by now the gap has probably narrowed.
@Stoori
=> More informations about this toot | More toots from osma@sigmoid.social
@osma 10,000 words is a typical length for essays and there are many explanations of how to get ChatGPT to do this.
I looked at the cost as well, and the dominant cost is not the electricity, it is the GPU servers. Electricity is only order of 10% of the total cost for inference.
But even if it is only 1,000 words, the analysis remains the same. And even if it was "only" as bad as GPT-3, it would still be very bad. Sure, driving a car is worse, but that seems a poor excuse to me.
@Stoori
=> More informations about this toot | More toots from wim_v12e@scholar.social This content has been proxied by September (ba2dc).Proxy Information
text/gemini