How Accurate Is ChatGPT at Estimating Calories From Food Photos?

The VerdictMODERATE CONVICTION

ChatGPT is roughly nutritionist-grade for ballpark, dangerous as your primary tracker if the number actually matters.

Next time you snap a meal photo for ChatGPT, type the ingredients and a portion reference (egg-sized, fist-sized, palm-sized) along with the photo. Then mentally discount the protein number it returns by 10 to 15 percent.

The number that changed my mind: in a JAMA Network Open benchmark of 222 food items, ChatGPT-3.5 and ChatGPT-4 only hit within 10% of true energy on 35 to 48% of items. About one in two — and that was on clean photos, not your phone shot of a half-eaten plate.
What most people get wrong: thinking the model is bad at identifying food. It is not. It identifies the food about 93% of the time. The problem is portion size and a protein bias the model carries silently.
What to actually do about it: pair the photo with text (ingredients, dish name, portion reference), use the strongest available model, and mentally discount any LLM protein number. Use a kitchen scale for anything where the corridor decides what you eat next week.

An LLM looking at your food photo is like a really fast cashier with no scale and no scanner. They can name almost everything on the tray (93% of foods correctly identified). They guess weights from across the counter. They round protein up because they over-trust the word "chicken." Useful for a sense check at the till. Not the system you want for the books.

The Practical Takeaway

Practical guidance for using LLMs to estimate calories from food photos

Ballpark onlyUse LLMs for "is this 500 or 1500 kcal" sanity checks. Not for "is this exactly 612 kcal."
Always pair photo with textType the ingredients, the dish name, and a portion reference (egg-sized, fist-sized, palm-sized). This is where the GPT-5 accuracy improvement actually came from.
Discount protein 10–15%Until a newer model is shown to have fixed the protein bias in independent testing, treat the LLM protein number as the upper bound, not the central estimate.
Use the strongest model availableGPT-5 > GPT-4 > GPT-3.5 in published data. If you are paying for AI anyway, prioritize the vision-capable frontier model.
Weigh real food in outcome-critical phasesLate cut, plateau diagnostics, contest prep. A 200–400 kcal compound daily error is the entire decision window in a corridor.
Add a reference object to the photoA standard fork, a known-size plate, a credit card. Give the model a depth anchor it currently does not have.

Next time you snap a meal photo for ChatGPT, type the ingredients and a portion reference along with the photo. Then mentally discount the protein number.

Pairing the image with text (ingredients, dish name, "egg-sized / fist-sized / palm-sized") is where the published accuracy improvement actually came from. The protein-bias discount is because ChatGPT-3.5/4 systematically overestimated protein in the JAMA benchmark and that has not been independently shown fixed in GPT-5 yet.

~10 extra seconds per upload · zero equipment

Conviction

MODERATE Modern LLMs are moderately accurate at calorie and macro estimation from food photos, with known systematic errors on portion size and protein.

Per-endpoint conviction: food identification (~93%) HIGH | single-item ±10% accuracy on 35–48% of items HIGH | poor portion estimation on medium/large portions HIGH | protein overestimation bias MODERATE (one peer-reviewed paper, not yet replicated for GPT-5) | GPT-5 "practically useful" claim MODERATE-to-LOW (single cross-sectional eval) | Claude vs Gemini vs ChatGPT relative accuracy LOW (no peer-reviewed head-to-head exists).

What would change my mind on the GPT-5 "practically useful" claim?

An independent (non-OpenAI-funded), pre-registered, three-arm benchmark of ≥500 real-world meal photos comparing GPT-5, Claude latest-vision, and Gemini latest-Pro. Primary endpoint: % of meals within ±5% of true kcal AND ±5% of each macro (true value via weighed ingredient log), stratified by portion size and dish type. Pass condition: ≥60% within ±5% kcal across all conditions, ICC ≥ 0.85 for protein, no systematic protein bias. That study would upgrade the GPT-5 case to HIGH.

What would change my mind on the protein bias?

An independent replication on GPT-5 using ≥150 protein-rich foods (mixed lean / fatty / processed / plant-based protein sources) showing zero systematic bias on protein estimation, with the bias also absent in head-to-head Claude and Gemini benchmarks. Until that exists, I treat the protein bias as live and applicable.

Go Deeper

Want evidence-scored answers to health and tracking questions like this? Join The Verdict — free weekly reviews.

Subscribe Free

Sources

Hoang Y, et al. (2023). Consistency and Accuracy of Artificial Intelligence for Providing Nutritional Information. JAMA Network Open. DOI 10.1001/jamanetworkopen.2023.50367. 222 food items, ChatGPT-3.5 vs -4 vs registered nutritionists. 35–48% within ±10% energy; ChatGPT-4 superior; protein systematically overestimated.
An Evaluation of ChatGPT for Nutrient Content Estimation from Meal Photographs (PMC11858203, ~2024–2025). Cross-sectional comparison vs dietitians. 93% food ID; ICC 0.31–0.67; poor on medium/large portions.
Image-Based Dietary Energy and Macronutrients Estimation with ChatGPT-5 (PMC12655113, ~2025). Cross-sectional eval. Image + non-visual descriptors reduced MAE / MedAE / RMSE vs image-only.
Biotics Research industry blog (2024). Informal qualitative comparison of ChatGPT, Claude, and Gemini on food-tracking tasks. "Significant errors" across all three; no quantified metrics. [industry-adjacent]

ChatGPT is roughly nutritionist-grade for ballpark, dangerous as your primary tracker if the number actually matters.

How Accurate Is ChatGPT at Estimating Calories From Food Photos?

The Practical Takeaway

Conviction

Sources

Get weekly verdicts — no fluff, just evidence

Related free research

There are 424 more inside