MT-Diet: Automated Smartphone based Diet Assessment with Infrared image

Department of Computer Science and Engineering, Arizona State University

Faculty Advisor

Sandeep K.S. Gupta
Professor
Department of Computer Science and Engineering
Arizona State University
sandeep.gupta@asu.edu

Research Faculty

Present Students

Sponsers


Objective

MT-Diet is an automated smartphone based diet assessment system using Mobile computing and Thermal images.

There are four main challenges:

  • Automated Food Segmentation
  • High Accurated Food Identification
  • Eating Gesture Recognition
  • Real-time Feedback related to Eating Habit and Recommended Diet

Overview

MT-Diet has two stpes: (1) Food identification using a thermal camera sensor and built-in smartphone camera and (2) Estimate food consumption using wrist sensors

MT-Diet Overview


Prototype

MT-Diet captures images of a food plate in both infrared and visual spectra through a thermal camera interfaced with an Android smartphone and the inbuilt camera

Prototype


Food Segmentation and Identification

Food segmentation process takes the color and thermal image of served hot food and plate in a background and outputs: a) number of different food items on the plate, and b) cropped region of each unique food item. Food identification process takes a food segment obtained from food segmentation as input and outputs the type of food items on the plate.

Food Segmentation & Identification


Estimate Food Consumption

Estimating food consumption process takes the outputs of food segmentation and identification as input and outputs: a) identifying the utensil b) an amount of each food the user ate.

Food Consumption


Recommended Diet

Given a food plate MT-Diet shows how far it is from a balanced diet in the form of a spider chart and what component of the diet should be changed in order to make it a balanced diet.

Balanced Diet Recommendation


MT-Diet Demo

The below video demonstrates MT-Diet application which provides the user with (a) segmented food images (removed plate and background), (b) food type in plate, and (c) each food’s nutrition information into USDA (United States Department of Agriculture) website.


MT-Diet Poster


Publications

  • J. Lee, A. Banerjee, and S. K. S. Gupta, MT-Diet : Automated Smartphone based Diet Assessment with Infrared Images, In: Pervasive Computing and Communications (PerCom), 2016 IEEE International Conference on. IEEE, 2016. [ PDF|PPT]

  • J. Lee, A. Banerjee, and S. K. S. Gupta, MT-Diet Demo: Demonstration of Automated Smartphone Based Diet Assessment System, In: Pervasive Computing and Communications (PerCom), 2016 IEEE International Conference on. IEEE, 2016. [ PDF|PPT]


Home | Projects | People | Publications | Courses | Resources | Books | News & Visitors | Contact

Last Updated: 12th May 2008