FruitSense

FruitSense

Evelyn

Evelyn

May 1, 2024

1 hour read

#mobile #android #CNN #google teachable machine #tensorflow lite

Important note: Feel free to admire my final year project, but please resist the urge to “borrow” it. I’ve spent countless sleepless nights perfecting this masterpiece. So unless you’re planning to give me full credit (and maybe a yacht), kindly keep your hands off.

Introduction

We all know picking the perfect fruit can be a hit-or-miss game, relying on how shiny, squishy, or “perfectly” round it looks. But let’s face it, this method is slow and often misleading. Enter AI! With the rise of machine learning, we now have an Android app that takes the guesswork out of fruit shopping. It analyzes key traits—color, shape, size, defects—to instantly tell you if that apple’s ripe or ready to toss. This app doesn’t just save time; it helps reduce food waste, promotes healthier choices, and educates us on what truly makes fruit fresh. This page dives into how current apps stack up, what’s missing, and how we can make fruit selection smarter than ever.

Objectives

  1. Analyze existing apps to spot what works and what doesn’t in fruit quality detection.
  2. Design a super user-friendly Android app for easy fruit freshness checks.
  3. Build the app using Android Studio, adding smarter features for better performance.
  4. Test the app’s usability to ensure it’s as intuitive as possible.

Methods

Design

This chapter covers the design and architecture behind FruitSense, turning the gathered requirements into a clear plan for implementation.

Flowchart

use_case_diagram.png
This flowchart outlines the entire fruit freshness detection process, from user interaction to backend processing. The project involves three roles: admin, user, and developer.

  • Admins log in to manage data and user details.
  • Users register, log in, and can update info, inspect fruit ripeness, check history, and change the language.
  • Developers handle datasets, train the model, and display fruit quality results.

Google Teachable Machine

google_teachable_machine.png
Google Teachable Machine is used to create machine learning models to classify fruit ripeness of apples, bananas, and tomatoes, into unripe, ripe, and overripe categories. By uploading labeled images of the fruits, the tool trains a model, which can then be exported as a TensorFlow Lite file for use in apps to detect ripeness in real-time.

Database

firebase_database.png
Firebase is the engine running FruitSense:

  • Realtime Database: Stores all the app’s data and syncs it in real-time across devices.
  • Authentication: Manages secure logins, handles password resets, and gives each user a unique ID.
  • Storage: Saves profile pics and fruit images, ensuring users can upload, download, and access them easily.
  • Firestore Database: Stores personal info and instantly updates changes across all devices.
MainActivity.java
mAuth = FirebaseAuth.getInstance();
mUser = mAuth.getCurrentUser();
mAuth = FirebaseAuth.getInstance();

This code initializes the Firebase Authentication instance (mAuth) and retrieves the current authenticated user (mUser) to handle Firebase Authentication and user management each time the app is opened.

1. Authentication

authentication.png
When users open the app, the sign-in page opens up to verify their identity. Users can enter email, password, and hit Log In button. If users forget their password, they can click ‘Forgot Password?’ to receive a reset link via email and get back to logging in with their new password.

register.png

If users don’t have an account, they can click Sign-Up to create one. Just fill in the username, email, password, and phone number, hit Sign Up, and then head back to the sign-in page to log in which then redirect to the home page.

2. Home

home_page.png
The homepage of FruitSense showcases a list of articles and educational content. Users can search for fruits or articles, click to explore more, or scroll through the “Explore the World of Fruits” and “Articles” sections. Plus, a handy menu bar lets users quickly jump to Home, Scan, or Account pages.

3. Fruit and Article

fruits_and_articles_page.png
This page shows the “Articles” and “Explore the World of Fruits” pages. Users can scroll horizontally, click on images to dive deeper into fruit types, read detailed descriptions, and explore a variety of related articles.

4. Scan

4.1 Scan Page

scan_page.png
Figure 4.16 showcases the Scan Page and Snap Tips feature. Users can either capture fruit images with the camera, upload from the gallery, or check out handy tips for taking the perfect shot. The tips guide users on lighting, steady hands, and proper fruit positioning to get the best scan results.

scan_picture.png
The camera scan page allows users to snap a fresh fruit pic or upload one from their gallery. Once the image is scanned, the system analyzes the fruit’s ripeness and estimates its shelf life, displaying the results on the screen.

result_page.java
import org.tensorflow.lite.support.tensorbuffer.TensorBuffer;

private void classifyImage(Bitmap imageBitmap) {
  Model model = Model.newInstance(getApplicationContext());

  TensorBuffer inputFeature0 = TensorBuffer.createFixedSize(new int[]{1, 224, 224, 3}, DataType.FLOAT32);
  ByteBuffer byteBuffer = ByteBuffer.allocateDirect(4 * imageSize * imageSize * 3);
  byteBuffer.order(ByteOrder.nativeOrder());

  Model.Outputs outputs = model.process(inputFeature0);
  TensorBuffer outputFeature0 = outputs.getOutputFeature0AsTensorBuffer();

  float[] confidences = outputFeature0.getFloatArray();
  int maxPos = 0;
  float maxConfidence = 0;
  for (int i = 0; i < confidences.length; i++) {
      if (confidences[i] > maxConfidence) {
          maxConfidence = confidences[i];
          maxPos = i;
      }
  }
  model.close();
  }

This block of code classifyImage(Bitmap imageBitmap) resizes an image, prepares it for classification, feeds it into a machine learning model, and retrieves the output to determine the most likely class from TensorBuffer (which is TensorFlow Lite’s Support Library) and associated confidence score. The model is then closed to free up resources.

5. Account

account_page.png
The profile page allows users to peek at their details, update their info, switch languages, or log out. They can hit “Edit Profile,” make your changes, and tap “Save” to save the changes.

6. Switch Language

switch_language.png
The language switch page allows users to switch between English and Malay. Users can select the language they prefer and hit “Save” to save the changes.

changelanguage_page.java
private void saveLanguageSelection() {
    if (selectedRadioButton.getId() == R.id.radioButtonEnglish) {
        languageCode = "en";
    } else if (selectedRadioButton.getId() == R.id.radioButtonMalay) {
        languageCode = "my";
    }

    SharedPreferences preferences = getSharedPreferences("AppPrefs", MODE_PRIVATE);
    SharedPreferences.Editor editor = preferences.edit();
    editor.putString("language", languageCode);
    editor.apply();

    // Apply language change
    Locale locale = new Locale(languageCode);
    Locale.setDefault(locale);
    Configuration configuration = new Configuration();
    configuration.setLocale(locale);
    getResources().updateConfiguration(configuration, getBaseContext().getResources().getDisplayMetrics());
  }
}

saveLanguageSelection() is used to save the selected language (in this case English or Malay) in SharedPreferences and redirect to the profile page.

Tech Stack

Android StudioAndroid Studio
FirebaseFirebase
JavaJava
Google Teachable MachineGoogle Teachable Machine
TensorFlow LiteTensorFlow Lite