Awesome
TFProfiler – ready to use 🚀
Overview
TFProfiler is an app that aims to profile your TensorFlow Lite and OpenCV DNN supported model right on the smartphone.
Measure model performance with model initialization time, inference time, FPS, memory consumption, etc.
Tweak model runs with different delegates (CPU, GPU, NNAPI, HEXAGON), XNNPACK option, and number of threads.
Apply a batch processing, adjusting the number of input images. TFLite batch inference is partially supported by TensorFlow.
Run neural models (*.caffemodel, *.onnx, *.pb, etc) supported by OpenCV DNN module.
- Android API SDK 21+
- Written in Kotlin
- TFLite 2.5.0 inside
- OpenCV 4.5.2 inside
Features
The app displays a list of models built inside the app that can be used to measure its performance on device. Also, you can upload your *.tflite model and see how it works on the smartphone.
<div> <img align="center" src="img/screenshot_config_options.jpg" alt="Configure delegate options" width="220"> <img align="center" src="img/screenshot_run_model.jpg" alt="Run model" width="220"> <img align="center" src="img/screenshot_report.jpg" alt="Report" width="220"> </div> <br/>Download
Latest apk release can be found on release page.
Or download app via Google Play:
<a href="https://play.google.com/store/apps/details?id=ru.igla.tfprofiler" target="_blank"> <img src="https://play.google.com/intl/en_gb/badges/static/images/badges/en_badge_web_generic.png" width=240 /> </a>Built-in models
App has built-in models that you can play with to see how inference performance is estimated.
Image Dataset
The app has built-in a subset of image dataset Caltech 101. It is used for running model inteferences.
Libraries
- TensorFlow Lite
- Android Architecture Components
- Timber for logging
- Room - Access your app's SQLite database with in-app objects and compile-time checks.
Performance tips
We can boost model performance and energy efficiency using various optimization techniques.
Model optimization aims to create smaller models that are generally faster and more energy efficient, so that they can be deployed on mobile devices.
-
Check ops compatibility for each delegate, CPU, GPU, XNNPACK.
-
Use XNNPACK to boost float-point inference.
-
Use GPU delegate to compare with CPU and other options. If some of the ops are not supported by the GPU delegate, the TF Lite will only run a part of the graph on the GPU and the remaining part on the CPU.
-
Experiment with NNAPI (Android API 27+) delegate to see whether the models works faster on Android.
-
Try to use quantization to optimize model.
-
Accelerate TFLite model on Qualcomm Hexagon DSPs
-
Experiment with version of TensorFlow Lite library
-
Deploy different models for different devices
Mentions
Featured in AppTractor
Credits
Launcher icon was built using this one. Icons made by Becris from https://www.flaticon.com/
Issues
If you find any problems or would like to suggest a feature, please feel free to file an issue