Awesome
<p align="center"> <img src="https://github.com/user-attachments/assets/bebd92b8-765e-4d63-bb3d-47e1bb8b51ad" width=500px> </p> <p align=center> <img src=https://img.shields.io/badge/HACS-Custom-orange.svg?style=for-the-badg> <img src=https://img.shields.io/badge/version-1.0.3-blue> <a href="https://github.com/valentinfrlch/ha-llmvision/issues"> <img src="https://img.shields.io/maintenance/yes/2024.svg"> <img alt="Issues" src="https://img.shields.io/github/issues/valentinfrlch/ha-llmvision?color=0088ff"/> </a> <p align=center style="font-weight:bold"> Image and video analyzer for Home Assistant using multimodal LLMs </p> </p> <p align="center"> <a href="#features">🌟 Features </a> · <a href="#resources">📖 Resources</a> · <a href="#installation">⬇️ Installation</a> · <a href="#roadmap">🚧 Roadmap</a> · <a href="#how-to-report-a-bug-or-request-a-feature">🪲 How to report Bugs</a> </p> <br> <br> <br>LLM Vision is a Home Assistant integration to analyze images, videos and camera feeds using the vision capabilities of multimodal LLMs.
Supported providers are OpenAI, Anthropic, Google Gemini, LocalAI and Ollama.
Features
- Compatible with OpenAI, Anthropic Claude, Google Gemini, LocalAI and Ollama
- Takes images and video from camera entities as input
- Takes local image and video files as input
- Images can be downscaled for faster processing
Resources
Check the docs for detailed instructions on how to set up LLM Vision and each of the supported providers as well as usage examples and service call parameters:
<a href="https://llm-vision.gitbook.io/getting-started"><img src="https://img.shields.io/badge/Documentation-blue?style=for-the-badge&logo=gitbook&logoColor=white&color=18bcf2"/></a>
Check 📖 Examples on how you can integrate llmvision into your Home Assistant setup or join the 🗨️ discussion on the Home Assistant Community.
Installation
- Search for
LLM Visionin Home Assistant Settings/Devices & services - Select your provider
- Follow the instructions to add your AI providers.
Detailed instruction on how to set up LLM Vision and each of the supported providers are available here: https://llm-vision.gitbook.io/getting-started/
Debugging
To enable debugging, add the following to your configuration.yaml:
logger:
logs:
custom_components.llmvision: debug
Roadmap
[!NOTE] These are planned features and ideas. They are subject to change and may not be implemented in the order listed or at all.
- New Provider: NVIDIA ChatRTX
- New Provider: Custom (OpenAI API compatible) Providers
- HACS: Include in HACS default
-
Feature: HTTPS support for LocalAI and Ollama -
Feature: Support for video files -
Feature: Analyze Frigate Recordings using frigate'sevent_id
How to report a bug or request a feature
[!IMPORTANT] Bugs: If you encounter any bugs and have followed the instructions carefully, feel free to file a bug report.
<div align = left> </div>
Feature Requests: If you have an idea for a feature, create a feature request.