Awesome
<img src='https://github.com/fabiomatricardi/OpenVINO-StableLM-3B-streamlit/blob/main/images/banner.png' width=900>OpenVINO-StableLM-3B-streamlit 
A streamlit ChatBot running StableLM Zephyr 3B with Openvino and Optimum Intel
- StableLM-3B Chatbot - A streamlit CHATBOT interface with stablelm-zephyr-3b quantized in 4bit and optimum-intel. The Interface has a kind text streaming effect, and the number of turns are handled to not exceed the context window. The Model used is published on Hugging Face Hub and was created with the free HF Space hosting the NCCF-quantization tool.
Requirements
python312 -m venv venv
.\venv\Scripts\activate
python -m pip install --upgrade pip
pip install openvino-genai==2024.4.0
pip install optimum-intel[openvino] tiktoken streamlit==1.36.0
Download the model
from HuggingFace
https://huggingface.co/FM-1976/stablelm-zephyr-3b-openvino-4bit
Run the app
streamlit run .\stappStableLM.py