Awesome

Task: TextGeneration Tags:

• TextGeneration
• Llama2-7b

Model-Llama2-7b-dvc

🔥🔥🔥 Deploy Llama2-7b model on VDP.

This repository contains the Llama2-7b Text Completion Generation Model in the vLLM and Transformers format, managed using DVC. For information about available extra parameters, please refer to the documentation on SamplingParams in the vLLM library.

Notes:

• Disk Space Requirements: 14G
• Memory Requirements: 40G (for fp32 in cpu mode)
• Following is an example of query parameters:

{
    "task_inputs": [
        {
            "text_generation": {
                "prompt": "The capital city of Franch is ",
                "max_new_tokens": "100",
                "temperature": "0.8",
                "top_k": "10",
                "seed": "42",
                "extra_params": {
                    "repetition_penalty": 1.8
                }
            }
        }
    ]
}