----------------------
To use the Llama2 model with LlamaIndex, you need to follow these steps:
- Install the necessary packages: You need to install the
llama-index and ipywidgets packages. You can do this using pip:
!pip install llama-index ipywidgets
- Sign in to HF hub: You need to sign in to the Hugging Face hub with an account that has access to the Llama2 models. You can do this using the
huggingface-cli login command in your console. For more details, please see: Llama Downloads ().
- Set up logging: You can set up logging to monitor the progress of your tasks. Here is an example of how to do it:
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
- Import necessary modules and set the model: You need to import the necessary modules from
llama_index and set the model you want to use. Here is an example of how to do it:
import torch
from llama_index.llms import HuggingFaceLLM
from llama_index.prompts import PromptTemplate
# Model names (make sure you have access on HF)
LLAMA2_7B = "meta-llama/Llama-2-7b-hf"
LLAMA2_7B_CHAT = "meta-llama/Llama-2-7b-chat-hf"
LLAMA2_13B = "meta-llama/Llama-2-13b-hf"
LLAMA2_13B_CHAT = "meta-llama/Llama-2-13b-chat-hf"
LLAMA2_70B = "meta-llama/Llama-2-70b-hf"
LLAMA2_70B_CHAT = "meta-llama/Llama-2-70b-chat-hf"
selected_model = LLAMA2_13B_CHAT
Please note that you need a decent GPU to run this notebook, ideally an A100 with at least 40GB of memory.
