LlamaIndex
Log in

Find answers from the community

Updated 3 months ago

Hello I came across the term Act Order

mmartinkozle
·
Hello, I came across the term "Act Order (desc_act)" here https://huggingface.co/TheBloke/vicuna-7B-v1.3-GPTQ in context of quantisation. I couldn't find any info on what that is. Does anybody happen to know what it means?
m
1 comment
mmartinkozle
To partially answer my question with extra info that I found. I found the parameter in the GPTQ-for-LLaMa repo, with the description Whether to apply the activation order GPTQ heuristic (https://github.com/qwopqwop200/GPTQ-for-LLaMa/blob/59cabb9a46db093f0fced4fe8329a3eb14e0a481/neox.py#L382)
And I found the bit of code where it is used https://github.com/qwopqwop200/GPTQ-for-LLaMa/blob/59cabb9a46db093f0fced4fe8329a3eb14e0a481/gptq.py#L150C4-L150C4.
Other than that it makes the quantized model better, I couldn't find any more detailed information on why.
Add a reply
Sign up and join the conversation on Discord