These YouTube video and article explain how to do it. He also provide an simple web application and quantized model files. The official llama2 only provides full model at this point.
I tried it and it worked well, and I would like to offer some tips.
If it stucked after downloading the model, it was necessary to use a privileged terminal/cmd to create the temporary folder on Windows,
otherwise it would get stuck after downloading the model.
As an alternative, you may get it work by disabling ‘Ransomware protection’, but I didn’t try. For more information, refer to the following link.
Hope your luck.
