Meus 2 cents expandidos:

1. Sim, roda direto via CPU sem precisar de GPU

2. Mas apenas para testes de desenvolvimento, nao serve para producao.

3. So aceita prompts em ingles (em pt-br gera alucinacoes)

4. Para processar o prompt "who are you" levou 2m30s em um i7-11800H e gerou a seguinte resposta:

User: who are you ? Assistant: I am a person who is interested in learning about the world, its history, and its culture. I am also a person who enjoys reading, writing, and exploring different places. I am a person who is curious and interested in learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different places. I am a person who enjoys learning about the world, its history, and its culture. I am a person who enjoys reading, writing, and exploring different

5. Ate da para usar em RAG, se o tempo de resposta de 2-3 mim for aceitavel

6. Multi-usuario ? Com 2 solicitacoes simultaneas, o tempo de resposta subiu para 3m30s

Enfim - para quem quer brincar, ja da para o gasto. Mas nao muda o fato que para producao precisa de GPU ou LLM providers.

Ainda faltam mais testes, se tiver novidades aviso.