zky
/
ai_model
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'e2444130b5'
${ noResults }
ai_model/runner
History
20918 5e0a8fedd0
Initial commit 		10 months ago
..
common Initial commit 10 months ago
llamarunner Initial commit 10 months ago
ollamarunner Initial commit 10 months ago
README.md Initial commit 10 months ago
runner.go Initial commit 10 months ago

README.md

runner

Note: this is a work in progress

A minimial runner for loading a model and running inference via a http web server.

./runner -model <model binary>

Completion

curl -X POST -H "Content-Type: application/json" -d '{"prompt": "hi"}' http://localhost:8080/completion

Embeddings

curl -X POST -H "Content-Type: application/json" -d '{"prompt": "turn me into an embedding"}' http://localhost:8080/embedding