Deploy Large Language Models as APIs using Zeblok Ai-MicroCloud’s Ai-API managed service and Zeblok SDK. Discover how to seamlessly serve LLMs without the burden of managing complex infrastructure.
Part 2 of Exploring the Zeblok-SDK series.
Recipe · Karan Pathak · 20 September 2024
Deploy Large Language Models as APIs using Zeblok Ai-MicroCloud’s Ai-API managed service and Zeblok SDK. Discover how to seamlessly serve LLMs without the burden of managing complex infrastructure.
Part 2 of Exploring the Zeblok-SDK series.
Minimal Notebook, a plan, a namespace and a bucket.
facebook/opt-125m into llm_deployment_example.
ubuntu:22.04, vLLM and the Dev API Server.
One create_and_spawn() call with the Zeblok SDK.
Append /generate to the endpoint and send a prompt from Postman.
Install the SDK with pip install zeblok-sdk.
The full recipe, with every code sample and screenshot, is on the author’s Substack: https://karanpathak.substack.com/p/effortless-llm-deployment-with-zeblok