Skip to content

Effortless LLM Deployment with Zeblok SDK and Ai-MicroCloud

Recipe · Karan Pathak · 20 September 2024

Deploy Large Language Models as APIs using Zeblok Ai-MicroCloud’s Ai-API managed service and Zeblok SDK. Discover how to seamlessly serve LLMs without the burden of managing complex infrastructure.

Part 2 of Exploring the Zeblok-SDK series.

What it walks you through.

Spawn an Ai-WorkStation

Minimal Notebook, a plan, a namespace and a bucket.

Download the LLM locally

facebook/opt-125m into llm_deployment_example.

Prepare the Dockerfile

ubuntu:22.04, vLLM and the Dev API Server.

Create and spawn the Ai-API

One create_and_spawn() call with the Zeblok SDK.

Query it

Append /generate to the endpoint and send a prompt from Postman.

Try it on Ai-MicroCloud®.

Download PDF