glm-5.3-flash open weights and deployment

Status: Available. Official FP8 and BF16 safetensors are published under the MIT License. The publisher lists SGLang, vLLM, TokenSpeed, Transformers, KTransformers and Unsloth for local serving.

Agent deployment prompt

You are my deployment agent. Deploy the official zai-org/GLM-5.3-Flash weights as a private, OpenAI-compatible service on this host. Use the publisher's model card as the source of truth: https://huggingface.co/zai-org/GLM-5.3-Flash.

Preflight
- Inspect the OS, GPU model and count, free VRAM, driver and CUDA versions, free disk space, and available ports. The default repository holds block-wise FP8 weights; zai-org/GLM-5.3-Flash-BF16 is the official BF16 alternative. Estimate whether the chosen weights and the requested context fit with safe headroom for runtime and concurrency; Z.ai publishes no minimum GPU requirement.
- If the host cannot safely run the official weights, stop before installing or launching anything. Explain the limiting resource and propose a separately reviewed alternative; do not substitute a community quantization automatically.

Deployment
- If preflight passes, create an isolated environment, install a current SGLang or vLLM release that documents GLM-5.3-Flash support, record the exact model revision, and serve the FP8 repository through its OpenAI-compatible API. Generate an API key and store it in a protected environment variable. Bind to 127.0.0.1 first; do not expose the endpoint publicly.
- Keep the publisher's sampling defaults (temperature 1.0, top_p 0.95). Thinking is always on for this model, so do not try to disable it. Start with a context length the preflight supports; the model supports up to 1,048,576 tokens.

Verification and handoff
- Send a real request to /v1/chat/completions and confirm that the model responds. Report the exact launch command, installed versions, model revision, local endpoint, test result, measured GPU memory and log location. Identify any reverse-proxy or firewall changes that still need my approval. Never print the API key or other secrets.
License
MIT
Hardware note
Not published by Z.ai; the Agent recipe performs a hardware preflight before serving.
Recommended runtime
SGLang or vLLM
Verified date
2026-09-24

Official repository

Published model card

Parameter count
320B total, 18B active (MoE)
Context window
1M tokens (1,048,576)
Maximum output
128K tokens (131,072 on the Z.ai API)
Input/output modalities
Video, image, text and file input; text output

Official sources