hy4-preview open weights and deployment

Status: Available. Tencent published BF16 and FP8 (MXFP8) weights, about 1.56 TB and 814 GB, under a plain Apache-2.0 licence with no extra use restrictions. The repositories ship no custom model code, and the model card gives vLLM and SGLang launch commands with tensor parallel 8 on the FP8 checkpoint. Tencent does not say in so many words that this checkpoint is the one behind the hy4-preview API.

Agent deployment prompt

You are my deployment agent. Deploy the official tencent/Hy4-preview weights as a private, OpenAI-compatible service on this host or cluster. Use the publisher's model card as the source of truth: https://huggingface.co/tencent/Hy4-preview.

Preflight
- Inspect the OS, GPU model and count on every node, the interconnect between nodes, free VRAM, driver and CUDA versions, free disk space, and available ports. The FP8 checkpoint (tencent/Hy4-preview-FP8, NVIDIA ModelOpt MXFP8) is about 814 GB and the BF16 checkpoint about 1.56 TB; the model card's vLLM and SGLang examples run the FP8 checkpoint with tensor parallel 8 but state no minimum GPU. Confirm the GPUs support MXFP8 and estimate whether the official weights and the requested context fit with safe headroom.
- If the hardware cannot safely run the official weights, stop before installing or launching anything. Explain the limiting resource and propose a separately reviewed alternative; do not substitute a community quantization automatically.

Deployment
- If preflight passes, create an isolated environment and use the official hy4-preview vLLM or SGLang image the model card names, with the parser settings it shows. The repository ships no custom model code; if a tool nevertheless asks for trust-remote-code, stop and ask me. Generate an API key and store it in a protected environment variable. Bind to 127.0.0.1 first; do not expose the endpoint publicly.
- Keep the publisher's reasoning default (high). Start with a context length the preflight supports; the model supports up to 1,048,576 tokens.

Verification and handoff
- Send a real request to /v1/chat/completions and confirm that the model responds. Report the exact launch command, installed versions, model revision, local endpoint, test result, measured GPU memory and log location. Identify any reverse-proxy or firewall changes that still need my approval. Never print the API key or other secrets.
License
Apache-2.0
Hardware note
Not published for inference beyond the tensor-parallel-8 launch examples; the FP8 weights alone are about 814 GB, and the Agent recipe performs a hardware preflight.
Recommended runtime
vLLM or SGLang (official hy4-preview images)
Verified date
2026-09-24

Official repository

Published model card

Parameter count
770B total, 49B active (MoE), plus a 10B MTP layer for speculative decoding
Context window
1M tokens (1,048,576 in the model config)
Maximum output
64K tokens on Tencent Cloud (960K maximum input)
Input/output modalities
Text input; text output (Tencent Cloud's guide says the API takes no image or video input)

Official sources