Developed by China Telecom Artificial Intelligence Technology Co., Ltd. and succeeding the TeleChat series, Xing4.0-29B-A4B stands out as a next-generation language model tailored for software engineering and complex task execution. With a total of 29 billion parameters of which only 4 billion are active per token (Mixture of Experts architecture), this model combines efficiency with power. It natively supports a 256K context length, extensible to 512K, while relying on cutting-edge architectural choices suited for demanding production environments.
This article provides an in-depth technical analysis of Xing4.0-29B-A4B, covering its internal architecture, its training on Ascend NPU hardware, its performance on state-of-the-art benchmarks, and its integration into modern development workflows.
Technical Architecture and Under-the-Hood Innovations
The architecture of Xing4.0-29B-A4B relies on a combination of several state-of-the-art technologies in neural networks and natural language processing. The objective is to offer an optimal balance between reasoning capability and execution speed, particularly for software agents and tool calling.
The mHC + MLA + MTP Structure
The model integrates several major structural innovations:
- Multi-head Latent Attention (MLA): Drastically reduces the memory footprint of the KV (Key-Value) cache while maintaining high-quality attention over very long sequences.
- mHC and MTP: Multi-stage planning and multi-token prediction mechanisms that reinforce the coherence of complex reasoning chains and stabilize execution over extended contexts.
Model Parameterization and Sizing
The design of Xing4.0-29B-A4B is based on a finely calibrated MoE (Mixture of Experts) configuration to maximize throughput while limiting latency per token.
| Technical Specification | Value |
|---|---|
| Total Parameters | 29 billion (29B) |
| Active Parameters per Token | 4 billion (4B) |
| Number of Layers | 40 |
| Hidden Size | 3584 |
| Dense FFN Intermediate Size | 9216 |
| Expert Intermediate Size | 1024 |
| Attention Type | MLA |
| Number of Routed Experts | 64 |
| Active Experts per Token | 4 |
| Number of Shared Experts | 1 |
| Native Context Length | 256K (extensible to 512K) |
This organization provides the responsiveness of a compact 4-billion-parameter model while benefiting from the semantic richness of a 29-billion-parameter model.
Training and Optimization on the Ascend NPU Platform
One of the major singularities of Xing4.0-29B-A4B lies in its training infrastructure. It is the first model of this scale trained entirely on the Ascend NPU hardware platform, using Huawei's MindSpore framework and MindFormers ecosystem.
Hardware and Software Co-Optimization
To overcome the inherent challenges of training large-scale MoE models on new accelerators, the China Telecom AI teams conducted extensive co-optimization work:
- Fine-grained MoE communication optimization to reduce cross-node bottlenecks on Ascend 910C clusters.
- Selective recomputation to optimize video memory (HBM) utilization.
- DVM graph-operator automatic fusion to accelerate tensor calculation execution.
- Development of Ascend C fused operators dedicated to mHC components.
Thanks to these low-level efforts, overall training throughput was improved by approximately 96% compared to standard initial performances, demonstrating the growing maturity of alternative software stacks for generative AI.
Comparative Evaluation and Benchmarks
Xing4.0-29B-A4B was evaluated on a demanding battery of benchmarks assessing mathematical reasoning, instruction-following, tool usage, and software engineering (SWE-bench, Terminal-Bench, etc.).
| Benchmark | Xing4.0-29B-A4B | Gemma4-26B-A4B | Qwen3.6-35B-A3B |
|---|---|---|---|
| IFBench | 69.67 | 72.67 | 65.50 |
| AIME2026 | 90.00 | 88.30 | 92.70 |
| AA.LCR | 61.00 | 66.00 | 62.00 |
| Tau3-Bench | 64.63 | 58.90 | 67.20 |
| Claw-Eval | 76.55 | 71.49 | 74.54 |
| SWE-bench Verified | 75.00 | 53.00 | 76.00 |
| Terminal-Bench 2.1 | 57.50 | 30.00 | 51.50 |
| SWE-bench Multilingual | 66.00 | 51.00 | 67.20 |
| DeepresearchBII | 60.80 | 39.30 | 59.70 |
Performance Analysis on Engineering Tasks
The results highlight the model's strengths in development environments and autonomous agent tasks:
- SWE-bench Verified (75.00%): The model demonstrates an excellent ability to resolve bug-fixing tickets on real software projects, rivaling larger models.
- Terminal-Bench 2.1 (57.50%): Top-tier performance in interacting with command-line environments and executing complex system tasks.
- Claw-Eval and DeepresearchBII: Confirm the model's aptitude for driving in-depth document research workflows and tool manipulation under an extended context.
Open Source Ecosystem and Development Integration
To facilitate the adoption of Xing4.0-29B-A4B by the developer community and enterprises, the model benefits from native compatibility with major market inference and fine-tuning frameworks.
Inference and Deployment
The model is compatible with the following high-performance serving engines:
- vLLM: For optimized production deployment with advanced KV cache management.
- SGLang: To maximize structured inference throughput and agent program execution.
- KTransformers: For local execution or on specific hardware architectures.
Adaptation to Agent Frameworks
The model integrates a specific format alignment to interface directly with cutting-edge agent frameworks such as OpenCode, Claude Code, OpenClaw, and Hermes. This adaptation ensures smooth support for feedback loops, tool calls, and the analysis of large source code repositories without loss of context.
Recommended Parameters by Use Case
To get the most out of Xing4.0-29B-A4B, it is advisable to adjust sampling hyperparameters according to the nature of the task:
| Usage Scenario | Temperature | Top-p | Repetition Penalty |
|---|---|---|---|
| Complex Reasoning / General Tasks | 1.0 | 0.95 | 1.05 |
| Software Development / Agents | 0.8 | 0.95 | 1.05 |
Fine-Tuning and Adaptation to Vertical Domains
Beyond its generalist capabilities, Xing4.0-29B-A4B has been designed to enable economical and rapid fine-tuning on specific business data. Thanks to its compatibility with LLaMA-Factory and MindFormers, technical teams can perform targeted adaptations for strict professional use cases:
- Legal contract auditing requiring the analysis of complex clauses over long context windows.
- Intent classification and financial table comprehension for banking and insurance services.
- Augmented conversational knowledge bases (RAG) exploiting dense and technical documentary corpora.
Conclusion
With Xing4.0-29B-A4B, China Telecom delivers a major contribution to the open-source large language model ecosystem. By combining an active-parameter-efficient MoE architecture, a native context extended to 256K, top-tier performance on coding and agent benchmarks, and real-world validation on Ascend NPU accelerators, this model directly targets engineers and architects looking to deploy robust, high-performing autonomous AI systems.