Are you tired of uploading private documents, company files, and personal data to cloud AI services? Many users are now exploring Local AI solutions that allow them to run powerful AI models directly on their own hardware.

The advantages are clear: private data stays on your device, no monthly AI subscription required, offline AI access, and full control over open-source models.

However, running modern LLMs locally has always required compromises. Even mid-tier dedicated GPUs hit memory limits when loading large dense LLMs like Qwen 3.5 32B, leaving most compact PC users stuck between slow cloud AI and bulky power-hungry workstations.

This is where the AMD Strix Halo Mini PC enters the discussion.

Built on the AMD Ryzen AI Max+ 395 architecture, Strix Halo combines 16 Zen 5 CPU cores, Radeon 8060S graphics, an XDNA 2 NPU (50 TOPS), and up to 128GB of LPDDR5X-8000 unified memory - all in a compact Mini PC form factor. Unlike traditional Mini PCs designed for office workloads, AMD Strix Halo targets a new category: a compact personal AI computer capable of running larger local models.

One of the most discussed use cases is running Qwen 3.5 locally on a Strix Halo Mini PC. But how practical is it? Can Strix Halo handle 7B, 14B, and 32B models? What about AI Agent workflows?

This article explores the real capabilities, limitations, and practical applications of Strix Halo for Local AI.

AMD Strix Halo Hardware Architecture: Why It Matters for LLMs

The Ryzen AI Max+ 395 is a single-chip solution that integrates a 16-core/32-thread Zen 5 CPU, Radeon 8060S graphics (40 compute units, RDNA 3.5), and an XDNA 2 NPU (50 TOPS) into one package. The defining feature for running LLMs locally is its 128GB LPDDR5X-8000 unified memory architecture, which eliminates the traditional split between CPU RAM and GPU VRAM.

Unlike discrete GPU setups with fixed VRAM caps (8GB, 12GB, 24GB), Strix Halo's shared memory pool lets users dynamically allocate memory to graphics and AI inference tasks - up to 96GB can be assigned as dedicated VRAM. This eliminates rigid memory segmentation bottlenecks that prevent compact PCs from running large LLMs. A 32B model quantized to Q4 requires roughly 20GB of memory; on a traditional system with 8GB VRAM, it simply cannot load. On Strix Halo, the same model fits comfortably within the shared pool alongside the operating system and other applications.

Benchmarking Qwen 3.5 on Strix Halo: Real Inference Speeds

To understand whether AMD Strix Halo can handle advanced Local AI workloads, community users have tested Qwen 3.5 models with llama.cpp under different quantization formats and acceleration backends.

Source: r/LocalLLaMA community benchmark

Source: r/LocalLLaMA community benchmark

All benchmark data below was captured under a fixed 30k context window using llama.cpp with ROCm 7.2 / Vulkan acceleration on 128GB Strix Halo hardware. MoE model figures come from public r/LocalLLaMA community benchmark logs; dense model estimates are based on Strix Halo memory bandwidth and ROCm inference characteristics.

Qwen 3.5 Inference Performance (Ryzen AI Max+ 395, 128GB)

MoE data: r/LocalLLaMA community benchmarks (Strix Halo laptop, llama.cpp, ROCm 7.2, 30k context). Dense model figures: hardware-based estimation. Actual performance varies by OS, backend, and power profile.

What the Numbers Mean

The 7B model is the everyday workhorse - fast enough for real-time chat, translation, and quick coding assistance. The 14B model represents the practical sweet spot for most users, offering meaningfully better reasoning and instruction following without excessive memory pressure.

The MoE models deserve special attention. The 30B-A3B variant achieves roughly 50 tok/s despite having 30B total parameters, because it only activates 3B parameters per token. This architecture makes MoE models exceptionally efficient on unified memory systems like Strix Halo. The larger 122B-A10B MoE model also runs at a usable 18-19 tok/s, demonstrating that Strix Halo can handle very large models when paired with efficient quantization.

The 32B dense model is where memory capacity becomes the true differentiator. At Q4_K_M, it requires 19-22GB - well within Strix Halo's limits. But real-world LLM usage introduces complexity: long-document analysis, extended code reasoning sessions, and multi-turn conversations with large context windows can push memory consumption significantly higher due to KV cache growth. On a 64GB system, this creates pressure. On a 128GB Strix Halo configuration, there is ample headroom to sustain these workloads without swapping.

Q4 vs Q8: Choosing the Right Quantization

Q4_K_M is the standard choice for Local AI on compact hardware. It reduces memory footprint by roughly 60-70% compared to full precision, with minimal quality degradation for most tasks. Q8_0 retains more of the original model's precision, offering marginal improvements on complex reasoning benchmarks - but at nearly double the memory cost and slower generation speeds.

Quick Start: Running Qwen 3.5 Locally on Strix Halo

Ollama has made local model deployment straightforward. The setup differs slightly between operating systems.

Step 1: Install Ollama

Linux users gain full ROCm GPU acceleration out of the box with most modern distributions. Windows users need to manually enable the Vulkan backend within Ollama configuration files for optimal token generation speed, as native ROCm support on Windows is still maturing.

Step 2: Download a Qwen 3.5 Model

Run ollama run qwen3.5:14b to download and start the 14B model. Choose a model size that matches your available memory:

Step 3: Adjust Context Length

Context length has a major impact on memory usage. The KV cache grows linearly with context window size, so a 32k context window uses roughly 4x the KV memory of an 8k window. A balanced 4K-8K configuration works well for most interactive use cases. Reserve larger windows (32k+) for document analysis sessions where the model needs to process long inputs.

Running Multiple AI Models on a 128GB Strix Halo System

A 128GB Strix Halo configuration can run a complete local AI stack without memory pressure. Here is an example setup on the ACEMAGIC M1A PRO:

This lightweight stack is designed for personal RAG knowledge bases - suitable for daily document retrieval and AI note organization without occupying excessive hardware resources. The remaining 90GB+ of headroom supports adding a second LLM for specialized tasks, extending context windows to 64k for deep research analysis, or running larger models like Qwen 3.5 32B alongside the embedding pipeline.

ROCm vs CUDA: Choosing the Right Software Stack

NVIDIA's CUDA remains the industry standard for AI workloads, with broader framework support, more optimisation tools, and deeper enterprise adoption. For users focused on model training, large-scale deployment, or CUDA-dependent frameworks, an NVIDIA RTX workstation is the clear choice.

For local inference, however, the gap has narrowed significantly. ROCm on Strix Halo now supports llama.cpp, Ollama, LM Studio, vLLM, and other open-source inference tools with competitive performance. Community benchmarks show ROCm delivering more than double the prompt processing throughput compared to Vulkan on the same hardware.

AMD Strix Halo vs Mac Studio vs NVIDIA RTX Workstation

Each platform - Strix Halo, Mac Studio, and RTX workstations - serves a different user profile. The comparison below focuses on practical Local AI capability rather than synthetic benchmarks.

AMD Strix Halo Mini PC

• Up to 128GB LPDDR5X unified memory
• Windows / Linux flexibility
• Best price-to-memory ratio (~£1,600-2,500 for 128GB)
• Strong open-source LLM compatibility (GGUF, Ollama)
• Weakness: ROCm software ecosystem lags behind CUDA for model training

Mac Studio

• Up to 192GB unified memory (M4 Ultra)
• Excellent power efficiency, polished macOS experience
• Strong MLX-optimized inference (~£2,600-3,200 for 128GB)
• Weakness: Locked macOS ecosystem, no Windows/Linux support

RTX 4090/5090 Workstation

• 24-32GB VRAM, maximum CUDA inference speed
• Full training and fine-tuning capability
• Broadest AI framework support
• Weakness: Bulky chassis, high idle power draw, expensive entry cost for large VRAM

Building AI Agent Workflows on Strix Halo

Local AI is evolving beyond simple chat. Modern AI Agent systems can search documents, call external tools, and complete multi-step tasks autonomously. Strix Halo's large memory pool makes it a practical platform for experimenting with these workflows locally.

Personal AI: Privacy-First Document Processing

For individuals handling sensitive files - legal documents, medical records, financial statements - a local AI Agent eliminates the need to upload private data to cloud services. Strix Halo can run a Qwen 3.5 model alongside a vector database, enabling private document search, automated summarization, and knowledge extraction entirely on-device.

Developer AI: Private Code and RAG Iteration

Developers working with proprietary repositories benefit from a local environment that keeps source code off external servers. A Strix Halo system can run a coding-focused LLM, index project documentation into a local vector store, and iterate on RAG pipelines without API costs or data exposure concerns. The 128GB memory pool supports running the LLM, embedding model, and test database simultaneously.

SME AI: Team Knowledge Base with UK GDPR-Compliant Data Isolation

A compact Strix Halo Mini PC can serve as a shared internal AI assistant for small teams / SMEs. Product documentation, customer FAQs, and internal procedures get indexed into a local vector database. Team members query the system through a simple interface, getting answers grounded in company-specific knowledge - with all data remaining on-premises. This approach costs a fraction of enterprise AI platforms while maintaining full data control.

Key Limitations: What Strix Halo Cannot Do

AMD Strix Halo is impressive for a compact AI system, but it has clear boundaries:

The primary limitation is software ecosystem maturity. ROCm has improved rapidly, but CUDA still offers broader framework support, more optimisation guides, and deeper enterprise tooling. Users who depend on CUDA-specific workflows (TensorRT, Triton Inference Server, large-scale distributed training) will find Strix Halo insufficient.

A secondary constraint: Strix Halo is not designed for large-scale full model training. LoRA and QLoRA fine-tuning on smaller models is feasible, but training a model from scratch or performing full-parameter fine-tuning on 7B+ models requires significantly more compute than a single APU can deliver.

A minor trade-off exists for running large LLMs at higher precision. Running Qwen 3.5 32B at Q8_0 or loading 70B+ models requires careful memory management and quantization compromises. Users who need maximum LLM quality at full precision should consider multi-GPU workstation setups.

Who Should Buy a Strix Halo Mini PC for Local AI?

Recommended For

• Users building private Local AI systems
• Developers testing open-source LLMs (Qwen 3.5, Llama, etc.)
• AI Agent experimentation and RAG development
• Private document processing with data isolation
• Small teams deploying internal knowledge assistants

Not Ideal For

• Users who rely entirely on cloud AI services
• Large-scale model training or fine-tuning at scale
• Enterprise production deployment requiring CUDA
• Users who need maximum benchmark performance

ACEMAGIC M1A PRO: A Strix Halo Mini PC for Local AI

The ACEMAGIC M1A PRO is a representative Strix Halo Mini PC that fully unlocks the platform's 128GB memory potential. Ships with AMD Ryzen AI Max+ 395, 128GB LPDDR5X-8000 unified memory, and 2TB SSD storage - a ready-to-use configuration for out-of-the-box Local AI testing with Qwen 3.5 and AI Agent development.

ACEMAGIC M1A PRO

AMD Ryzen AI Max+ 395 (16C/32T)

128GB LPDDR5X-8000 Unified Memory

Up to 96GB Dynamic VRAM Allocation

2TB NVMe SSD

Compact Strix Halo Mini PC with full 128GB unified memory - suitable for running Qwen 3.5 32B and other large LLMs, building local RAG pipelines, and experimenting with AI Agent workflows.

View Product (UK Store)

Frequently Asked Questions

Can AMD Strix Halo run Qwen 3.5 locally?

Yes. Strix Halo runs Qwen 3.5 models through llama.cpp, Ollama, or LM Studio. A 128GB configuration handles all Qwen 3.5 variants up to 32B dense and 122B MoE models. See the benchmark table above for specific performance figures.

Can Strix Halo run 32B LLMs?

At Q4_K_M quantization, Qwen 3.5 32B requires 19-22GB - well within a 128GB system's capacity. Q8_0 pushes usage to 34-38GB, still feasible but leaves less room for multi-model parallel loading.

Can Strix Halo run multiple AI models at the same time?

A 128GB configuration supports a language model, embedding model, and vector database simultaneously using roughly 30-35GB total. This leaves substantial headroom for adding models or extending context windows.

Does the Strix Halo NPU accelerate LLM inference?

Most LLM inference depends on GPU acceleration and memory bandwidth. The XDNA 2 NPU targets future low-power AI applications and hybrid NPU+GPU inference (currently limited to smaller models via tools like Lemonade). It does not replace GPU acceleration for large language model inference today.

Is Windows or Linux better for running AI models on Strix Halo?

Linux provides better ROCm support and more flexibility for advanced AI workloads. Windows offers easier daily usage but requires manual Vulkan backend configuration for optimal inference speed.

Is Strix Halo better than Mac Studio for AI?

It depends on your workflow. Mac Studio remains the premium choice for users embedded in the Apple ecosystem who value power efficiency and MLX optimisation. Strix Halo excels for users who prefer open-source AI tools, Windows/Linux flexibility, and a lower price-to-memory ratio. Mac Studio and Strix Halo serve different priorities - neither is universally better.

Conclusion

AMD Strix Halo creates a new category of Local AI hardware. It does not replace NVIDIA workstations or enterprise training systems. Instead, it fills a different role: a compact personal AI computer designed for running and experimenting with Local AI workloads.

For hobbyists, independent developers, and small teams / SMEs seeking offline private AI without bulky workstation hardware, Strix Halo fills a long-empty niche in the local LLM hardware market. Paired with open-weights models like Qwen 3.5 and tools like Ollama, a Strix Halo Mini PC makes practical local AI accessible in a form factor that fits on a desk.

References

• Alibaba. (2026). Qwen 3.5 Open-Weights LLM. Released February 2026 under Apache License. HuggingFace / Ollama
• r/LocalLLaMA Community. (2026). Some Qwen 3.5 benchmarks on Strix Halo and llama.cpp. Community benchmark logs, 30k context, ROCm 7.2 / Vulkan. Reddit
• AMD. (2025). AMD Ryzen AI Max+ 395 Processor Specifications. AMD Official
• Wikipedia. (2026). Qwen (language model). Model family history, sizes, and licensing. Wikipedia
• kyuz0. (2026). AMD Strix Halo Toolboxes. ROCm 7.2 setup guides and llama.cpp build configurations. GitHub
• Ollama. (2026). Run Qwen 3.5 locally. Open-source local model inference framework. ollama.com