MQSS-Selector: RL-Guided Pass Selection for an MLIR Compilation Pipeline
A new arXiv preprint introduces MQSS-Selector, a reinforcement-learning-guided method for selecting optimization passes in an MLIR-based compilation pipeline. The work is situated within the emerging convergence of high-performance computing and quantum computing into unified HPCQC infrastructure, where compiler and runtime layers must bridge classical and quantum workflows.
Why it matters
Compiler pass ordering and selection are typically fixed or heuristic, and poor pass sequences can significantly affect execution time and generated code quality. In emerging HPCQC toolchains built on MLIR, the space of classical and quantum passes is larger and less well understood than in either domain alone. Applying reinforcement learning to pass selection moves from hand-tuned pipelines toward learned, workload-specific optimization; if the method works, it could reduce manual burden and improve portability across heterogeneous HPCQC testbeds.
AI analysis — not reported by the source
What this could make possible
0–2 years
- Plausible
MQSS-Selector could become a reusable RL pass-ordering component for MLIR-based quantum compilation toolchains such as Xanadu's Catalyst or academic HPCQC compilers.
MLIR already has quantum dialects and pass-management infrastructure. RL pass selection has been demonstrated in LLVM and could be adapted to quantum passes with modest engineering. The near-term path depends mainly on integration effort rather than new fundamental results.
2–5 years
- Plausible
Learned pass selection could reduce end-to-end compile time or improve circuit-level metrics on representative hybrid classical-quantum workloads, but only if reward signals are tied to actual backend cost models.
RL policies can learn to avoid expensive or unhelpful transformations after seeing enough training instances. However, constructing meaningful rewards for quantum programs requires access to backends or accurate resource estimators; without that, the policy may optimize irrelevant proxies.
5+ years
- Speculative
This work could seed fully learned whole-stack compiler policies for HPCQC systems that jointly optimize classical and quantum passes, reducing the need for per-machine hand tuning as quantum resources scale.
A single MLIR-based policy that observes both classical and quantum IR could coordinate transformations across the stack. This requires standardized quantum IRs, stable reward definitions, and generalization across very different quantum backends, all of which are unresolved today.
What would have to be true
- The RL policy must be trained on a sufficiently diverse set of HPCQC workloads with meaningful performance or fidelity feedback; otherwise it may not generalize beyond the training set.
- MLIR quantum dialects and pass boundaries need to be stable and well-defined across the relevant hardware targets.
- A reliable cost model or execution feedback loop must exist to provide reward signals for both classical runtimes and quantum circuit quality.
Who’s positioned
- Xanadu — Its Catalyst compiler already uses MLIR for hybrid quantum-classical compilation, positioning it to integrate learned pass selection for quantum-classical programs.
- Oak Ridge National Laboratory — Active in HPCQC integration and co-design, it could leverage learned pass ordering to improve performance of hybrid workloads on large systems.
- IBM — Has Qiskit and HPC partnerships; a learned MLIR pass policy could complement its quantum runtime and compiler stack, especially for hybrid cloud HPC jobs.
What could change this
- The abstract does not report quantitative results; it is unclear whether RL-selected passes outperform strong heuristic baselines or random search.
- The definition of 'HPCQC workload' is broad, and the policy may not transfer between different quantum backends or problem domains.
- Reinforcement learning for compiler pass selection can be brittle and may require retraining for new MLIR dialects or hardware targets.