Abstract: While classical reinforcement learning (RL) has gained popularity for creating and optimizing variational quantum circuits (VQCs), there is still no consensus on the best model for the underlying problem of quantum architecture search (QAS). Specifically, how to effectively scope the iterative VQC adjustment steps of the RL policy remains an open question, with various approaches offering distinct benefits and challenges. In this work, we propose an RL approach that can cohesively predict entire circuit layers simultaneously, enabling rapid iterations in QAS. Our method allows for variable circuit lengths and is problem-agnostic, provided an objective function is available to guide the RL process. This makes it suitable for a broad range of applications. We evaluate our approach on the combinatorial optimization problem MaxCut, and achieve competitive results in terms of circuit solution quality when compared to both gradient-based and gradient-free circuit optimization baselines.