Manual work is a weak link within the intelligent manufacturing, however, it plays an important role in the highly customized and multi-variety assembling. Assisted by intelligent assembling technology such as augmented reality, a manual worker can integrate into the cyber-physics system to improve efficiency and reduce errors, which is of great engineering significance in the assembling field of industry 4.0. Assembly recognition is the initial part of progress analysis and it has predictable changing progress stages which can be matched with the digital model for recognition constraints. Therefore, based on the similarity between spatial increment information and part model, a real-time assembly recognition method is proposed in this paper. Firstly, the depth images from the multi-camera system were used to capture the assembling scene. Then, compared with the previous assembling scene, the spatial incremental information was used to quantitatively represent the assembled part. The spatial increment information and digital model are described with distance distribution. Finally, based on Earth mover’s distance algorithm, the matching between the spatial increment information and the part model indicates the part which had been assembled to realize the real-time assembly recognition. In the case study, an assembling process for 3D printing assembly which corresponded with the digital model was used to approve the feasibility of the real-time assembly recognition method.