ABSTRACT
The development of modern cyber-physical systems focused on user interaction with an intelligent environment has received a powerful impetus in the context of the COVID-19 pandemic. This has led to an urgent need for the creation of automated customer service systems in the catering sector, as well as the personalisation of user interaction with the cyber-physical environment. In an environment where most people are required to wear a mask, classical face recognition and tracking algorithms fail to cope with this task. During this study, a cyber-physical system for automated service of restaurant visitors was proposed. To create it, the following tasks were solved: human detection, identification of the face of the person wearing the mask, localisation and tracking of a person on the territory of a restaurant complex, determination of the status of a person. To solve the problem of face recognition in conditions of their partial overlap by personal protective equipment, an approach based on the FaceNet neural network model was proposed. For its training, a specialised semisynthetic data set Masked VGGFace2 was prepared. During the experiments, higher indicators of the quality of face recognition were obtained without masks (AP = 0.9635) and with masks (AP = 0.9488). In comparison with the original model, the increase in the recognition accuracy by the AP metric was more than 24% in the first case and about 3% in the second. A simulation of the developed algorithm is presented, which, when a person enters a restaurant complex, detects his face, checks for the presence of a mask on it, identifies it and tracks it. The system also allows assigning an order number to the person who made and paid for it. The developed solutions can serve as a basis for automated delivery of an order to a visitor using a mobile robot and correct planning of the trajectory of its movement.
