Application of Augmented Reality for Distance Learning to Teach Manufacturing Engineering during COVID-19 Social Distancing

The COVID-19 pandemic has created a shift in all our lives, turning in-class traditional educational systems into distance learning systems. Since distance learning is inevitable nowadays, the hands-on experience from in-class, particularly in practical sessions, is lacking. While there is a wide range of tools and technologies to enrich the student learning experience, the application of Augmented Reality (AR) is remarkable. This paper proposes a mobile app assisted with AR technology aimed to support the engineering faculty students to enhance their knowledge regarding the lathe machine and lathe operations during this pandemic period. The application consists of the augmented content of the separate parts of the lathe machine, exploded view and augmented animated content of the operations. The objective of this AR app is to guide the students regarding machineries and operations, and to introduce AR technologies to the local universities as a step of enhancing digital education. The adaptability of the AR app was experienced and verified by 72 students and the responses and feedback revealed their interest towards this educational approach. Conclusively, this study shows the effectiveness and importance of AR application specially for Manufacturing Engineering field during this distance learning period.


Introduction
The Covid-19 pandemic affected a large amount of the world population, that enforced businesses, educational institutions including schools and universities, and other organizations to migrate to digital platforms [15]. Social distancing has made in-class traditional education system into online teaching and learning as a method of delivering lectures, lab sessions and other learning activities to the students [1]. When considering in-class traditional systems, there are many advantages for students, like multiple learning opportunities and hands-on experiences. On the other hand, they have certain challenges also that need to be considered such as, students not getting a clear view when an expert is operating a machine due to the abundancy of students in a group, and heat management. However, social distancing has made the mode of education system into online learning system; it has restrictions, especially in practical sessions such as, inaptitude of handson experience, hardships in familiarizing with the machines and inability to develop practical skills [5].
Although there are many ways of delivering the subject content to the students through digital media during this social distancing, still there are many limitations and restrictions. This article discusses about minimizing these limitations and restrictions through Augmented reality-based tools and how the students can enhance their online learning experience through the new technologies.
With the continuous development of digital technologies, the current world is now entering a new era which is also known as Industry 4.0 [9]. Augmented reality (AR) is one of these digital technologies which interweaves digital data with physical spaces [6]. Digital data is laid onto the physical world, usually in the form of graphical augmentations. These augmentations have 3D effects, and they are interactive in real-time. Hence, there is a physical interaction between the user and the virtual environments.
AR offers tremendous benefits to various engineering fields such as mechanical engineering, industrial engineering, civil engineering, and aeronautical engineering [2]. Particularly, in Manufacturing and Industrial Engineering sector, AR is widely used for many purposes such as, in logistics, in training & learning and in layout planning etc. In industries, AR applications are used to visualize engineering concepts, visualize virtual prototypes, train employees, simulate the interactions of assemblies etc. [14]. Since AR technologies are used in various instances, this paper is focused on the educational applications in the manufacturing and industrial sector.
AR is being used in the higher education sector as well as in the educational sector where various modules of engineering are taught using this technology. AR is used when teaching engineering graphics and CAD models, machines and machine operations, product design, maintenance and planning [17]. Introduction of AR solutions for distance learning provide the remedies for challenges raised from the traditional online as well as inclass educational system. Rather than observing a video, the students can familiarize themselves with machines and machinery operations and even practice the operations and simulations virtually through these AR technologies. Even though these virtual technologies cannot provide the hands-on experience, they are pragmatic with the existing circumstances.
In order to overcome these challenges in terms of effectiveness and quality of teaching and learning during the social distancing, an AR application has been incorporated. Therefore, this paper describes the development of an AR based lathe machine to cover the knowledge regarding the identification of the machine parts, drilling and turning operations of the lathe machine. Also, this study aims to introduce the AR technology among the students and within the Department where AR is new to them. Further, the adaptability of the AR app was experienced and verified by 72 students in the Department of Manufacturing and Industrial Engineering.
Previous research reveals that it is essential to use Head Mounted Displays (HMDs) and various other hardware to obtain the AR experience [11]. Since it is difficult to facilitate this hardware during this pandemic period this research is focused only on mobile based AR applications. The existing AR apps related to machines and machine operations often fail to provide many options such as, AR lathe animations and AR models of separate parts [12,16]. Since AR based educational apps are rare in Sri Lanka, this study intends to introduce AR technology in the local context. In that respect, this research aims to support the engineering faculty students to enhance their knowledge regarding the lathe machine and lathe operations during this pandemic period using AR technologies. This AR app provides an adequate guide and information regarding the lathe machine components and operations.
The paper is organized as follows. In the literature review section, previous works related to AR educational apps for machinery technologies are presented. Thereafter, the limitations of these other apps and the research gaps are identified in the literature review section. The modelling, system architecture, app development and interface, and user guide and adaptability are presented in the methodology section. Subsequently, survey results and software validation are explained in the discussion section, and finally, conclusions and future directions are presented at the end of the paper.

2.
Literature Review AR and Virtual Reality (VR) technologies for educational purposes and training purposes are not new to the present world. This literature review section discusses the existing AR apps and their applications in education, advanced features of existing AR apps and limitations.
Reyes et al. [11] developed a mobile AR system to teach lathe and milling. Features like, incorporating the 3D models with real processes, are available in this application and to explain the setup pieces, tools and machine, augmented animations were used. The application was given to a group of mechanical undergraduate students, school teachers and technicians to get feedback and 75% of the users rated the app as good/very good.
Further, the study introduced a new method to explore 3D models of various equipment [4].
The course "cutting tools" is the module that the application was based on, and laboratory equipment, tooling, machine tools and measuring instruments were augmented giving a digital experience to the students.
AR applications are designed to alleviate the workload of the teachers when teaching manufacturing technology [12]. In this app, CA6140 lathe fork introduces the function and operation of the system and it is supposed to make the students understand and improve the convenience of the students. When the markers are being scanned, 3D-models are superimposed, and these augmented models can be rotated and scaled using gesture operation.
AR assisted CNC lathe app was developed that enables to rotate the virtual object around the x and y axes and left, right, up and down [7]. It avoids the necessity of rotating the mobile device or rotate the image target. The study discusses an AR application which helps the students to examine the lathe machine and its parts closely [16]. In this study, the product development, validation, revisions, product trials and final product development have been done. In this app, a virtual button is used to rotate the 3D model. The results show that 82% of the users categorized the app as good.
The authors have compared the 3D virtual simulation environment with the real laboratory classroom [8].
The virtual environment is based on measuring constructive and geometrical parameters of gear hobs. This new technology was designed as a free, safe, way to access and use 3D measuring tools in a virtual space. Thereafter, same measurements were taken in the conventional method and a comparison was conducted. The study reveals that virtual laboratory has some advantages such as possibilities of mathematical processing and analysis, preserving of measurements, and efficacy.
Liu et al. [10] presented a training app for lathe turning that allows to rotate and zoom by touch screen operations. The features considered in this study were safety precautions, teaching assignments, tools, training guidance, and comprehensive training. The survey was done, and the results reveal that this approach is very vivid, attractive, and a distinctive method of teaching. Also, the feedback said that even though hands-on experience is lacking, the application helped to strengthen the confidence in operational ability. In the relevant literature, several studies have been incorporated with application of AR and VR technologies into the learning systems which are listed in Table 1. In these studies, the most common hardware to obtain AR content is the mobile device, whereas, few studies used Head Mounted Displays (HMDs).
Reyes et al. [14] have developed an AR app that requires HMD device to experience the application. Also, this study facilitated these applications in the university premises. However, it is challenging to execute this application during this distance learning period due to the unavailability and cost of the HMDs [11]. Moreover, Alieve et al. [4] have created the AR app and facilitated the marker in student textbooks. However, the external students are unable to experience this app due to inaccessibility of marker. In addition, Pan et al. [12] have implemented an AR app only for CA6140 lathe fork that did not explore the details and other components of a lathe. Although the study implemented the AR app, its adaptability is not measured by students [7]. Further, the study that created the AR app for the lathe machine did not facilitate any augmented animations [16]. Therefore, to bridge these gaps identified from the literature, this study proposed a mobile app assisted with AR technology. In that respect, the study aims to support the engineering faculty students to enhance their knowledge regarding the lathe machine and lathe operations during this pandemic period. This AR app provides an adequate guide and information regarding the lathe machine components and operations. Also, this study intends to introduce AR technologies to the local universities as a step of enhancing digital education. Notably, this study created a marker that enables to experience the AR app from anywhere in the digital platform. Also, the hardware focused on this development process is a mobile device since it is accessible for many. Therefore, the authors believe that this research contributes to enhancing the existing educational system through the AR application.

Methodology
In this section, designing and implementation of an augmented reality application is presented. This app is developed for Android devices that provide 3D visualization of lathe machine and operations. The developed model shows similarities to the lathe machines that are being used in the University of Peradeniya. This 3D visualization is superimposed as the real-world equivalent, aiming to enhance the practical experience of the students during the coronavirus period. The methodology of this study consists of five main steps which include 3D Modelling and Texturing, Creation of AR image target, System architecture, App development, and User guide and evaluation of adaptability.

3D Modelling and Texturing
In this section, the development process of 3D lathe model and texturing is illustrated. In this modelling process, it is essential to consider few factors such as accurate measurements, proportionality, and inclusion of basic parts to provide quality distance learning experience. The 3D model of lathe developed in Blender was imported to Unity 3D software. The modelling process considered the experts who demonstrate the practical in the laboratory and their insights on machinery. The final lathe model consists of main components such as chuck, compound rest, carriage, headstock, tailstock etc.

3.2.
Creation of AR Image Target Image targets represent images that device can detect and track to produce an augmented content. Factors such as 1:1 ratio, image contrast, distribution in the texture, repetitive patterns and image dimensions were considered when creating the image target. The image target is of 1:1 ratio which is considered as the ideal ratio for an image target and only the colours black and white were used. While it is not mandatory to use only black and white, they were used because they have a high contrast from one to the other. In this markerbased AR application, this developed image target was used to obtain augmented content.

3.3.
System Architecture The architecture is based on views layer, app assets, Vuforia engine and rendering graphics. The interactions between the user and the device are depicted by the views layer. This AR app is a marker-based app where the camera of the portable device will detect the patterns of the marker (image target). The app will overlay digital information on the marker after it is recognized. The flow of the system architecture is given in Figure 1. Of the variety of software development kits (SDKs), Vuforia SDKs were selected in this study because it is free and the license Vuforia fulfils the required functionalities of this study. A further advantage of Vuforia AR SDKs is the ability to play videos and animations once the image target is identified.
When the camera tracks the image target along with the device, the camera is able to render the 3D content. Hence, with the graphic rendering, the user can obtain an augmented reality experience. To use Vuforia, it is mandatory to register at its website and create a unique individual license key after the registration. After the database of the target image is created, the database file of the target is transferred to Unity 3D. The orientation of 3D object on the image target can be adjusted according to the developer.

App Development
In the present world, smartphones, tabs, and other devices often read business cards, QR codes and other markers by scanning them [13]. Unity3D is an application creating system developed by Unity technologies and we have used this software to create the app in this study. The distinguishing characteristic of Unity is that the graphic engine can support file formats from Blender, 3DS Maya, Softimage, Cinema 4D etc. [3]. This engine is based upon JavaScript and C# and, in this study, scripts have been written in C#. This app has not been designed for multiple users, and the user is advised to follow the instructions depicted on the interface of the app. Under the Assets, 16 Scenes were created, and each scene represents each interface of the mobile application. Figure 2 shows the user flow diagram, and the forward arrows depict when the user is moving forward with the application while the backward arrows depict the flow when the user is moving backward in the app. At the image target detection stage, 3D rendering begins after the proper identification of the marker.
Also, the app consists of two 3D animations to demonstrate the drilling and turning operations. The application contains several CAD files and after importing the CAD files to Unity-3D, animations were performed as needed. For example, there were no animations for the separate parts but, for the drilling and the turning operations, animations were created using Unity tools.

3.5.
User Guide and Evaluation of Adaptability After the installation of the application, the user will obtain a splash screen and the user is directed to the image target. By clicking the menu button, the user will be able to access the functionalities such as parts of the lathe machine, exploded view, drilling operation, and turning operation. The "Back" button directs the user to move backwards. The application consists of 16 UIs and each UI gives practical knowledge about the lathe machine as well as the machining operations.
Finally, to validate the adaptability, 72 engineering undergraduate students participated in a survey. The survey was based on measuring the performance and capabilities In 14 of the 20 questions, students had to rate the system and the rating system ranged as follows. 5 -Very good, 4 -good, 3satisfactory, 2 -below average and 1unsatisfactory. Two questions were yes or no type questions and 4 questions out of 20 were written question where the students had to provide their comments. The basic sectors that were assessed in the survey are given below.
• User experience -Does the tool provide accurate information about the Lathe Machine? • Did you undergo a better practical/lab experience than other online learning methods? • System performance and the accuracy of the marker detection (Image target) • Has the user used an augmented reality system before or not? • The interface is attractive/not?
The above points assessed using a rating system while for the following questions the students had to provide open answers.
• Personal experience while using the app and suggestions to improve the app • Three positive experiences and three negative responses

Results and Discussion
In this section, the results of this study are discussed. In that respect, the developed 3D lathe model, Target image, User interfaces of AR app, and the survey responses are illustrated.

3D Lathe Model and Image Target
The 3D lathe model and its isometric view which is developed and textured by the blender software is presented in Figure 3. The model shows the level of lathe details considered in this study. Modelling in Blender has benefits as well as limitations. Although there are many 3D modelling software, Blender is the only open source software that is facilitated with Unity Software. However, Blender has assembly limitations and meshing issues. It does not offer mating options like other CAD software during the assembling process. Hence there can be issues in placing the parts in precise positions. The generated image target for the application is presented in Figure 4. This image target enables to visualize augmented content when the camera of the device recognises it.

AR app and Features
The AR app is developed and facilitated in the Department of Manufacturing and Industrial Engineering to provide quality learning opportunities especially on practical sessions. In that respect, the developed app interface of launch screen is presented in Figure 5. The interface of main functionalities of the app is presented in Figure 6. Also, the interface of machine components is presented in Figure 7.
The AR app was developed in Unity due to advanced rendering features, user friendliness, and animating options. However, it also has limitations such as unsupportive to many CAD software, limited features in certain versions, and problems with switching build targets. In the free version of Unity, high quality assets that aid to create virtual environments are inaccessible. Advanced and more realistic virtual environments could be created if sophisticated versions of Unity software are used.  The augmented reality content of all lathe components is included in this app, to experience each part descriptively. Accordingly, the augmented reality content of the compound rest is presented in Figure 8. This application is divided into several separate sections, so that the students can learn in a sequence. This application was developed as a e-learning tool which would help the students to grasp the basic knowledge of the subject while they are simultaneously learning from the conventional online teaching methods.
Further, two basic operations are facilitated in this app that provide the augmented animation of both lathe operations such as drilling and turning. Accordingly, Figure 9 shows the augmented reality of a lathe drilling operation. The adjustment of the device around the image target will provide the various views and details.
Unity 3D software which incorporates with Vuforia and Blender software was used to develop the app as well as lathe animations. Unity enables advanced animation tools and features that can create more realistic virtual environments [18]. Although, it can facilitate human intervention-based animations for machine operations, this research does not include the human intervention-based animations due to resource unavailability.

Survey Results
The summary of responses of all 72 students was analysed to evaluate the adaptability. The comparison of survey summary is presented in Figure 10. Also, remarkable feedback, including positive, and negative, is listed in Table 2. The survey results reveal constructive criticisms that aid to enhance this app in the future. Notably, the interest on exploring AR apps for the other machine shows the students' interest and enthusiasm to facilitate AR based distance learning. Overall, the results show the quality, necessity, and adaptability of facilitating this AR based distance learning approach in the manufacturing and industrial engineering context. However, there are few unsatisfactory remarks that can be attended to in future.

Conclusion
The emerging learning methods influenced the necessity to upgrade the traditional learning tools. Notably, AR based teaching has improved the student learning process in many countries nowadays. The purpose of this study is to introduce AR applications to distance learning in this pandemic period. Accordingly, in this research, AR based lathe application is developed and implemented to support the engineering faculty students to enhance their knowledge regarding the lathe machine and lathe operations. Also, this study aims to introduce AR technologies to the local universities as a step of enhancing digital education.
Hence, this study was developed and facilitated the mobile app assisted with AR technology for lathe machine and operations in the Department of Manufacturing and Industrial Engineering. The survey results reveal the student interest and enthusiasm towards these AR applications. Also, it has been requested by the students to enhance and expand this AR application to other machineries. There were a few negative feedbacks regarding this AR app and the authors aim to remedy these drawbacks and enhance these applications in the future.   Very useful in the home learning period.  3D models give a better understanding of the concept.  You do not have to go to the LAB to use the machine instead this is very useful to get used to the machine.  Can observe the machine parts from wherever you are.  Actually, it is good. First time I used this kind of app.  I had very good experience with the system.  That was my first mobile augmented reality system experience.  Rather than usual online lab videos that was very nice.  The AR models were awesome.
 Only for android.  Some details missing in the lathe machine.  Sometimes it crashes when moving the phone.  If possible, display it without scanning QR, it will be better.  Need to print image target.  Cannot replace the true experience.

Figure 10 -Students' Feedback Summary
However, some limitations of this AR based mobile app approach for distance learning do exist. The application was only developed for the android users because most of the students possess android devices. To learn the AR and VR technologies through a fully immersive environment, many more hardware is needed. Especially during the pandemic, this hardware is not accessible for students. The virtual environments often remain in the university premises and the students cannot have access to these resources during distant learning. Also, this AR app only facilitates the engineering faculty students at University of Peradeniya.
In future work, the application could be uploaded to the play store to make it accessible for everyone. Also, the application can be expanded for the IOS devices as well. Further, to enhance this approach and quality, the advanced devices like HMDs can be facilitated.