Robotics and AR Learning Experience

Introduction

The rationalization of the change in the education unit curriculum in Indonesia to K-13 was carried out due to various challenges faced including future challenges, competencies needed in the future, community perceptions, development of knowledge and pedagogy, as well as various negative phenomena that emerged. Education in accordance with future needs will only be realized if there is a shift or change in mindset in the learning process, with several points including: from one direction to interactive and from a single tool to multimedia tools. Thus, the authors conducted research on design of multimedia learning with the help of robot assistants. Multimedia learning contains material about the solar system, designed on the basis of marker based augmented reality using the Vuforia SDK in the form of Universal Windows Platform applications that are built on personal computer devices. The robot assistant is designed on a humanoid robot, with embedded intelligence that has the ability to detect and recognize text, using EAST Text Detector and Tesseract OCR, on the multimedia screen in the form of a planet name and the ability to find the fastest path to the destination planet to press buttons on the multimedia screen to change the augmented display reality planet. The entire design was tested and implemented at CoppeliaSim and managed to get a 100% accuracy and success rate.

Scope of Problem

1. Multimedia AR designed based on marker based tracking, in desktop application form, build on Microsoft Surface Pro 4;
2. Robot Assistant implemented on Humanoid Robot, planted embedded intelligence/artificial intelligence;
3. Testing is carried out on CoppeliaSim Simulator.

Scope of Embedded Intelligence

1. Able to detect (EAST Text Detector) and recognize (Tesseract OCR) text on multimedia screen (planet name);
2. Able to find the shortest path to the destination planet (Shortest Path Algorithm in CDLL Data Structure).

Block Diagram

Multimedia AR (Mercury)

Program Output on Console