ALBERT

Description: This paper presents an improved Convolutional Neural Network (CNN) architecture to recognize surface defects of the Calcium Silicate Board (CSB) using visual image information based on a deep learning approach. The proposed CNN architecture is inspired by the existing SurfNet architecture and is named SurfNetv2, which comprises a feature extraction module and a surface defect recognition module. The output of the system is the recognized defect category on the surface of the CSB. In the collection of the training dataset, we manually captured the defect images presented on the surface of the CSB samples. Then, we divided these defect images into four categories, which are crash, dirty, uneven, and normal. In the training stage, the proposed SurfNetv2 is trained through an end-to-end supervised learning method, so that the CNN model learns how to recognize surface defects of the CSB only through the RGB image information. Experimental results show that the proposed SurfNetv2 outperforms five state-of-the-art methods and achieves a high recognition accuracy of 99.90% and 99.75% in our private CSB dataset and the public Northeastern University (NEU) dataset, respectively. Moreover, the proposed SurfNetv2 model achieves a real-time computing speed of about 199.38 fps when processing images with a resolution of 128 × 128 pixels. Therefore, the proposed CNN model has great potential for real-time automatic surface defect recognition applications.

Description: Human detection and tracking is an important task in artificial intelligent robotic systems, which usually require a robust target detector to work in a variety of circumstances. In complex environments where a camera is installed on a wheeled mobile robot, this task becomes much more difficult. In this paper, we present a real-time remote-control system for human detection, tracking, security, and verification in such challenging environments. Such a system is useful for security monitoring, data collection, and experimental purpose. In the proposed system, a Kinect RGB-D camera from Microsoft is used as a visual sensing device for the design of human detection and tracking. We also implemented a remote-control system on a four-wheel mobile platform with a Robot Operating System (ROS). By combining these two designs, a wireless controlled mobile platform having the feature of real-time human monitoring can be realized to handle this task efficiently. Experimental results validate the performance of the proposed system for wirelessly controlling the mobile robot to track humans in a real-world environment.

Description: Three-Dimensional (3D) object pose estimation plays a crucial role in computer vision because it is an essential function in many practical applications. In this paper, we propose a real-time model-based object pose estimation algorithm, which integrates template matching and Perspective-n-Point (PnP) pose estimation methods to deal with this issue efficiently. The proposed method firstly extracts and matches keypoints of the scene image and the object reference image. Based on the matched keypoints, a two-dimensional (2D) planar transformation between the reference image and the detected object can be formulated by a homography matrix, which can initialize a template tracking algorithm efficiently. Based on the template tracking result, the correspondence between image features and control points of the Computer-Aided Design (CAD) model of the object can be determined efficiently, thus leading to a fast 3D pose tracking result. Finally, the 3D pose of the object with respect to the camera is estimated by a PnP solver based on the tracked 2D-3D correspondences, which improves the accuracy of the pose estimation. Experimental results show that the proposed method not only achieves real-time performance in tracking multiple objects, but also provides accurate pose estimation results. These advantages make the proposed method suitable for many practical applications, such as augmented reality.

Description: In this paper, a manipulation planning method for object re-orientation based on semantic segmentation keypoint detection is proposed for robot manipulator which is able to detect and re-orientate the randomly placed objects to a specified position and pose. There are two main parts: (1) 3D keypoint detection system; and (2) manipulation planning system for object re-orientation. In the 3D keypoint detection system, an RGB-D camera is used to obtain the information of the environment and can generate 3D keypoints of the target object as inputs to represent its corresponding position and pose. This process simplifies the 3D model representation so that the manipulation planning for object re-orientation can be executed in a category-level manner by adding various training data of the object in the training phase. In addition, 3D suction points in both the object’s current and expected poses are also generated as the inputs of the next operation stage. During the next stage, Mask Region-Convolutional Neural Network (Mask R-CNN) algorithm is used for preliminary object detection and object image. The highest confidence index image is selected as the input of the semantic segmentation system in order to classify each pixel in the picture for the corresponding pack unit of the object. In addition, after using a convolutional neural network for semantic segmentation, the Conditional Random Fields (CRFs) method is used to perform several iterations to obtain a more accurate result of object recognition. When the target object is segmented into the pack units of image process, the center position of each pack unit can be obtained. Then, a normal vector of each pack unit’s center points is generated by the depth image information and pose of the object, which can be obtained by connecting the center points of each pack unit. In the manipulation planning system for object re-orientation, the pose of the object and the normal vector of each pack unit are first converted into the working coordinate system of the robot manipulator. Then, according to the current and expected pose of the object, the spherical linear interpolation (Slerp) algorithm is used to generate a series of movements in the workspace for object re-orientation on the robot manipulator. In addition, the pose of the object is adjusted on the z-axis of the object’s geodetic coordinate system based on the image features on the surface of the object, so that the pose of the placed object can approach the desired pose. Finally, a robot manipulator and a vacuum suction cup made by the laboratory are used to verify that the proposed system can indeed complete the planned task of object re-orientation.

Description: In the natural science curriculum, chemistry is a very important domain. However, when conducting chemistry experiments, safety issues need to be taken seriously, and excessive material waste may be caused during the experiment. Based on the 11-year-old student science curriculum, this paper proposed a virtual chemistry laboratory, which was designed by combining a virtual experiment application with physical teaching materials. The virtual experiment application was a virtual experiment laboratory environment created by using selected experimental equipment cards in combination with augmented reality (AR) technology. The physical teaching materials included all virtual equipment required for experiment units. Each piece of equipment had corresponding cards for learners to choose from and utilize in specific experimental operations. It was hoped that students were able to achieve the desired learning effectiveness of experimental teaching while reducing the waste of experimental materials through the virtual experimental environment. This study employed the quasi-experimental and questionnaire survey methods to evaluate both learning effectiveness and learning motivation. Eighty-one students and eight elementary school teachers were surveyed as research subjects. The experimental results revealed that significant differences in learning effectiveness existed between the experimental group and control group, indicating that the application of AR technology to teaching substantively helped enhance students’ learning effectiveness and motivation. In addition, the results of the teacher questionnaire demonstrated that the virtual chemistry laboratory proposed in this study could effectively assist with classroom teaching.