I'm from Pakistan and currently working as a Graduate Fellow at Visual Computing Lab which is a part of the prestigious ISTI-CNR research institute located in Pisa, Italy. I am also pursuing my PhD studies at University of Pisa. My research focuses on user-assisted 3D reconstruction and in this context, I am working on the improvements in MoReLab.
I was recruited for ESR07 position in EVOCATION ITN. I Developed a PyQt based graphical user interface called MoReLab to perform user-assisted 3D reconstruction. MoReLab allows the user to add features manually instead of automatic feature detection and matching. Bundle adjustment computes a sparse 3D point cloud for manually added feature correspondences. After this, the user can use several primitive shape tools e.g., quadrilateral, cylinder, tube etc. to model different objects and export 3D mesh. MoReLab also allows the user to measure distance between any 3D points..
I worked as a Teaching Assistant for the course of Sensors, Perception and Actuation with instructor Ilya Afanasyev. My responsibility was to mark all homeworks, quizzes and exams according to instructor's guidelines.
ROS for beginners: Basics, Motion and OpenCV [certificate].
ROS for Beginners II: Localization, Navigation and SLAM [certificate].
MoReLab: A software for user-assisted 3D reconstruction [pdf]
Point cloud registration is a fundamental task in 3D reconstruction and environment perception. We explore the performance of modern Deep Learning-based registration techniques, in particular Deep Global Registration (DGR) and Learning Multi-view Registration (LMVR), on an outdoor real world data consisting of thousands of range maps of a building acquired by a Velodyne LIDAR mounted on a drone. We used these pairwise registration methods in a sequential pipeline to obtain an initial rough registration. The output of this pipeline can be further globally refined. This simple registration pipeline allow us to assess if these modern methods are able to deal with this low quality data. Our experiments demonstrated that, despite some design choices adopted to take into account the peculiarities of the data, more work is required to improve the results of the registration.
Evaluating deep learning methods for low resolution point cloud registration in outdoor scenarios [pdf] [code]
We present MoReLab, a tool for user-assisted 3D reconstruction. This reconstruction requires an understanding of the shapes of the desired objects. Our experiments demonstrate that existing Structure from Motion (SfM) software packages fail to estimate accurate 3D models in low-quality videos due to several issues such as low resolution, featureless surfaces, low lighting, etc. In such scenarios, which are common for industrial utility companies, user assistance becomes necessary to create reliable 3D models. In our system, the user first needs to add features and correspondences manually on multiple video frames. Then, classic camera calibration and bundle adjustment are applied. At this point, MoReLab provides several primitive shape tools such as rectangles, cylinders, curved cylinders, etc., to model different parts of the scene and export 3D meshes. These shapes are essential for modeling industrial equipment whose videos are typically captured by utility companies with old video cameras (low resolution, compression artifacts, etc.) and in disadvantageous lighting conditions (low lighting, torchlight attached to the video camera, etc.). We evaluate our tool on real industrial case scenarios and compare it against existing approaches. Visual comparisons and quantitative results show that MoReLab achieves superior results with regard to other user-interactive 3D modeling tools.
Deep learning based trajectory estimation of vehicles in crowded and crossroad scenarios [pdf] [code][video]
[talk]
Trajectory estimation of vehicles is an important part of traffic surveillance systems and self driving cars. The difficulty of task lies in variations in light intensity, sizes of objects and real time detection. In the past decade vision based Convolutional Neural Networks have shown some promising results in the area. However, these methods still face problems in crowded and crossroad scenarios when objects are very close together. We propose tracking by detection based trajectory estimation pipeline which consists of two stages: The first stage is the detection and localization of vehicles and the second stage is building associations in bounding boxes and track the associated bounding boxes. We analyze the performance of Mask RCNN benchmark and fYOLOv3 on UA DETRAC dataset which is a large scale real life traffic dataset. We evaluate certain metrics like inference time, Intersection over union (IoU), Precision Recall (PR) curve and mean Average Precision (mAP). Experiments show that Mask RCNN outperforms YOLOv3. After that, we analyze the performance of centroid tracker and SORT tracker. Experiments show that SORT tracker gives us smoother trajectory as compared to noisy trajectory obtained from centroid tracker. However, SORT tracker based trajectory gives 4 pixels more Euclidean distance loss than Centroid tracker based trajectory.
ROS-based integration of smart space and mobile robot as internet of robotic things [pdf] [code]
The rapid developments in the field of Artificial Intelligence are bringing enhancements in the area of intelligent transport systems by overcoming the challenges of safety concerns. Traffic surveillance systems based on CCTV cameras can help us to achieve safe and sustainable transport systems. Trajectory estimation of vehicles is an important part of traffic surveillance systems and self-driving cars. The task is challenging due to the variations in illumination intensities, object sizes and real-time detection. We propose tracking by detection based trajectory estimation pipeline which consists of two stages: The first stage is the detection and localization of vehicles and the second stage is building associations in bounding boxes and track the associated bounding boxes. We analyze the performance of the Mask RCNN benchmark and YOLOv3 on the UA-DETRAC dataset and evaluate certain metrics like Intersection over Union, Precision-Recall curve, and Mean Average Precision. Experiments show that Mask RCNN Benchmark outperforms YOLOv3 in terms of accuracy. SORT tracker is applied on detected bounding boxes to estimate trajectories. The tracker is evaluated using mean absolute error. We demonstrate that the developed technique works successfully in crowded and crossroad scenarios.
