Francisco Lourenço

3D Shape Classification System

This was a fully end-to-end solution I designed and delivered for Anything World. The objective was to develop a robust classification system to support both the Anything World API and the Animate Anything tool, which enables automatic rigging and animation of 3D characters.

Since different character types require distinct animation behaviors (e.g., a human moves on two legs, a dog on four, and a car uses spinning wheels), understanding the character's category is essential prior to animation. The tool supports over 50 categories, making accurate classification a foundational requirement.

I was responsible for researching, developing, and deploying a classification pipeline capable of taking an unaligned 3D mesh as input and outputting a single predicted class from the supported set.

After an in-depth literature review, I implemented and adapted a state-of-the-art multi-view graph convolutional neural network approach. The method renders multiple images of the 3D shape from evenly spaced viewpoints, extracts global features per image, and connects them in a graph structure processed by a GCN.

I introduced several custom modifications to the architecture to optimize performance and make it rotation-invariant, addressing the core challenge of working with unaligned meshes. These adaptations were specifically designed to meet the needs of real-world deployment within Anything World’s production environment.

The final model achieved a 95.6% F1-score on the rotated ModelNet40 benchmark — which, to the best of my knowledge, was a state-of-the-art result on this dataset at the time (2023).

📹 Classification system demo on the Animate Anything tool

3D Shape Automatic Alignment

This was a fully end-to-end project I led at Anything World, aimed at developing an automatic 3D alignment system to support both the Anything World API and the Animate Anything tool.

The objective was to rotate unaligned 3D characters into a canonical orientation before they were passed into the rigging and animation pipelines. Since these systems are rotation-variant, their performance was highly dependent on the initial pose of the character — making consistent alignment a critical pre-processing step.

I was tasked with researching, designing, and deploying a system that would take any unaligned 3D mesh as input and return a rotation matrix aligning the model to a predefined canonical pose.

After a comprehensive literature review, I found that existing methods did not meet the accuracy demands of our production requirements. As a result, I developed a custom hybrid method combining ideas from several academic approaches with in-house techniques I designed.

The resulting system — a proprietary method called “Symmetry-Aware and Base-Plane-Aided 3-Step Rotation Estimation” — achieved outstanding results. On a dataset of 220 diverse 3D characters across 56 categories (including people, animals, vehicles, and objects), it achieved a median mean angular error (MAE) of just 2.47°, meaning the system delivered near-perfect alignment in the vast majority of cases.

This solution is now a key component of the Anything World pipeline and is used in real-time for character animation and rigging in production.

📹 Automatic alignment system demo on the Animate Anything tool

3D Shape Part Segmentation System

This was another end-to-end project I developed at Anything World, focused on 3D shape part segmentation — a key step in the automatic rigging and animation of 3D characters. The objective was to identify the precise location of specific semantic parts (e.g., arms, legs, body) within 3D models to enable accurate joint placement and deformation.

Unlike the classification and alignment tasks I previously solved, a segmentation system was already integrated into both the Animate Anything tool and the Anything World API. My role was to either optimize the existing solution or design a more effective one from scratch.

After several iterations attempting to improve the current model, we determined that it had already been optimized to its full potential in terms of accuracy. As a result, I conducted a new round of literature review, identified a state-of-the-art method with potential advantages, and implemented a fresh solution for evaluation.

The newly developed model achieved similar accuracy to the existing system, thus revealing a likely dataset bottleneck. However, the key breakthrough was that my solution was significantly more efficient — offering reduced training time and substantially faster inference, which aligned better with our production requirements.

The final system was based on a graph convolutional network (GCN) for point clouds, enriched with ideas from multiple research papers. It achieved 88.8% mIoU on the ShapeNet Part benchmark — a competitive score, on par with the top-performing models on this dataset.

Thanks to its performance and speed, this new solution was successfully deployed into Anything World's production pipeline.

🧠 Part segmentation results across different categories

Custom Multi RGB-D Camera System Based 3D scanner

coatingAI is a startup pioneering the use of AI in the powder coating industry, with the goal of improving efficiency and reducing material waste while maximizing quality and precision.

A critical component of their pipeline involved obtaining a high-fidelity 3D understanding of the parts entering the powder coating line. I was brought on as their first 3D Computer Vision Engineer and was tasked with researching, designing, and deploying a custom multi-camera 3D scanning system — built entirely from scratch.

The system was based on multiple RGB-D cameras capturing metallic parts in motion, which were later fused into a full 3D model. Since I initially worked fully remotely from Coimbra, Portugal, I built a physical replica of the powder coating line in my office to prototype the solution. This setup included a metallic structure with movable mounts, allowing me to simulate the part transport mechanism and validate the camera placements.

In addition to the hardware design, I developed the full software pipeline responsible for processing the RGB-D data, estimating camera positions, and generating an accurate 3D scan of each part as it moved through the system.

The final solution delivered precise and consistent 3D reconstructions of metallic parts in real-time, and was successfully deployed into coatingAI’s production environment — enabling downstream automation for coating quality optimization.

📹 My first months working with coatingAI: Here you can see the first 3D scanner prototype and the sketchy metal structure I built to replicate the coating line.

Anomaly Detection System for a Glass Bottle Production Line

This was a research-focused project I contributed to at the Institute of Systems and Robotics (ISR), University of Coimbra, aimed at developing a computer vision-based anomaly detection system for a glass bottle production line.

My role centered on two key areas:

• Designing and implementing the anomaly detection software responsible for identifying visual defects or irregularities in real time.
• Contributing to the optimization of the camera setup, including the number, placement, and calibration of cameras to maximize inspection coverage and accuracy.

This project combined elements of classical and learning-based vision approaches, and was designed to operate under the high-throughput and precision requirements of industrial production lines.

6DoF Object Pose Estimation from RGB-D Images Using Machine Learning Approaches

Intel RealSense SR305, D415 and L515: Experimental Evaluation and Comparison of Depth Estimation

Evaluation of the Accuracy of Pose Estimation Based on Relative Pose