Francisco Lourenço

Aerial Roof Facet & Obstruction Instance Segmentation for 3D House Reconstruction

At Ellipse I worked on the computer vision backbone that turns aerial imagery into structured geometry for rooftop solar design. My focus was to reliably understand how a roof is shaped and what objects sit on top of it, so downstream tools can estimate usable solar area and generate accurate 3D models.

I developed two core segmentation systems:

• Roof facet segmentation – models that detect and separate individual roof planes (facets) from high-resolution imagery and elevation data, providing clean polygons that can be directly consumed for 3D reconstruction and solar layout.

• Object & obstruction segmentation – dedicated models for rooftops elements (chimneys, skylights, vents, HVAC units, nearby trees, etc.) to automatically flag areas that reduce usable panel surface or cast shadows.

To support these models long term, I also designed and built a new in-house dataset for Ellipse:

I created the annotation guidelines, class taxonomy and quality checks, then coordinated and reviewed labeling so that every polygon, facet and obstruction was consistent with how the models and solar design tools would consume them. The result is a production-ready dataset and segmentation stack that Ellipse can continue to expand as they scale to new cities and imagery sources.

🧠 Roof facet and obstruction segmentation results. Each image has original on the left, ground truth in the center and inference on the right.

City-Scale Point Cloud Colorization & Georeferencing Pipeline

EyeVi Technologies is a geospatial, AI-powered startup that offers an on-demand mobile mapping solution for road network intelligence, combining 360° imagery, LiDAR and GNSS/INS to help cities and infrastructure owners maintain roads more efficiently and build high-detail digital twins.

I joined the team as a 3D ML/CV engineer and took ownership of the all-in-one processing pipeline that turns raw sensor logs from EyeVi's survey vehicles into a single, coherent, city-scale representation of the street network. Each drive produces massive heterogeneous data: 360° RGB panoramas, dense LiDAR scans, and a navigation trajectory refined with LiDAR-based ICP.

My job was to design and implement a robust, production-ready pipeline that could:

• Ingest and validate raw survey logs
• Reconstruct continuous 3D street scans from LiDAR and trajectory
• Colorize every LiDAR point using the 360° imagery
• Georeference everything into a consistent global coordinate system
• Generate orthostrips: high-resolution top-down road mosaics created by stitching ortho photos

The pipeline starts with a strict synchronization and reconstruction stage. LiDAR packets and 360° camera frames are aligned with the refined GNSS/INS+ICP trajectory to rebuild a continuous, temporally ordered scan of the drive. This gives us a clean 3D backbone of the street geometry, even in challenging GPS conditions.

On top of this reconstruction, I implemented the colorization module. Each LiDAR point is transformed into the appropriate camera frame and projected onto the panoramic image, where the pipeline samples the corresponding RGB values. It handles multiple overlapping views, changing exposure, occlusions, and out-of-FOV points to produce dense, photorealistically colorized point clouds that keep both geometric accuracy and visual context.

Next, the georeferencing layer propagates the refined vehicle pose to every single point, anchoring the full dataset in a global coordinate frame. This is critical for EyeVi's use cases: road-asset inventories, condition monitoring, and digital twins that need centimeter-level consistency between passes, surveys, and external GIS data.

Finally, I extended the same pipeline to generate road orthostrips as a standard product. Using the colorized, georeferenced data, the system computes a ground-aligned sampling grid under the vehicle and produces high-resolution ortho photos of the road surface as it moves. These tiles are then stitched along the trajectory into long, continuous orthostrips.

Today, this all-in-one processing pipeline, from raw logs to colorized, georeferenced 3D scans and stitched orthostrips, sits at the heart of EyeVi's mapping stack and powers many of the AI and analytics tools delivered to clients.

📹 3D Reconstruction of a street using the all-in-one pipeline

3D Shape Classification System

This was a fully end-to-end solution I designed and delivered for Anything World. The objective was to develop a robust classification system to support both the Anything World API and the Animate Anything tool, which enables automatic rigging and animation of 3D characters.

Since different character types require distinct animation behaviors (e.g., a human moves on two legs, a dog on four, and a car uses spinning wheels), understanding the character's category is essential prior to animation. The tool supports over 50 categories, making accurate classification a foundational requirement.

I was responsible for researching, developing, and deploying a classification pipeline capable of taking an unaligned 3D mesh as input and outputting a single predicted class from the supported set.

After an in-depth literature review, I implemented and adapted a state-of-the-art multi-view graph convolutional neural network approach. The method renders multiple images of the 3D shape from evenly spaced viewpoints, extracts global features per image, and connects them in a graph structure processed by a GCN.

I introduced several custom modifications to the architecture to optimize performance and make it rotation-invariant, addressing the core challenge of working with unaligned meshes. These adaptations were specifically designed to meet the needs of real-world deployment within Anything World’s production environment.

The final model achieved a 95.6% F1-score on the rotated ModelNet40 benchmark — which, to the best of my knowledge, was a state-of-the-art result on this dataset at the time (2023).

📹 Classification system demo on the Animate Anything tool

3D Shape Automatic Alignment

This was a fully end-to-end project I led at Anything World, aimed at developing an automatic 3D alignment system to support both the Anything World API and the Animate Anything tool.

The objective was to rotate unaligned 3D characters into a canonical orientation before they were passed into the rigging and animation pipelines. Since these systems are rotation-variant, their performance was highly dependent on the initial pose of the character — making consistent alignment a critical pre-processing step.

I was tasked with researching, designing, and deploying a system that would take any unaligned 3D mesh as input and return a rotation matrix aligning the model to a predefined canonical pose.

After a comprehensive literature review, I found that existing methods did not meet the accuracy demands of our production requirements. As a result, I developed a custom hybrid method combining ideas from several academic approaches with in-house techniques I designed.

The resulting system — a proprietary method called “Symmetry-Aware and Base-Plane-Aided 3-Step Rotation Estimation” — achieved outstanding results. On a dataset of 220 diverse 3D characters across 56 categories (including people, animals, vehicles, and objects), it achieved a median mean angular error (MAE) of just 2.47°, meaning the system delivered near-perfect alignment in the vast majority of cases.

This solution is now a key component of the Anything World pipeline and is used in real-time for character animation and rigging in production.

📹 Automatic alignment system demo on the Animate Anything tool

3D Shape Part Segmentation System

This was another end-to-end project I developed at Anything World, focused on 3D shape part segmentation — a key step in the automatic rigging and animation of 3D characters. The objective was to identify the precise location of specific semantic parts (e.g., arms, legs, body) within 3D models to enable accurate joint placement and deformation.

Unlike the classification and alignment tasks I previously solved, a segmentation system was already integrated into both the Animate Anything tool and the Anything World API. My role was to either optimize the existing solution or design a more effective one from scratch.

After several iterations attempting to improve the current model, we determined that it had already been optimized to its full potential in terms of accuracy. As a result, I conducted a new round of literature review, identified a state-of-the-art method with potential advantages, and implemented a fresh solution for evaluation.

The newly developed model achieved similar accuracy to the existing system, thus revealing a likely dataset bottleneck. However, the key breakthrough was that my solution was significantly more efficient — offering reduced training time and substantially faster inference, which aligned better with our production requirements.

The final system was based on a graph convolutional network (GCN) for point clouds, enriched with ideas from multiple research papers. It achieved 88.8% mIoU on the ShapeNet Part benchmark — a competitive score, on par with the top-performing models on this dataset.

Thanks to its performance and speed, this new solution was successfully deployed into Anything World's production pipeline.

🧠 Part segmentation results across different categories

Custom Multi RGB-D Camera System Based 3D scanner

coatingAI is a startup pioneering the use of AI in the powder coating industry, with the goal of improving efficiency and reducing material waste while maximizing quality and precision.

A critical component of their pipeline involved obtaining a high-fidelity 3D understanding of the parts entering the powder coating line. I was brought on as their first 3D Computer Vision Engineer and was tasked with researching, designing, and deploying a custom multi-camera 3D scanning system — built entirely from scratch.

The system was based on multiple RGB-D cameras capturing metallic parts in motion, which were later fused into a full 3D model. Since I initially worked fully remotely from Coimbra, Portugal, I built a physical replica of the powder coating line in my office to prototype the solution. This setup included a metallic structure with movable mounts, allowing me to simulate the part transport mechanism and validate the camera placements.

In addition to the hardware design, I developed the full software pipeline responsible for processing the RGB-D data, estimating camera positions, and generating an accurate 3D scan of each part as it moved through the system.

The final solution delivered precise and consistent 3D reconstructions of metallic parts in real-time, and was successfully deployed into coatingAI’s production environment — enabling downstream automation for coating quality optimization.

📹 My first months working with coatingAI: Here you can see the first 3D scanner prototype and the sketchy metal structure I built to replicate the coating line.

Anomaly Detection System for a Glass Bottle Production Line

This was a research-focused project I contributed to at the Institute of Systems and Robotics (ISR), University of Coimbra, aimed at developing a computer vision-based anomaly detection system for a glass bottle production line.

My role centered on two key areas:

• Designing and implementing the anomaly detection software responsible for identifying visual defects or irregularities in real time.
• Contributing to the optimization of the camera setup, including the number, placement, and calibration of cameras to maximize inspection coverage and accuracy.

This project combined elements of classical and learning-based vision approaches, and was designed to operate under the high-throughput and precision requirements of industrial production lines.

6DoF Object Pose Estimation from RGB-D Images Using Machine Learning Approaches

Intel RealSense SR305, D415 and L515: Experimental Evaluation and Comparison of Depth Estimation

Evaluation of the Accuracy of Pose Estimation Based on Relative Pose