This research presents a geographical vector agent (GVA) approach, as an automated processing unit, to detect and extract 3D roof planes from color aerial images and light detection and ranging (LiDAR) data. In general, VAs are objects that can represent (non) dynamic and (ir) regular vector boundaries. Such dynamic geometry, together with thematic meanings of feature types, enables VAs to take advantage of the spatial relationships, such as connectivity, adjacency, and intersection that might define 3D roof planes. In this way, VAs can readily integrate and use the LiDAR and imagery datasets. First, a set of rules is defined in terms of spatial and non-spatial properties of 3D roof planes. These rules are then applied to detect and extract the boundaries of sample roof planes in a selected urban envrionment. The experimental tests show satisfactory performance and illustrate the capabilities of GVAs for generic applicability towards the extraction of building boundaries.