Integrating External Ground Points in Forests to Improve DTM from Dense-Matching Photogrammetry

The biggest problem of generating a Digital Terrain Model (DTM) from the photogrammetric point clouds that are produced from aerial imagery with dense-matching software such as SURE, Pix4D, or Photoscan is dense vegetation: when plants completely cover the terrain not a single point is generated on the ground. This is different for LiDAR point clouds as the laser can even penetrate dense multi-level tropical forests. The complete lack of ground points in larger vegetated areas such as closed forests or dense plantations means that the many processing workflows for vegetation analysis that have been developed for LiDAR cannot be used for photogrammetric point clouds  … unless … well unless we are getting those missing ground points some other way. In the following we see how to integrate external ground points to generate a reasonable DTM under a dense forest with LAStools. See this, this, this, this, and this article for further reading.

Here you can download the dense matching point cloud, the manually collected ground points, and the forest stand delineating polygon that we are using in the following example work flow:

We leave the usual inspection of the content with lasinfo and lasview that we always recommend on newly obtained data as an exercise to the reader. Using las2dem and lasgrid we created the Google Earth overlays shown above to visualize the extent of the dense matched point cloud and the distribution of the manually collected ground points:

las2dem -i DenseMatching.laz ^
        -thin_with_grid 1.0 ^
        -extra_pass ^
        -step 2.0 ^
        -hillshade ^
        -odix _hill_2m -opng

lasgrid -i ManualGround.laz ^
        -set_RGB 255 0 0 ^
        -step 10 -rgb ^
        -odix _grid_10m -opng

Attempts to ground-classify the dense matching point cloud directly are futile as there are no ground points under the canopy in the heavily forested area. Therefore 558 ground points were manually surveyed in the forest of interest that are around 50 to 120 meters apart from another. We show how to integrate these points into the dense matching point cloud such that we can successfully extract bare-earth information from the data.

In the first step we “densify” the manually collected ground points by interpolating them with triangles onto a raster of 2 meter resolution that we store as LAZ points with las2dem. You could consider other interpolation schemes to “densify” the ground points, here we use simple linear interpolation to prove the concept. Due to the varying distance between the manually surveyed ground points we allow interpolating triangles with edge lengths of up to 125 meters. These triangles then also cover narrow open areas next to the forest, so we clip the interpolated ground points against the forest stand delineating polygon with lasclip to classify those points that are really in the forest as “key points” (class 8) and all others as “noise” (class 7).

las2dem -i ManualGround.laz ^
        -step 2 ^
        -kill 125 ^
        -odix _2m -olaz

lasclip -i ManualGround_2m.laz ^
        -set_classification 7 ^ 
        -poly forest.shp ^
        -classify_as 8 -interior ^
        -odix _forest -olaz

Below we show the resulting densified ground points colored by elevation that survive the clipping against the forest stand delineating polygon and were classified as “key points” (class 8). The interpolated ground points in narrow open areas next to the forest that fall outside this polygon were classified as “noise” (class 7) and are shown in violet. They will be dropped in the next step.

We then merge the dense matching points with the densified manual ground points (while dropping all the violet points marked as noise) as input to lasthin and reclassify the lowest point per 1 meter by 1 meter with a temporary code (here we use class 9 that usually refers to “water”). Only the subset of lowest points that receives the temporary classification code 9 will be used for ground classification later.

lasthin -i DenseMatching.laz ^
        -i ManualGround_2m_forest.laz ^
        -drop_class 7 ^
        -merged ^
        -lowest -step 1 -classify_as 9 ^
        -o DenseMatchingAndDensifiedGround.laz

We use the GUI of lasview to pick several interesting areas for visual inspection. The selected points load much faster when the LAZ file is spatially indexed and therefore we first run lasindex. For better orientation we also load the forest stand delineating polygon as an overlay into the GUI.

lasindex -i DenseMatchingAndDensifiedGround.laz 

lasview -i DenseMatchingAndDensifiedGround.laz -gui

We pick the area shown below that contains the target forest with manually collected and densified ground points and a forested area with only dense matching points. The difference could not be more drastic as the visualizations show.

Now we run ground classification using lasground with option ‘-town’ using only the points with the temporary code 9 by ignoring all other classifications 0 and 8 in the file. We leave the temporary classification code 9 unchanged for all the points that were not classified with “ground” code 2 so we can visualize later which ones those are.

lasground -i DenseMatchingAndDensifiedGround.laz ^
          -ignore_class 0 8 ^
          -town ^
          -non_ground_unchanged ^
          -o GroundClassified.laz

We again use the GUI of lasview to pick several interesting areas after running lasindex and again load the forest stand delineating polygon as an overlay into the GUI.

lasindex -i GroundClassified.laz 

lasview -i GroundClassified.laz -gui

We pick the area shown below that contains all three scenarios: the target forest with manually collected and densified ground points, an open area with only dense matching points, and a forested area with only dense matching points. The result is as expected: in the target forest the manually collected ground points are used as ground and in the open area the dense-matching points are used as ground. But there is no useful ground in the other forested area.

Now we can compute the heights of the points above ground for our target forest with lasheight and either replace the z elevations in the file of store them separately as “extra bytes”. Then we can compute, for example, a Canopy Height Model (CHM) that color codes the height of the vegetation above the ground with lasgrid. Of course this will only be correct in the target forest where we have “good” ground but not in the other forested areas. We also compute a hillshaded DTM to be able to visually inspect the topography of the generated terrain model.

lasheight -i GroundClassified.laz ^
          -store_as_extra_bytes ^
          -o GroundClassifiedWithHeights.laz

lasgrid -i GroundClassifiedWithHeights.laz ^
        -step 2 ^
        -highest -attribute 0 ^
        -false -set_min_max 0 25 ^
        -o chm.png

las2dem -i GroundClassified.laz ^
        -keep_class 2 -extra_pass ^
        -step 2 ^ 
        -hillshade ^
        -o dtm.png

Here you can download the resulting color-coded CHM and the resulting hill-shaded DTM as Google Earth KMZ overlays. Clearly the resulting CHM is only meaningful in the target forest where we used the manually collected ground points to create a reasonable DTM. In the other forested areas the ground is only correct near the forest edges and gets worse with increasing distance from open areas. The resulting DTM exhibits some interesting looking  bumps in the middle of areas with manually collected ground point. Those are a result of using the dense-matching points as ground whenever their elevation is lower than that of the manually collected points (which is decided in the lasthin step). Whether those bumps represent true elevations of are artifacts of low erroneous elevation from dense-matching remains to be investigated.

For forests on complex and steep terrain the number of ground points that needs to be manually collected may make such an approach infeasible in practice. However, maybe you have another source of elevation, such as a low-resolution DTM of 10 or 25 meter provided by your local government. Or maybe even a high resolution DTM of 1 or 2 meter from a LiDAR survey you did several years ago. While the forest may have grown a lot in the past years, the ground under the forest will probably not have changed much …

LASmoons: Marzena Wicht

Marzena Wicht (recipient of three LASmoons)
Department of Photogrammetry, Remote Sensing and GIS
Warsaw University of Technology, Poland.

Background:
More than half of human population (Heilig 2012) suffers from many negative effects of living in cities: increased air pollution, limited access to the green areas, Urban Heat Island (UHI) and many more. To mitigate some of these effects, many ideas came up over the years: reducing the surface albedo, the idea of the Garden City, green belts, and so on. Increasing horizontal wind speed might actually improve both, the air pollution dispersion and the thermal comfort in urban areas (Gál & Unger 2009). Areas of low roughness promote air flow – discharging the city from warm, polluted air and supplying it with cool and fresh air – if they share specific parameters, are connected and penetrate the inner city with a country breeze. That is why mapping low roughness urban areas is important in better understanding urban climate.

Goal:
The goal of this study is to derive buildings (outlines and height) and high vegetation using LAStools and to use that data in mapping urban ventilation corridors for our case study area in Warsaw. There are many ways to map these; however using ALS data has certain advantages (Suder& Szymanowski 2014) in this case: DSMs can be easily derived, tree canopy (incl. height) can be joined to the analysis and buildings can be easily extracted. The outputs are then used as a basis for morphological analysis, like calculating frontal area index. LAStools has the considerable advantage of processing large quantities of data (~500 GB) efficiently.

Frontal area index calculation based on 3D building database

Data:
+ LiDAR provided by Central Documentation Center of Geodesy and Cartography
+ average pulse density 12 p/m^2
+ covers 517 km^2 (whole Warsaw)

LAStools processing:
1) quality checking of the data as described in several videos and blog posts [lasinfo, lasvalidate, lasoverlap, lasgrid, lasduplicate, lasreturnlas2dem]
2) reorganize data into sufficiently small tiles with buffers to avoid edge artifacts [lastile]
3) classify point clouds into vegetation and buildings [lasground, lasclassify]
4) normalize LiDAR heights [lasheight]
5) create triangulated, rasterized derivatives: DSM / DTM / nDSM / CHM [las2dem, blast2dem]
6) compute height-based metrics (e.g. ‘-avg’, ‘-std’, and ‘-p 50’) [lascanopy]
7) generate subsets during the workflow [lasclip]
8) generate building footprints [lasboundary]

References:
Heilig, G. K. (2012). World urbanization prospects: the 2011 revision. United Nations, Department of Economic and Social Affairs (DESA), Population Division, Population Estimates and Projections Section, New York.
Gal, T., & Unger, J. (2009). Detection of ventilation paths using high-resolution roughness parameter mapping in a large urban area. Building and Environment, 44(1), 198-206.
Suder, A., & Szymanowski, M. (2014). Determination of ventilation channels in urban area: A case study of Wroclaw (Poland). Pure and Applied Geophysics, 171(6), 965-975.

LASmoons: Gudrun Norstedt

Gudrun Norstedt (recipient of three LASmoons)
Forest History, Department of Forest Ecology and Management
Swedish University of Agricultural Sciences, Umeå, Sweden

Background:
Until the end of the 17th century, the vast boreal forests of the interior of northern Sweden were exclusively populated by the indigenous Sami. When settlers of Swedish and Finnish ethnicity started to move into the area, colonization was fast. Although there is still a prospering reindeer herding Sami culture in northern Sweden, the old Sami culture that dominated the boreal forest for centuries or even millenia is to a large extent forgotten.
Since each forest Sami family formerly had a number of seasonal settlements, the density of settlements must have been high. However, only very few remains are known today. In the field, old Sami settlements can be recognized through the presence of for example stone hearths, storage caches, pits for roasting pine bark, foundations of certain types of huts, reindeer pens, and fences. Researchers of the Forest History section of the Department of Forest Ecology and Management have long been surveying such remains on foot. This, however, is extremely time consuming and can only be done in limited areas. Also, the use of aerial photographs is usually difficult due to dense vegetation. Data from airborne laser scanning should be the best way to find remains of the old forest Sami culture. Previous research has shown the possibilities of using airborne laser scanning data for detecting cultural remains in the boreal forest (Jansson et al., 2009; Koivisto & Laulamaa, 2012; Risbøl et al., 2013), but no studies have aimed at detecting remains of the forest Sami culture. I want to test the possibilities of ALS in this respect.

DTM from the Krycklan catchment, showing a row of hunting pits and (larger) a tar pit.

Goal:
The goal of my study is to test the potential of using LiDAR data for detecting cultural and archaeological remains on the ground in a forest area where Sami have been known to dwell during historical times. Since the whole of Sweden is currently being scanned by the National Land Survey, this data will be included. However, the average point density of the national data is only 0,5–1 pulses/m^2. Therefore, the study will be done in an established research area, the Krycklan catchment, where a denser scanning was performed in 2015. The Krycklan data set lacks ground point classification, so I will have to perform such a classification before I can proceed to the creation of a DTM. Having tested various kind of software, I have found that LAStools seems to be the most efficient way to do the job. This, in turn, has made me aware of the importance of choosing the right methods and parameters for doing a classification that is suitable for archaeological purposes.

Data:
The data was acquired with a multi-spectral airborne LiDAR sensor, the Optech Titan, and a Micro IRS IMU, operated on an aircraft flying at a height of about 1000 m and positioning was post-processed with the TerraPos software for higher accuracy.
The average pulse density is 20 pulse/m^2.
+ About 7 000 hectares were covered by the scanning. The data is stored in 489 tiles.

LAStools processing:
1) run a series of classifications of a few selected tiles with both lasground and lasground_new with various parameters [lasground and lasground_new]
2) test the outcomes by comparing it to known terrain to find out the optimal parameters for classifying this particular LiDAR point cloud for archaeological purposes.
3) extract the bare-earth of all tiles (using buffers!!!) with the best parameters [lasground or lasground_new]
4) create bare-earth terrain rasters (DTMs) and analyze the area [lasdem]
5) reclassify the airborne LiDAR data collected by the National Land Survey using various parameters to see whether it can become more suitable for revealing Sami cultural remains in a boreal forest landscape  [lasground or lasground_new]

References:
Jansson, J., Alexander, B. & Söderman, U. 2009. Laserskanning från flyg och fornlämningar i skog. Länsstyrelsen Dalarna (PDF).
Koivisto, S. & Laulamaa, V. 2012. Pistepilvessä – Metsien arkeologiset kohteet LiDAR-ilmalaserkeilausaineistoissa. Arkeologipäivät 2012 (PDF).
Risbøl, O., Bollandsås, O.M., Nesbakken, A., Ørka, H.O., Næsset, E., Gobakken, T. 2013. Interpreting cultural remains in airborne laser scanning generated digital terrain models: effects of size and shape on detection success rates. Journal of Archaeological Science 40:4688–4700.

LASmoons: Muriel Lavy

Muriel Lavy (recipient of three LASmoons)
RED (Risk Evaluation Dashboard) project
ISE-Net s.r.l, Aosta, ITALY.

Background:
The Aosta Valley Region is a mountainous area in the heart of the Alps. This region is regularly affected by hazard natural phenomena connected with the terrain geomorphometry and the climate change: snow avalanche, rockfalls and landslide.
In July 2016 a research program, funded by the European Program for the Regional Development, aims to create a cloud dashboard for the monitoring, the control and the analysis of several parameters and data derived from advanced sensors: multiparametrical probes, aerial and oblique photogrammetry and laser scanning. This tool will help the territory management agencies to improve the risk mitigation and management system.

The RIEGL VZ-4000 scanning the Aosta Valley Region in Italy.

Goal:
This study aims to classify the point clouds derived from aerial imagery integrated with laser scanning data in order to generate accurate DTM, DSM and Digital Snow Models. The photogrammetry data set was acquired with a Nikon D810 camera from an helicopter survey. The aim of further analysis is to detect changes of natural dynamic phenomena that have occurred via volume analysis and mass balance evaluation.

Data:
+ The photogrammetry data set was acquired with an RGB camera (Nikon D810) with a focal length equivalent of 50 mm from a helicopter survey: 1060 JPG images
+ The laser scanner data set was acquired using a Terrestrial Laser Scanner (RIEGL VZ-4000) combined with a Leica GNSS device (GS25) to georeference the project. The TLS dataset was then used as base reference to properly align and georeference the photogrammetry point cloud.

LAStools processing:
1) check the reference system and the point cloud density [lasinfo, lasvalidate]
2) remove isolated noise points [lasnoise]
3) classify point into ground and non-ground [lasground]
4) classify point clouds into vegetation and other [lasclassify]
5) create DTM and DSM  [las2dem, lasgrid, blast2dem]
6) produce 3D visualizations to facilitate the communication and the interaction [lasview]

LASmoons: Chloe Brown

Chloe Brown (recipient of three LASmoons)
Geosciences, School of Geography
University of Nottingham, UK

Background:
Malaysia’s North Selangor peat swamp forest is experiencing rapid and large scale conversion of peat swampland to oil palm agriculture, contrary to prevailing environmental guidelines. Given the global importance of tropical peat lands, and the uncertainties surrounding historical and future oil palm development, quantifying the spatial distribution of ecosystem service values, such as climate mitigation, is key to understanding the trade-offs associated with anthropogenic land use change.
The study explores the capabilities and methods of remote sensing and field-based data sets for extracting relevant metrics for the assessment of carbon stocks held in North Selangor peat swamp forest reserve, estimating both the current carbon stored in the above and below ground biomass, as well as the changes in carbon stock over time driven by anthropogenic land use change. Project findings will feed directly into peat land management practices and environmental accounting in Malaysia through the Tropical Catchments Research Initiative (TROCARI), and support the Integrated Management Plan of the Selangor State Forest Department (see here for a sample).

some clever caption

Goal:
LiDAR data is now seen as the practical option when assessing canopy height over large scales (Fassnacht et al., 2014), with Lucas et al., (2008) believing LiDAR data to produce more accurate tree height estimates than those derived from manual field based methods. At this stage of the project, the goal is to produce a high quality LiDAR-derived Canopy Height Model (CHM) following the “pit-free” algorithm of Khosravipour et.al., 2014 using the LAStools software.

Data:
+ LiDAR provided by the Natural Environment Research Council (NERC) Airborne Research and Survey Facility’s 2014 Malaysia Campaign.
+ covers 685 square kilometers (closed source)
+ collected with Leica ALS50-II LiDAR system
+ average pulse spacing < 1 meter, average pulse density 1.8 per square meter

LAStools processing:
1) Create 1000 meter tiles with 35 meter buffer to avoid edge artifacts [lastile]
2) Remove noise points (class 7) that are already classified [las2las]
3) Classify point clouds into ground (class 2) and non-ground (class 1) [lasground]
4) Generate normalized above-ground heights [lasheight]
5) Create DSM and DTM [las2dem]
6) Generate a pit-free Canopy Height Model (CHM) as described here [lasthin, las2dem, lasgrid]
7) Generate a spike-free Canopy Height Model (CHM) as described here for comparison [las2dem]

References:
Fassnacht, F.E., Hartig, F., Latifi, H., Berger, C., Hernández, J., Corvalán, and P., Koch, B. (2014). Importance of sample size, data type and prediction method for remote sensing-based estimations of above-ground forest biomass. Remote Sensing. Environment. 154, 102–114.
Khosravipour, A., Skidmore, A. K., Isenburg, M., Wang, T., and Hussin, Y. A. (2014). Generating pit-free canopy height models from airborne LiDAR. Photogrammetric Engineering & Remote Sensing, 80(9), 863-872.
Lucas, R. M., Lee, A. C., and Bunting, P. J., (2008). Retrieving forest biomass through integration of casi and lidar data. International Journal of Remote Sensing, 29 (5), 1553-1577.

LASmoons: Elia Palop-Navarro

Elia Palop-Navarro (recipient of three LASmoons)
Research Unit in Biodiversity (UO-PA-CSIC)
University of Oviedo, SPAIN.

Background:
Old-growth forests play an important role in biodiversity conservation. However, long history of human transformation of the landscape has led to the existence of few such forests nowadays. Its structure, characterized by multiple tree species and ages, old trees and abundant deadwood, is particularly sensible to management practices (Paillet et al. 2015) and requires long time to recover from disturbance (Burrascano et al. 2013). Within protected areas we would expect higher proportions of old-growth forests since these areas are in principle managed to ensure conservation of natural ecosystems and processes. Nevertheless, most protected areas in the EU sustained use and exploitation in the past, or even still do.

lasmoons_elia_palopnavarro_0

Part of the study area. Dotted area corresponds to forest surface under protection.

Goal:
Through the application of a model developed in the study area, using public LiDAR and forest inventory data (Palop-Navarro et al. 2016), we’d like to know how much of the forest in a network of mountain protected areas retains structural attributes compatible with old-growth forests. The LiDAR processing tasks which LAStools will be used for involve a total of 614,808 plots in which we have to derive height metrics, such as mean or median canopy height and its variability.

Vegetation profile colored by height in a LiDAR sample of the study area.

Vegetation profile colored by height in a LiDAR sample of the study area.

Data:
+ Public LiDAR data that can be downloaded here with mean pulse density 0.5 points per square meter. This data has up to 5 returns and is already classified into ground, low, mid or high vegetation, building, noise or overlapped.
+ The area covers forested areas within protected areas in Cantabrian Mountains, occupying 1,207 km2.

LAStools processing:
1) quality checking of the data as described in several videos and blog posts [lasinfo, lasvalidate, lasoverlap, lasgrid, las2dem]
2) use existing ground classification (if quality suffices) to normalize the elevations of to heights above ground using tile-based processing with on-the-fly buffers of 50 meters to avoid edge artifacts [lasheight]
3) compute height-based forestry metrics (e.g. ‘-avg’, ‘-std’, and ‘-p 50’) for each plot in the study area [lascanopy]

References:
Burrascano, S., Keeton, W.S., Sabatini, F.M., Blasi, C. 2013. Commonality and variability in the structural attributes of moist temperate old-growth forests: a global review. Forest Ecology and Management 291:458-479.
Paillet, Y., Pernot, C., Boulanger, V., Debaive, N., Fuhr, M., Gilg, O., Gosselin, F. 2015. Quantifying the recovery of old-growth attributes in forest reserves: A first reference for France. Forest Ecology and Management 346:51-64.
Palop-Navarro, E., Bañuelos, M.J., Quevedo, M. 2016. Combinando datos lidar e inventario forestal para identificar estados avanzados de desarrollo en bosques caducifolios. Ecosistemas 25(3):35-42.

LASmoons: Jesús García Sánchez

Jesús García Sánchez (recipient of three LASmoons)
Landscapes of Early Roman Colonization (LERC) project
Faculty of Archaeology, Leiden University, The Netherlands

Background:
Our project Landscapes of Early Roman Colonization (LERC) has been studying the hinterland of the Latin colony of Aesernia (Molise region, Italy) using several non-destructive techniques, chiefly artefactual survey, geophysics, and interpretation of aerial photographs. Nevertheless large areas of the territory are covered by the dense forests of the Matese mountains, a ridge belonging the Apennine chain, or covered by bushes due to the abandonment of the countryside. The project won’t be complete without integrating the marginal, remote and forested areas into our study of the Roman hinterland. Besides, it’s also relevant to discuss the feasibility of LiDAR data sets in the study of Mediterranean landscapes and its role within contemporary Landscape Archaeology.

some clever caption

LiDAR coverage in Molise region, Italy.

Goal:
+ to study in detail forested areas in the colonial hinterland of Aesernia.
+ to found the correct parameters of the classification algorithm to be able to locate possible archaeological structures or to document appropriately those we already known.
+ to document and create new visualization of hill-top fortified sites that belong to the indigenous population and are currently poorly studied due to inaccessibility and forest coverage (Monte San Paolo, Civitalla, Castelriporso, etc.)
+ to demonstrate the archaeological potential of LiDAR data in Italy and help other scholars to work with that kind of data, explaining basic information about data quality, where and how to acquire imagery and examples of application in archaeology. A paper entitled “Working with ALS – LiDAR data in Central South Italy. Tips and experiences”, will be presented in the International Mediterranean Survey Workshop by the end of February in Athens.

Civitella hillfort (Longano, IS) and its local context: ridges and forest belonging to the Materse mountains and the Appenines.

Data:
Recently the LERC project has acquired a large LiDAR dataset created by the Italian Geoportale Nazionale and the Minisstero dell’Ambiente e della Tutella del Territorio e del Mare. The data was produced originally to monitor land-slides and erosive risk.
The average point resolution is 1 meter.
+ The data sets were cropped originally in 1 sq km. tiles by the Geoportale Nazionale for distribution purposes.

LAStools processing:
1) data is provided in *.txt files thus the first step is to create appropriate LAS files to work with [txt2las]
2) combine areas of circa 16 sq km (still fewer than 20 million points to be processed in one piece with LAStools) in the surroundings of the colony of Aesernia and in the Matese mountains [lasmerge]
3) assign the correct projection to the data [lasmerge or las2las]
4) extract the care-earth with the best-fitting parameters [lasground or lasground_new]
5) create bare-earth terrain rasters as a first step to visualize and analyze the area [lasdem]