LASmoons: Stéphane Henriod

Stéphane Henriod (recipient of three LASmoons)
National Statistical Committee of the Kyrgyz Republic
Bishkek, Kyrgyzstan

This pilot study is part of the International Climate Initiative project called “Ecosystem based Adaptation to Climate change in the high mountainous regions of Central Asia” that is funded by the Federal Ministry for the Environment, Nature Conservation, Building and Nuclear Safety (BMU) of Germany and implemented by the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH in Kyrgyzstan, Tajikistan and Kazakhstan.

lasmoons_Stephane_Henriod_1

Background:
The ecosystems in high mountainous regions of Central Asia are characterized by a unique diversity of flora and fauna. In addition, they are the foundation of the livelihoods of the local population. Specific benefits include clean water, pasture, forest products, protection against floods and landslides, maintenance of soil fertility, and ecotourism. However, the consequences of climate change such as melting glaciers, changing river runoff regimes, and weather anomalies including sharp temperature fluctuations and non-typical precipitation result in negative impacts on these ecosystems. Coupled with unwise land use, these events damage fragile mountain ecosystems and reduce their regeneration ability undermining the local population’s livelihoods. Therefore, people living in rural areas and directly depending on natural resources must adapt to adverse impacts of climate change. This can be done through a set of measures, known in the world practice as ecosystem-based adaptation (EbA) approach. It promotes the sustainable use of natural resources to sustain and enhance the livelihood of the population depending on those resources.

lasmoons_Stephane_Henriod_2 Goal:
In two selected pilot regions of Kyrgyzstan and Tajikistan, planned measures will concentrate on climate-informed management of ecosystems in order to maintain their services for the rural population. EbA always starts with identifying the vulnerability of the local population. Besides analyzing the socio-economic situation of the local population, this includes (1) assessing the ecological conditions of the ecosystems in the watershed and the related ecosystem services people benefit from, (2) identifying potential disaster risks, and (3) analyzing glacier dynamics with its response to water runoff. As a baseline to achieve this and to get spatially explicit, remote sensing based techniques and mapping activities need to be utilized.

A first UAV (unmanned aerial vehicle) mission has taken place in the Darjomj watershed of the Bartang valley in July 2016. RGB-NIR images as well as a high-resolution Digital Surface Model have been produced that now need to be segmented and analysed in order to produce comprehensive information. The main processing that will take advantage of LAStools is the generation of a DTM from the DSM that will then be used for identifying risk areas (flood zones, landslides and avalanches, etc.). The results of this approach will ultimately be compared with lower-cost satellite images (RapidEye, Planet, Sentinel).

Data:
+ High-resolution RGB and NIR image (10 cm) from a SenseFly Ebee
+ High-resolution DSM (10 cm) from a SenseFly Ebee

LAStools processing:
1)
classify DSM points obtained via dense-matching photogrammetry into a SenseFly Ebee imagery into ground and non-ground points via processing pipelines as described here and here [lastile, lassort, lasnoise, lasground]
2) create a DTM [las2dem, lasgrid, blast2dem]
3) produce 3D visualisations to facilitate the communication around adaptation to climate change [lasview]
lasmoons_Stephane_Henriod_0

Removing low noise from Semi-Global Matching (SGM) Points

At PhoWo and INTERGEO 2015 rapidlasso was spending quality time with VisionMap who make the A3 Edge camera that provides fine resolution images from high altitudes and can quickly cover large areas. Under the hood of their LightSpeed software is the SURE dense-matching algorithm from nframes that turns images into photogrammetric point clouds. We were asked whether LAStools is able to create bare-earth DTM rasters from such points. If you have read our first, second, or third blog post on the topic you know that our asnwer was a resounding “YES!”. But we ran into an issue due to the large amount of low noise. Maybe the narrow angle between images at a high flying altitude affects the semi-global matching (SGM) algorithm. Either way, in the following we show how we use lascanopy and lasheight to mark low points as noise in a preprocessing step.

We obtained a USB stick containing a 2.42 GB file called “valparaiso_DSM_SURE_100.las” containing about 100 million points spaced 10 cm apart generated by SURE and stored with an (unnecessary high) resolution of millimeters (aka “resolution fluff”) as the third digit of all coordinates was always zero:

las2txt -i F:\valparaiso_DSM_SURE_100.las -stdout | more
255991.440 6339659.230 89.270
255991.540 6339659.240 89.270
255991.640 6339659.240 88.660
255991.740 6339659.230 88.730
[...]

We first compressed the bulky 2.42 GB LAS file into a compact 0.23 GB LAZ to our local hard drive – a file that is 10 times smaller and that will be 10 times faster to copy:

laszip -i F:\valparaiso_DSM_SURE_100.las ^
       -rescale 0.01 0.01 0.01 ^
       -o valparaiso_DSM_SURE_100.laz ^

Then we tiled the 100 million points into 250 meter by 250 meter tiles with 25 meter buffer using lastile. We use the new option ‘-flag_as_withheld’ to mark all buffer points with the withheld flag so they can be easily removed on-the-fly via the ‘-drop_withheld’ command-line filter (also see the README file).

lastile -i valparaiso_DSM_SURE_100.laz ^
        -tile_size 250 -buffer 25 ^
        -flag_as_withheld ^
        -odir valparaiso_tiles_raw -o val.laz
250 meter by 250 meter tiling with 25 meter buffer

250 meter by 250 meter tiles with 25 meter buffer

Before processing millions to billions of points we experiment with different options to find what works best on a smaller area, namely the tile “val_256750_6338500.laz” pointed to above. Using the workflow from this blog posts did not give perfect results due to the large amount of low noise. Although many low points were marked as noise (violett) by lasnoise, too many ended up classified as ground (brown) by lasground as seen here:
excessive low noise affects ground classification

excessive low noise affects ground classification

We use lascanopy – a tool very popular with forestry folks – to compute four BIL rasters where each 5m by 5m grid cell contains the 5th, 10th, 15th, and 20th percentile of the elevation values from all points falling into a cell (also see the README file):
lascanopy -i val_256750_6338500.laz ^
          -height_cutoff -1000 -step 5 ^
          -p 5 10 15 20 ^
          -obil
The four resulting rasters can be visually inspected and compared with lasview:
lasview -i val_256750_6338500_*.bil -files_are_flightlines
comparing 5th and 10th elevation percentiles

comparing the 5th and the 10th elevation percentiles

By pressing the hot keys <0>, <1>, <2> and <3> to switch between the different percentiles and <t> to triangulate them into a surface, we can see that for this example the 10th percentile works well while the 5th percentile is still affected by the low noise. Hence we use the 10th percentile elevation surface and classify all points below it as noise with lasheight (also see the README file).
lasheight -i val_256750_6338500.laz ^
          -ground_points val_256750_6338500_p10.bil ^
          -classify_below -0.5 7 ^
          -odix _denoised -olaz
We visually confirm that all low points where classified as noise (violett). Note the obvious “edge artifact” along the front boundary of the tile. This is why we always recommend to use a buffer in tile-based processing.
points below 10th percentile surface marked as noise

points below 10th percentile surface marked as noise

At the end of the blog post we give the entire command sequence that first computes a 10th percentile raster with 5m resolution for the entire file with lascanopy and then marks all points of each tile below the10th percentile surface as noise with lasheight. When we classify all points into ground and non-ground points with lasground we ignore all points classified as noise. Here are the results:

DTM extracted from SGM points despite low noise

DTM extracted from dense-matching points despite low noise

corresponding DSM with all buildings and vegetaion included

corresponding DSM with all buildings and vegetaion included

Above you see the generated DTM and the corresponding DSM. So yes, LAStools can create DTMs from points that are result of dense-matching photogrammetry … even when there is a lot of low noise. There are many other ways to mix and match the modules of LAStools for more refined workflows. Sometimes declaring all points below the 10th percentile surface as noise may be too agressive. In a future blog post we will look how to combine lascanopy and lasnoise for a more adaptive approach.

:: compute 10th percentile for entire area
lascanopy -i valparaiso_DSM_SURE_100.laz ^
          -height_cutoff -1000 -step 5 ^
          -p 10 ^
          -obil

:: tile input into 250 meter tiles with buffer
lastile -i valparaiso_DSM_SURE_100.laz ^
        -tile_size 250 -buffer 25 ^
        -flag_as_withheld ^
        -odir valparaiso_tiles_raw -o val.laz

:: mark points below as noise
lasheight -i valparaiso_tiles_raw/*.laz ^
          -ground_points valparaiso_DSM_SURE_100_p10.bil ^
          -classify_below -0.5 7 ^
          -odir valparaiso_tiles_denoised -olaz ^
          -cores 4

:: ground classify while ignoring noise points
 lasground -i valparaiso_tiles_denoised\*.laz ^
          -ignore_class 7 ^
          -town -bulge 0.5 ^
          -odir valparaiso_tiles_ground -olaz ^
          -cores 4 

:: create 50 cm DTM rasters in BIL format
las2dem -i valparaiso_tiles_ground\*.laz ^
        -keep_class 2 ^
        -step 0.5 -kill 200 -use_tile_bb ^
        -odir valparaiso_tiles_dtm -obil ^
        -cores 4 

:: average 50 cm DTM values into single 1m DTM 
lasgrid -i valparaiso_tiles_dtm\*.bil -merged ^
        -step 1.0 -average ^
        -o valparaiso_dtm.bil

:: create hillshade adding in UTM 19 southern
blast2dem -i valparaiso_dtm.bil ^
          -hillshade -utm 19M ^
          -o valparaiso_dtm_hill.png

:: create DSM hillshade with same three steps
las2dem -i valparaiso_tiles_raw\*.laz ^
        -step 0.5 -kill 200 -use_tile_bb ^
        -odir valparaiso_tiles_dsm -obil ^
        -cores 4
lasgrid -i valparaiso_tiles_dsm\*.bil -merged ^
        -step 1.0 -average ^
        -o valparaiso_dsm.bil
blast2dem -i valparaiso_dsm.bil ^
          -hillshade -utm 19M ^
          -o valparaiso_dsm_hill.png

Creating DTMs from dense-matched points of UAV imagery from SenseFly’s eBee

Tim Sutton and his team at Kartoza work on flood modelling and risk assessment using Inasafe. They have been trying to generate a DTM from point cloud data derived via dense-matching from UAV imagery collected by an eBee of SenseFly in the “unplanned developments” or “slums” North West of Dar es Salaam, the capital city of Tanzania. Tim’s team was stuck after “other software” produced this result:

results for ground points classification with other software

poor ground classification of “Tandale” with “other software”

Tim reached out to us at rapidlasso asking whether LAStools could handle this better. After all, we had published two blog articles – namely this one and that one – showing how to generate DTMs from the point clouds generated by the dense-matching photogrammetry software of Pix4D. Below the workflow we devised and the results we produced for Tim and his team.

We obtained 3 different data sets of areas called “Tandale”, “Borahatward”, and “Bugurunni”. We added one new option to our lasground software called ‘-bulge 1.0’ (see README) to improve the removal of smaller buildings and got this result for “Tandale”.

ground classification with LAStools

DTM of “Tandale” from ground points classified with LAStools

Before you point out the “facetted” look of this DTM keep in mind that “Tandale” is a densely populated poor area. A first hand account of the rough life in this area can be found here. Most dense-matching points are on corrugated roofs that become voids that need to be interpolated across in the DTM. Take a look at the corresponding DSM where all objects are still present.

original data

DSM of “Tandale” from all dense-matching points

Below we give a detailed description at the example of the “Bugurunni” data set of the workflow that was used to generate DTMs for the three data sets. At the end of this article you will see some more results.

We first use lassort to quantize, sort, and compress on 4 cores the seven spatially incoherent LAS files of the “Bugurunni” data set (totalling 4.5 GB with excessive resolution of millimeters) into LASzip-compressed files with a more reasonable resolution of centimeters and points ordered along a space-filling curve. We also add the missing projection information with ‘-utm 37M’. The resulting 7 LAZ files occupy only 0.7 GB meaning we get a compression of 9 : 1. The option ‘-odir’ specifies the output directory.

lassort -i bugurunni_densified_*.las ^
        -rescale 0.01 0.01 0.01 ^
        -utm 37M ^
        -odir bugurunni_strips -olaz ^
        -cores 4

Next we tile the sorted strips into 500 meter by 500 meter tiles with 50 meter buffer using lastile. We use the new option ‘-flag_as_withheld’ to mark all buffer points with the withheld flag so they can easily be removed on-the-fly with the ‘-drop_withheld’ command-line filter (see the README file for more on this).

lastile -i bugurunni_strips\*.laz ^
        -files_are_flightlines ^
        -tile_size 500 -buffer 50 ^
        -flag_as_withheld ^
        -o bugurunni_raw\bugu.laz
Using lasnoise on 4 cores we classify isolated points that might hinder ground-classification as noise (class 7). The parameters ‘-isolated 15’ means that all points surrounded by less than 15 other points in their 3 by 3 by 3 = 27 cells neighborhood in a 3D grid are considered isolated. The size of each grid cell is specified with ‘-step_xy 2 -step_z 1’  as 2 meter by 2 meter by 1 meter. These parameters were found experimentally (see the README file for more on this).
lasnoise -i bugurunni_raw\*.laz ^
         -step_xy 2 -step_z 1 ^
         -isolated 15 ^
         -odir bugurunni_noise -olaz ^
         -cores 4
Then we run lasground on 4 cores to classify the ground points with options ‘-metro’ and ‘-bulge 1.0’. The option ‘-metro’ is a convenient short-hand for ‘-step 50’ that will remove all objects on the terrain (e.g. large buildings) that have an extend of 50 meters or less. The option ‘-bulge 1.0’ instructs lasground to be conservative and only add points that are 1 meter or less above a smoothed version of the initial ground estimate (see the README file for more on this)..
lasground -i bugurunni_noise\*.laz ^
          -ignore_class 7 ^
          -metro -bulge 1.0 ^
          -odir bugurunni_ground -olaz ^
          -cores 4
Now we use las2dem to raster a DTM from only those points that were classified as ground. The option ‘-step 0.5’ sets the output grid resolution to 0.5 meters, ‘-kill 200’ interpolates across voids of up to 200 meters, and ‘-use_tile_bb’ rasters only the original 500 meter by 500 meter tile interior but not the 50 meter buffer. This assures that the resulting raster tiling aligns without artifacts across tile boundaries. The option ‘-obil’ chooses BIL as the output raster format.
las2dem -i bugurunni_ground\*.laz ^
        -keep_class 2 ^
        -step 0.5 -kill 200 -use_tile_bb ^
        -odir bugurunni_dtm -obil ^
        -cores 4
As a simply form of anti-aliasing we average each four pixels of 0.5 meter resolution into one pixel of 1.0 meter resolution with lasgrid as all LAStools can read BIL files via on-the-fly conversion to points.
lasgrid -i bugurunni_dtm\*.bil -merged ^
        -step 1.0 -average ^
        -o bugurunni_dtm.bil

Finally we create a hillshade of the DTM adding back the projection that was “lost” in the BIL file generation so that blast2dem – the extremely scalable BLAST version of las2dem – can automatically produce a KML file for display in Google Earth.

blast2dem -i bugurunni_dtm.bil ^
          -hillshade -utm 37M ^
          -o bugurunni_dtm_hill.png

For comparison we also create a DSM with the same three steps but using all points.

las2dem -i bugurunni_raw\*.laz ^
        -step 0.5 -kill 200 -use_tile_bb ^
        -odir bugurunni_dsm -obil ^
        -cores 4
lasgrid -i bugurunni_dsm\*.bil -merged ^
        -step 1.0 -average ^
        -o bugurunni_dsm.bil
blast2dem -i bugurunni_dsm.bil ^
          -hillshade -utm 37M ^
          -o bugurunni_dsm_hill.png
DTM of "Bugurunni" from ground points classified with LAStools

DTM of “Bugurunni” from ground points classified with LAStools

Above you see the generated DTM and below the corresponding DSM. So yes, LAStools can create DTMs from points that are result of dense-matching photogrammetry … under one assumption: there is not too much vegetation.

DSM of "Bugurunni" from all dense-matching points

DSM of “Bugurunni” from all dense-matching points

Below also the results for the “Borahatward” data. In a future blog post we will see how to deal with the excessive low noise sometimes present in dense-matching points.

DTM of "Bo"

DTM of “Borahatward” from ground points classified with LAStools

DSM of "Borahatward" from all dense-matching points

DSM of “Borahatward” from all dense-matching points

Generating a DTM from Dense-Matching Photogrammetry

Point clouds from dense-matching photogrammetry are popular. Stored in the LAS format – or its compressed LAZ twin – these “near-LiDAR” points are easily ingested into LiDAR processing software to, for example, generate a Digital Surface Model (DSM). “Can LAStools also create a Digital Terrain Model (DTM) from such data?” is a question we often get asked by private email or via the LAStools user forum. At INTERGEO Euroasia 2014 we gave a demo of this at the UltraCam booth. We repeated this demo with different data at the Pix4D booth during the Geospatial World Forum 2014. Now the folks from Visionmap have also become curious …

pix4d_raw

Above the “topography_cadastre_switzerland_densified.las” point cloud we got at the Pix4D booth. You may have already seen this colorized point cloud at some other Pix4D event. The original points were in a highly incoherent spatial order that negativly affected all subsequent processing. Therefore we first use lassort to reorder the points into a space-filling Hilbert curve. At the same time we rescale the coordinate resolution to centimeters and compress the file into the LAZ format.

lassort -i topography_cadastre_switzerland_densified.las ^
        -rescale 0.01 0.01 0.01 -olaz
Next we use lasground to classify the ground points with options ‘-city’ and ‘-ultra_fine’. The option ‘-city’ is a convenient short-hand for ‘-step 25’ that will remove all objects on the terrain that have an extend of 25 meters or less. The option ‘-ultra_fine’ instructs lasground to spend more time on finding a good initial ground estimate. The option ‘-odix _g’ add the appendix ‘_g’ to the output file name.
lasground -i topography_cadastre_switzerland_densified.laz ^
          -city -ultra_fine ^
          -odix _g -olaz
pix4d_ground
Above you see only the points that were classified as ground (classification 2) and below you see how triangulating these points interpolates the voids where buildings and vegetation used to be.
pix4d_triangulated
Now we use las2dem to raster a DTM from only those points that were classified as ground. The option ‘-step 0.5’ sets the output grid resolution to 0.5 meters, ‘-ocut 2’ removes the ‘_g’ from the file name, ‘-odix _dtm’ adds the appendix ‘_dtm’ to the file name, and ‘-obil’ chooses BIL as the output raster format.
las2dem -i topography_cadastre_switzerland_densified_g.laz ^
        -keep_class 2 -thin_with_grid 0.125 -extra_pass ^
        -step 0.5 ^
        -ocut 2 -odix _dtm -obil
For comparison we also use las2dem to create the corresponding DSM. The option ‘-thin_with_grid 0.125’ makes the density of the points that are triangulated into a TIN more “compatible” with the 0.5 by 0.5 grid that the TIN is rastered with by keeping only the first point that falls within each 0.125 by 0.125 area or a maximum of 16 points per 0.5 by 0.5 raster cell. The option ‘-extra_pass’ counts all surviving points in a first pass to minimize the memory footprint needed for Delaunay TIN construction.
las2dem -i topography_cadastre_switzerland_densified_g.laz ^
        -thin_with_grid 0.125 -extra_pass ^
        -step 0.5 ^
        -ocut 2 -odix _dsm -obil

pix4d_dtm

Did you know that lasview can visualize BIL files via on-the-fly conversion from grids to points? Above you see the generated DTM and below the corresponding DSM. So yes, LAStools can create DTMs from points that are result of dense-matching photogrammetry … under one assumption: there is not too much vegetation.

pix4d_dsm

clone wars and drone fights

NEWSFLASH: update on Feb 5th and 6th and 17th (see end of article)

The year 2014 shapes to be a fun one in which the LiDAR community will see some major laser battles. (-: First, ESRI starts a “lazer clone war” with the open source community saddening your very own LAS clown here at rapidlasso with a proprietary LAZ clone. Then AHAB and Optech start to squirmish over the true penetration depth of their bathymetric systems in various forums. And now RIEGL – just having launched their Q1560 “crossfire” for battling Leica and Optech in the higher skies – is heading into an all-out “laser drone war” with FARO and Velodyne over arming UAVs and gyrocopters with lasers by rolling out their new (leaked?) RIEGL VUX-1 scanner … (-;

Some specs below, but no details yet on power-consumption.

  • about 3.85 kilograms
  • 225mm x 180mm x 125mm
  • survey-grade accuracy of 25mm
  • echo signal digitisation
  • online waveform processing
  • measurement rate up to 500 kHz
  • field of view 300 degrees
  • internal 360 GB SSD storage or real-time data via LAN-TCP/IP
  • collects data at altitudes or ranges of more than 1,000 ft
  • will be on display during ILMF 2014 (Feb 17-19, Denver, USA)
The new RIEGL VUX-1 laser scanner for UAVs ...

The new RIEGL VUX-1 laser scanner for UAVs

Below a proof-of-concept concept video by Phoenix Aerial on how such a system (here based on a Velodyne HDL-32E scanner) may look and operate.

A similar system introduced by Airbotix is shown below.

Aibot X6 with laser scanner from Velodyne HDL-32E

Aibot X6 with laser scanner from Velodyne HDL-32E

But before trusting these kind of drones with your expensive hardware you should make plenty of test flights with heavy payloads. Here a suggestion to make this more fun:

UPDATE (February 5th): It is reported by SPAR Point Group that the RIEGL VUX-1 LiDAR scanner for drones is to be shown at International LiDAR Mapping Forum in Denver later this month. Given the ceremony that the Q1560 “crossfire” was introduced with, it surely seems as if Geospatial World Magazine spilled the news early …

UPDATE (February 6th): RIEGL officially press-released the VUX-1 LiDAR. Its weight is below 4kg and its range up to 1000 ft …

UPDATE (February 17th): Today the new VUX-1 finally made it’s much anticipated world-premiere during ILMF 2014 in Denver. For more images of the official unveiling see RIEGL’s blog.

The VUX-1 LiDAR scanner for UAVs unveiled at ILMF 2014.

The VUX-1 LiDAR scanner for UAVs unveiled at ILMF 2014.