Release of PulseWaves Full Waveform LiDAR Format

PRESS RELEASE
June 21, 2013
rapidlasso GmbH, Gilching, Germany

Just in time for the RIEGL LiDAR 2013 user conference, technology start-up rapidlasso GmbH has released an open data exchange format called “PulseWaves” for storing full waveform LiDAR data. The new format has been in development since December 2011 through transparent discussions at http://pulsewaves.org that involved many stake holders. The result is a specification simple to implement, yet versatile enough to store many different waveform flavors. PulseWaves standardizes storage and delivery for applications that benefit from information about the waveform such as forest management, biomass estimation, and LiDAR bathymetry. It complements the ASPRS LAS format for discrete returns and radically simplifies the exchange of full outgoing and returning waveforms.

The initial PulseWaves release comes with an open source reference implementation (LGPL) and a set of basic processing tools to display and convert the data. A compressed version of the PulseWaves format based on LASzip technology is in progress. With their latest release of their RiPROCESS software, RIEGL GmbH has become the first hardware vendor to allow exporting full waveform LiDAR data to the PulseWaves format. The development effort was in parts supported by RIEGL GmbH and by Airborne Research Australia (ARA) at Flinders University in Adelaide.

About rapidlasso GmbH:
Technology start-up rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

An illustrative visualization of full waveform data as rendered by the latest version of pulseview.exe.

An illustrative visualization of full waveform data as rendered by the latest version of pulseview.exe.

sneak peak at PulseWaves

PulseWaves is a new exchange format for full waveform LiDAR data. Below you see a simple visualization of a small sample of full waveform LiDAR data exported from RIEGL’s RiProcess to the PulseWaves format via a prototype of the PulseWaves DLL interface.

sneak peak at PulseWaves full waveform LiDAR format

Illustrated in 3D are the waveforms of 50 pulses via a green to red shading of their intensity. Plotted in more detail in 2D from left to right are the waveforms of a single pulse that is in the very center of these 50 pulses. The outgoing waveform is red. The returning waveform – in this case consisting of two segments – is blue.

The 3D points (here mostly yellow apart from a few orange and blue ones) correspond to the first recorded waveform sample of each pulse. Through them a particular pulse (or a set of pulses) can be picked by pressing the ‘i’ hot key in the interactive 3D viewer – pulseview.exe – when hovering over a point. The color coding of these points indicates that different pulse descriptors are used for describing the waveforms of the corresponding pulse. For more details go to the discussion forum at http://pulsewaves.org/ where you find links to the specification, the tools, and some sample data.

LASindex – spatial indexing of LiDAR data

Salzburg is a beautiful city in December. The European LiDAR Mapping Forum coincided with the days when the “Krampus” (= “Christmas monsters”) are roaming the Christmas markets in the old town to scare children and adults alike. One gave me a painful whipping in the legs with its leathery tail when I tried to protect a LAStools user … (-;

More to the point, here is my talk at ELMF 2012 on “LASindex – simple spatial indexing of LiDAR data”. I first give a little update on LASzip, then talk about spatial indexing with LAX, before sneak-previewing PulseWaves – our new and open LiDAR format for storing full waveform data.

Some more detail:
Airborne LiDAR surveys collect large amounts of elevations samples, often resulting in Terabytes of data. The acquired LiDAR points are typically stored and distributed in the LAS format or – its lossless compressed twin – the LAZ format. However, managing a folder of LAS or LAZ files is not a trivial task when a survey consists, for example, of 500 flight strips containing around 200 million points each. Even a simple area-of-interest (AOI) query requires opening all files and loading all those whose bounding box overlaps the queried AOI. One solution is to copy the survey into a dedicated data base such as Oracle Spatial or PostgreSQL. We present a much simpler alternative that works directly on the original LAS or LAZ files.

Our minimal-effort spatial indexing scheme has very small setup costs, avoids creating a second copy of the data, and is already in use in the LAStools software suite. For each LiDAR file we generate a tiny LAX file that resides in the same folder as the *.las or *.laz file and has the same name but with a *.lax extension. The LAX files are generally as small as 0.01 percent (for a LAS file) or 0.1 percent (for a LAZ file) of the file containing the LiDAR data and they can be generated as fast as the points can be read off disk.

The LAX files describe an adaptive quadtree over the x and y coordinates of all points. Each occupied quadtree cell stores a list of point index intervals that together reference all points falling into this cell. By merging all intervals of a cell that are less than 1000 apart in point index space we significantly reduce the number of intervals, the size of the LAX files, and the number of file seek operations.

Although individual cells typically reference too many points this is usually amortized as a typical AOI query will require returning a union of all intervals from many quadtree cells. However, our in-place spatial indexing relies on a certain degree of spatial coherency to be present in the point order. A simple measure of the efficiency of the existing order is to calculate the overhead factor when loading each quadtree cell individually from disk.

The source code for LASindex is part of the open source library LASlib of LAStools. It has been extensively field-tested in the LiDAR delivery pipeline of Open Topography (OT) where it is used to efficiently gather data from folders of LAZ files in accordance to area-of-interest queries that are generated by users via OT’s popular web-based LiDAR download interface. Another important use is on-the-fly point buffering. When batch processing, for example, 2km by 2km LiDAR tiles to create DTMs via rasterization of a temporary TIN, it is beneficial to load a 100 meter point buffer around each tile to avoid tile boundary artifacts. The presence of LAX files allows doing so efficiently on-the-fly.

LAStools’ BLAST extension can process billions of LiDAR points

Often I get asked about the difference between las2dem.exe and blastdem.exe – the latter being part of the BLAST extension of LAStools. In the academic video from 2006 below, I outline the core technology details behind BLAST, which can seamlessly compute gigantic Delaunay Triangulations from billions of LiDAR points – using very little main memory. The blast2dem.exe module does not store the gigantic TINs that BLAST produces, but immediately rasters the finalized triangles streaming out of BLAST into equally massive DEMs. This avoids to ever having to store such huge triangulations – that are only needed temporarily – to disk. Currently blast2dem.exe can “only” process up to 2 billion points per file – a limitation that shall be lifted soon.

LASzip wins 2012 Geospatial World Forum Technology Innovation Award

PRESS RELEASE
April 26, 2012
rapidlasso consulting, Sommerhausen, Germany
The creators of LAStools are the recipient of the 2012 Technology Innovation Award for LiDAR processing at the Geospatial World Forum in Amsterdam for LASzip – a lossless LiDAR compression software. With LASzip rapidlasso consulting provides a free, efficient, open-source solution for squeezing LAS files into 5 to 13 times smaller LAZ files. LASzip has become a de-facto industry standard for exchanging compressed LiDAR that is used by NOAA, USGS, OpenTopography, Dielmo3D, Fugro, Blom, Riegl, Dewberry, USACE, DNR Minnesota, the Finnish Mapping Authority, and many more … saving government agencies, commercial companies, and end users many TeraBytes of disk storage and transmission bandwidth.
innovation_award_2012

LASzip: lossless compression of LiDAR data

Here is a video of my talk at ELMF 2011 on “LASzip: lossless compression of LiDAR data”.

Airborne or mobile laser scanning technology (LiDAR) collects billions of xyz points that are typically stored in the LAS(er) format. LASzip is a lossless compressor to turn large LAS into smaller LAZ files. Encoding and decoding speeds are 1 to 4 millions points per second and compressed files are only 7 to 25 percent of the original size.Compression and decompression happen on-the-fly in a streaming manner and random-access is supported. A reference implementation unencumbered by patents or intellectual property concerns is freely available with an LGPL-license here.