The dArc Force Awakens: ESRI escalates LiDAR format war

The empire has not changed their evil ways, despite an encouraging email from ESRI’s founder and president Jack Dangermond in response to the Open Letter by the OSGeo that was delivered to ESRI, OGC, and the ASPRS. Facing an incredible backlash by the LiDAR community over the release of their “LAZ clone” there was a new hope that unnecessary format fragmention could be avoided by working together within the Point Cloud Domain Working Group of the OGC. In fact only one thing happened: ESRI went silent on the controversy. They temporarily stopped promoting their “LAZ clone” and focused on locking in more content.

dArc_force_awakens

The message of the rebellion has been consistent and clear like in these two videos from the TC meeting of the OGC in Nottingham and the ASPRS side bar in Reno: a roadmap forward to avoid format fragmentation by exploiting the “natural break” in the format due to LAS 1.4. But there was zero technical contribution from ESRI during the past three PC-DWG meetings of the OGC. The slide sets that bored the audiences in Boulder and in Nottingham were not meant to contribute but merely stalled for time. Recently in Sydney ESRI was awefully quiet, knowing they were doing the exact opposite of what the OGC stands for. And now the empire strikes back.

laztozlas

There is a dArc force awakening that threatens the peace within the LiDAR community. ESRI has just released a new tool (see above) that enslaves point clouds by converting them from the open LAZ format to the near-identical but closed “LAZ clone” that they call “zLAS” or “Optimized LAS”. This comes just a few months after an entire nation‘s LiDAR was enslaved in this proprietary format. We have repeatedly warned about the ramifications of locking up Petabytes of LiDAR data in a closed format that is controlled by a single vendor.

ESRI is one of the largest GIS training organizations. By instructing LiDAR novices to “optimize” their LiDAR files and pushing LiDAR providers to switch from open LAS or open LAZ to closed zLAS, they effectively destroy the current success of our open formats. ESRI’s command of the GIS market can – little by little – turn their own proprietry format into the dominant way in which LiDAR point clouds are stored. Then we loose our open exchange formats. Hence, ESRI’s proprietary format threatens all that we have achieved with LAS (and LAZ) over the past years: compatible LiDAR data exchange and incredible LiDAR software interoperability.

ESRI is now escalating the LiDAR format wars. Join the rebellion, Jedis: download your lazer sabers and liberate some LiDAR.

This is not an anti-ESRI campaign. For the past three years we have been trying to resolve this situation. We have repeatedly reached out to ESRI to prevent format fragmentation. We have repeatedly offered to create a joint compressed format. We have plead, begged, and bargained for the sake of our LiDAR community and the sake of their ArcGIS user community not to promote a near-identical yet incompatible way for storing massive amounts of point cloud data.

Five Myths about LAS, LAZ, and “Optimized LAS”

The Open Letter by OSGeo was delivered to ESRI, OGC, and the ASPRS last week and the initial reponses – including an email from ESRI’s founder and president Jack Dangermond – are very encouraging. Attendees of last weeks’ ASPRS conference were discussing how to respond to ESRI’s proprietary “Optimized LAS” that threatens the achievements of the open LiDAR formats LAS and LAZ that the community has been using for many years now. Below five clarifications to five wrong statements overheard at these meetings:

1) Martin’s “LAZ” format is also proprietary.

Wrong. LAZ – just like LAS – is an open format. LAZ is defined by a well commented open reference implementation in C/C++ and described in a PE&RS paper published in February 2013. LAS is defined via a specification document but has no reference implementation. Both can be freely used by anyone and (re-)implemented on any operating system and in any programming language. For example, there is now a javascript version of LAZ that someone else created.

2) We have no argument because ESRI provides a free API for “Optimized LAS”.

Wrong. “Optimized LAS” can only be used via the mechanism, the programming language, and the operating system of ESRI’s choosing. This is the very definition of “proprietary format”. Here is what Wikipedia says:

A proprietary format is a file format of a company, organization, or individual that contains data that is ordered and stored according to a particular encoding-scheme, designed by the company or organization to be secret, such that the decoding and interpretation of this stored data is only easily accomplished with particular software or hardware that the company itself has developed. The specification of the data encoding format is not released, or underlies non-disclosure agreements.

In contrast an open format is a file format that is published and free to be used by everybody.

3) Martin’s “LAZ” format is only used by LAStools.

Wrong. Large parts of the LiDAR industry embrace LAZ and have added read & write support for the LAZ format using the open source code or the DLL. Examples are QT Modeler, Globalmapper, FME, Fugroviewer, ERDAS IMAGINE, ENVI LiDAR, Bentley Pointools, TopoDOT, FUSION, CloudCompare, Gexel R3, Pointfuse, …and many more. Notable exceptions are ArcGIS and the product line offered by Lewis Graham’s GeoCue group. We maintain an (incomplete) list of software with native LAZ support here.

4) ESRI has engineered “Optimized LAS” for the cloud and “LAZ” cannot compete.

Wrong. The extra functionality in “Optimized LAS” is a simple mash-up of LAZ with spatial indexing LAX, an optional spatial sort, and a few extra statistics. This is why ESRI’s format is also known as the “LAZ clone”. We were able to feature-match these minor engineering changes in an afternoon which – a few days later – resulted in this April Fools’ Day prank. In fact, LAZ has been used “in the cloud” for well over 4 years on OpenTopography – the first and probably the premier Web accessible LiDAR cloud service of our industry. It is also used by many other LiDAR download servers. We maintain an (incomplete) list of portals offering compressed LAZ here.

5) ESRI’s “Optimized LAS” does not prevent people from using LAS.

ESRI is one of the largest GIS training organizations. If they teach hundreds of LiDAR novices to “optimize” their “unoptimized LAS” files while simultaneously lobbying large LiDAR providers into switching from LAS or LAZ to zLAS they will effectively destroy the current success of our open formats. ESRI’s command of the GIS market can – little by little – turn their own proprietry format into the dominant way in which LiDAR point clouds are exchanged. Then we loose our open exchange formats. Hence, ESRI’s proprietary “Optimized LAS” format “threatens” what we have achieved with LAS (and LAZ): open LiDAR data exchange and incredible LiDAR software interoperability.

This is not an anti-ESRI campaign. We hope to work with ESRI to resolve this situation. Below an image and a quote from ESRI’s ArcNews Spring 2011 news letter about the importance of open formats, standards, and specifications …

ESRI: "Esri continues to advocate the need for open access to geographic data and functionality through support for widely adopted and practical standards and specifications. Esri follows an open system strategy for accessing and using geographic data and functionality."

“Esri continues to advocate the need for open access to geographic data and functionality through support for widely adopted and practical standards and specifications. Esri follows an open system strategy for accessing and using geographic data and functionality.” — ArcNews, Spring 2011

Geoinformatics magazine interviews rapidlasso

Eric van Rees, the editor of the Geoinformatics magazine, met with rapidlasso GmbH at INTERGEO 2013 in Essen to have a quick chat about LAStools, LASzip, and PulseWaves as well as our LiDAR processing toolboxes for ArcGIS and QGIS.

Geoinformatics magazine interviews rapidlasso

In the three-page article we talk about the beginnings of rapidlasso’s software, about when and how the company got started, about our various open and closed source products, and about current projects. The interview was published in the most recent October/November edition of the Geoinformatics magazine that was distributed in print at the SPARELMF 2013 conference in Amsterdam this week. If you did not get a copy you can read the online version of the magazine here.

rapidlasso selects LIDAR Technology Co. as first LAStools reseller

PRESS RELEASE
October 7, 2013
rapidlasso GmbH, Gilching, Germany

Just in time for INTERGEO 2013, the creators of LAStools and LASzip have partnered with LIDAR Technology Co., Ltd. of Zhubei City, Taiwan as their first exclusive reseller for the Taiwanese market. The two companies decided to strengthen their ties after holding a successful LiDAR processing workshop together with over 60 attendees following the International Symposium on Mobile Mapping Technologies (MMT 2013) in Tainan City last May. “We were looking for a partner with a strong technical mindset who is committed to capacity building,” says Dr. Martin Isenburg, founder of rapidlasso GmbH, “and LIDAR Technology Co., Ltd. is investing in R&D – including full waveform LiDAR – and has strong ties to academia. We couldn’t have found a better collaborator.” Dr. “James” Jin-King Liu, the CEO of LIDAR Technology Co., Ltd. concurs “our companies are a great match. LAStools is already widely popular in Taiwan, yet to employ them to their fullest potential can require some guidance. Our company has the technical expertise to provide this kind of support locally.”

About rapidlasso GmbH:

Technology start-up rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

About LIDAR Technology Co., Ltd.:

LIDAR Technology Co., Ltd. was founded in 2010 out of the airborne LiDAR team of the Industrial Technology Research Institute (ITRI), a National Laboratory in Taiwan. Their primary business is research, development, and application of earth observation, geoinformatics, and location-based services. LIDAR Technology Co., Ltd. performs acquisition, handling, and analysis of space-borne and airborne data with a focus on LiDAR, including full waveform, for which the company has developed own specialized solutions for their customers. Types of applications include mineral resources, geological mapping, geo-hazards studies, land-use and land-cover mapping, coastal zone mapping, forestry mapping, …  visit http://www.lidar.com.tw for more information.

With CEO "James" Jin-King Liu and CFO Ian Hsu in front of the office of LIDAR Technology Co., Ltd.

With CEO “James” Jin-King Liu and General Manager Ian Hsu in front of the office of LIDAR Technology Co., Ltd.

rapidlasso adds LAStools LiDAR processing toolbox to QGIS

PRESS RELEASE
September 30, 2013
rapidlasso GmbH, Gilching, Germany

At the code sprint following the FOSS4G 2013 conference, rapidlasso GmbH completed a toolbox that exposes the extensive LiDAR processing capabilities of the LAStools software suite within QGIS. QGIS (previously known as “Quantum GIS”) is a comprehensive desktop geographic information system that is free and open source software (FOSS). QGIS has seen rapid growth in acceptance as a viable and cost-effective alternative to the commerical ArcGIS products of ESRI. The newly integrated “Processing” framework (previously known as “SEXTANTE“) allows to combine the LiDAR functionality of LAStools with native raster and vector operations of QGIS as well as capabilities of other popular GIS packages such as GDALGRASS, SAGA, TauDEM, PostGIS and the (geo-)statistics package R. Detailed instructions on how to add the new LAStools toolbox to QGIS are available here.

About rapidlasso GmbH:
Technology start-up rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

Exhibiting at LiDAR Technologies 2013

PRESS RELEASE
August 22, 2013
rapidlasso GmbH, Gilching, Germany

This week rapidlasso GmbH is exhibiting its LAStools LiDAR processing software in tropical Cairns (Australia) at the LiDAR Technologies 2013 conference that is emerging as the premier LiDAR event in Australia. This marks the first time that rapidlasso is acting as a sponsor with its own exhibition booth on the showfloor. During the technical program rapidlasso is providing an update on the recently released PulseWaves open data exchange format for storing full waveform LiDAR data. The PulseWaves DLL is now fully integrated into the RASP software of Airborne Research Australia (ARA) at Flinders University in Adelaide and full waveform data is showcased that was recently acquired by ARA as part of a TERN AUScover campaign.

About rapidlasso GmbH:
Technology start-up rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

Toby Clewett Senior Geospatial Analyst at Sunshine Coast Regional Council

Having a laugh with Toby Clewett (Senior Geospatial Analyst at Sunshine Coast Regional Council) and Anne Henderson (catchement modeler and spatial analyst CSIRO Land and Water) at the rapidlasso booth

Release of PulseWaves Full Waveform LiDAR Format

PRESS RELEASE
June 21, 2013
rapidlasso GmbH, Gilching, Germany

Just in time for the RIEGL LiDAR 2013 user conference, technology start-up rapidlasso GmbH has released an open data exchange format called “PulseWaves” for storing full waveform LiDAR data. The new format has been in development since December 2011 through transparent discussions at http://pulsewaves.org that involved many stake holders. The result is a specification simple to implement, yet versatile enough to store many different waveform flavors. PulseWaves standardizes storage and delivery for applications that benefit from information about the waveform such as forest management, biomass estimation, and LiDAR bathymetry. It complements the ASPRS LAS format for discrete returns and radically simplifies the exchange of full outgoing and returning waveforms.

The initial PulseWaves release comes with an open source reference implementation (LGPL) and a set of basic processing tools to display and convert the data. A compressed version of the PulseWaves format based on LASzip technology is in progress. With their latest release of their RiPROCESS software, RIEGL GmbH has become the first hardware vendor to allow exporting full waveform LiDAR data to the PulseWaves format. The development effort was in parts supported by RIEGL GmbH and by Airborne Research Australia (ARA) at Flinders University in Adelaide.

About rapidlasso GmbH:
Technology start-up rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

An illustrative visualization of full waveform data as rendered by the latest version of pulseview.exe.

An illustrative visualization of full waveform data as rendered by the latest version of pulseview.exe.