New LASliberator “frees” LiDAR from Closed Format

PRESS RELEASE (for immediate release)
April 20, 2015
rapidlasso GmbH, Gilching, Germany

The latest product by rapidlasso GmbH – creators of LAStools and LASzip – is an open source tool aiming to liberate LiDAR points locked-up in proprietary “Optimized LAS” – a highly controversial, closed LiDAR format. The new LASliberator can be downloaded here. It comes as both, a simple command line tool for scripting and with an easy-to-use graphical interface.

The GUI version of the LASliberaor has a simple and easy-to-use interface.

The GUI of the “LASliberator” has a simple, easy-to-use interface.

The LASliberator reads LiDAR points from closed “Optimized LAS” files that use the “.zlas” extension and converts them to open ASPRS LAS files that use the “.las” extension. Alternatively, the points can be stored to compressed LAZ files – using the open source LASzip compressor – that use the “.laz” extension. In addition, the tool creates tiny spatial indexing files that use the “.lax” extension. These can then be exploited for accelerated area-of-interest queries via open source LASindex when using LAStools or the latest version of the LASzip DLL.

Note that the LASliberator cannot entirely be open source as it depends on a particular proprietry library. The closed nature of the “Optimized LAS” format does not allow for a full open source implementation. It is therefore not possible to port the LASliberator to other operating systems or into other programming languages.

Selecing open in the GUI pops up a file selection dialogue allowing the user to find the file that is to be set free.

The user can select a file to liberate by pressing “open” in the GUI.

The new LASliberator comes on the heels of an outcry in the community over the LiDAR format fragmentation “Optimized LAS” is creating. It provides an immediate solution to go from closed zLAS to open LAZ for people whose LiDAR got stuck in yet-another-proprietary-format.

About rapidlasso GmbH:
Technology powerhouse rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

The LAS format, the ASPRS, and the “LAZ clone” by ESRI

We are concerned about ESRI’s next moves in forcing yet another proprietary format into wide-spread deployment. Forwarded emails, retold conversations, and personal experiences suggest that sneaky tactics are being used to disrupt the harmony in open LiDAR formats that we have enjoyed for many years.

laz_and_lazclone_smallSome time has passed since we broke the news about the proprietary “LAZ clone” by ESRI. We were expecting the ASPRS to eventually comment on the issue. ESRI is promoting their lock-in product by the name of the open LAS specification (for which the ASPRS holds the copyright) calling their closed format “Optimized LAS“. We have been asked (in various forums) about the position of the ASPRS on this issue. ESRI’s use of LAS (*) makes it seem as if their “LAZ clone” was somehow an ASPRS thing (as evidenced by Harold’s comment). Despite ESRI’s media-blah-blah about “open and interoperable” they are – once again – luring the geospatial community to fall for a new proprietary format. So far the ASPRS has not released a statement on ESRI’s closed version of the LAS format.

The LAS Working Group (LWG) is part of the Lidar Division of the ASPRS. It has been maintaining the evolving LAS format from its 1.0 version that was (apparently as early as 1998) created by the LiDAR industry’s pioneers and eventually donated to the ASPRS (more recent LAS history is linked here). The good and early decisions of the LWG have created an incredible successful open data exchange standard for discrete LiDAR points that is nowadays supported by practically every software. “Kudos” to the original members for this achievement.

We did not join the LWG until 2011 to help avoid broken compatibility in LAS 1.4. After weathering the following “laser storm of 2011” the working group has been rather quiet. Its most recent activity was in 2013 for tendering the development of an official ASPRS LAS Validation Suite (LVS) that eventually resulted in ‘lasvalidate‘  – an open source LAS validator.

So who is this LWG? And why are they not commenting on such an important controvery like this “LAZ clone” with the seductive name “Optimized LAS”? The latest document on the Web pages of the LAS Working Group (LWG) lists the following people as members:

asprs_lascontrovery_1_members

This list from 2011 is hopelessly out of date, but it should give you an idea of the composition of the LWG. Most likely rapidlasso is still a member of the LWG but it is hard to tell because there have not been any emails recently and because there are no regular meetings. In the past we had some real bad luck with bringing up issues directly with the LWG, so here we go:

Dear ASPRS and LWG,
we are the guardians of the open LiDAR data exchange
specification, the LAS format. What is our response
to the proprietary format called "Optimized LAS" that
is being agressively promoted by ESRI?

Dear concerned ASPRS member,
how would you like your organization to respond now
that a large geospatial company uses its dominance
to push a closed format into the market, sabotaging
the accomplishments of an open data exchange standard
maintained by the ASPRS.

We are worried that ESRI – beyond lobbying agencies to convert their current holdings to the proprietary “LAZ clone” or to tender future deliveries in the closed zLAS format – may also be trying to form strategic alliances with vendors of popular LiDAR processing packages. Many of these vendors are also members of the LAS Working Group and would be in a conflict-of-interest if they were to “sell out” to ESRI’s lock-in ambitions.

You can imagine the red flag that went up a few days ago when we saw a technical comment on a LinkedIn post by Gene Roe that suggested intimate familiarity with the capabilities of the “LAZ clone” by Lewis Graham who has been leading the LAS effort since 1998 and who is the chair of the LAS Working Group. That Lewis’ comment has since been removed did little to calm our worries. As a side note: Gene’s posts being titled “LAS Data Format” further dilutes the difference between open LAS and closed zLAS.

Please inform us (or comment below) about any lobbying you hear about. Given the agressive moves by ESRI – in face of our repeated attempts to reach out – we do not think we can afford to err on the side of caution any longer … (-;

——————-

(*) It is fair to note that our products such as LAStools, LASlib, and LASzip also use the name “LAS”. This is for historic reasons. That is what we called the simple package for reading, writing, and processing LAS files we created back in 2005 for our own research before releasing them as open source in 2007. During our postdoc years at UC Berkeley we did not anticipate that these tools would become so or that we would start a company a few years later …

Can you copyright LiDAR?

A few weeks ago, I wanted to demonstrate how geometric compression can shorten download times for online dissemination of 3D archeological artifacts. The demo failed. My web page was gone. The web admin told me later that he had to delete it after receiving an email from a director at CyArk stating that I was “[…] hosting unauthorized content from CyArk […] ” and that “[…] Dr. Isenburg gained unauthorized access to our information and the re-posted it to his webpage […]”. Bummer. Unintentionally, I had become “the raider of the CyArk”. A point plunderer. A LiDAR looter. A scan scrounger. A laser pirate … arrr … (-;

How did I fall so low? After reading this LiDAR news article about CyArk’s new online 3D viewer I invested serious time into understanding their content delivery system and suggested how to shorten download times as I had done a lot of prior research on this particular topic (see this, this, this or this page). So, I created several interactive java-based web pages for them to demonstrate how – with some quantization, simple prediction, and clever scripting – more web-efficient 3D content might be possible. I did these experiments with their data sets to allow an apples-to-apples comparison: a point model (Ti’kal) and a mesh model (Mount Rushmore).

cyark_mount_rushmore_small

After a long technical exchange the person at CyArk suddenly demanded that I take down the compressed content. I was surprised and asked why I should have to delete these illustrative examples on 3D compression that represented a significant investment in volunteered time and energy.

Me: “What you mean with take down? Delete it from my webpages? But I am using it as a purely educational example for scanner precision and coordinate resolution. I am not promoting it in any context that would interfere with the mission of CyArk. I do not quite follow the imperative here. Aren’t you a non-profit site dedicated to science and education? And anyone could download those points clouds from your site just the way I did it. It’s not rocket science … (-:”

Person at CyArk: “And, yes, to clarify my request, I would like you to delete any content from your server or webpages. Sorry if I was vague. Thanks!”

Me: “I believe I am in accordance with both Ben (Kacyra)’s vision and the creative commons license with my educational use of the 3D content (see http://archive.cyark.org/copyright). Is there something I am missing?”

I considered myself well informed about CyArk’s mission on providing open access to 3D data for research, education, and virtual tourism through various media such as Wikipedia and Ben Kacyra’s visionary TED talk. I assumed that my creative commons argument had resonated because I did not hear back from them. I only realized that CyArk was not interested in explaining their licensing but simply had my pages removed when I tried to access this demo.

A few days ago I saw Tom Greaves, executive director at CyArk, commenting “Sweeeet use of CyArk data.” on their blog entry which describes the creation of a sugary fudge replica of Ti’kal – the very same data set that I had been using – for the launch event of a new sugar series by British-based multinational agribusiness Tate & Lyle.

Tik’al Fudge Cake made with golden caster sugar, over 80 cm tall

I like to have fun with LiDAR and appreciate the educational factor of such events. Yet I wonder whether the Guatemalan people would be that much happier to see their ancient cultural heritage presented as a piece of cake to promote a new line of sugars than to see it used as a demo on how to Web-optimize 3D content … (-;

I took this as an opportunity to – once more – inquire about the creative commons license of CyArk and I finally received an answer from Tom.

Dear Dr. Isenburg,

Please understand that only some of […]     […] have any questions.

Sincerely,
Tom Greaves

Executive Director
CyArk

Unfortunately Tom did “not recall giving” me his “permission to publish” his “private correspondence” as he pointed out shortly after this blog article went live, so I had to remove the reprint of his email. It essentially said that much of the data collected by CyArk remains property of the site owners and that Cake for Breakfast obtained permission to use the 3D scan as the secret ingredient for their Mayan bake job.

copyright cartoon

After inquiring with Tom “So which models are creative commons and which not?” I quickly got the surprising response from Tom that: “None of our 3D point cloud is available under Creative Commons. Only some of the 2D image data is covered by this.

Now this is certainly not what I had been reading into their press releases and news articles. The data-generous openness in access to 3D data that is advertised for example on their mission statement: “Digital Preservation is ‘Preserving cultural heritage sites through collecting, archiving and providing open access to data created by 3D laser scanning, digital modeling, and other state-of-the-art technologies,’ the CyArk Mission.” is apparently not the practiced reality.

So I asked in my LAStools user forum about the experiences of others: What are the most and least permitting licenses for such data and what do they mean in practice? How do I know what is open and what not? Can you help clarifying what “creative commons” licensing means and what it allows and forbids so I don’t violate anyone’s license in the future. This sparked discussions with interesting outcomes:

  1. the creative commons (non-commercial) license is useless
    The folks behind @OpenAccessArch picked up the story to provide their view of the particularities of the creative commons (non-commercial) license used by CyArk in a long blog post titled “Creative Commons Non-commercial A Cruel Joke.
  2. it is in general not possible to copyright a LiDAR scan
    Doug Rocks-Macqueen from @OpenAccessArch also started the fundamental discussion whether it is even possible to copyright a LiDAR scan in the first place. Apparently not – at least not for an object whose copyright has already expired and for details read this message thread. His closing argument was that in this legal battle between Meshwerks, Inc. and Toyota Motor Sales U.S.A., Inc. the Court of Appeals affirmed the district court’s opinion that “3D models of physical objects, if faithfully and accurately representing the original, are not original enough to warrant copyright protection.”
  3. do not engage in open-washing … (-:
    In CyArk’s defense it needs to be said that there is probably a collision between their vision and the contract terms specified by the different site owners that they have to respect in order to get the permission to scan a site. What is lacking are clear terms of use in their communications and proper protection mechanisms. The academic pioneers of 3D scanning at Stanford university had to deal with similar issues during their Digital Michelangelo Project and created ScanView: a secure client / server rendering system that permits users to examine 3D models, but not extract the underlying data. In summary: do not claim your data is open and allow access to it when it is not.

With all the publicity I was worried that my little rabble rousing might be perceived as disruptive instead of constructive by the community until someone reassured me that: “I think we’re all quite happy that the discussion is happening. Between you and me, CyArk have a reputation as being rather less open with their data than their publicity would suggest.” … (-;

Martin @rapidlasso

PS: Be aware that all comments to this article will be considered “creative commons”. Or maybe not … (-;

cyark_tikal_sugarcubes_400The (loosely related) image shown above was obtained here and is courtesy of CyArk. I assume this use is allowed under their copyright and does not violate the creative commons (non-commercial) license … (-;

Addendum (May 1st, 2013): As a result of this article CyArk not only updated their copyright notice to exclude point clouds from the Creative Commons license but also added a very clear data use policy statement.