The LAS format, the ASPRS, and the “LAZ clone” by ESRI

We are concerned about ESRI’s next moves in forcing yet another proprietary format into wide-spread deployment. Forwarded emails, retold conversations, and personal experiences suggest that sneaky tactics are being used to disrupt the harmony in open LiDAR formats that we have enjoyed for many years.

laz_and_lazclone_smallSome time has passed since we broke the news about the proprietary “LAZ clone” by ESRI. We were expecting the ASPRS to eventually comment on the issue. ESRI is promoting their lock-in product by the name of the open LAS specification (for which the ASPRS holds the copyright) calling their closed format “Optimized LAS“. We have been asked (in various forums) about the position of the ASPRS on this issue. ESRI’s use of LAS (*) makes it seem as if their “LAZ clone” was somehow an ASPRS thing (as evidenced by Harold’s comment). Despite ESRI’s media-blah-blah about “open and interoperable” they are – once again – luring the geospatial community to fall for a new proprietary format. So far the ASPRS has not released a statement on ESRI’s closed version of the LAS format.

The LAS Working Group (LWG) is part of the Lidar Division of the ASPRS. It has been maintaining the evolving LAS format from its 1.0 version that was (apparently as early as 1998) created by the LiDAR industry’s pioneers and eventually donated to the ASPRS (more recent LAS history is linked here). The good and early decisions of the LWG have created an incredible successful open data exchange standard for discrete LiDAR points that is nowadays supported by practically every software. “Kudos” to the original members for this achievement.

We did not join the LWG until 2011 to help avoid broken compatibility in LAS 1.4. After weathering the following “laser storm of 2011” the working group has been rather quiet. Its most recent activity was in 2013 for tendering the development of an official ASPRS LAS Validation Suite (LVS) that eventually resulted in ‘lasvalidate‘  – an open source LAS validator.

So who is this LWG? And why are they not commenting on such an important controvery like this “LAZ clone” with the seductive name “Optimized LAS”? The latest document on the Web pages of the LAS Working Group (LWG) lists the following people as members:

asprs_lascontrovery_1_members

This list from 2011 is hopelessly out of date, but it should give you an idea of the composition of the LWG. Most likely rapidlasso is still a member of the LWG but it is hard to tell because there have not been any emails recently and because there are no regular meetings. In the past we had some real bad luck with bringing up issues directly with the LWG, so here we go:

Dear ASPRS and LWG,
we are the guardians of the open LiDAR data exchange
specification, the LAS format. What is our response
to the proprietary format called "Optimized LAS" that
is being agressively promoted by ESRI?

Dear concerned ASPRS member,
how would you like your organization to respond now
that a large geospatial company uses its dominance
to push a closed format into the market, sabotaging
the accomplishments of an open data exchange standard
maintained by the ASPRS.

We are worried that ESRI – beyond lobbying agencies to convert their current holdings to the proprietary “LAZ clone” or to tender future deliveries in the closed zLAS format – may also be trying to form strategic alliances with vendors of popular LiDAR processing packages. Many of these vendors are also members of the LAS Working Group and would be in a conflict-of-interest if they were to “sell out” to ESRI’s lock-in ambitions.

You can imagine the red flag that went up a few days ago when we saw a technical comment on a LinkedIn post by Gene Roe that suggested intimate familiarity with the capabilities of the “LAZ clone” by Lewis Graham who has been leading the LAS effort since 1998 and who is the chair of the LAS Working Group. That Lewis’ comment has since been removed did little to calm our worries. As a side note: Gene’s posts being titled “LAS Data Format” further dilutes the difference between open LAS and closed zLAS.

Please inform us (or comment below) about any lobbying you hear about. Given the agressive moves by ESRI – in face of our repeated attempts to reach out – we do not think we can afford to err on the side of caution any longer … (-;

——————-

(*) It is fair to note that our products such as LAStools, LASlib, and LASzip also use the name “LAS”. This is for historic reasons. That is what we called the simple package for reading, writing, and processing LAS files we created back in 2005 for our own research before releasing them as open source in 2007. During our postdoc years at UC Berkeley we did not anticipate that these tools would become so or that we would start a company a few years later …

Restricting Access to National LiDAR: Is it worth it?

Policy makers in many countries are debating whether it is worthwhile to open their National LiDAR holdings – collected with tax-payer money – for free (or cheap) open access or whether they should continue to restrict access and charge potential users of the data to recoup expenses. Most would agree that hurdle-free, instant online access allows to exploit this valuable resource to the fullest and to maximize its benefit to the citizens. But some argue that opening this data eliminates the revenue stream the government needs to finance future surveys. Is this really true? No!

On November 21st, 2014 Louise Huby made this “Freedom of Information” request to the Environment Agency which are responsible for collecting, processing, and selling LiDAR data and derivatives for England with a focus on flood mapping applications. This seems to mainly be handled by the Environment Agency Geomatics, a specialist business unit within the Environment Agency.

Dear Environment Agency,
Please could you provide me with a breakdown by year of
all revenue made from the sale of LiDAR Data?
Yours faithfully,
Louise Huby

Later she added

Please could you provide me with a breakdown by year
of all revenue made from the sale of LiDAR Data?
Could you provide me with the information requested for
the following years: 2004, 2005, 2006, 2007, 2008, 2009,
2010, 2011, 2012, 2013, 2014.

A week later, on November 28th, Louise Huby received this answer:

EA_annual_sales_turnover

Hence the annual sales turnover (*) for LiDAR data was around £323,000 per year between 2007 and 2014.  It would be interesting to know how the annual average of £323,000 divides up further in terms of delivered products (raw LiDAR points or derivative), individual sale volume, price per square kilometer, type of customer, … Another request anyone?

(*) Sales turnover is the total amount of revenue generated during the calculation period. The revenue included in this calculation is from both cash sales and credit sales.

According to Wikipedia, the Environment Agency had an operational budget of £1,025,000,000 in 2007/08 (about half of this for flood risk management). This means the renevue from LiDAR sales is equivalent to 0.03 percent of the Agencies’s operating budget. I find this number shockingly low. Is this meager sales revenue an acceptable reason to keep the LiDAR locked up and inaccessible to the public?

Finland opened its LiDAR data in May 2012 and Denmark freed their national holdings in March 2013. Also Holland has turned around 180 degrees on its original LiDAR access policy by releasing its comprehensive AHN2 National LiDAR data set (including the raw point cloud) as free and open online data that was at first guarded and sold at high cost. I was given several reasons for this:
  • The main customer of LiDAR are the many branches of government from municipalities to federal agencies who often did not use it before, because it was “too difficult to obtain”. If someone needed geospatial data to make a decision but first had to fill out lots of paperwork and justification forms, possibly find a budget, and wait weeks for the delivery, then many questions had become irrelevant by the time the data arrived and decision were made without or with less good data.
  • Open LiDAR brings a bigger “return of investment” because the data – whose value is highly inflationary – gets used immediately for all purposes and not only when the anticipated potential of exploiting it sufficiently outweighs the upfront investment in time and money for aquiring it.
  • New business cases become worthwhile for which the LiDAR becomes “raw material” used to create new products and provide new services. This creates additional high-tech companies that pay taxes and desirable high-skilled jobs that would otherwise not exist. This can only happen if the resource “LiDAR” is either free or very cheap. It benefits both government and citizens as additional services and products are becoming available.
  • Overall: faster, better, cheaper service for all, full exploitation of the available resource, and higher return (even financially) in the long run.
One of the biggest drawbacks of trying to monetize the LiDAR is not only that there will be fewer services and products created by private sector businesses … but they will also be much more expensive. Those companies that took the risk to buy the data in the first place will charge a high premium on their services and products. And their biggest customer will be the government. Hence, an initial sale (quick money) will lead to years of high expenses for the government when it has to buy back the derived data products and services … costing many times more than was earned in the initial data sales.
In contrast, if the data is free for commercial use, then (a) the industry can only charge for the added value, (b) many more business cases suddenly make sense and even smaller added-value service or product can be offered, and (c) existing data is exploited for all decisions it is needed, (d) other agencies, universities, and private companies can focus on collecting complementing (instead of duplicate) data, …
To summarize: Opening data for free (or cheap) online access is seen as more cost-effective in the long run. It also allows the government to make better decisions, provide the citizens better service, and assures their national geospatial data holdings are exploited to the fullest …
Here some further reading on this topic by MetroGIS based on ideas of the NSGIC (National States Geographic Information Council) who is committed to efficient government through adoption of geospatial information by making “… all non-sensitive geospatial data, produced or maintained using taxpayer funds, a part of the public record.”