2m DEM for England: Environment Agency LIDAR

[SandTigerUK]

Looks like us Celts have been given LIDAR also now -

http://www.scottish-orienteering.org/so ... -available
http://lle.wales.gov.uk/Catalogue/Item/ ... t/?lang=en

[Danny252]

Playing around with the LIDAR data, I tried using the LIDAR Tiles DSM/DTM, which I think are an unprocessed (or at least less processed) version of the data in the LIDAR Composite DSM/DTM. In the areas I was looking at, there are several surveys included in Tiles that weren't available as composite, extending coverage to areas not covered by the Composite data (which focuses on floodplains, etc.). Whilst this data mostly works well, there are a few problems or difficulties:

1) A major problem is that the Tiles dataset contains elevations in millimeters, rather than meters - however, TransDEM interprets the data as meters, giving heights of 110,000m in rural England! I'm not sure if DEM data should contain some sort of meta-data about heights, or if it is always assumed elevation is measured in meters. If the metadata should handle this, then this is a problem with the data, as MicroDEM also reads the elevation scale incorrectly. I corrected the data by using MicroDEM, with Edit > Single Grid Arithmetic > Divide Z Values. Is it worth adding an option to set z scale for TransDEM?

2) All LIDAR data are in .asc files. Whilst TransDEM is able to read these, options such as "Add DEM" and "Fill up DEM" will accept only .dem files. As a result, I had to open each file and resave as a .dem before combining the tiles into a larger area.

3) Most of the available data in the area I looked at was 1m, which is obviously overkill for Trainz! Whilst this in itself wasn't an issue, 1m resolution is good enough to start picking up the shapes of individual trees, which results in very bumpy terrain in forested areas when exported to Trainz with the default parameters. The only solution I've found to this is to use very harsh smoothing (filter width of 10 or so).

[geophil]

1) A major problem is that the Tiles dataset contains elevations in millimeters, rather than meters - however, TransDEM interprets the data as meters, giving heights of 110,000m in rural England! I'm not sure if DEM data should contain some sort of meta-data about heights, or if it is always assumed elevation is measured in meters. If the metadata should handle this, then this is a problem with the data, as MicroDEM also reads the elevation scale incorrectly. I corrected the data by using MicroDEM, with Edit > Single Grid Arithmetic > Divide Z Values. Is it worth adding an option to set z scale for TransDEM?

.asc is a rather basic DEM format without much meta data at all. So it would have to be a manual option, I think. (I have added it to the wish list.) But why are they coding z as mm anyway? The .asc format supports floating point notation, so data representation in m shouldn't be an issue.

However, even with formats that support many meta options, data providers will still find ways not to adhere to common practice and invent all sorts of deviations. The main reason I guess is that data providers often have not much other data to compare with and are not familiar with typical usage. I have encountered quite a few such "exceptions" in the almost fifteen years I am working with geo data now.

2) All LIDAR data are in .asc files. Whilst TransDEM is able to read these, options such as "Add DEM" and "Fill up DEM" will accept only .dem files. As a result, I had to open each file and resave as a .dem before combining the tiles into a larger area.

Unfortunately, yes. Opening any other format than MicroDEM .dem and converting from other projections to UTM needs full access to the DEM "engine" in TransDEM. That's why any open DEM needs to be / will be closed before proceeding with a new one. The DEM merge function and its "fill" sibling work differently, with the help of a substitution mechanism.

3) Most of the available data in the area I looked at was 1m, which is obviously overkill for Trainz! Whilst this in itself wasn't an issue, 1m resolution is good enough to start picking up the shapes of individual trees, which results in very bumpy terrain in forested areas when exported to Trainz with the default parameters. The only solution I've found to this is to use very harsh smoothing (filter width of 10 or so).

That's one of the challenges with hi-res data. You can try to run the low pass filter (smoothing) over the entire DEM, and that's what you did I guess. When exporting to Trainz, TransDEM simply samples DEM points through bilinear addressing, without applying any filter at that stage.

[PFX72]

Having just discovered the joy of 2m LIDAR data, I started to experiment with an area a few miles long. Is there any way to open the individual files and do the conversion to DEM automatically or does it all need to be done one at a time? Doing it one at a time, it has taken me 2 hours to do just 3 grid squares.

Thanks,
Innis

[geophil]

If you have the latest TransDEM version, 2.6.3, you can select multiple files when opening the DEM, e.g. all the files in the folder, and TransDEM will merge them automatically. I added this new functionality to 2.6.3 because LIDAR files are often relatively small and you need lots of them.

[PFX72]

Thanks Roland. That makes me most happy, especially as i just upgraded.

Innis