TransDEM Forum

TransDEM News, Support, Hints and Resources
It is currently 28 Mar 2024 21:04

All times are UTC + 1 hour




Post new topic Reply to topic  [ 4 posts ] 
Author Message
PostPosted: 28 Sep 2017 21:43 
Offline

Joined: 14 Sep 2017 00:43
Posts: 16
I am trying to save a map from the sas.planet program, and it offers so many choices on output I don't know where to begin.

I am trying to use sas.planet to do my map downloads because it can download files at 40Mbps, with up to 64 download threads running all at once.

I have tried to export the map as a geo referenced tiff, but i don't see anyway to open that in transdem.

Have you used this program before?

Any tips on how to export the geo referenced maps from sas.planet into transdem?
I know this isn't your program, but there has got to be some way I can import the georef image into transdem, it has so many output types.

I looked at one of your .trf files, but it doesn't match any of the georeferencing files that sas.planet can output.
sas.planet puts out many georef files that are readable in text format like the .trf file.
.map .dat .kml .tab .w



Format Options:
Image

Export Options:
Image

Coordinate Options:
Image








I also am working with DEM files that need to be merged, so I am using the latest 64-bit version of microdem.
No matter what DEM format I save my dems into, transdem will not read them.
Any tips on what format i should save my merged dems as so transdem can read the file?

Or how one can go about merging dems using microdem and then save it in a format transdem can read.
I am merging 100+ dem files, transdem don't allow multiple dem file selection when adding dems.
Which is why im using microdem.


I JUST found the answer to my DEM file format issue.

1. use .FLT (File Menu --> SAVE DEM --> Grid Float

Open .FLT file directly with tranddem.

I thought I already tried the .flt format with microdem, but apparently not or my file was just corrupt.





Still trying to figure out how to get the georeferenced raster from sas.planet into transdem...
From looking at the TransDEM .TRF file, and the .map .dat .kml .tab .w files, I can see coordinate info in all the files.


But I dont see how the numbers relate so i can create a .trf file manually for the raster image created with sas.planet.


Top
 Profile  
 
PostPosted: 29 Sep 2017 09:39 
Offline

Joined: 05 Jan 2011 16:45
Posts: 1463
One of the objectives of TransDEM was to avoid other 3rd party software as much as possible. Instead DEM, raster and vector map sources should be processed directly. This has its limitations in practice, of course. There just too many options for coding geo data.

For DEMs TransDEM internally writes and reads the MicroDEM .dem format. It should be able to formally read MicroDEM .dem versions 1 to 4, but the projection has to be UTM and the geodetic datum WGS84, plus elevations in metres. I have never worked with with other file formats as written by MicroDEM. Almost every DEM data format I ever came across - apart from the most trivial ones - offers several options of encoding. GeoTIFF, for instance, is one of the most complex ones. In my experience, about 50% of newly discovered DEM sources will not work out of the box. And quite surprisingly, it's not always a shortage in TransDEM itself.

But why are you merging in MicroDEM and not in TransDEM directly?

For map tiles (or tile maps) no real international standard exists. It's more by convention and tile geo-referencing is not intended for file distribution. However, many providers stick to one of the three patterns supported in TransDEM. But TransDEM will probably not recognise these specific map tile projections when present in a raster data file.

You can try to produce the geo-referenced format directly and create the .trf file via a script. Again, projection has to be UTM and geodetic datum WGS84. The .trf file is more or less self explaining. It contains the parameters for translation and scaling, there is no rotation or shearing. It also has a bit of redundancy in it, since image dimensions could be taken directly from the image file. That is due to its usage in the Zusi simulator environment where it could happen that the image files were resampled without also adjusting the .trf file.


Top
 Profile  
 
PostPosted: 30 Sep 2017 04:08 
Offline

Joined: 14 Sep 2017 00:43
Posts: 16
geophil wrote:
But why are you merging in MicroDEM and not in TransDEM directly?


I don't want to use transdem to do the DEM merging because trying to merge 100+ dem files one at a time using your software is painfully painful.

First I am required to let transdem convert the dem to .dem format. and then save into that format.


1.Open dem, let load, save dem as microdem format.
Repeat 100 times.

2.THEN I must also open the first dem, and ADD DEM 100+ MORE TIMES. Each time i must let the dem load and repaint, by the 100th dem, transdem is slow and too painful to keep using it to merge dems.

Your software won't allow multi-selection dem adding, which forces one to move to other programs to do mass dem merging.

SIMPLY PUT, MICRODEM allows one to choose ALL THE DEMS THEY WANT TO MERGE in a single operation.


geophil wrote:
One of the objectives of TransDEM was to avoid other 3rd party software as much as possible. Instead raster and vector map sources should be processed directly.



The current transdem orthographic map server resources, suck.

Also the transdem interface for downloading map files displays each file and loads them all into ram WHEN DOWNLOADING, if you download 1,000+ tiles, transdem becomes unusable (slow) for the download part.
IT can USE that many tiles, but when it DOWNLOADS them, its programming slows to a crawl due to all the maps being loaded into ram.

Other programs that cache map tiles do NOT keep them in RAM, ONLY WHILE downloading so the download remains constant and speedy.

Im not complainin, the program is cheap in cost, fine, but it is what it is.
Its, limited, and I am trying to work around those limits.


--->Many of the map tile and wms services no longer work properly, someone needs to go thru them and update them and add more, since many were retired for newer better ones that transdem dosen't have listed.

For instance, none of this ortho imagery is accessable in transdem, the server isn't listed, even tho its available and free, and not restricted by GOOGLE.
https://basemap.nationalmap.gov/arcgis/ ... /MapServer

I am not smart enough to figure out how these services work to add them to your program, but I am smart enough to find the servers and see that they exist.

I could post them here and ask you how to add them...


Google has some nice orthographic, but they HIGHLY restrict its use, and even restrict the amount of tiles you can download to a very very restricted level if you want ANY KIND OF ZOOM CLARITY AT ALL.

The google service is painstakingly slow also. And EVEN if you pay them for it, it won't work because it requires their signature, which you don't support, and they still restrict the bandwidth and total number of tiles.




So I found a ortho resource that allows high speed and high volume x20 images

I would like to add this tile map server to TransDEM, but I can't figure out how.



http://server.arcgisonline.com/ArcGIS/r ... rver/tile/

Which has a 20 zoom level available.


The url is structured like this:

Base: http://server.arcgisonline.com/ArcGIS/r ... rver/tile/

Code:
GetUrlScript
begin
 ResultURL:=GetURLBase+inttostr(GetZ-1)+'/'+inttostr(GetY)+'/'+inttostr(GetX);
end.



Code:
[PARAMS]
pnum=
GUID={7B743985-BC5F-4AB6-8915-AC5DBBB8F552}
name=ArcGIS.Imagery
NameInCache=ArcGIS.Imagery
ParentSubMenu=ESRI
DefURLBase=http://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/
ContentType=image/jpeg;image/jpg
Ext=.jpg
projection=1
sradiusa=6378137
sradiusb=6356752


Code:
Info
ESRI<br>
<a href="http://www.arcgis.com/home/webmap/viewer.html?useExisting=1">www.arcgis.com</a>



I see there are tons, ESRI servers would be good ones to add... but i don't know HOW.

You can download these two files that contain all the different urls and how they are formatted.
https://bitbucket.org/sas_team/sas.plus.maps/downloads/
https://bitbucket.org/sas_team/sas.maps/downloads/

Inside the files you will find tons of different map resources.
I would add them to transdem and share the export file here, BUT i don't know how the map servers work at all.

You show one of them, The usa Topo Raster 1:24k Raster tile service uses the arcgisonline.com sites, but DOSENT WORK.

Of that huge list from the sas zip file downloads, i got ONE to work.

And it only works to 16x, making it not that useful for placing trainz objects.

Image

So clearly all these companies that got all their data from satellites paid for by the us citizens now are somehow blowing thru all the rights we have to the data, and are now charging an arm and a leg for it all, or requiring users to "sign up" and get the data thru their proprietary software?

Is this where its all going now?

Its about time the data, is revoked, by the American people, that should stir up some attention and get it all back to where it should be, free.


So ones only option is to do what im doing, try to download the data while YOU STILL CAN.

Using programs like SAS.Planet. and TILE MAP SERVERS that you run yourself.

Thats where I am now, trying to get the data i acquired with sas.planet into transdem.

So far, i can only figure out that I must manually georeference each raster i create from sas.planet into transdem using geo points and the like..

I was just trying to make that more accurate by not doing it manually with the human error involved.

So a few reasons im not using trans dem to merge dems and to download the tiles, is:

1. Transdem cannot merge multiple dems at once.

2. Tansdem cannot do multithreaded tile downloading, and must keep all tiles in RAM while downloading, slowing the download logic, each url request slows down incrementally based on the amount of tiles currently loaded. So each tile loaded will slow down the rest of the tiles downloading, after 300 tiles, transdem's 300th tile request rate is measurably slower than the first requested tile. Other map tile downloaders, downloads 63 threads at 50Mbps... On the same pc/internet.

3. It also will not reuse already downloaded tiles for the same zoom and area, other programs DO, like sas.planet.
Wasting so much data that googles or other map services become prohibitively expensive. When your tile map pathway has gaps, and many of them you must re do the path and re-download.
Do that 3 times, and you have exhausted your quota, because transdem will download tiles for the exact same identical area, again. But just rename the file, and keep both.
Even if you "fill in the gaps" you now have much more downloaded overlaps, causing much more hd use and ram use, and pc slow down.
U can't win without GRID BASED map caching. checksum the tile, grid the checksums, checksum the grid of check sums, and you quickly scan. scan the checksum grid for missing... download only missing, have option to download only if different.


Top
 Profile  
 
PostPosted: 30 Sep 2017 10:56 
Offline

Joined: 14 Sep 2017 00:43
Posts: 16
My solution to this was to download my maps with Sas.Planet and use GEOSERVER to give a WMS service to TRANSDEM via localhost

I saved my entire map at zoom level 20 in the GeoTiff format once completely downloaded in sas.planet.

I then installed GeoServer, moved my 40 gig geotiff (and all the georef files) to the proper directory in the geo server data folder.

I then followed the tutorials on creating a new vector layer, but instead i created a geotiff layer (raster)

I then added a new WMS server to transdem. My url is this:
Code:
http://localhost:8080/geoserver/PA/wms?service=WMS&version=1.1.0&request=GetMap&layers=PA:padem&srs=EPSG:3395


Your url will be different ONLY where my url says "PA" and "PA:padem" if you used the default geoserver install config.


If anyone wants help getting maps downloaded in sas.planet, and then get geoserver to serve your downloaded maps to transdem, ask for help in a reply here.
And I will make the effort to do a screen shot tutorial.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: Ahrefs [Bot], SemrushBot [Bot] and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
cron

Imprint & Privacy

Powered by phpBB® Forum Software © phpBB Group