Moscow Importing Data

A variety of data formats are supported for input of the DEM data. 

The IMPORT menu lists options for the import of TIN data (i.e. irregularly spaced data) which is then gridded for use within SIBERIA. If possible you should not use these options for the input of data that is already gridded. You might want to do this, for instance, to convert from one grid resolution to another. This is because of limitations within the gridding code that result in pits being occasionally generated along the original grid, because of internal numerical roundoff errors.

The SIBERIA menu provides options for input of  gridded data and should be used for all gridded data, if possible.

For TIN data the data processing generates a series of intermediate files which remain after the run as (for this example the input file will be called EXAMPLE.XYZ):

  1. The imported TIN data is output in a file with extension .RAW. These files are identical except in the format of the files. The filename generated for our example will be EXAMPLE.RAW. If a file with this name already exists it will be overwritten.
  2. The TIN data is then gridded and a file with extension .GRID.RAW is output. This file has in the gridded data data in .RAW format. The filename generated for our example will be EXAMPLE.GRID.RAW. If a file with this name already exists it will be overwritten.
  3. In preparation for a SIBERIA run a SIBERIA restart file is generated with elevation that it is in the input gridded data. The restart file consists of a merger of the DEM, the run parameters, and environmental states from SIBERIA. The filename generated by default in the example is EXAMPLE-START.RST2. This first part of this filename (the bit before the hypthen '-') is the generic filename specified when setting SIBERIA run parameters. By default this generic filename is taken from the DEM name, but can be changed when setting the run parameters. The *-START.RST2 file will NOT overwrite a preexisting file of the same name.
For GRID data the data processing generates a series of intermediate files which remain after the run as (for this example the input file will be called EXAMPLE.GRID.XYZ):
  1. If an XYZ file is input the DEM in the file will thinned according to options requested when you input this file (every data point, every second data point, etc) and used as the basis for the RST2 format file. For all other gridded file formats the data in the file will be used as is.
  2. In preparation for a SIBERIA run a SIBERIA restart (RST2) file is generated with the elevation that it is in the input gridded data. The restart file consists of a merger of the DEM, the run parameters, and other environmental states from SIBERIA. The filename generated by default in the example is EXAMPLE-START.RST2. This first part of this filename (the bit before the hypthen '-') is the generic filename specified when setting SIBERIA run parameters. By default this generic filename is taken from the DEM name, but can be changed when setting the run parameters. The *-START.RST2 file will NOT overwrite a preexisting file of the same name.

DXF format (IMPORT TIN menu)

This is text file format used by AutoDesk's AutoCAD for interchange with other packages. AutoDesk regularly upgrade and extend the output options for this format so we cannot claim to support all current options. We do believe that we support all capabilities up to V11 of AutoCAD. The file should have the extension .DXF.

     Known Limitations:

  1. All x,y,x points in the file are interpretted. However, all LAYER commands are ignored and the file processed as if all x,y,z points are in the same layer. This means that if you have a layer containing x,y,z data that are NOT part of the landform surface (e.g. you might have a layer for power pole heights, roof elevations, etc) then EAMS will erroroneously assume that these points are part of the landform surface. In this latter case you will need to only output those layers that are the landform elevations from within AutoCAD
  2. Some V11 DXF commands are recognised but ignored. Recognised DXF entities are POINT, POLYLINE, LINE, 3DFACE, VERTEX and LWPOLYLINE. Ignored entities are SECTION, TABLE (and ENDTAB), LAYER, BLOCK (and ENDBLK), INSERT, TEXT, SEQEND, VPORT, STYLE, CLASS, LTYPE, APPID, DIMSTYLE, BLOCK_RECORD, DICTIONARY, MLINESTYLE, CIRCLE. All other entities are unrecognised and trigger a warning message. This warning is output because we cannot guarantee that subsequent data read from the DXF file is not impacted by the codes inability to understand the entity.
  3. Note even though the dxf  file may have, in addition to the original x,y,z data, a triangulation for this x,y,z data the triangulation data (though not the x,y,z data) is ignored and the gridding program uses its own triangulation. This is usually why the entity 3DFACE is used (each 3DFACE being a triangle).

XYZ format (IMPORT TIN menu)

This is a common format used in packages for the exchange of x,y,z data. It consists of a text file with each line having the x,y,z data for a point in free format (i.e. the 3 numbers for the x,y,z are seperated by either one of a comma, space or tab). There are as many lines in the file as there are points. The file extension is .XYZ.

RAW format (IMPORT TIN menu)

This is an exchange  format used in several mine management packages (e.g. VULCAN, MINDRAFT, ARGUS) and is functionally equivalent to the XYZ format, though the format itself is less user friendly and the file generated for an equivalent data set is much bigger. If possible the use of the XYZ format is preferred. The file extension is .RAW.

Gridded RAW format (SIBERIA menu)

This is gridded version of the RAW format (see TIN description above). The file extension is .GRID.RAW. Generally this file format is only used as an intermediate between the various parts of EAMS. Given the time consuming nature of the gridding step the user should try and reuse this gridded file than go back and regrid everytime they do a run.

Gridded XYZ format (SIBERIA menu)

This is a gridded version of the XYZ file format where the the x,y,z data are the x,y poisiton of a grid point, and z the gridded elevation for that point. The file extension is .GRID.XYZ. This file format is a useful way to import data from GIS systems such as ARC and MapInfo and is optimised to input very large files (we know of users who have sucessfully input files from LIDAR data with up to 200 million data points). NOTE that this file has no explicit information about the grid spacing nor the x,y dimensions of the grid, the import code has to work this out from the file. Accordingly the importer expects the points in the file in a specific order. These constraints are
  1. The grid must be square: i.e. the x and y grid spacing must be equal.
  2. The x,y directions must be aligned in the East and North directions. i.e. The grid cannot be at an angle to the E-N directions.
  3. The first point in the file is the point at the S-W corner of the grid (i.e. x,y, grid coordinates (1,1)). The 2nd point in the file is that grid point directly adjacent to it in the east direction (i.e. x,y grid coordinates (2,1)). The subsequent points in the file are those in the east direction (i.e. (3,1), (4,1), etc) until the eastern boundary is encountered at which time the data increments in the northern direction to the next grid line and repeats this process (i.e.(1,2), (2,2), (3,2), (4,2), etc). This is generally referred to as input of the data X-wise. Specifically, the x coordinate must increase between the first two points in the file (i.e. NOT decrease) and the y coordinates must increase though the file (again NOT decrease), so that the first point MUST the SW corner (this was done as its the default for ARC gridded output).
  4. The x and y grid spacing is determined by the difference in the x positions of the first two points in the file. The x size of the grid is determined by the number of points read before the y position of the input point changes for the first time. Subsequently the y size of the grid is determined by the number of lines of data input. 
  5. If you have asked for the data to be thinned (even though the importer can input very large files computer memory limitations typically limit SIBERIA runs to less than about a few million points) then the data is thinned as the data is read in so that the original very large data set never fully resides in computer memory.
  6. Error checking: Because this format is designed for automatic interchange bewteen GIS and EAMS there is almost no error checking in this format so that if there are missing data points in the grid they will not be picked up and all subsequent points in the grid will be offset from their correct position. Nor, for instance, are there checks that each x line of the input grid is the same length.


Back to EAMS Moscow