Annex: trend.aeronomie.be/trend4/format/
TREND-4 MF format: File specifications
The (new) UNIRAD tabular outputs are sequentially organised files consisting of variable-length records whose length is indicated by an explicit terminator embedded in the data. The record terminator is the 2-character sequence consisting of a carriage-return and a line-feed (ASCII codes 13 and 10, respectively). Except for the 2-character terminator, the records only include readable ASCII characters with code between 32 and 127. Such files can be created by a FORTRAN programme with the following statement:
OPEN(UNIT=n, FILE='project.csv', FORM='FORMATTED',
: ORGANIZATION='SEQUENTIAL', RECORDTYPE='STREAM')
where the text in italics depends on the user application.
Each record contains comma (,) separated values that can
be read by a FORTRAN list-directed READ
statement. The integer and floating values are represented
by their decimal forms (e.g. 123, -98,
734.5, -3.2345E+10, +6.2345D+68),
optionally preceded by white space characters (ASCII code 32).
The character strings are delimited by apostrophes (') without
other characters before or after the apostrophes.
When an apostrophe has to be included in the character string, it is
represented by two consecutive apostrophes.
Example
The use of the file specifications is illustrated by the following FORTRAN programme.
PROGRAM sample_1
C
INTEGER k1, k2
REAL r1, r2
CHARACTER*32 s1
C
DATA k1, k2, r1, r2/ -547, 12, -5.978E-18, 6371.2/
C
C File creation
C
OPEN(UNIT=1, FILE='sample_1.txt', FORM='FORMATTED',
: ORGANIZATION='SEQUENTIAL', RECORDTYPE='STREAM')
WRITE(1,1000) k1, r1, k2, r2
CLOSE(1)
C
C format
C
1000 FORMAT( i6, 1h,, 1p, e13.6, 0p, 2h,',
: 'Text with an apostrophe:''''.', 2h',, i8, 1h,, f10.3)
C
C File read
C
OPEN(UNIT=1, FILE='sample_1.txt', STATUS='OLD')
READ(1,*) k1, r1, s1, k2, r2
CLOSE(1)
C
PRINT*,k1
PRINT*,r1
PRINT*,s1
PRINT*,k2
PRINT*,r2
END
When imported in a spreadsheet programme, the content of the
sample_1.txt file appears as
Internal organisation
A UNIRAD output file contains one or more independent parts, each being associated with a section of the corresponding UNIRAD namelist. Note that, for some applications, more than one part can be associated to a single section of the namelist. Each part includes three sections: a header, a body, and a footer. The following sub-sections describe the general structure of these three sections.
The general structure is illustrated with the help
of an example file.
Header section
The records included in the header section contain general
information on the data (project title, main parameters of
the programme used to generate the data,...), and a description
of the content of the body section. The header section includes
also a reserved area that can be extended for futur usage.
The following table give a detailled description of the structure
of the header section. Note that the number of records varies
with the application.
| Record | Elements | Description |
|---|---|---|
| 1 | '*',
nheader, ntext,nmeta, nfut, nvar, ncol, nbody, npart |
The first record of the header section consists of
a character string and eight integers. The character
string serves as guard and shall always be set to
a single asterisk ('*'). The integers
provide the information on the size of the header
and body sections:
|
| 2 ... (ntext+1) | atext | Each record includes a single character string. These
character strings can be used to indentify the programme used
to produce the file (e.g. 'UNIRAD v3.1 - SAPRE -
OpenVMS (BIRA/IASB)'), the project
title, or other information such
as the name of the model used to produce the data. |
| (2+ntext) ... (1+ntext+nmeta) |
ameta, ktyp, val_1, val_2, ... |
Each record contains the information on a specific metavariable, and starts by a character string (ameta) followed by an integer (ktyp). The string ameta contains the name of the metavariable. The integer ktyp specifies the type of the metavariable. When ktyp is equal to -1, the metavariable contains a character string, the value of which is set in the third element of the record (val_1). When ktyp is greater than 0, the metavariable contains a vector of ktyp real numbers, the values of which are set in the third and following elements of the record (val_1 ... val_[ktyp]). |
| (2+ntext+nmeta) ... (nheader-nvar) |
not defined | This area is reserved for future usage and is therefore usually not present in the header section. |
| (nheader-nvar+1) ... nheader |
aname, aunit, nelem, atitle | Each record includes information on a specific
variable of the data set. A variable is defined as a single
real number or as a vector of real number.
Each record contains four elements:
|
The header section is directly followed by the body section.
Future extension
According to the above specification, the body can only include single real variables or one dimensional variables. In future developments, multi-dimensional variables could be introduced. In that case, nelem should correspond to the total number of elements and information on the dimensions can be added after the argument atitle. For instance: a list of metavariables, each one of which corresponds to one of the dimensions and contains as many elements as the corresponding dimension. For example, a density map could be stored in a variable defined as'DENS','cm-3',
200,'Density','LATITUDE','LONGITUDE' where LATITUDE and
LONGITUDE are metavariables that contain a set of
10 latitudes and 20 longitudes, respectively (10 x 20 = 200).
Body section
Each record of the body section contains the values associated to all the variables of the data set. All records of the body section have the same structure: they contain a suite of real numbers, the number of which is specified by the 7th element of the first record of the header section (ncol). For the sake of clarity, the values are usually written in field with fixed with such that the fields from the different records are aligned in columns. In each record, the values are ordered with the same order that the variables are listed in the header section.
The number of records contained in the body section is
not explicitly determined. An expected value of this number
is given by the 8th element of the first record of
the header section (nbody).
Practically, the end of the body section is defined
by the detection of the footer section.
For the sake of uniformity the following character strings
are suggested:
The usage of the header, body and footer sections are illustrated
by an example file.
Footer section
The footer section consists of a single record that contains
a character string. This character string shall not start
with an asterix (*) and is used to detect the end of the
body section.
The text of the footer string can be use to pass information
on the next part of the output file.
Note that both character strings '*ERROR*'
and '*CONTINUE*'
will be used by IDL and FORTRAN interface routines.
'End of Block', to indicate that the next part
of the output file is related to the next namelist section;
'End of File', to indicate that the end of the
output file has been reached;
'Section Break', to indicate that the next
part of the output file is related to the same namelist
section (see npart in the header
section);
'End of Namelist', to indicate that, next to the
current part, parts have added in the output file which are
not related a common namelist section.
Example file
This small example illustrates the specifications
on the UNIRAD output file structure. Different colors have been used
in order to better highlight the different part of the structure.
1:'*', 13, 2, 3, 4, 3, 8, -1, 0 2:'Title of this project' 3:'This is an example file' 4:'EPOCH', 1, 1995.0 5:'ENERGY', 6, 0.10, 0.50, 1.00, 2.00, 5.00, 10.00 6:'MODEL', -1,'IRI-90' 7:########################## 8:# Specific area reserved # 9:# for futur use # 10:########################## 11:'AMJD ','day ', 1,'Modified Julian Day' 12:'FLUX_EL ','1/cm2/s', 6,'Integral electron flux' 13:'L ','Re ', 1,'McIlwain''s shell parameter' 14: 17888.07465, 1.2E+06, 1.0E+06, 5.4E+05, 2.9E+05, 4.2E+04, 9.8E+03, 2.067 15: 17890.78901, 6.0E+05, 5.0E+05, 2.7E+05, 1.4E+05, 2.1E+04, 4.9E+03, 1.076 16: 17892.87572, 3.2E+05, 2.0E+05, 2.4E+05, 1.9E+05, 2.2E+04, 4.8E+03, 1.085 17: 17894.36543, 1.2E+04, 1.0E+04, 5.4E+03, 2.9E+03, 4.2E+02, 9.8E+01, 2.094 18: 17896.43453, 6.0E+05, 5.0E+05, 2.7E+05, 1.4E+05, 2.1E+04, 4.9E+03, 3.103 19: 17898.88785, 1.2E+06, 1.0E+06, 5.4E+05, 2.9E+05, 4.2E+04, 9.8E+03, 3.112 20: 17900.68776, 1.2E+04, 1.0E+04, 5.4E+03, 2.9E+03, 4.2E+02, 9.8E+01, 2.121 21: 17902.76786, 6.0E+05, 5.0E+05, 2.7E+05, 1.4E+05, 2.1E+04, 4.9E+03, 1.130 22:'End of Block' |
Comments
- The first record (in black) indicates that
- the header section extends from record 1 to 13;
- the header section includes 2 text lines, 3 metavariables, 4 reserved records, and 3 variable definitions;
- each record of the body section contains 8 elements corresponding to the values of the 3 variables;
- the number of records of the body section was unknown when this record has been written;
- there is only a single part associated to the current namelist section.
- Records 2 and 3 (in green) contain the two text lines.
- Records 4 to 6 (in red) contain the three metavariables:
EPOCH,ENERGYandMODEL. The metavariableEPOCHcorresponds to a single real number while the metavariableENERGYis a vector of 6 elements. The metavariableMODELrefers to a character string. - Records 7 to 10 (in silver) correspond to an area, the format of which is not defined by the current specification.
- Records 11, 12 and 13 (in blue, brown and
magenta, respectively) contain the definition of the three
variables. Both first and third variables (
AMJDandL) correspond to single real numbers. The second variable (FLUX_EL) is a 6-element vector of real numbers. - Records 14 to 21 correpond to the body section. Each record contains 8 elements associated
to the three variables
AMJD,FLUX_ELandL. The colors highlight the association between the columns and the variables. - Record 22 is the footer section.
