close
تبلیغات در اینترنت

پشتیبانی

aggregate data files
loading...

free grads

Using Templates grads allows you use a single data descriptor file to aggregate multiple data files and handle them as if they were one individual file. The individual data files must be identical in the X, Y, and Z dimensions and have the same list of variables. The time range of each individual file must be indicated it its filename. Beginning with version 2.0, data files may also be aggregated in the ensemble dimension.    First, the DSET entry has a substitution template instead of…

aggregate data files

admin بازدید : 259 چهارشنبه 10 دي 1393 نظرات ()

Using Templates

grads allows you use a single data descriptor file to aggregate multiple data files and handle them as if they were one individual file. The individual data files must be identical in the X, Y, and Z dimensions and have the same list of variables. The time range of each individual file must be indicated it its filename. Beginning with version 2.0, data files may also be aggregated in the ensemble dimension.  

 First, the DSET entry has a substitution template instead of a filename. See below for a description of all the possible components of the template. Second, the OPTIONS entry contains the template keyword. Third, the TDEF entry describes the time range for the entire set of data files.

Templating works on any grads data type for which you can write a descriptor file. If you specify any additional OPTIONS keywords in the data descriptor file, make sure the options apply equally to each file included in the template.

Templating over TIME

Valid components of the substitution template for the TIME axis are as follows:

    %x1   1 digit decade 
    %x3   3 digit decade 
    %y2   2 digit year 
    %y4   4 digit year 
    %m1   1 or 2 digit month 
    %m2   2 digit month (leading zero if needed) 
    %mc   3 character month abbreviation 
    %d1   1 or 2 digit day 
    %d2   2 digit day (leading zero if needed) 
    %h1   1 or 2 digit hour 
    %h2   2 digit hour 
    %h3   3 digit hour (e.g., 120 or 012) 
    %n2   2 digit minute; leading zero if needed
    %f2   2 digit forecast hour; leading zero if needed; more digits added for hours >99; hour values increase indefinitely
    %f3   3 digit forecast hour; leading zeros if needed; more digits added for hours >999; hour values increase indefinitely
    %fn2  2 digit forecast minute; leading zero if needed; more digits added for minutes > 99; minute values increase indefinitely (2.0.a9+)
    %fhn  forecast time expressed in hours and minutes (hhnn) where minute value (nn) is always <=59
          and hour value (hh) increases indefinitely. If hh or nn are <=9, they are padded with a 0 
          so they are always at least 2 digits; more digits added for hours >99. (2.0.a9+)
    %fdhn forecast time expressed in days, hours, and minutes (ddhhnn) where minute value (nn) is always <=59, 
          hour value (hh) is always <=23 and day value (dd) increases indefinitely. If dd, hh, or nn are <=9, 
          they are padded with a 0 so they are always at least 2 digits; more digits added for days >99. (2.0.a9+)
    %j3   3 digit julian day (day of year) (2.0.a7+)
    %t1   1 or 2 digit time index (file names contain number sequences that begin with 1 or 01) (2.0.a7+)
    %t2   2 digit time index (file names contain number sequences that begin with 01) (2.0.a7+)
    %t3   3 digit time index (file names contain number sequences that begin with 001) (2.0.a7+)
    %t4   4 digit time index (file names contain number sequences that begin with 0001) (2.0.a8+)
    %t5   5 digit time index (file names contain number sequences that begin with 00001) (2.0.a8+)
    %t6   6 digit time index (file names contain number sequences that begin with 000001) (2.0.a8+)
    %tm1  1 or 2 digit time index (file names contain number sequences that begin with 0 or 00) (2.0.a7+)
    %tm2  2 digit time index (file names contain number sequences that begin with 00) (2.0.a7+)
    %tm3  3 digit time index (file names contain number sequences that begin with 000) (2.0.a7+)
    %tm4  4 digit time index (file names contain number sequences that begin with 0000) (2.0.a8+)
    %tm5  5 digit time index (file names contain number sequences that begin with 00000) (2.0.a8+)
    %tm6  6 digit time index (file names contain number sequences that begin with 000000) (2.0.a8+)

When specifying the initial time (e.g., NWP model output), use these substitutions:

    %ix1   initial 1 digit decade 
    %ix3   initial 3 digit decade 
    %iy2   initial 2 digit year 
    %iy4   initial 4 digit year 
    %im1   initial 1 or 2 digit month 
    %im2   initial 2 digit month (leading zero if needed) 
    %imc   initial 3 character month abbreviation 
    %id1   initial 1 or 2 digit day (leading zero if needed) 
    %id2   initial 2 digit day 
    %ih1   initial 1 or 2 digit hour 
    %ih2   initial 2 digit hour 
    %ih3   initial 3 digit hour 
    %in2   initial 2 digit minute (leading zero if needed)
Starting with version 1.9b4, a new template option has been added that allows for any user-specified string substitution, instead of only date string substitution. This is useful when none of the above template options match the time ranges in the files you wish to aggregate, or if the files are located on different disk pathnames. The syntax is as follows:
    %ch   substitute string
If you put the %ch template in your DSET entry, then you also need to put additional CHSUB entries in the descriptor file that contain two integers (t1 and t2) followed by a string which will be substituted for %ch in the data file names for the time steps beginning with
t1 and ending with t2. The CHSUB descriptor file entries have the following syntax:

    chsub  t1  t2  string

     

Templating over ENSEMBLE

In version 2.0, with the introduction of the extra grid dimension for ensembles, support was also added for file templating over E. The sole substitution template is %e and the substitution string is the ensemble name, which is provided in the EDEF entry in the descriptor file. Note that the ensemble names are limited to 15 characters -- keep this limit in mind when designing your data directory structure and file naming conventions (or use symbolic links to create short aliases for longer filenames). If you are templating over the ensemble dimension, there can be only one ensemble member per file. If your data set has an ensemble dimension, and you are using templating over T but not E (i.e., there is no %e in the DSET entry), then all ensemble members are presumed to have identical time axes, and all members must be contained in the data file for a given time. Templating over T but not E is not supported for data sets in flat binary or GRIB1 formats.


Examples

  1. Here's a set of binary files spanning a single month, where each day's worth of hourly data is contained in individual files: 
       1may92.dat
       2may92.dat
       ...
       31may92.dat 
    Three records must be modified in the data descriptor file. Note that the TDEF entry reflects the entire month's worth of data: 
       DSET ^%d1may92.dat
       OPTIONS template
       TDEF 744 linear 0z1may1992 1hr 

  2. If your data set expanded, and there were more files containing hourly data for other months and years: 
       1jun92.dat
       2jun92.dat
       ...
       1jan93.dat 
    Then you would add a template for month and year in your DSET entry and extend the length of your TDEF: 
       DSET ^%d1%mc%y2.dat
       OPTIONS template
       TDEF 6624 linear 0z1may1992 1hr


  3. Suppose you have a set of seven netcdf files, each containing monthly data spanning a decade:
       pr.1880_1889.nc
       pr.1890_1899.nc
       pr.1900_1909.nc
       pr.1910_1919.nc
       pr.1920_1929.nc
       pr.1930_1939.nc
       pr.1940_1949.nc
    Then your descriptor file would include the following entries:
       DSET ^pr.%x30_%x39.nc 
       OPTIONS template 
       DTYPE netcdf 
       TDEF 840 linear jan1880 1mo

  4. Here are two netcdf files, one containing 50 years of monthly data (600 time steps), the other 100 years (1200 time steps): 
       pr.1851-1900.nc
       pr.1901-2000.nc
    Your descriptor file should include the following entries:
       DSET ^pr.%ch.nc
       CHSUB   1  600 1851-1900
       CHSUB 601 1800 1901-2000
       OPTIONS template
       DTYPE netcdf
       TDEF 1800 linear jan1851 1mo
    If these two data files were located on different disks, you could write out the relevant descriptor file entries this way instead:
       DSET %ch
       CHSUB   1  600 /disk1/pr.1851-1900.nc
       CHSUB 601 1800 /disk2/pr.1901-2000.nc

  5. Your forecast model output looks like this:
      MMOUT_DOMAIN1_00
      MMOUT_DOMAIN1_01
      MMOUT_DOMAIN1_02
    so your DSET enry will look like this: 
      DSET ^MMOUT_DOMAIN1_%tm2
برچسب ها aggregate data files ,
ارسال نظر برای این مطلب

نام
ایمیل (منتشر نمی‌شود) (لازم)
وبسایت
:) :( ;) :D ;)) :X :? :P :* =(( :O @};- :B /:) :S
نظر خصوصی
مشخصات شما ذخیره شود ؟ [حذف مشخصات] [شکلک ها]
کد امنیتی
اطلاعات کاربری
نام کاربری :
رمز عبور :
  • فراموشی رمز عبور؟
  • آمار سایت
  • کل مطالب : 155
  • کل نظرات : 0
  • افراد آنلاین : 2
  • تعداد اعضا : 4
  • آی پی امروز : 18
  • آی پی دیروز : 34
  • بازدید امروز : 61
  • باردید دیروز : 287
  • گوگل امروز : 0
  • گوگل دیروز : 0
  • بازدید هفته : 348
  • بازدید ماه : 1,781
  • بازدید سال : 37,067
  • بازدید کلی : 58,457
  • کدهای اختصاصی