Seismic time series: Acceleration and mocking SG signal

  Preparation of environment:
     Assuming the production is to target 10 Samples per second BIN-ts files,
     and using tcsh as the interactive shell:
cd ~/Seismo/gcf
source CMGT3AR-10Hz.env
 Short-cut—an example:
 
     source CMGT3AR-10Hz.env
     gcf2ts -F -DL -N 27 -X -gz 20220521 23
     make-zacc-likeg
     make-exact-hour -S 20220522 00
 

gcf2ts:

  USAGE: 

  (alt.1)   gcf2ts [options]  <stream-dir>/<YYYYMMDD_HH00>{e|n|z}.gcf 

  (alt.2)   gcf2ts [options] -g{z|e|n} <YYYYMMDD>_<HH>
          gcf2ts [options] -g{z|e|n} <YYYYMMDD> <HH>
 
  PURPOSE:
           Make 100-S/s ts-files from one hour of gcf-data. Streams are in our case at Onsala
           3u93z2 3u93e2 3u93n2  for 100 samples/s vertical, east and north components, respectively
           Output files will be named  (stream to STREAM upper-case):
           G<YYYY>.<DDD>/GCF.<YYYY>:<DDD>:<HH>:00:00:<STREAM>-100Hz.ts
  
           The working directory is /data/hgs/Seismo/gcf
           while the stream directories 3u93z2 etc. and time series directories Gyyyy.ddd/
           are subdirectories.

           A file (default name gcf2ts-outfiles.lst  is prepared containing the commands to
           produce extended length 100Hz-files. This is necessary in order to avoid end-effects in
           ZACC and LIKEG files. One file covering 1 h is appended to the previous one.
           Thus, processing of 27 hours (option -N 27) is required in order to produce 24 LIKEG-files.
           Use option -X to include the length-extending stage in the tasks.
 
           To convert date formats from YYYY-MM-DD to YYYY.DDD:  Utility GCF4ymd may be helpful,
           GCF4ymd -h  will show usage.

           If final files after make-zacc-likeg are going to be reprocessed in order to start at exact hours,
           issue
              gcf2ts [options] -N 27 3u93z2/<PPPPPPPP>_2300z.gcf 
           with PP  the day before YYYYMMDD for getting it right at midnight.  
 
  OPTIONS:
 
   -z  -e  -n  - Process one of these components: c
  -gz -ge -gn  - For one of these components c, using the defaults for
                                stream 3u93<c>2 and suffix 00<c>.gcf
                 the argument(s) YYYYMMDD HH construct the file name.

  -F           - If not locally available, fetch gcf-data from archive, from
                                Elder in the first place.
 
  -N <n>       - Process n hours of data. n = 25 would also retrieve the
                                first one of the next day.  Default is 1.
 
  -renew       - Renew the ts-files. Default is to keep them untouched.

  -list <file> - The file with the list of commands for further processing.
                                A pre-existing file will not be over-written unless the
                 -DL option is specified.
                                Default is gcf2ts-outfiles.lst
  -X           - Exec the commands in the list-file.
                                Default is to leave it to the user to launch
                   source <file>
   -DL                    - Allow over-writing a pre-existing list-file.

Still missing:
  1. A time shift less than 0.1 s  - must be applied at the 100Hz-stage. A 0.1 s shift that is applied at the stage of creating ZACC-10Hz.ts files. Its purpose is to mimic the delay of the SG digital voltmeters. It is specified in the CMT3AR-10Hz.env file: setenv LIKEG_SHIFT "0 1"
  2. The scripts are still is an early development stage. Alternatives for output file names and directories might be desirable. 
  3. Detection of missing-record symbols in gcf-ascii and .sac.asc is not yet coded up.

make-zacc-likeg:

 USAGE:   
           make-zacc-likeg [options] [<gcf2ts-listfile>]
 
 PURPOSE:
           Scanning the lsit-file produced by gcf2ts, makes 10Hz.ts-files from 100Hz-files
           and filters them to match SG-1Hz series, "likeg".
           The resulting files are marked -LIKEG-10Hz.ts
           e.g. G2022.136/GCF.2022:136:05:00:00.3U93Z2-LIKEG-10Hz.ts
           They are 36000 samples long and start 9 min after the full hour.
           The main purpose of producing 10Hz is liberty with respect to a delay between
           seismometer and SG. Reduction to 1Hz can be accomplished with primitive
           sub-sampling of the resulting ZACC-LIKEG-10Hz.ts files after
           setting the best-fitting shift:   setenv LIKEG_SHIFT "0 <shift>"
 
           On the fly, takes the list file produced  by gcf2ts and makes
           extended-length -100Hz files.  Default list-file = gcf2ts-outfiles.lst
 
           Makes calibrated ZACC-10Hz files on the way.  If this is the only task,
           the likeg-stage can be skipped using option -ZACC
 
 OPTIONS:
 
  -sel <selection> 
                     - the range of statements in the list-file. Default = all.
                       Valid specifications are
                        i-j   (i and j integers!)
                        i-
                        1-j
 
  -s <scl> - An adjusting scale factor, probably near 1.0
                        for best fit of SG-like data to SG-original. Default=1.0

  -ZACC    - Process up to stage  GCF-ZACC-files type ts (calibrated acceleration)
                        and exit. Default is to produce the SG-like files.
  -LIKEG   - Skip re-processing the ZACC-files.
 

make-exact-hour:

  USAGE:
           make-exact-hour [options] {<likeg-file.ts> | <ZACC-file.ts>}
 
  PURPOSE:
           Takes two contiguous ts-files at a time, crops them to start
           exactly at the full hour and comprise exactly one hour of data.
           By default, processes 24 contiguous hourly file pairs.
           The specified file is the first in this row; the file for the
           hour before must be available.
           For instance,  G2022.136/GCF.2022:136:00:00:00.3U93Z2-LIKEG-10Hz.ts
           needs        G2022.135/GCF.2022:135:23:00:00.3U93Z2-LIKEG-10Hz.ts
           If it does not exist, the required commands will be shown, and
           the script will exit.
 
           This script must be launched from working directory ~/Seism/gcf
           or one that resides directly above the G<YYYY>.<DDD> data directories.
 
           The tslist processes produce much output; it is written to a log file:
           logs/make-exact-hour.log
 
           The output files are marked with  -XH-10Hz.ts
           and with -S or -OS marked with -XH-1s.ts 
            
  OPTIONS:
   -N <n>   - The number of hours to process. Missing files' names will be shown.
    
   -S       - Make -1s.ts files and the daily addition to
             ~/TD/d/G1_garb_$ymd-1s.mc -> ~/TD/d/G1_garbs_...
   -OS      - Like -S  except, if the -XH-10Hz.ts files are already available,
             skip reprocessing.
 
   -o <output-file>
            - Unless -N 1 has been specified, the structure of the name must
             comply with the GCF-conventions, including the directory.
             Default is adding a mark -XH before the -*Hz.ts substring of the
             input file's name and stepping the date part at each cycle.
              


 EXAMPLE:
 
 #1 - decide the output sampling rate. Setup the environment100hhz.
     source CMGT3AR-10Hz.env
        OBS! setenv statements in there. This is only good for tcsh; change to  export var=val for bash
        We also have  CMGT3AR-1Hz.env  and  CMGT3AR-100Hz.env
        File names must/will be marked with the sampling rate (e.g. -10Hz.ext) accordingly.

 #2 - create 100Hz.sac and 100Hz.ts files
     gcf2ts -gz -DL -N 27 3u93z2/20220516_0000z.gcf
 # -> gcf2ts-outfiles-z.lst

 #3 - create calibrated 100Hz.ts file of 4200 s duration
     source gcf2ts-outfiles.lst
OBS! DEFUNCT. gcf2ts doesn't write commands into it, only "File nn filename-x-100Hz.ts
i.e.those 1:20 minutes files we wanted; what the next step would do is thus already done.
 # or one-by-one, make the extended files comprising 42000 seconds  
     tslist-app -x -Un420000 -I -o G2022.136/GCF.2022:136:00:00:00.3U93Z2-x-100Hz.ts \
              + G2022.136/GCF.2022:136:00:00:00.3U93Z2-100Hz.ts \
                G2022.136/GCF.2022:136:01:00:00.3U93Z2-100Hz.ts

 #4 - create 10Hz acceleration
     tslist.x G2022.136/GCF.2022:136:00:00:00.3U93Z2-x-100Hz.ts -I \
             -Eguralp.tse,CMGT3AR -Edeci.tse,R \
             -Un36000 -o G2022.136/GCF.2022:136:00:00:00.3U93Z2-ZACC-10Hz.ts |\
          tee logs/GCF.2022:136:00:00:00.3U93Z2-ZACC-10Hz.log
        OBS! In the case of -100Hz the tsfedit-stage -Edeci.tse,R must not be included. 
# or with a script
     make-zacc-likeg [-select i-j] -ZACC [<gcf2ts-outfiles-list>] 
# or  create 10Hz acceleration and 10Hz SG-like data    
          make-zacc-likeg [-select i-j] gcf2ts-outfiles.lst

#5 - make 1-second data
          make-exact-hour -N 24 -S G2022.136/GCF.2022:136:00:00:00.3U93Z2-LIKEG-10Hz.ts