# info on the 6-hrly CHIRPS, originally created to be read into the NASA Land Information System, hence the strange format to work with a different precip 'reader'
# NASA LIS-LDT now has capabilities for temporal disagregation of rainfall. Hopefully, by Fall 2018 these 6-hrly .bin files will no longer be needed for FLDAS. 
# if you need these data for other purposes let us know incase pete p. at UCSB decides to stop producting them. questions? amy.l.mcnally@nasa.gov and pete peterson <geogpete@gmail.com>


;;read in the 6 hrly CHIRPS (temporally downscaled w/ CFS)
;gridded binary float, no header
;min_lat = -40S
;max_lat = 40N
;min_lon = -20 W
;max_lon = 55 E
;resolution = 0.1 degree
;nx = 751
;ny = 801
;need to confirm units are mm (kg/m2)
;byte order is BIG ENDIAN (note change order of bytes)

indir = '/discover/nobackup/projects/fame/MET_FORCING/CHIRPSv2/6-hrly/201704/'
ifile = file_search(indir+'rfe_gdas.bin.2017043018')

ingrid = fltarr(751, 801)

openr,1, ifile
readu,1, ingrid
close,1

;A = SWAP_ENDIAN(A) should also work.
byteorder,ingrid,/XDRTOF
print, min(ingrid) ;min value = -1
ingrid(where(ingrid lt 0)) = !values.f_nan
temp = image(ingrid, rgb_table=4)

%%how to read into matlab (thanks Jossy Jacob)
ncol=801;
nrow=751;
fname='rfe_gdas.bin.2017052300'
fid = fopen(fname, 'r','b'); 
%data1 = fread(fid, [nrow, ncol], 'int');
data1 = fread(fid, [nrow, ncol], 'float');
fclose(fid)
% mask the ocean
data2=data1; 
data2(data2==-1)=NaN;
figure; pcolor(data2'); shading flat; colorbar;


there was an update the the CHIRPS on Nov3.
The previous version had, in some locations, some large discrepancies between the monthly totals (which are computed initially in the CHRIPRS process...) and the daily values summed to month. email amy.l.mcnally@nasa.gov w/ questions and she will email the developer Pete Pterson.

the files from 1981-2015 July come from here:
ftp://chg-ftpout.geog.ucsb.edu/pub/org/chg/experimental/africa_6-hourly.p1_bin.01mm/

and then it switches to the files in here in Aug 2015:
ftp://chg-ftpout.geog.ucsb.edu/pub/org/chg/products/CHIRPS-2.0/africa_6-hourly/p1_bin/extra_step/

*******temporal downscaling explaination from pete Aug 21, 2014**********

The downscaling to daily, and later 6-hry has become a multi-step marvel....

(fortunately I already had this written up...)

1st, start with CHIRPS pentads, downscale to daily based on CCD%.
Wherever CCD% is zero, rainfall becomes zero. (This was done wrong last time)
Where there are missing CCD% values, downscaled rainfall becomes a missing value (-1)
This removes rainfall from the total.
 
2nd, use FCP (CHPclim * %anom CFS) to fill in missing data from step 1.
In this step used our CCD% (1981-1999) histogram to make 12 monthly FCP cutoff maps.
(last time made the cutoff maps based on Sheffield daily histograms)
For this step clip cutoff maps to 3.5 mm/day (places where cutoff > 3.5 set to 3.5)
This should prevent the introduction of FCP from disrupting the number of rain events.
This step insures data exists everywhere in Africa.

3rd, rescale daily values from step 2 to match CHIRPS pentad totals.
This sometimes produces significant residuals when all days in the pentad = zero.

4th step rescale the daily data from step 3 by month to match CHIRPS monthly data.
Then only get residuals where every day in the month = zero (still happens)

That's how we come up with our,
   daily_downscaled_by_monthly_full_rescale
which we refer to as daily.

You can see the latest IDL code used to top these off here,
         /home/source/cscd/mqb_v1.8/runem.post_mqb_61_make_final_dailies.pro

Now, to get 6-hry, start with the above daily CHIRPS and repartition into four 6-hourly time steps using the ratio of the 6-hourly to daily in the CFS reanalysis ds093/ds094 data while preserving the daily total.

Viola!

Happy to add any other clarification to this.

Take care,
- pete