$Header: /cvsroot/nco/nco/doc/ANNOUNCE,v 1.693 2014/02/20 21:46:29 zender Exp $ -*-text-*-

The netCDF Operators NCO version 4.4.2 are ready. 

http://nco.sf.net (Homepage)
http://dust.ess.uci.edu/nco (Homepage "mirror")

This release is mainly a bugfix release. Memory issues that caused
problems on LLVM/clang (usually Mac OS) and RHEL/CentOS (but not
trusty old Fedora!) have been fixed. Thank you to everyone who
reported issues. nces --nsm_grp now works as advertised with
coordinates. 

The one new feature is support for key parts of the CF cell_methods  
convention. 

Work on NCO 4.4.3 is underway, focused on stability and speed.   
There will be more netCDF4 mop-up, improved group broadcasting
algorithms, and, possibly, improved HDF4 support, and cache
manipulation for chunking.

Enjoy,
Charlie

"New stuff" in 4.4.2 summary (full details always in ChangeLog):

NEW FEATURES:

A. cell_methods: ncwa and ncra now support key parts of the CF
   cell_methods convention. Averaging foo(time,lat,lon) now
   results in ncwa creating or appending the cell_methods attribute
   "time, lat, lon: mean" to foo in the output file. 
   Pre-existing cell_methods are preserved and appended to.
   cell_methods are turned-on by default. Turn-off with --no_cll_mth.
   ncwa --op_typ=max in.nc out.nc # Output gets cell_methods
   ncwa --no_cll_mth in.nc out.nc # Turn-off cell_methods
   http://nco.sf.net/nco.html#cll_mth

BUG FIXES:

A. nces --nsm_grp now outputs only the group statisitics. In 4.4.0 and
   4.4.1 nces --nsm_grp would, inadvertently, also sometimes copy
   coordinate variables of member groups to member groups in the
   output file. This did not take much space (because only coordinates
   were copied), nor did it affect nces --nsm_fl, so we did not notice
   it for some time. Now fixed.  

B. Groups matched via regular expressions in group-subsetting now
   have their associated variables flagged in intersection mode. 
   This mumbo-jumbo means that regular expressions work better now.
   Any variable(s), group(s), or intersection of the two can be
   obtained with surgical precision with regular expressions.

C. Fix build problems exposed by LLVM on Mac OS X 10.9 Mavericks.
   The underlying memory problems could cause segfaults for other
   OSs including RHEL and CentOS. Issues reported by George White.

D. ncwa now always outputs averages of coordinates, even when
   operation on (non-coordinate) variables is other like max, min.
   This complies with longstanding intent & documentation that
   coordinate variables always return averages, never extrema or other
   statistics. ncra and nces have long-implemented this convention.

E. The -X option for auxiliary coordinates now works fully when
   latitude and longitude coordinates are nested in group hierarchies.
   The most in-scope latitude/longitude are used for each variable.  
   We said this worked in 4.4.1. This time we mean it :)
   http://nco.sf.net/nco.html#aux

F. ncap2 sometimes failed when converting netCDF3->netCDF4 files
   because it had uninitialized deflation levels. Fixed.

G. Chunking: Matt Thompson reported a problem that manifested when
   multiple --cnk_dmn options were employed. Strange chunk-sizes or
   segfaults could result. Fixed.

KNOWN ISSUES NOT YET FIXED:

   This section of ANNOUNCE reports and reminds users of the
   existence and severity of known, not yet fixed, problems. 
   These problems occur with NCO 4.4.2 built/tested with netCDF
   4.3.1 on top of HDF5 hdf5-1.8.9 with these methods: 

   cd ~/nco;./configure --enable-netcdf4  # Configure mechanism -or-
   cd ~/nco/bld;make dir;make allinone # Old Makefile mechanism

A. NOT YET FIXED (NCO problem)
   Correctly read arrays of NC_STRING with embedded delimiters in ncatted arguments

   Demonstration:
   ncatted -D 5 -O -a new_string_att,att_var,c,sng,"list","of","str,ings" ~/nco/data/in_4.nc ~/foo.nc
   ncks -m -C -v att_var ~/foo.nc

   20130724: Verified problem still exists
   TODO nco1102
   Cause: NCO parsing of ncatted arguments is not sophisticated
   enough to handle arrays of NC_STRINGS with embedded delimiters.

B. NOT YET FIXED (netCDF library problem)
   Probe hidden attributes (chunking, compression) of HDF4 files

   Demonstration:
   ncdump -h -s ~/nco/data/hdf.hdf # (dies)
   ncks -m ~/nco/data/hdf.hdf # (works by avoiding fatal calls)

   20131230: Verified problem still exists

   Cause: some libnetCDF library functions fail on HDF4 file inquiries.
   Bug report filed: netCDF #HZY-708311 ncdump/netCDF4 segfaults probing HDF4 file
   Tracking tickets NCF-272, NCF-273

C. NOT YET FIXED (would require DAP protocol change?)
   Unable to retrieve contents of variables including period '.' in name
   Periods are legal characters in netCDF variable names.
   Metadata are returned successfully, data are not.
   DAP non-transparency: Works locally, fails through DAP server.

   Demonstration:
   ncks -O -C -D 3 -v var_nm.dot -p http://thredds-test.ucar.edu/thredds/dodsC/testdods in.nc # Fails to find variable

   20130724: Verified problem still exists. 
   Stopped testing because inclusion of var_nm.dot broke all test scripts.
   NB: Hard to fix since DAP interprets '.' as structure delimiter in HTTP query string.

   Bug report filed: https://www.unidata.ucar.edu/jira/browse/NCF-47

D. NOT YET FIXED (would require DAP protocol change)
   Correctly read scalar characters over DAP.
   DAP non-transparency: Works locally, fails through DAP server.
   Problem, IMHO, is with DAP definition/protocol

   Demonstration:
   ncks -O -D 1 -H -C -m --md5_dgs -v md5_a -p http://thredds-test.ucar.edu/thredds/dodsC/testdods in.nc

   20120801: Verified problem still exists
   Bug report not filed
   Cause: DAP translates scalar characters into 64-element (this
   dimension is user-configurable, but still...), NUL-terminated
   strings so MD5 agreement fails 

"Sticky" reminders:

A. Pre-built, up-to-date Debian Sid & Ubuntu packages:
   http://nco.sf.net#debian

B. Pre-built Fedora and CentOS RPMs:
   http://nco.sf.net#rpm

C. Pre-built Windows (native) and Cygwin binaries:
   http://nco.sf.net#windows

D. Pre-built AIX binaries:
   http://nco.sf.net#aix

E. NCO support for netCDF4 features is tracked at

   http://nco.sf.net/nco.html#nco4

   NCO supports netCDF4 atomic data types, compression, chunking, and groups.

F. Reminder that NCO works on most HDF4 and HDF5 datasets, e.g., 
   HDF4: AMSR MERRA MODIS ...
   HDF5: GLAS ICESat Mabel SBUV ...
   HDF-EOS5: AURA HIRDLS OMI ...

