=================
= ESPRESSO NEWS =
=================

ESPResSo 4.2.2
==============

This release provides a number of corrections for the ESPResSo 4.2 line.
We recommend that this release be used for all production simulations.
The interface has not been changed between ESPResSo 4.2.1 and 4.2.2.
However, some bugs were discovered which can affect simulation results.
Please find the list of changes below. The numbers in brackets refer to
ticket numbers on https://github.com/espressomd/espresso

Improved documentation
----------------------

* Installation instructions now mention the FFTW3 MPI dependency
  of long-range solvers and provide recommended version numbers
  for Jupyter Notebook dependencies (#4790).

* Installation instructions now mention Python environments (#4922).

* Observables not properly document return values, array shapes,
  and use a more consistent mathematical notation (#4898).

Bug fixes
---------

* Fatal runtime errors due to MPI global variables lifetime
  were addressed (#4858). Older ESPResSo releases built with
  Boost 1.84 or later might randomly crash when exiting
  the Python interpreter.

* Virtual sites no longer contribute to the kinetic energy
  of the system (#4839). The regression was introduced
  in April 2021 and affected the 4.2 branch of ESPResSo.

* Inertialess tracers are now integrated along the z-axis (#4714).
  The regression was introduced in February 2022 and affected
  the 4.2 branch of ESPResSo.

* Inertialess tracers now throw an exception when attempting to use
  LB GPU with 2 or more MPI ranks (#4714). Before, tracers on non-root
  MPI ranks would be silently ignored by the CUDA kernels,
  and would have a constant velocity, either 0 if the particle never
  visited the fluid domain on the root rank, or the last known velocity
  if the particle was once on the root rank. This bug affected all
  ESPResSo versions.

* Particles close to the faces of the simulation box are now properly
  coupled to the LB fluid (#4827). Due to numerical instability, it was
  previously possible for particles to be outside the box simulation by
  a tiny amount and skip LB particle coupling. The probability of this
  bug occurring was low, but could be enhanced in simulations that
  purposefully placed particle near the faces of the simulation box:
  polymers sheared by Lees-Edwards boundary conditions, raspberry
  particles (colloids, bacteria, etc.) when crossing a periodic
  boundary, or cell membranes placed close to a periodic boundary.

* Resizing the box now throws a runtime error if there are constraints
  present (#4778), since constraint preconditions might no longer be
  fulfilled. For example, a wall constraint might end up outside the
  box boundaries when the box shrinks.

* Resizing the box via `system.box_l = new_box_l` now throws
  a runtime error if there are particles present, because particle
  position folding cannot be guaranteed to be correct (#4901);
  use `system.change_volume_and_rescale_particles()` instead,
  which properly rescales particle positions.

* The velocity Verlet NpT propagator doesn't apply friction and noise
  on angular velocities. ESPResSo now throws an error when NpT
  encounters a rotating particle (#4843). This bug affected all
  ESPResSo versions.

* The Brownian thermostat can no longer be configured with
  `act_on_virtual=True` due to an unresolved bug (#4295)
  that will be addressed in the next minor release.

* Restrictions on the number of MPI ranks have been lifted from the
  checkpointing mechanism (#4724). It is now possible to use
  checkpointing again in MPI-parallel simulations when the system
  contains LB boundaries or `Union` shape-based constraints.
  These restrictions had been introduced in 4.2.0 for technical
  reasons that have since been resolved.

* When passing an invalid value to a function that expects an input
  parameter of type `list` of size 3, an exception is now raised (#4911).
  Previously, some functions would print an error message and continue
  their execution with uninitialized data.

* The per-`type` and per-`mol_id` contributions from
  `system.analysis.energy()`, `system.analysis.pressure()`
  and `system.analysis.pressure_tensor()` now return the correct
  values (#4788). Older version of ESPResSo were confusing the
  particle `mol_id` with the particle `type`. The total pressure
  was unreliable when `mol_id` properties were set to non-zero values.

* The OpenGL visualizer now extracts the correct non-bonded potential
  parameter `sigma` when feature `WCA` is compiled in but `LENNARD_JONES`
  isn't (#4720). The regression was introduced in 4.2.1.

* Method `OifCell.elastic_forces()` no longer throws a `TypeError` (#4813).

* Benchmark scripts were adjusted to support large particle numbers (#4753).

Under the hood changes
----------------------

* Several Clang 16 and GCC 13 compiler diagnostics have been addressed
  (#4715).

* A non-critical GCC C++20 deprecation warning in Cython-generated code
  was disabled (#4725).

* Several deprecation warnings emitted by CMake 3.27 have been silenced
  (#4792).

* Add support for setuptools version 67.3.0 and above (#4709).

* Add support for Python 3.12 in testsuites run by CTest (#4852).

* Python requirements have been updated (#4924).

* CI pipeline URLs have been fixed (#4736).

ESPResSo 4.2.1
==============

This release provides a number of corrections for the ESPResSo 4.2 line.
We recommend that this release be used for all production simulations.
The interface has not been changed between ESPResSo 4.2.0 and 4.2.1.
However, some bugs were discovered which can affect simulation results.
Please find the list of changes below. The numbers in brackets refer to
ticket numbers on https://github.com/espressomd/espresso

Added functionality
-------------------

* P3M and DipolarP3M can now be used with the hybrid decomposition cell
  system with 1 MPI rank (#4678).

* Lattice-Boltzmann can now be used with the N-square and hybrid
  decomposition cell systems with 2 or more MPI ranks (#4676).

Changed requirements
--------------------

* The nbconvert version requirement was bumped to 6.5.1 to patch an XSS
  vulnerability (#4658).

Improved documentation
----------------------

* The user guide now documents how to improve the reproducibility of
  simulations that have checkpointing enabled (#4677).

* The user guide now reflects that the lattice-Boltzmann profile observables
  can be used in parallel (#4583).

* The active matter tutorial now uses an adequate engine dipole for the
  swimmer particle (#4585).

* The error analysis tutorials have been improved (#4597).

* The tutorials can now be used in VS Code Jupyter (both the desktop and web
  versions) and the mathematical formula are now correctly displayed (#4531).

* All ESPResSo-specific CMake options are now documented in the installation
  chapter of the user guide (#4608).

* Python package installation instructions no longer feature package version
  numbers; instead, `requirements.txt` is used as a constraint file (#4638).

* MMM1D algorithms now properly document their parameter names (#4677).

* Reaction methods now cite the relevant literature (#4681).

* Caveats for chain analysis methods are now documented (#4698).

* Minor formatting issues in Sphinx and typos in Python docstrings were
  addressed (#4608).

Interface changes
-----------------

* A new boolean property `System.virtual_sites.override_cutoff_check` was
  introduced to allow disabling the cutoff range checks from virtual sites
  (#4623).

Removed functionality
---------------------

* The unused and untested `Analysis.v_kappa()` method was removed (#4534).

Improved testing
----------------

* Improve unit testing of core functionality: P3M, MMM1D, OIF, virtual sites,
  script interface factory (#4631).

Bug fixes
---------

* The checkpointing mechanism now properly restores the particle quaternion
  and all derived quantities (#4637). Release 4.2.0 introduced a regression
  that caused checkpoint files to overwrite the particle quaternion/director
  by a unit vector pointing along the z direction, when the `DIPOLES` feature
  was part of the myconfig file. This lead to incorrect trajectories when
  reloading a simulation from a checkpoint file, if the particle director
  played a role in the simulation (ex: relative virtual sites, Gay-Berne
  potential, anisotropic particles, active particles, etc.). In addition,
  the angular velocity in body frame was restored with the wrong orientation.
  Since the default myconfig file contains `DIPOLES`, most ESPResSo users
  were affected.

* The checkpointing mechanism now properly restores LB boundaries (#4649).
  Release 4.2.0 introduced a regression where reloading LB populations
  would accidentally reset LB boundary flags.

* The checkpointing mechanism now restores P3M and DipolarP3M solvers without
  triggering a re-tune (#4677). In previous releases, the checkpointing code
  would automatically re-tune these algorithms during a reload, causing tiny
  deviations in the forces that were problematic for trajectory reproducibility.

* Brownian dynamics now integrates the rotational dynamics of rotatable
  particles whose position is fixed in 3D space (#4548).

* Langevin dynamics now properly integrates particles with anisotropic
  friction (#4683, #4690).

* A regression that caused virtual sites to incorrectly count their image box
  when crossing a periodic boundary has been fixed (#4564, #4707).

* Particles can no longer be created or updated with a negative mass or a
  null mass (#4679).

* Particles created without a user-specified type can now participate in
  reactions (#4589).

* When a Monte Carlo displacement move is rejected, the original particle
  velocity is now restored (#4589).

* Reaction methods now raise an exception when accidentally calling
  `method.reaction(steps=20)` instead of `method.reaction(reaction_steps=20)`
  (#4666). Since 4.2.0 the `steps` argument was ignored, in which case the
  default value `reaction_steps=1` would used by the core. Note that in the
  next minor release of ESPResSo, the `reaction_steps` argument will be
  renamed to `steps`.

* Reaction methods now rebuild the list of free particle ids every time
  `WidomInsertion::calculate_particle_insertion_potential_energy()` and
  `ReactionAlgorithm::do_reaction()` are called (#4609). This was needed to
  allow multiple concurrent reactions, as well as avoiding subtle bugs when
  both the user and a reaction method tried to create a new particle with
  an id that used to belong to a deleted particle.

* When all particles are cleared, the reaction methods type map is now also
  cleared (#4645). In the past, it was possible to attempt a reaction on
  particles that had just been cleared from the system, which would raise
  an exception. This bug affected all ESPResSo releases since 4.0.

* The `System.part.pairs()` method now returns the correct particle pairs
  when particle ids aren't both contiguous and starting from 0 (#4628).
  The regression was introduced in release 4.2.0.

* The auto-exclusions feature no longer adds spurious exclusions to particle
  ids in the range [1, distance] (#4654). This bug would potentially break
  the physics of the system and potentially raise an exception in a system
  with non-contiguous particle ids. This regression was introduced in release
  2.2.0b.

* The structure factor analysis code no longer double-counts particles when
  the same particle type is provided twice (#4534).

* The minimal distance distribution analysis code no longer has an arbitrary
  cutoff distance when the simulation box is aperiodic (open boundaries);
  this would cause spurious artifacts to appear in the histogram at
  `r = np.sum(system.box_l)` when particles were further apart than this
  arbitrary distance (#4534).

* The cluster analysis functions are now disabled for systems with
  Lees-Edwards periodic boundaries, since the cluster analysis position
  wrapping code doesn't properly handle the shear offset (#4698).

* The chain analysis methods now raise an error when the number of chains or
  beads per chain is invalid (#4708).

* The observable tests now longer rely on deprecated numpy options that were
  removed in numpy 1.24 (#4635).

* The visualizer `*_arrows_type_materials` options now have an effect on
  arrow materials (#4686).

* The visualizer exception handling mechanism has been made less brittle
  (#4686).

* The visualizer no longer raises exception when the optional dependency
  `freeglut` isn't installed (#4691).

* The visualizer can randomly freeze when using collision detection or bond
  breakage; a temporary workaround has been introduced that fixes the issue
  for simulations that use only 1 MPI rank (#4686).

* The `__dir__()` method of script interface objects no longer raises an
  exception (#4674).

* Compilation and testsuite issues involving missing or incorrect feature
  guards were addressed (#4562, #4648).

* The build system no longer silently ignores invalid external feature
  definitions in `myconfig.hpp` and CMake files (#4608). This issue would
  only affect feature developers, as well as users of very old compilers,
  and would lead to ESPResSo builds missing features.

Under the hood changes
----------------------

* The Clang 14 and AppleClang 14 compilers are now supported (#4601).

* Several Clang 14 compiler diagnostics have been addressed (#4606).

* Boost 1.81 and later versions are now supported (#4655).

* Compiler errors on non-x86 architectures were addressed (#4538).

* Test tolerances were adjusted for non-x86 architectures (#4708).

* The pypresso script now prints a warning when running with MCA binding
  policy "numa" on NUMA architectures that are not supported in singleton
  mode by Open MPI 4.x (#4607).

* The config file generator has been rewritten to properly handle external
  features and compiler errors (#4608).

* Security hardening for GitHub Workflows (#4577, #4638) and Codecov (#4600).

* Deployment of the user guide to GitHub Pages now relies on cloud providers
  to fetch JavaScript dependencies (#4656).

ESPResSo 4.2.0
==============

This is a feature release, i.e., new functionality is added to ESPResSo.
New thermostats, cell systems and boundary conditions have been introduced
to simulate systems with Stokesian Dynamics, Brownian Dynamics, strongly
inhomogeneous particle sizes or translation-invariant shear flow. The
interface underwent (non-silent) changes, therefore scripts will have to
be slightly adapted. Most notably, particle access by id and particle
slices have a new syntax, and electrostatic/magnetostatic layer correction
and reaction methods have a different setup. All errors are also now
emitted as Python exceptions and are recoverable with minimal effort.

An additional focus of this release is the simplification of both the C++ core
and the Python script interface to facilitate future extensions of ESPResSo.
The testing of ESPResSo's functionality has been extended considerably.

We recommend that this release be used for all production simulations.
No further bug fix releases will be provided for the 4.1 line, and not
all fixes are present in ESPResSo 4.1.4.

Please carefully read the detailed list of changes below before using this
release. Issues can be reported at https://github.com/espressomd/espresso.

Added functionality
-------------------

* `P3MGPU` now supports energy and pressure calculation via the CPU
  kernels (#4506).

* `ELC` now works with `P3MGPU` (#4506).

* The LB grid now supports slicing operations (#4195) and LB slices are
  equality comparable (#4268).

* Lees-Edwards boundary conditions can be used for particle-based
  simulations (#4457). Lattice-Boltzmann support will be added in
  the 4.3.0 release.

* The non-bonded energy of a single particle can be calculated (#4401).

* The list of close neighbors of a single particle can be extracted (#4401).

* Brownian Dynamics simulations can be carried out with the newly added
  Brownian integrator and Brownian thermostat (#1842).

* Stokesian Dynamics simulations can be carried out with the newly added
  Stokesian integrator and Stokesian thermostat (#3790, #3987).

* Bonded interactions can now be automatically broken when the bond length
  exceeds a critical value (#4456). This feature can be combined with
  collision detection to model reversible bonds (#4464).

* A new cell system `HybridDecomposition` was introduced to speed up
  simulations with inhomogeneous particle interaction ranges (#4373).

* Shapes can be merged into meta-shapes (#3493, #3538).

* The `HollowConicalFrustum` can now be sliced open, made thick and rotated
  to model quarter pipes in any orientation (#4179). The main application is
  in the construction of complex microchannel geometries via `LBBoundaries`.

* A parametric weight function was added to the DPD interaction (#3570).

* H5MD output files now support a unit system (#3751).

* H5MD output files now support custom specifications to control which
  particle and box properties to write to disk (#4480).

* The `H5md` class is now checkpointable and usable in an interactive
  Python session (#3751).

* MDAnalysis integration now provides bond information (#3801).

Changed requirements
--------------------

* The minimal version of all dependencies was increased (#3375, #3687,
  #3878, #3984, #3994, #4115, #4312, #4337, #4489): Python 3.8, Cython
  0.29.14, CMake 3.16, Boost 1.69, Sphinx 2.3.0, and Python packages
  versions are pinned on versions available in the Ubuntu 20.04 repository.

* CMake no longer emits a warning about the deprecated `distutils` Python
  package, which is also no longer a requirement (#4433).

* CUDA 11 support was added (#3870).

* CUDA 8 and CUDA 9 support was removed (#3984).

* AMD GPU support via ROCm (HCC and HIP-Clang compilers) was removed (#3966).

* library `libcuda` is no longer a dependency in CUDA builds (#4095).

* Installation instructions for ESPResSo on Microsoft Windows via WSL are
  now available (#4348).

* LaTeX is no longer a requirement for building the Sphinx documentation
  and running the tutorials (#3256, #3395).

Feature configuration at compile time
-------------------------------------

* GPU support is now opt-in (#3582). Pass the CMake flags `-DWITH_CUDA=ON`
  to compile CUDA code and optionally `-DWITH_CUDA_COMPILER=<compiler>` to
  select the CUDA compiler: NVCC (default), Clang.

* Optional features HDF5, ScaFaCoS and Stokesian Dynamics are now opt-in
  (#3735, #4112). If they are requested with their `-DWITH_<FEATURE>=ON`
  flag and their dependencies are not found, CMake will raise an error.
  In the older 4.1 build system, CMake would silently ignore these features
  when their dependencies were not found, causing confusion as to what was
  exactly compiled.

* Experimental support for fast-math mode was added (#4318). Some features
  might break depending on the compiler used to build ESPResSo. Please
  quantify the numerical stability of your simulations before enabling
  fast-math mode in production.

* The `LANGEVIN_PER_PARTICLE` feature was renamed to
  `THERMOSTAT_PER_PARTICLE` (#4057).

* The magnetostatic extension `DLC` now depends on feature `DIPOLES`
  instead of `DP3M`, since FFTW is not a dependency of `DLC` (#4238).

* The electrostatic extension `ICC` now depends on feature `ELECTROSTATICS`
  instead of `P3M`, since FFTW is not a dependency of `ICC` (#4238).

* The `MMM1D_MACHINE_PREC` feature was added to enable Chebychev series for
  MMM1D on CPU without the need to define the (now removed)
  `BESSEL_MACHINE_PREC` macro (#4311).

* The `EXPERIMENTAL_FEATURES` feature was removed (#4482).

Improved documentation
----------------------

* Tutorials have been renamed and organized by difficulty level (#3993).

* Tutorials Lennard-Jones, electrostatics, lattice-Boltzmann, raspberry
  electrophoresis and constant-pH have been improved (#3408, #3881, #3914,
  #3893, #4302, #4262).

* Tutorial lattice-Boltzmann was split into three tutorials: polymer
  diffusion, Brownian motion and Poiseuille flow (#4052, #4329).

* The active matter tutorial was rewritten into a Jupyter notebook
  (#3395, #4304).

* An error analysis tutorial was added (#4174).

* Tutorials now use the `exercise2` plugin to hide solutions (#3872);
  since this plugin only exists for the classic Jupyter Notebook,
  a conversion script is provided for JupyterLab users (#4522).

* The user guide now includes a button on Python code samples to hide
  terminal output and Python prompt symbols (`>>>` and `...`), so as to
  facilitate copy-pasting examples directly in the terminal (#4386).

* The user guide now uses a responsive theme for mobile/tablet users (#4504).

* The user guide chapter on thermostats was moved to the chapter on
  integrators, since they are tightly coupled (#4080).

* Mentions to non-existent functions were removed from the user guide (#4482).

* Scientific publications referenced in comment lines in the core have been
  converted to BibTeX citations and integrated into Doxygen blocks to make
  them accessible in the Doxygen HTML documentation (#3304).

* The Reaction Field electrostatic method is now documented (#4218).

* The H5MD feature is now better documented (#4480).

* A Gibbs ensemble sample was added to simulate the exchange of particles
  between two ESPResSo systems via the `multiprocessing` Python module (#4243).

* A reaction ensemble sample was added to simulate a complex chemical
  reaction involving 5 chemical species (#3778).

Interface changes
-----------------

* The `system.set_random_state_PRNG()` method was removed (#3482).

* The []-operator on `system.part` was removed (#4402). Use
  `system.part.by_id(1)` to fetch a specific particle,
  `system.part.by_id([1, 3])` to fetch a group of particles, or
  `system.part.all()` to fetch all particles. This change was necessary
  to resolve the ambiguity of particle slices containing non-contiguous
  particle ids.

* The domain decomposition cell system was renamed to regular decomposition
  (#4442). The `system.cell_system.set_domain_decomposition()` function
  was renamed to `system.cell_system.set_regular_decomposition()`.

* Bonds are now immutable (#4350). Bonds added to the list of bonds can
  no longer be overwritten by a bond of a different type, as it could
  lead to undefined behavior when the number of bonded partners was
  higher in the overwriting bond. Bonds can now be removed from the
  list of bonds, after they have been removed from particles.

* Observable parameters are now immutable (#4206, #4211).

* The `Electrokinetics` actor parameters are now immutable (#4327).

* The `LBFluid`, `LBFluidGPU`, `Electrokinetics` and `Species` methods
  `print_*()` have been renamed to `write_*()` (#4049).

* The `ELC` actor is no longer an electrostatics extension (#4125, #4506).
  The `ELC` actor now takes a `P3M` or a `P3MGPU` actor as argument and
  modifies it. Only the `ELC` actor needs to be added to the system list
  of actors. The `ELC` actor can now be removed from the list of actors.

* The `DLC` actor is no longer a magnetostatic extension (#4506). The
  `DLC` actor now takes a magnetostatic actor as argument and modifies it.
  Only the `DLC` actor needs to be added to the system list of actors.

* The NpT thermostat now uses the Philox random number generator and
  requires a random seed on first instantiation (#3444).

* The analysis module `energy()` function now returns the lower triangle
  of the non-bonded interaction matrix, to be consistent with `pressure()`
  and `stress_tensor()` (#3712).

* The analysis module `energy()`, `pressure()` and `pressure_tensor()`
  functions now return only two slots for electrostatics and magnetostatics:
  short-range contribution in the first slot and long-range contribution +
  layer correction in the second slot (#3770).

* The analysis module `pressure()` and `pressure_tensor()` functions no
  longer provide a velocity-compensation flag to compute the pressure at
  half the time step in NpT simulations (#3756).

* The `espressomd.reaction_ensemble` module was renamed to
  `espressomd.reaction_methods` (#4482).

* The argument `temperature` in reaction methods was renamed to `kT`
  for clarity (#4305).

* All reaction methods now take keyword arguments instead of positional
  arguments (#4451).

* The constant pH method now implements a symmetric proposal probability
  instead of an asymmetric proposal probability (#4207).

* The reaction method parameter `exclusion_radius` was renamed to
  `exclusion_range` (#4469).

* Reaction method now take an optional parameter `exclusion_radius_per_type`
  for better control of the exclusion radius in simulations involving
  different particle sizes (#4469).

* The `WidomInsertion.measure_excess_chemical_potential()` method was
  replaced by `WidomInsertion.calculate_particle_insertion_potential_energy()`,
  which returns the instantaneous value of the excess chemical potential
  instead of the accumulated mean and standard error (#4374). The mean value
  and standard error of the excess chemical potential must be now be
  calculated by `WidomInsertion.calculate_excess_chemical_potential()`.

* Reaction method constraints can now be safely changed from cylindrical
  to slab and can be removed (#4310). They will also raise an error when
  created with invalid parameters.

* The `mpiio` global variable was removed (#4455). The MPI-IO feature is now
  used by creating a local instance with `mpiio = espressomd.io.mpiio.Mpiio()`.

* The MPI-IO feature now raises an exception from which the user can
  recover when the simulation script runs on 1 MPI rank, instead of
  an unrecoverable fatal error (#4455). This change is meant to help
  debugging read/write errors in simulation scripts. On 2 or more MPI
  ranks, exceptions still lead to a fatal error.

* The `H5md` class takes new arguments during instantiation (#3785).

* The `system.cell_system.get_pairs_()` method was renamed to
  `system.cell_system.get_pairs()` and now supports filtering particle
  pairs by type (#4035).

* The polymer setup code was moved from the core to Python (#3477). The
  `espressomd.polymer.positions()` function was renamed to
  `espressomd.polymer.linear_polymer_positions()` and the
  `espressomd.diamond.Diamond` class was converted to function
  `espressomd.polymer.setup_diamond_polymer()`. For diamond polymers,
  counter-ions must now be added manually by the user.

* The particle director can be set from the Python interface (#4053).

* The particle method `vs_auto_relate_to()` can take a particle as
  argument instead of a particle id (#4058).

* Particles can be serialized and deserialized in the Python interface with
  `particle_dict = p.to_dict()` and `system.part.add(particle_dict)` (#4060).

* It is no longer necessary to manually reshape the output of `Observable`
  objects. The `Observable` classes now return multi-dimensional numpy
  arrays and the documentation clearly indicates the shape and size of
  the calculated data (#3560). The same applies to accumulators and
  time series (#3578).

* `Accumulator` and `Correlator` classes now return the data in suitably
  shaped multi-dimensional numpy arrays; dependent properties such as lag
  times and sample sizes need to be obtained separately by calling methods
  `lag_times()` resp. `sample_sizes()` (#3848).

* `Profile` observables provide methods `bin_centers()` and `bin_edges()`
  to facilitate plotting (#3608).

* The observable `ComForce` was renamed to `TotalForce`, so as to better
  reflect what it actually calculates (#3471).

* The RDF feature was removed from the analysis module and converted to
  an Observable class (#3706). Time averages can be obtained using the
  `TimeSeries` accumulator.

* All occurrences of "Stress Tensor" in the analysis module, LB module and
  EK module were renamed to "Pressure Tensor" to better reflect what is
  actually calculated (#3723, #4228).

* The `MeanVarianceCalculator` interface changed (#3996).

* Observables now check their input parameters (#4211, #4255) and raise
  an exception when an invalid value is detected (e.g. `min_x > max_x`
  in profile-based observables).

* Cylindrical observable classes have an extra `transform_params` argument
  to change the orientation of the cylindrical coordinates systems and
  control the origin of the phi angle (#4152).

* Incompatible thermostat/integrator combinations raise an exception (#3880).

* The `system.cuda_init_handle.list_devices()` feature is now a function,
  and the `system.cuda_init_handle.list_devices_properties()` function
  disabled in 4.0.0 was restored (#4095).

* CUDA errors now halt the flow of the program by throwing a Python
  exception with a clear error message (#4095).

* Parameter `particle_scales` of coupling-based fields `PotentialField`
  and `ForceField` now takes a `dict` object instead of a list of tuples
  (#4121).

* The `System` class no longer has a `globals` member (#4276). Global
  variables are still accessible from other members of the `System` class.

* Methods from the cluster analysis class `Cluster` no longer returns
  `False` when a string passed to `call_method()` doesn't match the name
  of a core method; instead `None` is returned (#4234).

* Methods from the cluster analysis class `ClusterStructure`, integrator
  classes and interaction classes no longer returns `True` when the
  corresponding core method doesn't return a value; instead `None`
  is returned (#4234, #4516).

* Several parameters of the `ICC` class are no longer optional: `epsilons`,
  `normals`, `areas`, `sigmas` (#4162).

* The electrostatic actors charge neutrality check tolerance can be modified
  via `actor.charge_neutrality_tolerance`; this is mostly relevant to actors
  coupled to `ICC`, whose induced charges can have values spanning several
  orders of magnitude (#4506).

* Electrostatic and magnetostatic methods that support tuning now have a
  `timings` argument to control the number of integration loops to run
  during tuning (#4276).

* The Drude helpers (global variables and free functions) have been
  gathered into a checkpointable class `DrudeHelpers`, which now relies
  on particle handles instead of particle ids (#4353).

* ScaFaCoS integration now supports activating an electrostatics ScaFaCoS
  actor at the same time as a magnetostatics ScaFaCoS actor (#4036).

* The list of actors can no longer end up in an invalid state: updating
  an electrostatic or magnetostatic actor with an invalid parameter now
  automatically restores the original parameter; inserting an invalid
  electrostatic or magnetostatic actor in the list of actors now
  automatically removes the actor from the list of active actors,
  even when the exception happens during tuning (#4506).

* The `LBBoundaries` slip velocity check was lowered to Mach 0.35, or 0.2 in
  LB units (#4376).

* The OpenGL visualizer allows changing the radius of LB velocity arrows,
  documents all LB-related keyword arguments, and no longer suffers from
  a division-by-zero error that used to trigger a runtime warning for
  fluid inside boundaries (#4376).

* The `Electrokinetics` class got an optional `ext_force_density` parameter
  for consistency with other LB implementations (#4203).

* MDAnalysis integration now checks if the MDAnalysis package version is
  supported (#4386).

Removed functionality
---------------------

* The `ENGINE` shear torque calculation feature deprecated in 4.1.1 was
  removed (#3277).

* The `MEMBRANE_COLLISION` and `OifOutDirection` features were removed
  (#3418).

* The `AFFINITY` feature was removed (#3225).

* The unused and untested `UMBRELLA` feature was removed (#4032, #4079).

* The unused and untested `VIRTUAL_SITES_COM` feature was removed (#3250).

* The unused and untested `EK_DOUBLE_PREC` feature was removed (#4192).

* The unused and untested MD metadynamics feature was removed (#3563).

* The unused and untested `Stomatocyte` shape was removed (#3730).

* The `PdbParser` feature deprecated in 4.1.1 was removed (#3257).

* The incorrectly implemented and untested `HarmonicDumbbellBond`
  interaction was removed (#3974, #4079).

* The layered cell system was removed (#3512).

* The unused Wang-Landau reaction ensemble algorithm was removed (#4288).

* The reaction ensemble tutorial deprecated in 4.1.1 was removed (#3256).

* The per-particle temperature feature was removed (#4057).

* The `Current` observable was removed in favor of the `FluxDensityProfile`
  observable (#3973).

* The incorrectly implemented analysis function `cylindrical_average` was
  removed in favor of the `CylindricalDensityProfile` observable (#3470).

* The `minimize_energy` member of the `System` class was removed (#3390,
  #3891). The steepest descent algorithm is now a regular integrator that
  is set up via the `system.integrator.set_steepest_descent()` method.

* The MMM2D electrostatics feature was removed (#3340). Electrostatics in
  slab geometries can still be achieved by ELC, with significantly better
  performance.

* The dipolar direct sum with replica method is now disabled on periodic
  systems with zero replica, as it does not apply minimum image convention
  (#4061).

* The analysis module `min_dist2()` function was removed and the `dist_to()`
  function was merged into `system.distance_vec()` (#3586).

* The analysis module `nbhood()` function slab search mode was removed
  (#4516) since it was incorrect (all ESPResSo versions were affected).

* The number of cells for the link cell algorithm can no longer be
  constrained to a range of values (#3701).

* The global Mersenne Twister RNG was removed (#3482). All thermostats
  are now Philox-based. Local Mersenne Twister RNGs are still used in
  the linear polymer position generator (now with proper warmup) and
  in the `ReactionAlgorithm` class.

* It is no longer possible to checkpoint an ESPResSo system instance that
  contains `Union` shape-based constraints when the simulation is running
  with 2 or more MPI ranks. An error will be raised (#4287, #4510).

* It is no longer possible to checkpoint an `Electrokinetics` instance (#4327).

* The unmaintained `lj-demo.py` sample was removed (#4482).

* The unmaintained `mayaviLive` visualizer was removed (#4515).

Improved testing
----------------

* The C++ core of ESPResSo is covered by unit tests and integration tests
  at 98% coverage (#4426, #4479, #4489).

* The structure factor code is tested against simple lattices (#4205).

* The MMM1D GPU code is tested (#4064).

* The reaction method core classes are unit tested (#4164).

Performance enhancements
------------------------

* The `Particle` memory footprint was reduced and the MPI serialization
  code was improved (#4414). The structure size is now 584 bytes instead
  of 640 bytes on maxset configuration (10% reduction). All substructures
  in `Particle` are bitwise serializable and dynamic vectors are compact
  vectors. The performance gain is about 9% for a LJ liquid on both maxset
  and empty configurations, for both 1 000 and 10 000 particles per core.

* Particle creation happens in constant time instead instead of monotonically
  increasing with the number of particles already in the system (#4493).

* When only one MPI rank is used, the maximum cutoff of bonded interactions
  is ignored when initializing the cell properties, since the bond partners
  are always accessible on the same node, regardless of the cell size; if
  the system also doesn't have short-range interactions, the short-range
  loop is skipped (#4452).

* The `ReactionAlgorithm::do_reaction()` function used by reaction methods
  now caches the potential energy of the system and only updates it after
  a successful reaction trial move (#4374).

* Reaction methods can delegate the particle neighbor search to the cell
  system when evaluating the exclusion range of inserted particles (#4401).
  This leads to better performance only on 2 or more MPI ranks.

Bug fixes
---------

* The `transform_vector_cartesian_to_cylinder()` now calculates the correct
  phi angle (#4094). The bug was present since ESPResSo 4.0.0 and affected
  observables `CylindricalVelocityProfile`, `CylindricalFluxDensityProfile`,
  `CylindricalLBVelocityProfile`, `CylindricalLBVelocityProfileAtParticlePositions`,
  `CylindricalLBFluxDensityProfileAtParticlePositions`.

* Several memory leaks were fixed in the `TabulatedBond` interactions
  (#3961), electrostatics and magnetostatics tuning functions (#4069),
  lattice-Boltzmann code (#4108) and Barnes-Hut code (#4404).

* The `system.actors.clear()` method was broken and would only remove half
  of the actors since 4.0.0. This is now fixed (#4037).

* The `ClusterStructure` feature did not properly handle box periodicity
  since 4.0.0 and would under rare circumstances calculate a center of mass
  to be outside a fully periodic simulation, and would incorrectly fold
  coordinates in aperiodic systems. This is now fixed (#4363).

* Adding a LB thermostat when any other thermostat was already active would
  silently fail since 4.0.0. This is now fixed (#4116).

* Setting the NpT or steepest descent integrators with incorrect parameters
  no longer leaves the system in an undefined state (#4026).

* The OpenGL visualizer had a tendency to slow down after pausing and
  resuming the simulation, or freezing when using the steepest descent
  integrator. This was due to a race condition between two threads that
  has been fixed (#4040).

* The OpenGL visualizer no longer raises an exception when activating the
  `LB_draw_boundaries` option without any other `LB_draw_*` option (#4479).

* The OpenGL visualizer now correctly updates bond information when the
  collision detection and bond breakage features are used (#4502).

* It is no longer possible to accidentally set a non-cubic NpT integrator
  with P3M (#4165).

* The NpT integrator used to work with `P3MGPU` even though it didn't
  implement long-range energy calculation and therefore couldn't contribute
  to the virial; now the long-range energy is calculated and added to the
  virial (#4026, #4506).

* Illegal LB node access is now properly caught by exceptions (#3978).

* EK node access no longer accepts floating-point values for node indices
  (#4228), and always requires exactly three integers (#4482).

* Accessing the flux property of EK species no longer throws an error
  (#4106).

* Accessing the `boundary` field of LB nodes from a `LBFluid` actor when
  `LB_BOUNDARIES` is not compiled in now returns 0 instead of a random
  integer (#4479).

* The LB grid in the GPU implementation is now automatically resized when
  the simulation box size changes (#4191).

* The LB code now throws an error when adding a LB boundary to the
  `LBFluid` actor when `LB_BOUNDARIES` is not compiled in, or to the
  `LBFluidGPU` actor when `LB_BOUNDARIES_GPU` is not compiled in (#4472).

* The lattice-Boltzmann Python interface no longer ignores runtime errors,
  nor converts them to cryptic system errors (#4355).

* The script interface no longer silently ignores runtime errors when
  converting Python objects to C++ data types (#4387, #4492).

* The system now throws an error when a non-bonded interaction cutoff is
  too large for the local box size in MPI-parallel simulations; in older
  releases the error was queued and deferred to the integration loop (#4479).

* The system now throws an error when a virtual site tracks a real particle
  too far away for the local box size in MPI-parallel simulations; in older
  releases the error was queued and deferred to the integration loop (#4479).

* It is no longer possible for a virtual site to track itself (#4479).

* It is no longer possible for a particle to exclude itself (#4493).

* It is no longer possible to accidentally add the same bond twice on the
  same particles (#4058).

* Fatal errors triggered by stale references in virtual sites, invalid
  particle ids and null quaternions have become runtime exceptions (#4479).

* Virtual sites now contribute to the rotational kinetic energy of the
  system (#4198).

* Particle creation no longer raises `numpy.VisibleDeprecationWarning`
  (#4493).

* The EK feature now generates VTK files that are compliant with the VTK 2.0
  standard (#4106).

* The `ELC` and `DLC` actors now throw an error when a particle enters the
  gap region (#4051).

* The `ELC` actor is now updated when the box size changes in the
  z-direction (#4231).

* The `DLC` actor now raises an exception when tuning fails instead of
  causing a fatal error (#4238).

* The `MMM1D` actor now raises an exception for incorrect periodicity or
  cell system instead of causing a segfault (#4064).

* The `DipolarP3M` checkpointing mechanism was fixed (#3879).

* The `DipolarP3M` method now recalculates the energy correction when the
  box length changes (#4506).

* P3M-based actors now sanitize the user-provided `alpha` and `accuracy`
  parameters and no longer allow constraining the `alpha` parameter during
  tuning (`alpha` was always derived from the other parameters at the end
  of tuning) (#4118).

* A buffer overflow in the `DipolarP3M` tuning function lead to random
  failures during tuning, this is now fixed (#3879).

* A buffer overflow in the LB code could lead to incorrect results in grids
  of size 9x9x9 or larger with open boundaries, this is now fixed (#4078).

* Providing incorrect parameters to the ScaFaCoS actors no longer cause
  ESPResSo to crash (#4068).

* FENE, harmonic and quartic bonds now throw an error when the bond length
  is zero and the equilibrium bond length is non-zero, since the direction
  of the force cannot be determined (#4471).

* Immutable parameters `default_scale`, `particle_scales` and `gamma` of
  coupling-based fields `PotentialField`, `ForceField`, `FlowField` and
  `HomogeneousFlowField` now throw an error when an attempt is made to
  change their value via the class setter, instead of silently ignoring
  the new value (#4121).

* The `CylindricalLBFluxDensityProfileAtParticlePositions` observable now
  measures the correct quantity (#4152).

* The Boost 1.74 bug was patched (#3978).

* A bug involving an access out of bounds was fixed in the structure factor
  code (#4205).

* A bug in the collision detection feature that lead to a harmless warning
  being printed to the terminal upon collision was fixed (#4484).

* Calling `collision_detection.set_params()` with invalid arguments no
  longer leaves the collision detection feature in an indeterminate state;
  the previous state is automatically rolled back (#4484).

* Setting the collision detection mode `glue_to_surface` or
  `bind_at_point_of_collision` when feature `VIRTUAL_SITES_RELATIVE`
  is not compiled in now generates the correct error message (#4484).

* Passing a particle chain-based observable object (`ParticleDistances`,
  `BondAngles`, `BondDihedrals`, `CosPersistenceAngles`) that doesn't have
  enough particle ids for the calculation (e.g. only 1 particle id when 2
  are needed for the bond distance calculation) to a `Correlator` object
  no longer causes a memory overflow (#4255).

* Calculating the energy of the system when an IBM object is present no
  longer terminates ESPResSo, instead a warning is issued (#4286).

* The `Sphere` shape no longer returns NaN values in the distance vector
  for particles located exactly in its center (#4384).

* Runtime errors raised when the maximal bonded interaction range becomes
  larger than the simulation box are no longer ignored when dihedral bonds
  are added to the list of interactions (#4383).

* Runtime errors about incorrectly initialized electrostatic/magnetostatic
  methods are no longer silently ignored at integration start (#4383).

* Runtime errors about incorrectly initialized GPU dipolar direct sum and
  Barnes-Hut are no longer silently ignored when the actors are instantiated
  (#4404).

* A bug that could potentially lead to stale references in the script
  interface was fixed (#4476).

* `TabulatedNonBonded.is_active()` now returns `False` instead of `None`
  when the interaction is inactive (#3586).

Under the hood changes
----------------------

* The Python code is now checked with Pylint to prevent the introduction
  of unused code and dangerous anti-patterns (#3293, #3203).

* The CMakeLists.txt files are now formatted automatically with
  `cmake-format` (#3622).

* The Python code and C++ code were checked with LGTM to detect and fix
  coding errors and anti-patterns (#3851, #3856, #4300).

* Compiler warnings and diagnostics from GCC 11, 12, from Clang 10, 12, 13, 14
  and from Intel 19.0.4 were addressed (#4084, #4426, #4510, #4526).

* The `Particle` struct was moved to a dedicated header file `Particle.hpp`
  to improve separation of concerns in the core (#3251, #3164).

* The `Observable_stat` structs were moved to a dedicated header file
  `Observable_stat.hpp` and decoupled from the pressure/energy/coulomb/dipolar
  frameworks (#3712) and made stateless (#3723).

* Observables based on particle ids have been rewritten using particle traits
  to decouple the `Particle` struct from `Observable` classes (#3667).

* The Python `Integrator` class was split into multiple classes, one for each
  integrator, with a structure similar to actor and interaction classes
  (#3390). This layout better reflects the structure of integrators in
  the core and will make it easier to include new integrators in the future.
  This change doesn't break the API.

* The ghost communication infrastructure was simplified (#3216, #3399).

* The LB coupling for the regular decomposition scheme was rewritten (#4470).

* Thermostats are now fully object-oriented in the core to reduce code
  duplication (#3438, #3444, #3461).

* Bonded interactions are now fully object-oriented in the core to
  facilitate the development of new interactions (#4161).

* Bonded interactions are now communicated between MPI processes
  automatically and transparently by the script interface (#4350).

* The custom `MpiCallbacks` framework has been simplified and the callbacks
  made more homogeneous (#4383).

* The custom `MpiCallbacks` framework is being progressively replaced by
  `boost::mpi` communication (#4506, #4511).

* The `local_particles` global variable is no longer accessible directly
  (#3501).

* The Python tests now use specialized assertions to generate more helpful
  error messages (#3419).

* The tutorial tests were simplified using AST to parse Jupyter notebooks
  (#3408).

* The CMake logic for tutorials has been simplified (#3408, #3486).

* The Cython interface was thoroughly cleaned up from unused imports
  (#3496, #3510).

* The `ScriptInterface` framework was rewritten (#3794).

* The `ScriptInterface` framework is now the preferred way to implement
  new features. Existing features were converted to `ScriptInterface`
  objects: bonded interactions (#4350), bond breakage (#4464), collision
  detection (#4484), reaction methods (#4451), MPI-IO (#4455), H5MD (#4480),
  cell system (#4511), actors, scafacos, electrostatics and magnetostatics
  (#4506). The corresponding Cython files were converted to Python files.

* It is now possible to extend the list of available specifications in the
  H5MD feature at the C++ level (#4480).

* The duplicated functions between `P3M` and `DipolarP3M` were factored out
  (#3879).

* Statistical tests are no longer executed in coverage and sanitizers
  builds (#3999).

* The `Utils::Mpi::gather_buffer()` function was fixed (#4075). The bug
  didn't affect ESPResSo.

* Parameters can be passed to CTest at configuration time via the new
  `CTEST_ARGS` CMake option (#3862). This replaces the deprecated and
  non-portable `ARGS` Makefile variable expansion.

* A superfluous and non-portable CMake `target_compile_options()` statement
  was removed (#3852).

ESPResSo 4.1.4
==============

This release provides a number of corrections for the ESPResSo 4.1 line.
We recommend that this release be used for all production simulations.
The interface has not been changed between ESPResSo 4.1.3 and 4.1.4.
However, some bugs were discovered which can affect simulation results.
Please find the list of changes below. The numbers in brackets refer to
ticket numbers on https://github.com/espressomd/espresso

General corrections and improvements:

* Fix a bug in the LB CPU implementation that lead to incorrect
  LB shear stress tensors in thermalized fluids since 4.1.0 (#3847)

* Fix a bug in the LB CPU and GPU implementations that lead to
  `gamma_bulk` being used in place of `gamma_shear` in the second
  order term for forces in all previous ESPResSo versions (#3885)

* Fix a bug that always set the epsilon value to zero in `P3M` and
  `P3MGPU` actors since 4.0.0 (#3869)

* Fix an issue in the Python script interface that rejected integer
  epsilon values in the `P3M` and `P3MGPU` actors, and the `'metallic'`
  epsilon value in the `P3M` actor (#3869)

* Fix the exception mechanism in the `P3M`, `P3MGPU` and `DipolarP3M`
  code to forward errors to the Python interface instead of silencing
  them or running infinite loops (#3869)

* Fix range checks in the OIF code that failed to raise `TypeError`
  exceptions (#3846)

Documentation and tutorials corrections and improvements:

* Explain the discrepancy between the Gay-Berne formula from the user
  guide and from the original paper (#3839)

* Add note explaining the P3M algorithm works with non-metallic epsilon
  values only when the box is cubic (#3869)

Build system and platform-related corrections and improvements:

* Fix several CMake warnings raised by CMake 3.17 and above (#3830, #3859)

Improved testing:

* Add a test to check the off-diagonal elements of the LB stress tensor
  in long simulations of thermalized fluids (#3847)

Under the hood changes:

* Remove unnecessary memory allocation on GPU from MPI worker nodes (#3911)

ESPResSo 4.1.3
==============

This release provides a number of corrections for the ESPResSo 4.1 line.
We recommend that this release be used for all production simulations.
The interface has not been changed between ESPResSo 4.1.2 and 4.1.3.
However, some bugs were discovered which can affect simulation results.
Please find the list of changes below. The numbers in brackets refer to
ticket numbers on https://github.com/espressomd/espresso

Feature configuration at compile time

* The number of features which need to be defined at compile time in
  myconfig.hpp has been reduced. Features without performance impact
  are now always present. These are:

   * ``OIF_LOCAL_FORCES``
   * ``OIF_GLOBAL_FORCES``

General corrections and improvements:

* Many bonded interactions were not considered in the bond cutoff calculation:
  umbrella, OIF local, OIF global, IBM tribend, IBM volcons, angle harmonic,
  angle cosine, angle cossquare, tabulated angle, bonded Coulomb, subtracted
  bonded Coulomb, subtracted LJ, quartic, harmonic dumbbell. This can lead
  to sub-optimal skin values when such bonds are used with an inter-particle
  distance that is longer than other bonded (FENE, harmonic bond, rigid bond,
  thermalized distance, tabulated bond, tabulated dihedral, dihedral, IBM
  triel), non-bonded (LJ, Morse, Buckingham, etc.) and long-range
  (electrostatics, magnetostatics) interactions in the same system. All
  bonded interactions are now considered in the cutoff calculation (#3443).

* Fix a bug in a rotation function that resulted in improper treatment of
  rotation vectors with norm different from unity (#3559); all observable
  classes inheriting from `CylindricalProfile` are affected

* Fix a bug in the LB GPU implementation that lead to incorrect velocity
  interpolation near LB boundaries (#3593)

* Fix a bug in the LB CPU implementation that lead to incorrect grid sizes
  (#3678)

* Object-in-fluid bugfixes have been backported from the OIF development
  branch; in particular, the bending force between two triangles is now
  torque-free (#3385)

* Rewrite the linear polymer position generator, which was inefficient and
  frequently rejected valid positions (#3402, #3484, #3491)

* Fix an error in the distance calculation of the `SpheroCylinder`
  shape (#3629)

* Fix a sign flip in the surface normal calculation of the `Torus`
  shape (#3728)

* Fix an `IndexError` when running `system.number_of_particles()` without a
  value for the argument `type` (#3496, #3494) and fix the range check (#3536)

* Fix a `NameError` when running `system.analysis.rdf()` without a value for
  the argument `r_max` (#3496, #3494)

* Fix a `NameError` raised by the OpenGL visualizer when drawing bonds in
  periodic images of the unit cell (#3511)

* Correctly calculate the orientation of bonds cut by the faces of the
  simulation box in the OpenGL visualizer (#3511)

* Fix a memory leak in the OpenGL visualizer when drawing shapes containing
  cylindrical elements (`Cylinder`, `SpheroCylinder`, `SimplePore`,
  `Slitpore`) and drawing bonds between particles (#3533)

* Fix an issue in the OpenGL visualizer that drew the channel of the
  `Slitpore` shape at the center of the box, instead of using the
  `dividing_plane` attribute (#3728)

* Fix a bug in the ELC algorithm that ignored the Coulomb prefactor (#3731).
  The same bug is also present in MMM2D but could not be fixed.

* Correctly check the P3M parameter `mesh` (#3676)

* The LB checkpointing argument `binary` now takes a boolean value (#3541);
  integers values 0 and 1 are still accepted (integers are implicitly cast
  to boolean values)

* Reinitialize the P3M and dipolar P3M solvers when the box size or skin
  changes (#3717)

* Clarify error messages in the Steepest Descent integrator (#3763)

* Fix an incorrect formula in the `tensor_product` mode of the `Correlator`
  class that always returned an array of 0's since 4.1.0 (#3781)

* Fix a runtime error when calling the `get_params()` method of a
  ScaFaCoS-based actor (#3784)

Documentation and tutorials corrections and improvements:

* Fix paragraph formatting in Jupyter notebooks and update Sphinx
  bibliography (#3395).

* The Sphinx documentation generation doesn't run in parallel any longer
  due to plugin `sphinxcontrib.bibtex` throwing a warning when executed
  with more than one thread in Sphinx v2.3.1 (#3393). The slowdown is not
  significant (less than a second).

* Fix compatibility issues with Sphinx 2.4.0 (#3480), 3.0.1 (#3642, #3659)
  and 1.6.7 (#3743)

* Clarify the quaternion formalism used in ESPResSo (#3392, #3748)

* The `p3m.py` sample showcased an incorrect usage of the ELC actor
  (the gap region was missing). The actor was removed and a new,
  stand-alone sample `visualization_elc.py` was created (#3468)

* The `visualization_constraints.py` sample showcased an incorrect
  usage of the `Slitpore` and `Wall` shapes that lead to a discontinuous
  potential; this is now fixed (#3728)

* Correct errors in the documentation of the constructor parameters for
  shape classes `Cylinder`, `SpheroCylinder`, `Rhomboid` (#3567), for
  class `System` (#3542) and for cylindrical observables (#3569)

* Correct an error in the formula of the electrostatic prefactor in the
  electrostatics documentation, give the full expression of the
  electrostatic prefactor in tutorials and samples (#3673)

* Improve documentation of the `Slitpore` shape and document the
  `Torus` shape (#3728)

* Improve installation instructions (#3673, #3699, #3732)

* Document `BoxGeometry`-related functions (#3747)

* Explain release workflow and how to obtain released versions of
  ESPResSo (#3745)

* Improve citation instructions with examples (#3745)

* General improvements (#3740, #3743)

Build system and platform-related corrections and improvements:

* The benchmarks can now be run with any MPI library supported by
  ESPResSo (#3412)

* The CMake logic was simplified (#3574, parts of #3582). The minimal
  required Cython version is now checked. CMake now generates an error
  message if `WITH_CLANG_TIDY` is `ON` but no Clang-Tidy can be found
  with a version matching the Clang version. The CUDA library installed
  via the Ubuntu package `nvidia-cuda-toolkit` is now correctly detected.

* Add support for ROCm versions 3.0 (#3386), 3.1 (#3574) and 3.3 (#3623)

* Fix compiler errors with HDF5 > 1.10.2 (#3604)

* Fix compiler errors with Boost 1.73 (#3725)

* Fix a deprecation warning from the `collections.abc` that will become
  an error in the upcoming Python 3.9 interpreter (#3568)

* Fix a compatibility issue with `pint` 0.10.1 in tutorial
  12 - constant pH (#3423)

Improved testing:

* Fix a tolerance value that was incorrectly divided by 100, causing unit
  tests to fail on i586 architectures (#3427)

* Compile CUDA code in the Travis-CI image to detect more compiler
  errors (#3699). GPU tests are skipped on Travis-CI.

* Add a test for the ``Utils::get_n_triangle`` function used in OIF (#3391)

* Add a test for sample `visualization_constraints.py` (#3533)

* Add missing `LENNARD_JONES` and `GPU` feature guards in Python
  tests (#3403, #3676)

* Fix a few non-functional Python tests (#3419) and sample tests (#3791)

* Improve testing of ELC (#3731)

* Improve testing of the `Slitpore` shape (#3728)

* Fix an issue in a core test (#3677)

* Add cleanup function in the checkpointing tests (#3699)

* Add a test for `fold_position()` (#3747)

* Improve testing of observables, correlators and accumulators
  (#3781, #3783, #3784)

Under the hood changes:

* Remove unused code (#3556, #3738)

* Remove the unused `FindPythonModule` CMake module (#3736)

* Update the espresso-ci bot scripts (#3613)

ESPResSo 4.1.2
==============

This release provides a number of corrections for the ESPResSo 4.1 line.
We recommend that this release be used for all production simulations.
The interface has not been changed between ESPResSo 4.1.1 and 4.1.2.
However, some bugs were discovered which can affect simulation results.
Below, please find the list of changes. The numbers in brackets refer to
ticket numbers on https://github.com/espressomd/espresso

General corrections and improvements:

* Remove correlation between the rotational noise and translational
  noise in the Langevin thermostat (#3355)

* Fix a bug that may cause the wrong temperature to be set by the
  Langevin and DPD thermostats in the first time step after the system
  was altered from the Python level, e.g., by changing particles or
  interactions (#3341)

* Fix a bug that caused the DPD thermostat to generate an incorrect
  velocity distribution when used together with the Langevin thermostat
  (#3352)

* Fix a bug in MMM2D and ELC with potential boundary conditions, where
  one of the correction factors was over-counted resulting in wrong
  energies (#3310)

* Fix a bug that caused the wrong bonds to be deleted when removing
  particles from the system (#3356)

* Fix an ambiguity in `ParticleSlice`: the values in the square brackets
  refer to particle ids, not array indices (#3367). This means the
  ill-defined syntax `system.part[0:-1]` is no longer valid. See the
  User Guide section on Setting up particles for more information.

* Remove the mass prefactor in the `ComForce` observable and use
  the correct Particle ids in the `ParticleAngularVelocities`
  and `ParticleBodyVelocities` observables (#3380)

* Fix a rounding error that caused debug builds of ESPResSo running with
  multiple MPI threads to crash when a particle was placed exactly on
  the boundary between two cells (#3377)

* Fix `espressomd.has_features()` for the corner case where the list of
  all compiled-in features is passed as argument, returning ``False``
  instead of ``True`` (#3318)

* Refactor the random number generator code (#3349)

* Minor fixes (#3351, #3336)

Documentation and tutorials corrections and improvements:

* Improve documentation of Monte Carlo methods (#3254, #3330)

* Minor fixes (#3342, #3334)

Build system and platform-related corrections and improvements:

* List all Python dependencies in `requirements.txt` with the supported
  version numbers (#3300). Please note that most of them are optional.

* Add `MPIEXEC_PREFLAGS` and `MPIEXEC_POSTFLAGS` to the command lines of
  parallel tests (#3221)

* Add the `-oversubscribe` flag to the command lines of parallel tests
  running with OpenMPI v2.X to avoid exiting early from a Python test
  configured without `MAX_NUM_PROC` on a machine with a hyperthreaded
  CPU where OpenMPI is configured such that the number of threads cannot
  exceed the number of cores (#3335)

* Refactor the CI, maintainer, Doxygen and pypresso shell scripts to
  make them more portable and support filepaths containing whitespaces
  (#3326, #3373)

* Fix a nvcc compiler warning on the empty config (#3329)

Improved testing:

* Add a test for ELC and MMM2D using analytic expressions of the force
  and energy (#3331)

* Sped-up seven Python tests (#3319)

* Fix a test that broke on s390x architectures with Fedora 31 (#3312)

* Fix tests that broke on i586 architectures with OpenSUSE Tumbleweed
  (#3327, #3358)

ESPResSo 4.1.1
==============

This release provides a number of corrections for ESPResSo 4.1.
We recommend that this release be used for all production simulations.
The interface is mostly unchanged between ESPResSo 4.1.0 and 4.1.1; the
two exceptions are limited to these experimental features:

* `Integrator.set_isotropic_npt()`: input value `direction=[0,0,0]`
  now throws an error instead of being silently changed to `[1,1,1]`

* `ParticleHandle.swimming`: deprecated value `'rotational_friction'`
  is now disabled

These changes are unlikely to affect production simulations.
However, some bugs were discovered which can affect simulation results.
Below, please find the list of changes. The numbers in brackets refer
to ticket numbers on https://github.com/espressomd/espresso

General corrections and improvements:

* Restore checkpointing mechanism for the steepest descent and NPT
  integrators, LB and NPT thermostats (#3245)

* Increase the minimum MPI version to 3.0; OpenMPI versions 1.6.5 and
  lower are no longer supported (#3236)

* Fix `Integrator.set_isotropic_npt()`: remove the silent conversion of
  the incorrect input parameter `direction=[0,0,0]` to `[1,1,1]` in the
  core; the function now throws an exception for fixed-volume boxes;
  this change is unlikely to break pypresso scripts since not providing
  a value to `direction` or providing `[1,1,1]` were the two standard
  ways to set up a box with all directions allowed to rescale (#3253)

* Fix `Integrator.set_vv()`: this function failed to set the velocity
  Verlet integrator if the NPT integrator was active; this is now
  resolved (#3274)

* Fix the random segmentation fault triggered by the removal of a
  particle with a bond or a virtual site relationship to another
  particle (#3288)

* Fix `system.part.writevtk()`: the function now writes down all
  particles when using `types="all"` (#3290)

* Disable the deprecated and broken ENGINE shear torque calculation
  feature; the feature will be completely removed from the core in
  the upcoming 4.2 release (#3277)

* Fix unit conversion for the LB fluid viscosity (#3287)

Documentation and tutorials corrections and improvements:

* Add more detailed installation instructions for ESPResSo and its
  Python dependencies on MacOS X (#3236)

* Add links to Dockerfiles providing installation instructions for
  ESPResSo and its Python dependencies on CentOS 7, Fedora 30, Debian 10
  and OpenSUSE Leap 15.1 (#3244)

* Add instructions to read PDB files with `MDAnalysis`, which is one of
  the recommended tools to read/write molecular dynamics topologies and
  trajectories in ESPResSo; the PdbParser feature will be removed in the
  upcoming 4.2 release (#3257)

* Add a new tutorial on the constant pH method; the reaction ensemble
  tutorial will be removed in the upcoming 4.2 release (#3184)

Build system and platform-related corrections and improvements:

* Fix a PYTHONPATH error when ESPResSo is built in a directory
  containing whitespace characters (#3238)

* Fix several issues with the command `make install` that lead to import
  errors in Python (incorrect runtime path, missing shared objects,
  name collision for submodule `cluster_analysis`) and deprecate the
  `make install DESTDIR=/path/to/espresso` command in favor of the
  standard `cmake .. -DCMAKE_INSTALL_PREFIX=/path/to/espresso` command
  (#3228), install espressomd module in a platform-dependent python
  path, typically `lib{,64}/python3.X/{dist,site}-packages` (#3258)

* Fix an issue in mpiio that triggered an assertion in systems with no
  bonds when ESPResSo is built with stdlibc++ range checking enabled
  (#3234)

* Fix the pypresso script to correctly parse filepaths containing
  whitespaces passed after a pypresso flag, such as `--gdb`, and make
  conditional statements cross-platform (#3292)

Improved testing:

* Test checkpointing of integrators and thermostats (#3245)

* Fix and improve `check_cmake_install` test (#3228, #3258) and add a
  new CI job to test an installed version of ESPResSo (#3228)

* Test engine LB (#3277)

* Add more LB tests (#2748)


ESPResSo 4.1.0
==============

This is a feature release, i.e., new functionality is added to ESPResSo.
An additional focus of this release is quality assurance and
modernization. The testing of ESPResSo's functionality has been extended
considerably. Also, sample and tutorial scripts are now automatically
tested. Moreover, a large effort was put into modernizing the C++
simulation core. Work has been done, e.g., on particle sorting, the MPI
communication infrastructure, and the lattice-Boltzmann implementations.
Electrostatic and magnetostatic methods now have a clear and common
interface. These changes will facilitate future extensions of ESPResSo
and make the code more understandable to new developers.

We recommend that this release be used for all production simulations.
No further bug fix releases will be provided for the 4.0 line, and not
all fixes are present in ESPResSo 4.0.2.

Please carefully read the detailed list of changes below before using
this release. Issues can be reported at
https://github.com/espressomd/espresso.


Changed requirements
--------------------

* Python 2 support has been dropped. ESPResSo now requires Python 3.
  For additional information, please see
  https://github.com/espressomd/espresso/wiki/python_2_3_transition.

* ESPResSo now needs a C++14-capable compiler, such as GCC 4.9 and later
  or Clang 4 and later.

* It is discouraged to use ESPResSo with Boost versions below 1.67.


Added functionality and documentation
-------------------------------------

* The distance between a shape (such as sphere) and a position can now
  be queried via ``shape.calc_distance()``.

* The lattice nodes of a lattice-Boltzmann fluid can now be iterated
  using ``LBFluid.nodes()``.

* A tutorial on magnetic fluids has been added.

* The stress created by the dissipative particle dynamics interaction
  (DPD) can now be obtained via the ``DPDStress`` observable.

* The stress of a lattice-Boltzmann fluid can now be obtained via the
  ``LBFluidStress`` observable.

* A torus shape has been added.

* Two new accumulators for observables have been added:
  ``MeanVarianceCalculator`` and ``TimeSeries``.

* An ``ElectricPlaneWave`` constraint was added.

* Experimental support for AMD GPUs via HIP. The future of this feature
  is unclear. Please do not base hardware buying decisions on its
  presence.

* Visualization of slit pores in the OpenGL visualizer.

* A Weeks-Chandler-Anderson short-range potential has been added
  (``WCA``).

* The external force density applied to a lattice-Boltzmann fluid
  can now be changed during the simulation.

* Sanity checks for Mach limits and unequal MD and lattice-Boltzmann
  time steps have been added.

* The ``system.cell_system.tune_skin()`` method now has a keyword
  argument ``adjust_max_skin``. If set to ``True``, the maximum skin
  to be tested will be reduced such that it is compatible with the local
  box size.

* For the CPU lattice-Boltzmann implementation, the limit on the Verlet
  list skin (<0.5 agrid) has been lifted.

* A new observable ``CosPersistenceAngles`` has been added for the bond
  angles of a polymer (needed, e.g., for determining the persistence
  length).


Feature configuration at compile time
-------------------------------------

* The number of features which need to be defined at compile time in
  myconfig.hpp has been reduced. Features without performance impact
  are now always present. These are:

  * ``PARTIAL_PERIODIC``
  * ``LB``
  * ``LB_GPU`` (for builds with CUDA)
  * ``IMMERSED_BOUNDARY``
  * ``BOND_ANGLE``

* For most compilers, it is checked that only known features are
  declared in myconfig.hpp.

* The default feature configuration applied when no myconfig.hpp is
  present has been extended significantly. In particular, all tutorials
  can now be run with the default feature configuration. For production
  simulations, it is still recommended to use a custom myconfig.hpp
  containing only necessary features. This is true in particular, if
  particle rotation is not needed.

* Features that do not have automated tests now require
  ``EXPERIMENTAL_FEATURES`` to be defined in myconfig.hpp.


Interface changes
-----------------

* Several parts of ESPResSo now use a method-specific seed for random
  number generation. For the following individual methods a random
  number seed has to be passed using the keyword argument ``seed``:

   * [Langevin thermostat](http://espressomd.org/html/doc4.1.0/espressomd.html?#espressomd.thermostat.Thermostat.set_langevin)
   * [DPD thermostat](http://espressomd.org/html/doc4.1.0/espressomd.html?#espressomd.thermostat.Thermostat.set_dpd)
   * [LB Thermostat](http://espressomd.org/html/doc4.1.0/espressomd.html?#espressomd.thermostat.Thermostat.set_lb)
   * [Thermalized bond](http://espressomd.org/html/doc4.1.0/espressomd.html?#espressomd.interactions.ThermalizedBond)
   * [Wang-Landau reaction ensemble](http://espressomd.org/html/doc4.1.0/espressomd.html?#module-espressomd.reaction_ensemble)
   * [Constant pH ensemble](http://espressomd.org/html/doc4.1.0/espressomd.html?#espressomd.reaction_ensemble.ConstantpHEnsemble)
   * [Widom insertion method](http://espressomd.org/html/doc4.1.0/espressomd.html?#espressomd.reaction_ensemble.WidomInsertion)

* Changes in the lattice-Boltzmann (LB) interface:

   * By default the LB fluid is not thermalized. A temperature can be
     set using the ``LBFluid``'s keyword parameter ``kT``. If
     ``kT > 0``, an additional ``seed`` keyword parameter has to be
     provided.
   * The LB thermostat gets its temperature from the LB fluid.
   * The frictional coupling coefficient ``gamma`` is now a keyword
     parameter of the LB thermostat.

  For more detailed information on how to set up a LB fluid and
  thermostat, please see http://espressomd.org/html/doc4.1.0/lb.html.

* The method for polymer creation has been replaced. Now
  ``espressomd.polymer.positions()`` can be used to obtain particle
  positions for one or more polymer chains. Based on these positions,
  polymers can be created. For an example please see
  http://espressomd.org/html/doc4.1.0/particles.html#setting-up-polymer-chains.


Changed and removed functionality
---------------------------------

* The ``remove_total_momentum()`` method for lattice-Boltzmann fluids
  has been removed. The overall velocity of a fluid can be changed
  using the ``lb_fluid.nodes()`` iterator.

* The ``CATALYTIC_REACTIONS`` feature has been removed.

* The method for creating a polymer has been replaced.
  ``espressomd.polymer.positions()`` can now be used to obtain particle
  positions for one or more polymer chains.

* Checkpointing has been added for the electrokinetics method.

* The global random number seed has been partly replaced by
  method-specific ones. These are specified when activating the relevant
  feature such as the Langevin, DPD and lattice-Boltzmann thermostats
  via a ``seed`` keyword argument.

* The random number generator has been switched to Philox for most
  algorithms requiring random numbers.

* Limitations on the exclusion radius have been relaxed in the reaction
  ensemble method.

* A new observable ``CosPersistenceAngles`` has been added for the bond
  angles of a polymer (needed, e.g., for determining the persistence
  length).

* ELC has been disabled for non-neutral systems with constant potential.

* The calculation of the linear particle momentum included the forces of
  the last time step. The function ``system.analysis.linear_momentum()``
  now returns the sum of the product of mass and velocity of all
  particles, if no lattice-Boltzmann fluid is coupled.


Performance enhancements
------------------------

* Speedup in the short-range force calculation in situations where the
  short-range cutoff varies strongly for different pairs of particles,
  e.g., in a bidisperse fluid.

* Speedup in particle resorting triggered when particles have moved by
  more than a skin.

* Significantly faster back-transfer of particle forces from GPU-based
  methods such as the GPU implementations of lattice-Boltzmann and P3M.


Bug fixes
---------

* Lattice-Boltzmann boundaries, constraints and auto_update_accumulators
  are now included in checkpointing. (#2915)

* Collision detection is now checkpointed. (#2342)

* The rhomboid shape was fixed. (#2756)

* Deadlocks on certain GPUs have been resolved for the dipolar
  Barnes-Hut method. (#2719)

* The visualization of dihedrals has been fixed. (#2677)

* The ``ENGINE`` implementation for CPU LB has been fixed. (#3025)

* The external force density in lattice-Boltzmann fluids is no longer
  ignored in the first integration step after setting the force density.
  (#3144)

* The positions of virtual sites and the charges of ICC particles are
  now updated before observable calculation. (#3128)

* The forces and torques for the Gay-Berne potential have been
  corrected. (#3091)

* Remove undocumented behaviour in the case of using a cylindrical
  sampling area in the reaction ensemble, constant pH ensemble,
  Wang-Landau ensemble, Widom-Insertion method. (#3174)

* The ELC tuning error calculation has been rearranged to produce
  correct results for higher accuracies. (#3123)


New tutorials
-------------

* Tutorials for simulating ferrofluids and for using the constant-pH
  method have been added.


Under the hood changes
----------------------

* Automated testing has been enhanced. It now also includes samples and
  tutorials. The overall test coverage for the simulation core has
  increased by ~12% since ESPResSo 4.0.2.

* The CPU LB and LB-particle coupling have been refactored.

* Particle resorting has been simplified and sped-up.

* The MPI callback mechanism has been simplified. Furthermore, reduction
  operations such as summing values from all MPI ranks can now be
  performed.

* Nearly all manual memory management and C-style arrays have been
  removed.

* The rotation-related code has been simplified.

* Long-range electrostatic and magnetostatic methods now have a common
  interface.

* The kernels for short-range and bonded interactions have been
  simplified.

* The CMake build system has been refactored and dependencies between
  different parts of the code have been made clear.

* Python code formatting: the autopep8 version now matches the one in
  Ubuntu 18.04 (autopep8 v1.3.4 with pycodestyle v2.3.1).




ESPResSo 4.0.2
==============

This release provides a number of corrections for the ESPResSo 4.0 line.
We recommend that this release be used for all production simulations.
Please note that a sign error in tabulated interactions was fixed.
Simulation scripts which worked around this problem might have to be changed.
Below, please find the list of changes. The numbers in brackets refer to
ticket numbers on http://github.com/espressomd/espresso


Corrections for bugs that may harm simulation results:
  * A sign error in tabulated interactions was corrected such that
    the force equals the negative gradient of the potential. (#2519,2520)

  * The flow field of the CPU lattice-Boltzmann implementation was deleted
    when aspects of the molecular dynamics cell grid were changed; E.g., when
    interactions, the skin or the parallelization setup were changed.
    ESPResSo now terminates with an error, when this happens.
    To avoid this, please setup the CPU lattice-Boltzmann after all
    other aspects of the system. The GPU LB is not affected in the 4.0
    release, but was affected in the current development branch. (#2728, #2736)

  * Corrected the force acting on LB Boundaries for the case of
    agrid and density not equal to 1 (#2624).

  * Corrected the cutoff calculation for the soft sphere interaction. In the
    previous implementation, the offset parameter was ignored. (#2505)

  * The "three point coupling" of particles to the lattice-Boltzmann method
    has been removed. While it works in most environments, for some compilers
    the calculation gives wrong values. This is likely caused by undefined
    behavior. A corrected implementation is available in
    ESPResSo's development branch. It cannot be safely backported to 4.0.2,
    because the code has diverged too far. (#2516, #2517)
    Users who did not explicitly activate this coupling via couple="3pt" are
    not affected.

  * The velocity of existing particles was changed when setting or changing
    the simulation time step (#2480)


Further changes:
  * Fixed the electrokinetic Python interface (#2486)

  * Correction to the installation instructions for mac (#2510)

  * Corrected file permissions (#2470)

  * Minor corrections and extensions to the test suite (#2477, #2552)

  * Fixed a dead-lock in the dipolar Barnes Hutt method on the GPU for
    recent NVIDIA cards such as RTX 2080 (#2719).

  * Restored Mayavi visualizer's API-compatibility with OpenGL visualizer
    (#2751)




ESPResSo 4.0.1
==============

This release provides a number of corrections for ESPResSo 4.0.
We recommend that this release be used for all production simulations.
The interface has not been changed between ESPResSo 4.0.0 and 4.0.1
However, some bugs were discovered which can affect simulation results.
Below, please find the list of changes. The numbers in brackets refer to
ticket numbers on http://github.com/espressomd/espresso


Physics related corrections:
  * The GPU lattice Boltzmann method produced incorrect results when
    EXTERNAL_FORCES was not declared in myconfig.hpp.
    This issue was present since around June 2018 (#2241)

  * The temperature fluctuations for the GPU lattice Boltzmann implementation
    were larger than the ones for the Cpu lattice Boltzmann implementation.
    The cause was likely weak or incorrect random number generation.
    It is not clear since when this issue existed.
    It has been resolved by using library code for a counter-based
    random number generator (Philox) rather than the existing custom code.

  * Particles which were moved with the configuration changing moves (MC) implemented in the reaction ensemble
    module did not get assigned a random velocity. This was not a problem if you were looking at observables
    which do not depend on velocity.

  * Particles which were created in the Reaction Ensemble module were assigned a random velocity which was not distributed according to the Maxwell-Boltzmann distribution. This was not a problem if you were looking at observables which do not depend on velocity. If you looked at velocity-dependent observables but used a thermostat for thermalization before taking a sample you are also fine.  (#2377).

  * Under some conditions, the torque on self-propelled particles
    in a lattice-Boltzmann fluid was incorrect due to a sign error in the
    ENGINE feature. This was the case since the introduction of the feature
    (#2383)

  * The SimplePore shape was incorrect (#2379)

  * The parameters passed from Python to some features were narrowed to single
    precision. I.e., the values used were only accurate up to the 7th-8th
    significant digit. Further calculations with those values still were
    done using double precision.
    Classes backed by the script interface were affected. This includes
    shapes, LB boundaries, pair criteria and the collision detection.
    This issue was likely present since the introduction of Python support
    for the relevant features. (#2379)

  * Forces on LB boundaries retrieved via the LbBoundary.get_force() method
    in a Python script were incorrect for the CPU LB implementation.
    The GPU implementation was not affected.
    It is not clear, when the issue was introduced. (#2366)

  * In highly dense systems, the Widdom insertion scheme reported wrong values
    (#2294)

  * Fixed the Gay-Berne potential for sigma != 1. Downgraded the GAY_BERNE feature to an experimental feature due to insufficient testing (#2424)

Further general corrections and improvements:

* Fix restoring checkpointed simulations which contain exclusions (#2418)

* Allow neutral systems in reaction ensemble (#2378)

* Support re-tuning the dipolar P3M method (#2351, #1937)

* Support checkpointing of the collision detection parameters (#2342)

* Virtual sites tracers: don't complain about missing lb for non-virtual
  part (#2341)

* Prevent access to non-existent lb nodes from Python

* Opengl wildcard imports + mac fix  (#2295)

* Fix segfaults in virtual sites based collision detection schemes on more than one processor core (#2195)

* Fixed find_current_cell() and particle deletion (#2410, #2441)

* Fixed coordinate folding for corner cases (0 - epsilon on 32 bit architectures) (#2415)

Documentation and tutorials corrections and improvements:

* Mention how to read h5md files in the docs (#2353)

* Correct docs for setting the skin (#2340)

* Corrections to the Lennard-Jones tutorial (#2339)

* doc: Fixed Gay-Bern formula (#2234)

* Update PDF of tutorial 6 with bibliography (#2285)


Build system and platform related corrections and improvements:

* Fix Cython detection on Fedora 29 (#2371)

* Properly detect CUDA libraries if multiple are available (#2352)

* Guard tests and compilation for different feature sets
  in myconfig.hpp (#2344, #2326, #2350, #2349, #2335, #2337, #2266)

* install libH5mdCore (#2277)

* Fix MPI on Ubuntu 18.04 with CUDA (#2271)

* Fix PARTIAL_PERIODIC on big-endian platforms (#2259)

* Fixes for OpenSUSE rpm build (#2255)

* ctest: add ARGS option to custom 'make check' (#2252)

* cmake: workaround for cmake bug#18349 (#2244)

* Remove some superfluous executable bits (#2242)

* Silence arm32 warnings (#2267)

* Fix unit test without SSE instructions (#2265)

* Turn off ScaFaCoS support by default. Has to be enabled explicitly using -DWITH_SCAFACOS=on in CMake. (#2417)

* Fix builds with the Gnu Scientific Library in non-standard locations (#2429)

* Resolve some floating point rounding issues on I386 32-bit (#2454)


Improved testing:

* Test the LB thermostat (#2391, #2362)

* Better stability of the LB test (#2360)

* testsuite: fix checkpoint test (#2336)

* Test the installation routine (#2268)

* Improved testing of the cylindrical lb profile observables (#2272)

* testsuite: Check planar shear profile of LB (#2263)

* Test builds without optional dependencies (#2406)

* Test builds on non-Amd64 architectures (#2401)

* Improved test coverage of the Wang-Landau reaction ensemble (#2404)

* Testing of the electrostatic pressure (#2409)




ESPResSo 4.0.0
==============

ESPResSo 4.0 is the first release of ESPResSo with a Python interface.
I.e., the language in which simulation scripts are written is now Python,
and support for the TCL-interface has been dropped.
We recommend all users switch to ESPResSo 4.0, there will be no more fixes to
TCL-based versions.

Transitioning from TCL-based versions
-------------------------------------

The following can serve as a starting point for the new interface:
* The "Introduction" chapter in the user's guide
* The tutorials provided with ESPResSo
Both can be found at http://espressomd.org/wordpress/documentation

Please note that, starting from ESPResSo 4.0, enabling additional features in myconfig.hpp no longer changes the behavior of a simulation scripts. Features such as particle rotation and virtual sites have to be activated explicitly in the simulation scripts.

New features:
-------------
* Reaction ensemble methods for the modelling of chemical reactions, including constant pH and Wang-Landau schemes
* Polarisable particles via the Drude-oscillator scheme
* Steepest-descent energy minimization scheme
* Methods for active particles (swimmer_reaction, dipolar swimmers)
* GPU-accelerated P3M method for electrostatic interactions
* GPU-accelerated direct summation and Barnes-Hutt schemes for dipolar interactions under open boundary conditions
* Support for the electrostatic and magnetostatics method provided by the ScaFaCoS library
* Immersed boundary method for soft immersed objects in an LB fluid
* Virtual sites acting as inertialess tracers in an LB fluid usable as part of the immersed boundary method or independently
* Online cluster analysis based on criteria for pairs of particles
* New bonded quartic and Coulomb interactions
* Possibility to tune the skin parameter
* Support for saving to the h5md file format for molecular dynamics data.
* Connection to the MD Analysis Python package
* A rotate_system command was added, which rotates the particle positions around the system's center of mass

Visualisation
-------------
ESPResSo contains two online visualizers, one based on Mayavi, one on the OpenGl Python module.
Support for online visualization in VMD has been dropped, but particle data can be stored in file formats which VMD can read (vcf/vtf, h5md).

User-visible changes
--------------------

* When the ROTATION feature is compiled in, the rotation of a particle around each of its Cartesian axes in the body-fixed frame can be enabled
* The rotational degrees of freedom of virtual sites is now integrated. Virtual sites no longer use the quaternion attributes to store their relative orientation
* Alternatively, the orientation of the virtual site can be specified relative to that of the particle it is derived from.
* The random number generator was changed to Mersenne Twister as implemented by C++11.
* ESPResSo now depends on the Boost library including Boost-Mpi and a C++11 capable compiler
* The build system is now based on CMake.

Removed features
----------------
The following features are not part of the ESPResSo 4.0 release, but support is expected to be re-added in a subsequent release.
* Generalized hybrid Monte Carlo thermostat
* Virtual sites located at the center of mass of a group of particles
* Metadynamics, umbrella sampling and parallel tempering
* Non-equilibrium molecular dynamics (shear boundary conditions implemented by moving slabs of particles at the boundaries)
* The memd/maggs electrostatic solver
* The com_force feature
* The Shan Chen lattice Boltzmann extension
* Lees-Edwards (shear) boundary conditions
* The chain rdf analysis

The following functionality is removed permanently:
* Blockfile support, which is replaced by support for the h5md format as well as checkpointing based on the pickle Python module.
* Some of the convenience functions for particle creation such as salt and counterions, as these can be replaced by a very few lines of Python in the simulation script.
* The lj_angle and angledist (not-quite)-pair potentials
* The following analysis methods:
  diffusion_profile, cwvac, p_inst, ..._mol, cel_gpb, dipmom_normal, MSD, angularmomentum, cluster_size_dist, mol, lipid_orient_order, get_lipid_orients, current, wall_stuff,
  necklace, bilayer, modes2d
* The uwerr error estimator



Known issues and limitations
----------------------------
The numbers refer to issue numbers at http://github.com/espressomd/espresso
* Lattice Boltzmann interpolated velocities are only accurate up to one
  lattice cell next to a boundary (#1865)
* Adding boundaries in the electrokinetics method resets species densities
  to their initial values (#1899)
* The collision detection feature does not support checkpointing (#1943)
* Changing interaction parameters does not automatically update the value of
  the force property stored on the particles. Forces are re-calculated in
  the next integration step. (#1973)
* The domain decomposition cell system does not always choose the node grid
  yielding the best performance in parallel simulations (#1970)
* The dipolar P3M method can only be tuned once in a simulation script,
  unless method parameters are explicitly reset to 0 before the second tuning
  (#1937)
* In rare cases, P3M produces an rs mesh overflow warning (#2067)
* The H5MD file writer cannot handle bounds that involve more than two bond
  partners (#1715)
* Pickle checkpoints are not guaranteed to be compatible between
  different ESPResSo versions (#1181)
* Switching from the GPU-Based MMM1D and MMM2D methods to a different
  electrostatic method within a single simulation script may not work (#396)
* The membrane_collision interaction of the object in fluid framework may
  use an outward direction which is outdated by one time step (#2214)

ESPResSo 3.3.0
==============

New user-visible features
-------------------------

* SHANCHEN: a bicomponent lattice Boltzmann fluid, with support for rigid
  boundaries and coupling to particle dynamics.

* ELECTROKINETICS: An algorithm to treat species of ions on a mean-field
  level. Implementation takes advantage of a GPU.

* MMM1DGPU: The mmm1d electrostatics algorithm is now available as GPU version.

* Support of P3M on GPU.

* Provide the preliminary Python interface. Configure with
  --with-python-interface.

* Allow the p3m to dump the mesh in x, y, and z (before it only dumped
  the mesh in x). This allows for the dumping of the p3m in blockfile
  format in the case of non-cubic boxes. It will also now be possible
  to tune the settings of a non-cubic mesh.

* Stomatocyte lb-boundary and constraint implemented.

* MEMD electrostatics can now handle spatially varying dielectrc
  constants.

* Pore constraint / lbboundary can have a two outer radii now to create nozzles.

* Slitpore constraint and dielectric for IL based supercaps implemented.

* New command time_integration to get the runtime of the integration
  loop.

* New harmonic well that runs on the GPU.

Known bugs
----------

* The implementation of dielectric contrasts in conjunction with ELC
  seems to have an error. Please do not use it if you do not know
  exactly what you are doing until we have fixe the problem.

User-visible changes
--------------------

* Added new arguments "recalc_forces" and "reuse_forces" to the
  command "integrate". These arguments can be used to enforce or
  suppress the recalculation of the forces at the beginning of the
  call to "integrate". This is important for doing checkpoints, where
  the forces have to be stored and reloaded.

* Removed command "invalidate_system".

* Comfixed now works with periodic boundary conditions.

* The pressure contribution due to rigid bodies constructed by means of the
  virtual sites relative mechanism is included, both for pressure and stress
  tensor. Note that the corresponding contribution for rigid bonds is still
  not implemented.

* The configure option "--with-myconfig" has been removed. Instead,
  the configure variable "MYCONFIG" can be set to give the name of a
  myconfig file.

* Generic LJ can now be turned into a soft potential.

* Renamed torque to torque_{lab,body}, improved torque handling in
  blockfiles.  Clearer distinction between the reference frames for
  torque and angular momentum.

* Lattice-Boltzmann now has a additional 3-point coupling scheme.

* The noise type in thermalized Lattice-Boltzmann is now selectable via Tcl.

* Interaction with a wall can be restricted to the outside.

* Removed compiler switch GRANDCANONICAL.

* Removed outdated ADRESS code.

* external_potential tabulated to include arbitrary potentials applied to all
  particles.

* thermostat inter_dpd ignore_fixed_particles 1 allows the user to swith on DPD
  with fixed particles.

* New Observable concept that includes running averages and other observables
  with history


Changes visible for developers
------------------------------

* Feature GHOSTS_HAVE_BONDS allows for ghost particles to also store the bonds.

* The code has been switched to using a C++ compiler.

* Introduced object-oriented interfaces for forces/constraints/external fields:
  SystemInterface and Actor.

* Improved Lattice structure

* Generic TclAppend function

ESPResSo 3.2.0
==============

New user-visible features
-------------------------

* Added a new immersed boundaries code that allows to simulate
  extended, flexible objects in conjunction with the LB code.  This is
  described in the UG in the new chapter "Object-in-fluid".

* The script tools/blockfile2vtf.tcl can be used to convert ESPResSo
  blockfiles into VTF files.

* Two new short-ranged nonbonded potentials have been added:

  * HAT: the classical conservative DPD interaction, a simple force
    ramp.

  * GAUSSIAN: A Gaussian potential.

* The feature CATALYTIC_REACTIONS and the command "reaction" model a
  simple chemical reaction: when a particle of the reactant type comes
  into the vincinity of a particle of the catalysator type, the
  reactant is transformed into a particle of the product type.

* The feature GHMC implements a generalized hybrid Monte-Carlo
  thermostat.

* The feature GRANDCANONICAL provided functions to simplify grandcanonical
  simulations in ESPResSo. To that end it provides functions to find and
  delete random particles of a certain type.

* Added new feature ROTATION_PER_PARTICLE that allows to choose
  whether a particle has rotational degrees of freedom or not.

User-visible changes
--------------------

* Removed non-working tools and samples.

* Bond angle potentials are now defined on the Tcl-level instead of on
  the feature-level, i.e. to choose the type of the bond-angle
  potential, you should use the corresponding Tcl command instead of a
  feature. To allow for any bond angle potential, activate the feature
  BOND_ANGLE.

* We have removed the forcecaps for the different interaction
  types. Instead, there is now a single global forcecap, plus it is
  possible to define individual forcecaps on a particle pair level.

* By default, CUDA is now automatically detected and activated when it
  is there and usable.

* The functions for galilei transformations have been overhauled and
  renamed. The old functions are deprecated.

* DPD (as thermostat) now works without any other interaction, so a pure
  DPD fluid is possible.

ESPResSo 3.1.2
==============

* Fixed bugs #38475, #37725, #36434.

* Added workaround for bug in VTF reader plugin of VMD 1.9.1 that made
  all particles look the same when ELECTROSTATICS is turned on.

* Added Tcl-scripts of the tutorials to the distribution.

* Added forgotten part of the UG on the correlators.

* Removed buggy readline code tclline.tcl. Fixes #36432.

* Removed Ewald code from ESPResSo that never worked and was never
  documented.

ESPResSo 3.1.1
==============

* The "Espresso" binary now outputs the header to STDERR instead of STDOUT.

* Fixed bugs #36431, #37120, #37214, #37374, #37306, #35767.

* Fixed generation of myconfig-sample.h.

* Fixed AdResS example.

* Added new logo.

* The documentation sources are now left out of the distribution
  package, only the PDF files are included. This fixes problems when
  calling "make doc" in a distribution package (#35958).

ESPResSo 3.1.0
==============

New user-visible features
-------------------------

* While so far, observables were typically computed and stored on the
  Tcl-level after a call to "integrate", a new observable concept now
  allows to compute observables while the C-core is running. This is
  particularly useful for observables that have to be computed very
  frequently, as for example in the case of time-correlations
  (e.g. the RMSD of particles).
  The new concept is documented in the User's Guide in Section 8.4
  ("Correlations and observables").

* The new feature COLLISION_DETECTION allows to dynamically add new
  bonds between colliding particles.

* Added new feature LANGEVIN_PER_PARTICLE that allows to set the
  Langevin parameters temperature and gamma per particle.

* Added new constraint and LB boundary condition "rhomboid".

* Lattice-Boltzmann:

  * nonzero velocity boundary conditions are available

  * forces on boundaries can be calculated

  * wall constraints work

* Added basic tutorial on Lennard-Jones liquid, made second tutorial
  (simple charged systems) better available (all in doc/tutorials/)

* The ICC* algorithm is now ready to use. It is used with the command
  "iccp3m" and allows to take into account dielectric boundaries of
  arbitrary shape. The command "dielectric" allows to create the
  boundaries in a similar fashion as constraints and lbboundaries.

User-visible changes
--------------------

* The blockfile C-library has been removed. In theory, there was a
  library usable from C that could be used to read and write
  blockfiles. Since several years, it was broken, so apparently it
  wasn't used anyway.

* The NPT barostat now works in many more combinations of
  algorithms than so far (e.g. MMM2D, ELC, ...)

* The Lattice-Boltzmann CPU implementation now works when Verlet lists
  are used.

* Removed unneccessary overhead of neutral particles in simulations
  with P3M.

* Removed unneccessary overhead of activated but ununsed interaction
  features.

* Some changes in the Verlet list construction yield a performance
  gain of up to 90% in systems where the interaction ranges differ
  significantly, or where some particle types do not interact at all
  (phantom particles).

* Renamed Coulomb method maggs to MEMD (inter coulomb maggs => inter
  coulomb memd).

Changes visible for developers
------------------------------

* code_info does now not only show the main package version, but also
  the exact git commit id and whether or not the code was modified
  ("dirty"). Also, the distribution package provides this information.

* The various features are now defined in the file
  src/features.def. From this file, a couple of Python scripts
  automatically generate myconfig-sample.h and what was config.h and
  config.c. Also, this allows to check whether all features are
  documented, tested and defined.
  Note that due to this change, ESPResSo development now requires
  Python.

* The domain decomposition, P3M, LB and MEMD now use MPI cartesian
  communicators. This will hopefully speed up the simulations on some
  platforms.

* Split interaction code into .c and .h files.

* Split off the Tcl interface (into src/tcl/) from the C core code (in
  src/)

* Added ESPResSo logos, cover issue images, and some other material to
  the repo (doc/).

* Started new LaTeX-Developer's Guide (DG), and moved all "Related
  pages" from the doxygen docs. The doxygen stuff still exists and is
  available via doc/doxygen/.

ESPResSo 3.0.2
==============

* Fixed features ADRESS and VIRTUAL_SITES_COM that did not work for a while.

* Fixed bugs #33489, #34238.

* Fixed a few bugs that were never reported via the bug tracker,
  mainly in the documentation.

ESPResSo 3.0.1
==============

* Fixed bugs #33375,#33376,#32005

* Fixed a few bugs that were never reported via the bug tracker.

ESPResSo 3.0.0
==============

User-visible changes
--------------------

* The Lattice-Boltzmann implementation in ESPResSo has been
  significantly overhauled and it has been documented for the first
  time.

* A CUDA implementation for NVIDIA GPUs of the Lattice-Boltzmann
  algorithm is included.

* The Maggs algorithm (or Maxwell Equation Molecular Dynamics, MEMD)
  for computing electrostatic interactions has been included and
  documented. The algorithm is a fast and scalable alternative for P3M
  or other such algorithms.

* The P3M algorithm for electrostatic and dipolar (magnetostatic)
  interactions has been extended.

  * Electrostatic P3M allows for non-cubic boxes.

  * The electrostatic P3M pressure tensor is computed correctly now.

  * Dipolar P3M has been parallelized (electrostatics P3M was
    parallelized already!)

  * The MDLC method (magnetic dipolar layer correction) has been
    parallelized.

* The virtual sites feature has been significantly extended and
  documented.  Virtual sites are particles that can interact with
  other particles but whose positions are not integrated via the
  normal integrator.  Instead, the positions are determined by the
  positions of other, "real" particles.  Virtual sites can either be
  set into the center of mass of a set of real particles (feature
  VIRTUAL_SITES_COM), or it can be set to an arbitrary location
  relative to another particle (VIRTUAL_SITES_RELATIVE).  This allows
  to create rigid structures within ESPResSo.

* The User's Guide has been significantly updated and extended.

* Some constraints can now be made penetrable and reflecting.

* "Espresso" is now the actual binary instead of a wrapper shell
  script that automatically calls MPI. This means that it is necessary
  to call "mpiexec" or "mpirun" youself to run ESPResSo in an MPI
  environment.

* The directory where the scripts are installed is now compiled into
  the binary, i.e. it is not necessary anymore to set the environment
  variable ESPRESSO_SCRIPTS. Still, it will heed the environment
  variable if it is set.

* The build system has been overhauled and simplified.

  * Running "make check" will run the testsuite with a single number
    of processors, which gives a significant speedup. "configure" will
    try to determine the available number of CPUs/cores. If it can't,
    it will use 1 by default.

  * MPI recognition has changed. The build system first needs to find
    out how to compile an MPI binary. If it doesn't find out
    automatically, you can help it by setting the variable MPICC or
    LDFLAGS correctly.  Furthermore, it needs to know how to run an MPI
    binary.  If "mpiexec" is available, everything is fine.  If it is
    not, you can provide a script "mympiexec.sh" that behaves like
    mpiexec.

  * So far, when you built in the source directory, all files ended up
    in a subdirectory "obj.XXX", where XXX was some description of the
    processor you used.  This non-standard behavior has been removed.
    Instead, the files are generated directly in the source directory
    when compiling there.
    If you want to compile several binaries from a single source
    directory, you can use build directories as described in the
    User's Guide.

* A number of functions have been deprecated, as they are not
  maintained anymore. Plase do not use them in your code and replace
  them with appropriate alternatives if you use them in your code!
  The following functions are deprecated:
  checkpoint_*, polyBlock*, calcOb*, calcObs*, plot*, polyConf*

* FFTW2 is no longer supported.

Organizational changes
----------------------

* The home page of ESPResSo has been relocated to

    http://espressomd.org

* The mailing list and source code repository have been moved to GNU
  Savannah. Furthermore, we have a bugtracker now. The development
  homepage is

    https://savannah.nongnu.org/projects/espressomd

* The maintenance of ESPResSo has been relocated from Torsten Stühn
  from the Max-Planck-Institute for Polymer Research at Mainz to

    Olaf Lenz <olenz@icp.uni-stuttgart.de>

  from the Institute for Computational Physics at Stuttgart University.

* The old "ESPResSo license" was removed, as it was not really a
  license, but only contained additions that were either non-binding
  or already contained in the GPL.

* The license was upgraded to GPLv3.

* The package was adapted to GNU standards. RELEASE_NOTES was moved
  aside to old/RELEASE_NOTES, instead, we will have the files
  ChangeLog and NEWS. NEWS will contain a description of the most
  noteworthy changes since the last release.

Changes visible for developers
------------------------------

* The source code repository has been moved from CVS to git, and it
  has moved to GNU Savannah (see above) and github. The main
  development code repository is

    https://github.com/espressomd/espresso

* We have an automated build server (Jenkins) at

    http://espressomd.org/jenkins/

  that automatically builds and checks the ESPResSo package whenever
  new code is pushed to the main development repository.  Every night,
  a number of additional tests are performed

* The .c/.h/.cu source files have been moved to the subdirectory
  src/. The "Espresso" binary is still placed in the top-level build
  directory. myconfig.h may be placed eitther in the src/ or top-level
  dirs.

* The automatically generated files configure and Makefile.in are not
  part of the repository anymore.  A developer can generate them using
  the command "bootstrap.sh", which requires the GNU autotools
  (autoconf and automake).

* All functions that represent the interface to Tcl should follow the
  same naming conventions. These are described here:

    git log 3d4d6f31655a8d96e47c8f80e10f27f6b764f8df

* Adding a new MPI function to communication.c has been significantly
  simplified. It is enough to add the function in the C file at a
  single location.

========================================================================
For older changes to ESPResSo, see ./old/RELEASE_NOTES.

========================================================================
Copyright (C) 2010-2022 The ESPResSo project

Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved.  This file is offered as-is,
without any warranty.
