More thorough unit tests for non-Parmed writers

[summeraz]

Current unit tests for writers that don't go through Parmed (lammps, hoomdxml, and soon gsd) test only that files are written without error. More thorough tests that check to make sure the expected sections are present (and contain the expected number of atoms, bonds, etc.) should probably be included.

[summeraz]

This issue has been partially addressed, but we could still use more thorough unit testing. I'm thinking it would be nice to have some single point energy tests to make sure we're getting consistent results from all three writers. We could maybe get some inspiration from unit tests used by ParmEd and Intermol.

[ahy3nz]

Just curious, what kind of energy tests were you thinking? Since we're constructing compounds in 'code' and then writing them for a particular MD engine, it's not like we can compare before and after. If it were converting from MD engine to MD engine, we'd have the same system with the same force field, but just converted to fit the MD engine's form. If this were an mBuild compound, it's not like we could evaluate the bond energies of an mB compound (no force field) and compare it to the output gsd/hoomdxl/gro/lammps compound's bond energies (with a force field applied)

[summeraz]

I was thinking something along the lines of what I did locally to test the GSD writer, which would basically be:

1. Create an mBuild compound
2. Save to .lammps, .hoomdxml, and .gsd formats (applying a force field)
3. Read these structures into their respective simulation engines (LAMMPS or HOOMD)
4. Calculate various energy terms (total potential energy, LJ, bond, etc.)
5. Check to make sure the results from each of these engines agree.

We could even check against some of the ParmEd writers as well.

We would probably want to check a few different systems with different force fields.

One problem with this approach would be figuring out how to get the tests to run on Travis because we would need to access the LAMMPS binary (and GROMACS or whatever other engines from the ParmEd writers we want to test). We might be able to create a LAMMPS package within our conda channel similar to how omnia hosts several PACKMOL versions, but I'm not sure about that.

[summeraz]

These would not be exclusively mBuild tests, but tests of the full workflow of mBuild + Foyer + either ParmEd or mBuild for writing. However, I think these fit better within mBuild then Foyer.

[ctk3b]

Not sure if these are mature enough yet but eventually I think this will be the way to go:
https://docs.travis-ci.com/user/docker/
https://github.com/cb-geo/lammps-container
https://hub.docker.com/r/gromacs/gromacs/

[ctk3b]

Also for actually evaluating energies, these scripts may be a good starting point:
https://github.com/ctk3b/validate/blob/master/validate/gromacs.py

MolSSI is also working on a fully fledged replacement for InterMol/the conversion part of ParmEd. Don't think they're quite there yet but they're building off of some of that code:
https://github.com/MolSSI/EEX
https://github.com/MolSSI/EEX/blob/master/eex/translators/lammps/lammps_utility.py#L42

[ctk3b]

And last but not least: https://www.biorxiv.org/content/biorxiv/early/2018/04/22/306043.full.pdf

[chrisiacovella]

I think comparing identical, static structures is a good idea (as opposed to comparing the averages from MD runs and doing the "physical validation" stuff...that is something we should think about long term though).

In reality, if the goal is to just test the writers, we could feed it any arbitrary set of particles/parameters to make sure things are being parsed correctly. For these types of tests, we should actually think about doing something similar to how we derive forcefield parameters. E.g., calculate the tosional energy landscape by spinning a molecule around a bond or stretch a bond out and see how the various components to the energy change. That is, not rely on a single configuration, but also look at a series of configurations to ensure the trends are as expected.

I definitely agree with Christoph about using docker for the simulation engines.

[summeraz]

👍 to the idea of using Docker for LAMMPS and Gromacs. I still don't know much about how to actually use Docker, so this might be a good time to get started.

I'll have to look over those packages you linked @ctk3b . At first glance, they appear to maybe be a bit more involved than we need for just single point energy calculations, but if this is something we can take advantage of then no need to reinvent the wheel. The Gromacs Python API is something I've been meaning to check out for awhile, so I'll look into that, could be useful here.

And right, @chrisiacovella, I wasn't thinking of anything as involved as the whole physical validation package. Just some single point energies to validate that for an atomtyped compound (really a parametrized ParmEd Structure) that the topology has been correctly defined in the simulation data file.

[ctk3b]

I think comparing identical, static structures is a good idea

As long as you're running tests on structures with more than a handful of atoms, this is the only test required to validate your writers. If you want to be extra sure, you can readily achieve this:

look at a series of configurations to ensure the trends are as expected

by building a system that has a handful of molecules and running a few steps of equilibration to get a sampling of different realistic configurations.

a bit more involved than we need for just single point energy calculations

Yeah I don't think you need to use most of the packages but I would advocate against re-inventing the wheel that "runs a single point energy calc and parses the output into a canonical form". That being said, none of these packages thus far provide scripts that cover LAMMPS, HOOMD and GROMACS which is what you're probably after.

Might make sense to start a little utility library that simply lets you run evaluate_energy(structure, engine='HOOMD')

^ this is what we recommended to the EEX developers as well but I don't think they'll get to all the packages you're interested in in the near future.

[summeraz]

Might make sense to start a little utility library that simply lets you run evaluate_energy(structure, engine='HOOMD')

👍 I like this idea for an approach. And then we can basically just parametrize a unit test to look at different structures and engines.

[mattwthompson]

Just wanted to add that LAMMPS is available on conda-forge and bioconda maintains a version of GROMACS (not that this conda is necessarily an easy way to go for this problem).

https://anaconda.org/conda-forge/lammps
https://anaconda.org/bioconda/gromacs

[CalCraven]

As we're moving away from ParmED, the writers here become less relevant.