Articles on ruhi's Blog

Post 14: Final Report and Recollections

ms20110@iisermohali.ac.in (ruhi) — Mon, 28 Aug 2023 17:52:50 +0000

I have finalized my report here:

https://gist.github.com/RuhiRG/9a33441072660bd62b987f77b9d88367

I also submitted my final google form.

Since the semester has begun in earnest I haven't had much free time for FOSS work. I do have plans for the winter and other vacations. I thought I'd wrap up with some scattered thoughts on what the project was (from my running notes throughout the coding period which never made it into a full blog before).

By offering Python bindings through pyseams, d-SEAMS is now accessible to a wider audience, including researchers who are more familiar with Python than C++ or Lua. This will potentially speed up the analysis of complex ice nucleation and solvated chemistry simulation and analysis.
I found working with C++ to be quite challenging initially. Getting my head around its syntax and idiosyncrasies took some time, but it gave me a new appreciation for type safety and memory management.
The project is closely aligned with my academic focus on computational biology, and I would love to work on the scientific uses more.

It would be an honor to work on a GSoC project under the PSF next year as well. It has been a pleasure.

Week 13: Addenum and Capstone

ms20110@iisermohali.ac.in (ruhi) — Wed, 23 Aug 2023 18:38:24 +0000

Overview

This is my last week working on d-SEAMS and my freshly minted `pyseams` project as a GSoC student for 2023. I have been working on polishing up my PRs so they can be merged and finalizing my work product. It has been a pleasure to share my progress with the community each week. Moreover, I've been discussing the design of `d-SEAMS`, which my mentor tells me is a good sign of taking responsibility. This has been my first project in the public eye, and it has truly been a pleasure..

For my final report, I opted for a Github Gist:

gsoc23_pyseams_fin_report.md

I plan to keep revising this for a while now, with inputs from my mentor as well.

Issues Opened

- RFC: Removing lua bindings

PRs Merged

- BLD: Work on MacOS, use as a subproject

week 12: wrapping up

ms20110@iisermohali.ac.in (ruhi) — Wed, 16 Aug 2023 17:55:11 +0000

Initially, the plan was to write the python binding along side of lua so that we could use both. while, doing the python bindings, we (me and my mentors) realized that the lua pointer to objects function design led to very awkward (not pythonic) bindings. To this end I worked on refactoring the functions inside seams-core itself, though this in-turn broke the lua interface which did not support constructing C++ objects from lua (and by extension could not use the new value return-type functions). So, this week as discussed with my mentors, we removed the lua interface from the C++ command line interface.

At one point in an earlier week I had floated the idea of eventually embedding the python interpreter into the C++ CLI instead of the lua interpreter and using the new bindings, however, the development team felt that this is a cleaner approach. The C++ CLI remains capable of reproducing the older D-SEAMS results, and a major release will clarify the role of the C++ core as a backend engine with Python bindings.

Check-in Questions

What did you do this week?

Wrote the last of example function binding chillPlus.
Wrote code to call it directly from C++ and python without any involvement of lua.
The problem with installation of dSEAMS in Macos is fixed.

What is coming up next?

Clear documentation for the whole work.
Final report which will also detail my next steps for collaboration.

Did you get stuck anywhere?

No, I wasn't stuck anywhere.

week 11: Bulk systems and 2-D ice analysis

ms20110@iisermohali.ac.in (ruhi) — Wed, 09 Aug 2023 23:18:50 +0000

This week, I was testing more in-built examples from seams-core by calling it from python. Unfortunately, I ran into a bug while writing the pybind for bulkTopologicalCriterion example. I raised an issue about it.

Check-in Questions

What did you do this week?

Wrote working pybinds for topo2d , BPring and bulkTopologicalCriterion in-built examples.
Raised an issue for a bug in bulkTopologicalCriterion example.

What is coming up next?

Better documentation of pyseams
Visualising workflows.
Look into replacing the lua interpreter in the C++ CLI with pybind11.

Did you get stuck anywhere?

No, I wasn't stuck anywhere.

week 10: Testing examples with pybind

ms20110@iisermohali.ac.in (ruhi) — Wed, 02 Aug 2023 21:48:48 +0000

This week, I was testing in-built examples from seams-core by calling it from python. I was also working on updating the new installation methods for seams-core with meson and building the executable with lua and meson.

Check-in Questions

What did you do this week?

Ran an in-built example from seams-core from python and compared to the lua implementation.
Made changes in the README.md file for installing updates.
Updated the environment.yml and meson.build file for version updates.
Started working on atomic visualizations.

What is coming up next?

Adding plots.
Better documentation of pyseams
Visualizing workflows.
Adding pytests.
Continuing the CI test to see if the results are same.
Doing the rest of the two workflows.

Did you get stuck anywhere?

No, I wasn't stuck anywhere.

week 9: Basic testing

ms20110@iisermohali.ac.in (ruhi) — Wed, 26 Jul 2023 19:57:15 +0000

This week, as my mentor suggested, I was working on testing pyseams on linux in the cloud with the help of github actions. While doing that it came to my attention that the function named "keepAxialRingsOnly" is unused and not defined in any file but it is declared in "topo_one_dim.hpp". I created an issue about it here

Check-in Questions

What did you do this week?

Ran a basic test in github for pyseams.
Created an issue for an unsued function.
Started working on atomic visualizations
Was reading the d-SEAMS paper to write the lua workflow

What is coming up next?

Running the dSEAMS workflow in python
Visualizing it.
Adding pytests.

Did you get stuck anywhere?

No, I wasn't stuck anywhere. My meetings with my mentors are regular and frequent which helps me to clear my doubt ASAP.

week 8: More python binding for functions

ms20110@iisermohali.ac.in (ruhi) — Mon, 17 Jul 2023 23:44:54 +0000

This week I wrote function bindings for functions in the namespaces sinp, nneigh, bond, primitive, ring, tum3 and gen. I made some commits as well over in pyseams. I worked a little bit on string representation for users while using the dSEAMS in python, it looks like this but this would be confusing for the users as the representation was <build.cyoda.pointdouble 0x7fbab9a75a30="" at="" object=""> </build.cyoda.pointdouble>which is terrible in lists.

So, we changed it to <pointdouble mem_loc:1eb6b20=""></pointdouble> by using this:

.def("__repr__", [](const molSys::Point<double> &a) {
std::uintptr_t ptr_val = std::uintptr_t(&a);
return fmt::format("<pointdouble mem_loc:="">"</pointdouble>, static_cast<uint>(ptr_val));
.def("__str__", [](const molSys::Point<double> &a) {
return fmt::format(
"x: {} y: {} z: {} type: {} molID: {} atomID: {} inSlice: {}",
a.x,
a.y,
a.z,
a.type,
a.molID,
a.atomID,
a.inSlice);
});</uint>

this looks more clear and specific. It also allows for users to get the memory location fairly easily if needed with aa.pts[0].split(':')[:-1].

Since we implemented __str__ as well, by using the following command, they can get more information by calling str() on a PointDouble.

However we might change this to expose a print() or info function instead.

Check-in Questions

What did you do this week?

python bindings for functions. The following are the commits made in response to that.
1. commit 1.
2. commit 2.
3. commit 3.
4. commit 4.
5. commit 5.
6. commit 6.
7. commit 7.
What is coming up next?
1. Writing python scripts
2. Writing pytests
3. More repr and str work
Did you get stuck anywhere?

No, I didn't get stuck anywhere.

Week 7: Admin, Python distribution, testing

ms20110@iisermohali.ac.in (ruhi) — Thu, 13 Jul 2023 22:56:11 +0000

Overview

This week I was rather unwell. However, I spent my time going over python packaging rules for eventual distribution via PIP, and I looked into alternate ways to fill in the core `yodaStruct` object because of segfaults in d-SEAMS. I am splitting my time now between the C++ core (with my mentor mostly) and the Python (pure and Pybind11) bindings.

Check-in Questions

What did you do this week?

- Moved `pyseams` into the d-SEAMS organization

- Looked into `ase` atoms and how to read an `xyz` in Python to populate a`yodaStruct`

- Discussed the (re-)design of the readers in D-SEAMS for the bindings

- Started writing some basic `pytest` fixtures, since the data needed will have to be shared across tests

What is coming up next?

- More re-writing of the C++ core

- Adding more tests and continuous integration (I was asked to look into this)

- Understanding more of `ase` and how the Atoms class maps to the d-SEAMS yodaStruct

Did you get stuck anywhere?

No, I was not well, and couldn't raise any PRs this week, but I'm on track, and have discussed the next steps with my mentors.

Week 6: Pointers and Pybind11

ms20110@iisermohali.ac.in (ruhi) — Wed, 05 Jul 2023 23:56:17 +0000

Overview

This week I spent a lot of time on debugging issues with pointers in d-SEAMS. As noted on the issue I opened and discussions with the core-team, some refactoring is required which I began working on in tandem.

Check-in Questions

What did you do this week?

Worked on binding the functions for input in d-SEAMS
Opened an issue to discuss issues with the code: https://github.com/d-SEAMS/seams-core/issues/35
Added bindings and cleaned up https://github.com/RuhiRG/pyseams for migration.
Opened a pull request to seames-core : https://github.com/d-SEAMS/seams-core/pull/36

What is coming up next?

More fixing of the d-SEAMS core
Add bindings and fix build
Setting up an issue with the order in which I will implement the bindings

Did you get stuck anywhere?

I was able to meet my mentor and the project organisers. Pointers and their usage are a bit confusing to me so far.

week 5: Core compound type bindings

ms20110@iisermohali.ac.in (ruhi) — Wed, 28 Jun 2023 23:32:31 +0000

Overview

This week I started by making a github repository named pyseams. Me and my mentor worked together on writing a meson build file. After this, I realised that dSEAMS had some enums which had to be converted to enum class to prevent the implicit conversion to other types i.e., to another enum or int and made a pull request for the same thing. Then I proceeded with writing bindings using pybind11, tested it locally, and committed to the git repo pyseams.

// The following bindings are still in progress:

#include <pybind11/pybind11.h>
#include "../subprojects/seams-core/src/include/internal/mol_sys.hpp"

namespace py = pybind11;

PYBIND11_MODULE(cyoda, m) {
  py::class_<molSys::Point<double>>(m, "PointDouble")
          .def(py::init<>())
          .def_readwrite("c_type", &molSys::Point<double>::type)
          .def_readwrite("molID", &molSys::Point<double>::molID)
          .def_readwrite("atomID", &molSys::Point<double>::atomID)
          .def_readwrite("x", &molSys::Point<double>::x)
          .def_readwrite("y", &molSys::Point<double>::y)
          .def_readwrite("z", &molSys::Point<double>::z);

  py::enum_<molSys::bond_type>(m, "BondType")
          .value("staggered", molSys::bond_type::staggered)
          .value("eclipsed", molSys::bond_type::eclipsed)
          .value("out_of_range", molSys::bond_type::out_of_range);

  py::enum_<molSys::atom_state_type >(m, "AtomStateType")
          .value("cubic", molSys::atom_state_type ::cubic)
          .value("hexagonal", molSys::atom_state_type ::hexagonal)
          .value("water", molSys::atom_state_type ::water)
          .value("interfacial", molSys::atom_state_type ::interfacial)
          .value("clathrate", molSys::atom_state_type ::clathrate)
          .value("interClathrate", molSys::atom_state_type ::interClathrate)
          .value("unclassified", molSys::atom_state_type ::unclassified)
          .value("reCubic", molSys::atom_state_type ::reCubic)
          .value("reHex", molSys::atom_state_type ::reHex);

  py::class_<molSys::Result>(m, "Result")
          .def(py::init<>())
          .def_readwrite("classifier", &molSys::Result::classifier)
          .def_readwrite("c_value", &molSys::Result::c_value);
}

Check-in Questions

What did you do this week?

Made a Github repository named pyseams.
Pull request for enum to enum class change.
Made a meson build file.
Started compiling the extensions and running the python bindings.

What is coming up next?

Finishing the python bindings for the first workflow in the lua design as discussed earlier.
Writing documentation and tests
Moving the pyseams repository to the dseams-core.

Did you get stuck anywhere?

I was a bit stuck at the meson build file process, but my mentor helped me with it. I think I am on track.

Week 4: Local builds and Python API design

ms20110@iisermohali.ac.in (ruhi) — Thu, 22 Jun 2023 03:44:01 +0000

Overview

This week I was setting up d-seams in my Mac, but that did not seem like a great idea as some requirements for d-seams were not supported on the M1 Mac (I opened an issue and PR regarding this). I had to figure out a way to run it on a remote linux machine.

I was also working with how the code design would look like.

import pyseams

# Configuration Variables
cutoffRadius = 3.5
oxygenAtomType = 2
hydrogenAtomType = 1
targetFrame = 1
finalFrame = 1
frameGap = 1
maxDepth = 6
isSlice = False
sliceLowerLimits = [0, 0, 0]
sliceUpperLimits = [0, 0, 0]
outDir = "runOne/"
trajectory = "input/traj/exampleTraj.lammpstrj"
lowestAtomID = 0

print("\nWelcome to the Ice Type Determination Module\n")

resCloud = None  # Initialize resCloud variable

for frame in range(targetFrame, finalFrame + 1, frameGap):
    resCloud = pyseams.readFrameOnlyOne(trajectory, frame, resCloud, oxygenAtomType, isSlice, sliceLowerLimits, sliceUpperLimits)
    nList = pyseams.neighborList(cutoffRadius, resCloud, oxygenAtomType)
    hbnList = pyseams.getHbondNetwork(trajectory, resCloud, nList, frame, hydrogenAtomType)
    hbnList = pyseams.bondNetworkByIndex(resCloud, hbnList)
    rings = pyseams.getPrimitiveRings(hbnList, maxDepth)
    pyseams.prismAnalysis(outDir, rings, hbnList, resCloud, maxDepth, lowestAtomID, targetFrame, frame, False)

print("Ice Type Determination completed.")

This is just a rough draft. I will be working on it to make it more perfectly.

Check-in Questions

What did you do this week?

build issues in GitHub.
did a pull request on GitHub.
worked a little bit on pybind design.

What is coming up next?

I will make a repository with python bindings which will be linked with d-seams as a shared library.
d-seams and python binding uses cmake. I am not that comfortable with cmake so I was planning to use meson for the python bindings. Meson supports pybind11 as mentioned here
I will also work on the python design a bit more.

Did you get stuck anywhere?

I was stuck as my M1-macos machine didn't support certain requirements of d-seams which set me back a little bit. I had to work on a linux machine. My mentor helped me out with the process.

Week 3: Embedding and automated bindings

ms20110@iisermohali.ac.in (ruhi) — Thu, 15 Jun 2023 18:18:45 +0000

Overview

Since last week, I was going over the Pybind11 documentation and also looking into automated binding generators. I have been mulling over what it means to provide bindings and how the interface is going to change. I tested out cppyy and its examples, but my mentor suggested that it may not be the right fit for the project as it is unlikely to be able to handle templates without more involved bindings.

However, as I began setting up a new repository for the bindings, it occurred to me that for backwards compatibility, though providing a compiled extension will be useful for more users, switching over from Lua really needs an embedding of the Python interpreter! I am fairly sure of this, as the binary (from last week's main.cpp) essentially copies into C++ data from Lua. Embedding is part of Pybind11's feature-set:

// Starts the interpreter and keeps it alive
py::scoped_interpreter guard{};

However, there's a catch, we need both the bindings (to run seams-core) from python so we can transfer data into the C++ in a robust manner! This means my timeline is a little altered, I will need to write the compiled extension, and then interface that to the main.cpp. So:

from pyseams import PointCloud

pinp = PointCloud(...)

Which will then be read in on the C++ side by the embedded interpreter. Sort of like the existing examples:

print("\n Welcome to the manual lua function evaluation environment.\n");

for frame=targetFrame,finalFrame,frameGap do
   resCloud=readFrame(trajectory,frame,resCloud,oxygenAtomType,isSlice,sliceLowerLimits,sliceUpperLimits) --- Get the frame
   nList=neighborList(cutoffRadius, resCloud, oxygenAtomType); --- Calculate the neighborlist by ID
   ---
   resCloud=chillPlus_cij(resCloud,nList,isSlice); --- Calculate Cij (cloud,slice)
   resCloud=chillPlus_iceType(resCloud,nList,outDir,targetFrame,isSlice,chillPlus_noMod); --- Write out data (cloud,slice,name)
   writeDump(resCloud,outDir,dumpChillP); --- Dump the rescloud which currently has CHILL Plus classifications
   --- Modified CHILL+
   avgQ6=averageQ6(resCloud,nList,isSlice); --- Average Q6 (cloud,slice)
   resCloud=modifyChill(resCloud,avgQ6); --- Modification (cloud,q6)
   percentage_Ice(resCloud,outDir,targetFrame,isSlice,chillPlus_mod); --- Post reclassification writeOut
   writeDump(resCloud,outDir,dumpSupaaP); --- Dump the rescloud which now has the supaa CHILL Plus Trajectory
   --- Get the largest ice cluster. Here, iceNeighbourList is the neighbour list by index.
   clusterAnalysis(outDir, clusterCloud, resCloud, nList, iceNeighbourList, cutoffRadius, targetFrame, "q6");
   --- Recenter the cluster such that the centroid is at the center of the simulation box
   recenterCluster(clusterCloud, iceNeighbourList);
   writeDump(resCloud,outDir,largestClusterDump); --- Dump the recentered largest ice cluster
end

This is a tentative design which I will discuss with my mentor this week.

Check-in Questions

What did you do this week?

Looked into automated bindings (cppyy)
Realized I need to embed an interpreter for backwards compatibility
Started designing the classes based on pypotlib
What is coming up next?
I'm still planning on that personal site
I will publicly post my repo for the python-bindings
- My mentor will move it into the dseams workspace
- This will be my first time working in an organization on Github
I will start a design document
I will go over the C-Python documentation to understand the automated / manual bindings system in a bit more detail
- My mentor pointed me to his post on interoperability between Python C and Fortran for some background information

The goal is to have working code examples by my midway evaluation as per the original plan.

Did you get stuck anywhere?

The embedding of the interpreter was something which I hadn't considered, not even in my proposal
I was interested (and a bit caught up on) how automatic bindings work
I am having some trouble (the "tyranny of a blank page") getting started with runnable code
- Especially when I realized I cannot run the exact code (main.cpp) until after the bindings are complete!
I was having trouble formatting my posts on the PSF GSoC blogroll
- Figured out I can use this site (or pandoc)

Week 2: Into the Lua Bindings

ms20110@iisermohali.ac.in (ruhi) — Wed, 07 Jun 2023 21:58:38 +0000

Overview

I have been going over the source code of d-SEAMS and puzzling out the minimal required objects and functions. It seems that for the `lua` bindings, the objects were mostly constructed in `C++` and then populated in `lua`, however, for the `python` bindings each class will need to have separate bindings. I have narrowed my focus to a single workflow, not the entire driver (which is controlled by the `yaml` file), that is as discussed in the paper (from here):

```cpp
if (config["topoTwoDim"]["use"].as<bool>()) {
// Use the variables script
lua.script_file(vars);
// -----------------
// Variables which must be declared in C++
//
// Newer pointCloud (rescloud -> ice structure, solcloud -> largest cluster)
molSys::PointCloud<molSys::Point<double>, double> resCloud;
// Some neighbor lists
std::vector<std::vector<int>> nList, hbnList;
// For the list of all rings (of all sizes)
std::vector<std::vector<int>> ringsAllSizes;
std::vector<std::vector<int>> rings;
// RDF stuff
std::vector<double> rdfValues; // RDF vector
// -----------------
// This section basically only registers functions and handles the rest in
// lua Use the functions defined here
auto lscript = lua.get<std::string>("functionScript");
// Transfer variables to lua
lua["doBOP"] = config["bulk"]["use"].as<bool>();
lua["topoOneDim"] = config["topoOneDim"]["use"].as<bool>();
lua["topoTwoDim"] = config["topoTwoDim"]["use"].as<bool>();
lua["topoBulk"] = config["bulk"]["use"].as<bool>();
//
lua["nList"] = &nList;
lua["hbnList"] = &hbnList;
lua["resCloud"] = &resCloud;
lua["trajectory"] = tFile;
// Confined ice stuff
lua["ringsAllSizes"] = &rings;
// RDF stuff
lua["rdf"] = &rdfValues;
// -----------------
// Register functions
//
// Writing stuff
// Generic requirements
lua.set_function("readFrameOnlyOne", sinp::readLammpsTrjreduced);
lua.set_function("readFrameOnlyOneAllAtoms", sinp::readLammpsTrj); // reads in all atoms regardless of type
lua.set_function("neighborList", nneigh::neighListO);
// -----------------
// Topological Network Method Specific Functions
// Generic requirements (read in only inside the slice)
lua.set_function("getHbondNetwork", bond::populateHbonds);
lua.set_function("bondNetworkByIndex", nneigh::neighbourListByIndex);
// -----------------
// Primitive rings
lua.set_function("getPrimitiveRings", primitive::ringNetwork);
// -----------------
// Quasi-two-dimensional ice
lua.set_function("ringAnalysis", ring::polygonRingAnalysis);
// --------------------------
// RDF functions
lua.set_function("calcRDF", rdf2::rdf2Danalysis_AA);
// --------------------------
// Use the script
lua.script_file(lscript);
// --------------------------

} // end of two-dimensional ice block
```

It is clear that the bindings might be a mite more complicated in `python`, especially as I realized many of the core structures are templated.

Check-in Questions

What did you do this week?

Went over the `d-seams` documentation and examples locally
- Settled on a workflow to bind
- Looked into the classes and functions needed
Went over the `sol3` documentation on the `lua` bindings

What is coming up next?

1. I intend to have a personal site with more details of the internals
2. I shall start designing the code and determine a style guide with my mentor
3. I will finish enumerating the functions and objects needed to be bound for this workflow

Did you get stuck anywhere?

I was a bit confused about the manner in which memory was managed between `lua` and `cpp` and how it would change with the `python` bindings. I hope to clarify this further, and have a minimal toy setup on my machine to work with when I meet my mentor later this week.

Community Bonding and Week 1

ms20110@iisermohali.ac.in (ruhi) — Thu, 01 Jun 2023 12:09:33 +0000

Overview

As this post is at the start of the first week of the coding period, there haven't been any concrete pull requests, though over the course of the community bonding period I pointed towards a project called pypotlib (GH source) as an example of the split binding / core library design I will be implementing.

Check-in Questions

What did you do this week?

1. Went over the pybind11 documentation and examples locally

2. Looked into style and design guides for PyPI a bit

What is coming up next?

1. Local setup

2. Setting up an issue with the order in which I will implement the bindings

Did you get stuck anywhere?

I was able to meet my mentor and the project organisers. I think I am on track and do not feel stuck so far.