~/karan.sh
I am a PhD candidate in the Machine Learning for Materials Design group at the Center for Advanced Systems Understanding
(CASUS), Helmholtz-Zentrum Dresden-Rossendorf (HZDR) in Görlitz, Germany, under the supervision of Dr. Attila
Cangi.
I work on accelerating quantum dynamics simulations through machine learning. My research goal is to develop
Density Functional Theory (DFT) based simulation frameworks for electron dynamics for matter under extreme conditions.
I like to think about the promises and pitfalls of machine learning for science. This web page serves as my extended CV. I can elaborate on the details without the space constraints. You can reach out to me at k.shah@hzdr.de.
CV: Link to my pdf CV (updated April 2024)
Update Apr 2023: Attended an Open Science Retreat. We studied the impact of LLMs for science. You can access the document here and on GitHub.
Update Feb 2023: Announcing a new nuptial collaboration with my wonderful Co-PI. More information here: Wedding Invite
Update Jan 2023: I have been selected once again as a APS DSECOP Fellow.
Update Oct 2022: Our work "Physics-Informed Neural Networks as Solvers for the Time-Dependent Schrödinger Equation" was accepted to the Machine Learning and the Physical Sciences workshop at NeurIPS 2022. Links: arXiv:2210.12522
Update Aug 2022: I am now part of the Helmholtz AI funded project "SynRap – Machine-learning based synthetic data generation for rapid physics modeling"
Update Apr 2022: Our work "Deep dive into machine learning density functional theory for materials science and chemistry" was published in Physical Review Materials. Links: Phys. Rev. Materials, arXiv:2110.00997
Update Feb 2022: I received APS DSECOP Fellowship to develop data science/machine learning materials for the undergraduate physics curriculum.
Update Aug 2021: I moved to Germany to begin my PhD at CASUS
Update Dec 2020: Our work "Fourier Spectrum Discrepancies in Deep Network Generated Images" was accepted to NeurIPS 2020. Links: NeurIPS, arXiv:1911.06465
I am always excited to explore new fields. I am broadly interested and have worked in the following areas:
The intersection of scientific computing and machine learning is my favorite place to work in. I believe that machine learning techniques are a necessity for current and future data intensive experiments. ML techniques can also serve as surrogates for computationally expensive simulations. On the other hand, insights from physics could be helpful for interpretability and uncertainty quantification of ML models.
I worked at Lawrence Livermore National Laboratory (LLNL) under Dr. Michael Schneider during 2017-2019. Here is a overview of my projects:
I joined LLNL as a fellow in their Data Science Summer Institute (DSSI). I worked with Dr. Michael Schneider on a Hierarchical Bayesian Model to solve the inverse problem of inferring cosmic shear and intrinsic galaxy properties from data that will be collected by the Large Synoptic Survey Telescope (LSST). I used Probabilistic Graphical Models and wrote MCMC algorithms (Metropolis Hastings within Gibbs) to infer the posterior distribution of cosmic shear and intrinsic ellipticity (shape) from observed galaxies in the weak lensing regime. I am now working on an extended hierarchical model incorporating non-shape galaxy properties like flux, Sersic profile and half light radius along with ellipticity. Our end goal is to determine the multivariate population statistics, including correlations amongst different properties of all four billion galaxies in LSST. I gave a talk on this project in the Bay Area LSST & Machine Learning Meeting at UC Berkeley in December 2018.
Presentation, Short Class Report, Old LLNL Symposium Poster
Gaussian process for cosmological parameter estimation: It has been shown that certain gaussian process kernels are equivalent to infinitely wide deep neural networks while also providing robust uncertainty quantification. Under Dr. Schneider, I am also exploring the use of neural network equivalent Gaussian Processes (NNGPs) to estimate cosmological parameters (such as ratio of baryonic matter to dark matter) from observed 3D mass maps of galactic surveys. I am currently testing my pipeline on simulated toy models. The main challenges I face here are efficient representation of 3D volumes and improving Gaussian Process scalability.
I also briefly worked in the Gravity group at Georgia Tech, on Gaussian Processes as surrogates for expensive gravitational wave simulations.
During my undergrad, I worked in Otte group at GT on simulation code for large scale cosmic ray showers. I added functionality for creating new telescope models in GrOptics (specifically included Schwarzschild-Couder model for Chernekov Telescope Array). I also wrote scripts to run large scale HPC simulations.
I received my first President's Undergraduate Research Award (PURA) for this work.
GitHub Project Link (I created the GrOptics GitHub account and repo)
Really old: Here is a freshman year report that I am proud of. I observed the location of sunsets throughout the semester and explained the trends.
Link to report (115/100 points)
I worked with Professor Andrew Medford on deep learning approaches to density functional theory from 2017-2019. For my undergraduate thesis, I focused on the effectiveness of machine learning techniques in finding a unique mapping between the electron density and the exchange-correlation potential of a system. My current project is uncertainty quantification for machine learned surrogate functionals. I used bootstrap aggregation with an ensemble of weak learners (shallow neural nets) to perform uncertainty quantification. Concurrently, I am building a data management framework to make all stages of the machine learning workflow smoother on the Georgia Tech HPC clusters. I received my second President's Undergraduate Research Award (PURA) from Georgia Tech for this work in Fall 2017.
Undergraduate Thesis
For my Advanced Scientific Computing class, I worked on using neural network equivalent Gaussian Processes to solve the Schrödinger equation. I used it to solve the ground state energy for 2D electron systems. It works well with robust uncertainty quantification, but suffers from the scalability issues inherent to Gaussian Processes.
Link to project report: Gaussian Processes and the Schrodinger equation
Complimentary to my SciML projects, I am also interested in interpretable machine learning. I feel that the black box nature of current deep models is not ideal for scientific discovery. I am looking into robust uncertainty quantification, Bayesian methods, etc.
Along with collaborators from Texas A&M University, I recently submitted a paper to NeurIPS 2020, on the difference in Fourier spectrum of real and generated images. We demonstrated a low (computational) cost technique to detect synthetic images, which could also be used to interpret GANs and VAEs.
arXiv:1911.06465
In order to better understand the societal impacts of AI, I am filling the social science gaps in my knowledge. I am currently enrolled in the excellent MIT Micromasters in Data, Economics and Developmental Policy program. I am learning about the design of social experiments, which are sometimes trickier than the computational science experiments I am used to.
Previously, as a member of the Data Driven Education team, I worked on statistical methods to tailor GT’s pioneering online courses to student performance. In my senior year, I led a team of junior undergrads and we presented one such scheme at the Georgia Tech STEM Education Research Expo.
Poster
I have been fascinated with cellular automata ever since I encountered Conway’s game of life in freshman year. This lead to adventures in non-linear dynamics (chaos theory).
Currently, I am intrigued by neural cellular automata. I have been playing around with it while under lockdown. I will post about my experiments soon.
I took a great course on emergent phenomena with Prof. L. S. Schulman in Spring 2020. The class consisted of four students and the professor. I learnt about the relevance of statistical mechanics in a wide variety of fields (from finance to syncing fireflies). I gave a presentation of the emergence of Zipf’s law from simple assumptions about human settlements.
You can find the presentation here: Cities and Galaxies Note: It ends with aliens.
I worked on “predicting” chaotic systems as a project in my deep learning class. Inspired by this article and the linked papers, I explored the use of reservoir computing for predicting time series. It worked excellently for repeating series, but also worked eerily well for chaotic systems.
Draft here: Tiamath's Swimming Pool (plan on going beyond preliminary results some day, sigh)
I was fortunate to be accepted into Wolfram Summer School 2016. It was an intense experience with some of the brightest people I have met. I worked on automatic classification of cellular automata, under Dr. Georgia Fortuna, with some advice from Stephen Wolfram. I got inducted in the cult of Mathematica (feels so good when you solve a complicated problem with a paragraph long one liner, which feels great when you write it, but takes 20 minutes to decipher when you read it two years later) and functional programming.
Alumni Link, GitHub Repo
I combined cellular automata (Schelling's Segregation Model) and n-body simulation (Barnes-hut algorithm) to model population movement, taking into account multiple factors such as wealth, race etc.
GitHub Repo
For my electronics class, I made Chua’s circuit. It exhibits chaotic behavior. Looking at phase plots on a three decade old oscillator (because newer digital ones can't handle that) was the highlight of my course. I also wrote a small matlab simulator.
Here’s a short report:
Chua's Circuit: A chaotic analog computer (contains pretty phase plots)
I wrote a cellular automata simulator in Java for my computational physics class, complete with an Arduino implementation on rings and grids of LEDs.
GitHub repo, Report
In decreasing order of proficiency:
Python: Bread and butter. I use the “data-science stack” (numpy, scipy et al) everyday. I alternate between TF and PyTorch depending on the project. I experiment with code in Jupyter notebooks and convert finished code into modules (sometimes). I am very comfortable with python, but need to learn better testing and documentation practices.
Mathematica: I use it for hairy math problems. I also use it because it is always satisfying to get non trivial things done with a few function calls.
Matlab: Mostly use it for coursework that requires matlab.
Julia: More out of intellectual curiosity. It has an active scientific computing community and claims to combine the ease of python with the performance of C.
C, C++: Used it for GrOptics. Also for learning HPC (MPI) and my systems and networking classes. I can get things done after many Stack Overflow look ups.
Java: Language of choice when starting out my degree. Learn OOP on Java and C++. Can’t believe I wrote a GUI cellular automata simulator in it a long time ago. Won’t be able to do that today.
R: Trying it for my social science classes. It feels like a lower friction way to do statistics than python.
FORTRAN: Tried it for that one physics professor who won't use anything else. Don’t remember it at all now.
Shell Scripting: Writing good bash scripts is an under appreciated skill. These are helpful for managing multiple simulations and making sure ML pipelines work smoothly.
LaTeX: Documents look beautiful, once I figure out how to put that image below the paragraph I want.
Hardware: I am comfortable with basic electronics and soldering. I like tinkering with Arduino and Raspberry Pi boards. I recently setup a pi-hole for my home network. I also built my own PC for deep learning and gaming, christened ada.
Some HTML/CSS: Mostly to make personal websites, and to look for easter eggs on Mr. Robot and Westworld websites. Hugo and Jupyter and Markdown are easier for me.
I have been a teaching assistant in the following courses:
I believe that giving back is an important part of a career in science. As a lowly grad student, I try to volunteer wherever I can. Some of the highlights are:
I serve as a reviewer for President’s Undergraduate Research Award in the fields of physics and computer science.
I was also elected Senator to the Graduate Student Senate from the School of Computational Science & Engineering for 2018-2019 and 2019-2020.
In my youth (6 years ago in 2014), I experimented with a start up, cryptomen.com (the website is dead, we had some articles about us though). It was five of us scattered throughout North America, and we traded crypto currencies back when bitcoin was about $400. We raised about $47000 from individual investors. I learnt a lot about crypto currencies, dealing with people, marketing and trading stresses.
Dead domain, an article about us
I also cofounded Georgia Tech's first Cryptocurrency club around that time. We invited people in Atlanta's cryptocurrency scene to give talks at GT, left 3D printed wallets with 0.0000001 BTC at random spots around campus and just spread awareness about cryptocurrencies in general.
In high school, I made a Jarvis like voice automation system with a friend. This was 2010, before Siri and AWS and home automation were cool. The whole system was made out of junk parts. An ancient desktop with a parallel port was the brain. The parallel port pins were connected to relays which were driving the mains current to various appliances. We used the hilariously bad Windows speech recognition to manipulate some cmd scripts which turned the parallel pins on and off. As stupid teenagers, we had open mains lines on the floor. I am happy to be alive.
Embarrassing video
Feel free to submit anonymous feedback on this form.