unlocking Black Hole Secrets with AI and Distributed Computing: A New Era in Astrophysical research
The quest to understand the universe’s most enigmatic objects - black holes – has entered a new phase, powered by a groundbreaking combination of artificial intelligence (AI) and a revolutionary distributed computing system called Livny. This innovative approach is not only refining our understanding of black holes like Sagittarius A at the centre of our Milky Way, but also demonstrating the immense potential of large-scale data analysis in modern astrophysics.
For years,the Event Horizon Telescope (EHT) Collaboration captivated the world with the first-ever images of black holes,first M87 in 2019 and then Sagittarius A in 2022.However, these images represent just the tip of the iceberg. The underlying data held a wealth of information, locked away in its complexity. Extracting this hidden knowledge required a new paradigm – one that could handle the sheer volume of data and the nuanced uncertainties inherent in astrophysical modeling.
Livny: The Engine Behind the Breakthrough
Enter Livny, a novel distributed computing framework developed by the Center for High Throughput Computing (CHTC) at the University of Wisconsin-Madison. Livny doesn’t rely on a single, monolithic supercomputer. Rather, it intelligently distributes computing tasks across a network of thousands of computers, effectively transforming a massive challenge into a fleet of smaller, manageable ones. This approach is proving invaluable across a wide range of scientific disciplines,from searching for cosmic neutrinos and subatomic particles to tackling the growing crisis of antibiotic resistance.
The power of Livny was crucial to a recent project funded by the National Science foundation (NSF) through the Partnership to Advance Throughput computing (PATh) project. Previously, EHT Collaboration studies where limited by the availability of only a handful of realistic synthetic data files. Livny enabled researchers to dramatically expand this dataset, feeding millions of these files into a complex Bayesian neural network. This network, capable of quantifying uncertainties, allowed for a far more rigorous comparison between the EHT observations and theoretical models.
New Insights into sagittarius A – and a Challenge to Existing Theory
the results are compelling. The AI-driven analysis suggests that Sagittarius A, the black hole at the heart of our galaxy, is spinning at nearly the maximum possible rate, with its rotational axis pointing directly towards Earth. Furthermore,the research indicates that the bright emission surrounding the black hole is primarily generated by extremely hot electrons within the accretion disk - the swirling mass of gas and dust falling into the black hole – rather than from a powerful jet of particles.
Perhaps most substantially, the analysis reveals that the magnetic fields within the accretion disk behave in a way that challenges current theoretical models.”That we are defying the prevailing theory is of course exciting,” explains Dr. Michael Janssen, lead researcher from Radboud University Nijmegen in the Netherlands. “However, I see our AI and machine learning approach primarily as a first step. Next, we will improve and extend the associated models and simulations.”
Scaling to Millions: A Testament to throughput computing
The success of this project hinges on Livny’s ability to scale. “The ability to scale up to the millions of synthetic data files required to train the model is an impressive achievement,” notes Dr. Chi-kwan Chan, an Associate Astronomer at the University of Arizona and a long-time PATh collaborator.”It requires dependable workflow automation, and effective workload distribution across storage resources and processing capacity.”
Professor Anthony Gitter, a Morgridge Investigator and PATh Co-PI, emphasizes the broader impact: “We are pleased to see EHT leveraging our throughput computing capabilities to bring the power of AI to their science. Like in other science domains, CHTC’s capabilities allowed EHT researchers to assemble the quantity and quality of AI-ready data needed to train effective models that facilitate scientific revelation.”
Livny’s infrastructure, powered by the NSF-funded Open Science Pool and contributions from over 80 institutions across the United States, has already processed over 12 million computing jobs for the Event Horizon black hole project in the last three years. As Livny’s director, Dr. Miron Livny, states, “A workload that consists of millions of simulations is a perfect match for our throughput-oriented capabilities that were developed and refined over four decades. We love to collaborate with researchers who have workloads that challenge the scalability of our services.”
Looking Ahead: The Future of Astrophysical Discovery
This research marks a pivotal moment in astrophysics. The combination of advanced AI techniques, powered by the scalable infrastructure of Livny, is opening up new avenues for exploring the universe’s most complex phenomena. The findings are detailed in a series of papers published in Astronomy & Astrophysics* (Janssen
