The Worldwide LHC Computing Grid (WLCG), formerly (until 2006)[1] the LHC Computing Grid (LCG), is an international collaborative project that consists of a grid-based computer network infrastructure incorporating over 170 computing centers in 42 countries, as of 2017[update]. It was designed by CERN to handle the prodigious volume of data produced by Large Hadron Collider (LHC) experiments.[2][3]
Server racks of the Worldwide LHC Computing Grid in CERN
By 2012, data from over 300 trillion (3×1014) LHC proton-proton collisions had been analyzed,[4] and LHC collision data was being produced at approximately 25 petabytes per year. As of 2017[update] the LHC Computing Grid is the world's largest computing grid comprising over 170 computing facilities in a worldwide network across 42 countries[4][5][6] scattered around the world that produce a massive distributed computing infrastructure with about 1,000,000 CPU cores, providing more than 10,000 physicists around the world with near real-time access to the LHC data, and the power to process it.
According to the WLCG Website as of 2024: "WLCG combines about 1.4 million computer cores and 1.5 exabytes of storage from over 170 sites in 42 countries [...] It runs over 2 million tasks per day and [...] global transfer rates exceeded 260 GB/s." Indicating substantial upgrades to WLCG over time beyond its initial release.
Background
The Large Hadron Collider at CERN was designed to test the existence of the Higgs boson, an important but elusive piece of knowledge that had been sought by particle physicists for over 40 years. A very powerful particle accelerator was needed, because Higgs bosons might not be seen in lower energy experiments, and because vast numbers of collisions would need to be studied. Such a collider would also produce unprecedented quantities of collision data requiring analysis. Therefore, advanced computing facilities were needed to process the data.
Description
A design report was published in 2005.[7]
It was announced to be ready for data on 3 October 2008.[8]
A popular 2008 press article predicted "the internet could soon be made obsolete" by its technology.[9]
CERN had to publish its own articles trying to clear up the confusion.[10]
It incorporates both private fiber-optic cable links and existing high-speed portions of the public Internet. At the end of 2010, the Grid consisted of some 200,000 processing cores and 150 petabytes of disk space, distributed across 34 countries.[11]
The data stream from the detectors provides approximately 300 GByte/s of data, which after filtering for "interesting events", results in a data stream of about 300 MByte/s. The CERN computer center, considered "Tier 0" of the LHC Computing Grid, has a dedicated 10 Gbit/s connection to the counting room.
The project was expected to generate multiple TB of raw data and event summary data, which represents the output of calculations done by the CPU farm at the CERN data center. This data is sent out from CERN to thirteen Tier 1 academic institutions in Europe, Asia, and North America,[12] via dedicated links with 10 Gbit/s or higher of bandwidth. This is called the LHC Optical Private Network.[13]
More than 150 Tier 2 institutions are connected to the Tier 1 institutions by general-purpose national research and education networks.[14]
The data produced by the LHC on all of its distributed computing grid is expected to add up to 200 PB of data each year.[15] In total, the four main detectors at the LHC produced 13 petabytes of data in 2010.[11]
The Tier 1 institutions receive specific subsets of the raw data, for which they serve as a backup repository for CERN. They also perform reprocessing when recalibration is necessary.[14] The primary configuration for the computers used in the grid is based on CentOS.[16] In 2015, CERN switched away from Scientific Linux to CentOS.[16]
Distributed computing resources for analysis by end-user physicists are provided by multiple federations across the Europe, Asia Pacific and the Americas.
^Worldwide LHC Computing Grid about page 2017: "Even after filtering out 99% of it, in 2017 we're expecting to gather around 50 petabytes of data. That's 50 million gigabytes, the equivalent to nearly 15 million high-definition (HD) movies."
^Worldwide LHC Computing Grid main page 2017: "WLCG is the world's largest computing grid. It is supported by many associated national and international grids across the world, such as European Grid Initiative (Europe-based) and Open Science Grid (US-based), as well as many other regional grids."
