What is the Open Science Grid?
Open Science Grid: A User Introduction
This page is an introduction to the O
) from a user point-of-view. See also Information on joining and using OSG
and User Documentation
We start with the mission of the OSG:
The Open Science Grid (OSG) advances science through open distributed computing. The OSG is a multi-disciplinary partnership to federate local, regional, community and national cyberinfrastructures to meet the needs of research and academic communities at all scales.
) provides provide common service and support for resource providers and scientific institutions using a distributed fabric of high throughput computational services. The OSG does not own resources but provides software and services to users and resource providers alike to enable the opportunistic usage and sharing of resources. The OSG is jointly funded by the Department of Energy
and the National Science Foundation
High-throughput computing needs of our scientific users are being met by a large number of computing resources
that can be accessed directly through Globus
services or more conveniently through grid technologies build on top of them. Storage resources provide easy
access to terabytes of storage space through the common SRM
protocol. The resources contributing to the OSG are primarily Linux clusters with Ethernet interconnects.
Succeeding on the OSG
The OSG is primarily used as a high-throughput grid
where scientific problems are solved by breaking them down into a very large number of individual jobs that can run independently. The most successful opportunistic applications
run on the OSG share the following characteristics:
- The application is a Linux application for the x86 or x86_64 architecture.
- The application is single- or multi-threaded but does not require message passing.
- The application has a small runtime between 1 and 24 hours.
- The application can handle being unexpectedly killed, and restarted.
- The application is build from software that does not require to contact licensing servers.
- The scientific problem can be described as a workflow consisting of jobs of such kind.
- The scientific problem requires to run a very large number of small jobs rather than a few large jobs.
Other applications may succeed, but users may have difficult barriers to surmount. Some characteristics of applications that do not run well
opportunistically on the OSG are:
- The application requires message passing and a low-latency local network.
- The application requires a long runtime and cannot be checkpointed through signal TERM.
- The application workflow cannot tolerate preemptions or individual jobs crashing at remote sites.
- The application requires complex software and specific environments that cannot be easily met on resources.
- The application requires write access to a code base pre-installed at sites and shared among the worker nodes.
- The application requires administrator privileges for its operation.
A Virtual Organization
) is a set of groups or individuals defined by some common cyber-infrastructure need. This can be a scientific experiment
, a university campus
or a distributed research effort
. A VO represents all its members and their common needs in a grid environment. A VO also includes the group's computing/storage resources and services. We typically use the terms site, computing element (CE), and/or storage element (SE) to refer to the resources owned and operated by a VO.
OSG brings together many VOs for contracted and/or opportunistic sharing of resources in a grid environment, allowing for more effective use of their collective resources. Use of resources at a site is determined by a combination of the site's local policies and the user VO's policies. VOs are responsible for contracting individually with each other for guaranteed access to resources.
Many VOs in the OSG address the specific requirements of their users with their own user support that provides more in-depth help than the OSG does (current VO list
). Additionally some VOs provide dedicated resources to the OSG that provide preferred access to their members. Some VOs provide a problem specific user interface to their members.
The OSG assumes that each user is preferably supported by their membership VO. Each VO is expected to register with the OSG and provide support for their members. OSG support is maintained through a VO representative for each VO rather than a member of a VO.
The OSG welcomes researchers that are not associated with a VO!
They are welcome to join the OSG
VO. The OSG VO is inviting all users who don't need support running their application on the OSG. The Engage VO provides strong support for scientists who wish to bring their applications to the grid but who are unfamiliar with grid technologies.
If you don't know what VO you belong to or should belong to, we suggest doing that before moving on, email the Grid Operations Center
The Open Science Grid is proud to support a wide range of scientific research activities, among them:
A complete list of VOs can be found in the OIM
(OSG Information Management System).
Recent statistics show
that one third
of the resources are used opportunistically by VOs that do not own resources and who benefit from the grid effort of the OSG. The OSG continues to grow and is currently supporting 30 VOs
at 80 sites
. More than 1,000,000 cpu hours
are turned into scientific results by their members every day.