Data Access

We facilitate open access to the alert data by making it available through multiple channels and in multiple formats. Our cloud-based broker is flexible and modularized, with access hooks at multiple stages of the pipeline so that users can access the data at any stage from raw to processed, value-added, and/or filtered. We provide the data in two formats: live alert streams (filtered and unfiltered) and long-term databases. Our filtering options include both predefined filters (based on our object classifications and basic properties of the raw data) and custom, user-defined filters to support analysis by advanced users.

Our Google cloud implementation also makes it easier for users to create, deploy, and manage their analysis pipelines in astronomy's big-data era. Users have access to Google's scalable, high-performance compute resources at the location of the data eliminating the need for individual users to manage the bandwidth and storage resources and complicated server environments required for sophisticated analysis of large datasets.

Planned APIs

  • Direct connections to the Kafka stream or Pub/Sub from within Google Cloud or the outside world ($$$). Provide the full stream to significant number of users/sites outside GCP will require finanical support, but little technical risk.
  • SQL access to the accumulated AlertDB through BigQuery.
  • A higher-level Python API to support more abstract operations
  • A higher-level Python API to support integration with additional external databases and resources
  • A service to which Target and Observations Monitor (TOM) system can be connected.