SP-1758: Update RTN-002: user profiles, analysis use cases, and support use cases

.github/workflows/ci.yaml

@@ -15,7 +15,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v2
         with:
-          python-version: 3.7
+          python-version: 3.11
 
       - name: Python install
         run: |

RTN-002.tex

@@ -21,7 +21,7 @@
 % \setDocSubtitle{A subtitle}
 
 \author{%
-Melissa Graham, Christina Adair, Jim Annis, Jeff Carlin, Alex Drlica-Wagner, Leanne Guy, Greg Madejski, Andrés A. Plazas Malagón, Robert DePeyster
+Melissa Graham, Andrés A. Plazas Malagón, Jeff Carlin, Leanne Guy, Christina Adair, Greg Madejski
 }
 
 \setDocRef{RTN-002}
@@ -33,7 +33,7 @@
 % \setDocCurator{The Curator of this Document}
 
 \setDocAbstract{%
-Draft user profiles and use cases by the Community Science Team (CST).
+User profiles and analysis and support use cases (workflows) for Rubin community science.
 }
 
 % Change history defined here.
@@ -43,6 +43,7 @@
 \setDocChangeRecord{%
   \addtohist{1}{2021-03-05}{Released.}{Melissa Graham}
   \addtohist{2}{2023-04-19}{Add user profiles.}{Andrés A. Plazas Malagón}
+  \addtohist{3}{2025-01-21}{Reorganize document (SP-1758).}{Melissa Graham, Andrés A. Plazas Malagón}
 }
 
 \begin{document}
@@ -57,66 +58,71 @@
 
 \section{Introduction}
 
-This document is a work in progress.
+The contents of this document are under development and are subject to change.
 
-The Community Science Team (CST) in the Rubin System Performance department is generating a set of user profiles and use cases
-to capture the range of anticipated needs regarding community support for Rubin software and data products.
-These user profiles and use cases are being used to guide the developement of science validation processes for
-user-facing resources such as the Rubin Science Platform, documentation, tutorials, and activities.
-They are also being used to refine the Model for Community Science \citedsp{RTN-004}.
+The Community Science Team (CST) in the Rubin System Performance department maintains this set of user profiles and use cases
+to capture the range of anticipated needs regarding scientific analyses with the Rubin software and data products.
+
+This document guides the development of a process to scientifically validate user-facing resources for scientific analysis,
+such as the Rubin Science Platform (RSP), documentation, tutorials, and activities.
+To ``scientifically validate" the RSP means to confirm that the RSP's functionality meets the scientific needs
+of its users, and that users are able to do their science with the RSP's tools.
+This document also guides the development of Rubin's Model for Community Science \citedsp{RTN-004}.
 
 \textbf{User profiles} are designed to represent the diverse user community of the RSP:
 different experience and expertise backgrounds, scientific motivations, and accessibility needs.
-Four user profiles for each of the Notebook, Portal, and API aspects of the RSP have been drafted along
-the axes of user experience and science motivation.
-Two additional user profiles have been drafted: one to represent integrated use of the Notebook and Portal aspects,
-and one to represent accessibility needs.
 
-These fourteen user profiles are subject to change.
+\textbf{Analysis use cases} are designed to represent the variety of types of analysis that
+users will do with the RSP, such as subsetting, visualization, model fitting, and reprocessing.
+These use cases help to define the limitations of the RSP as well, and identify analysis
+that requires external resources (independent data access centers; IDACs).
 
-\textbf{Use cases} for the support of scientists using the Rubin software and data products have been drafted
-in a format defined by model-based systems engineering (MBSE) practices.
-The seven use cases represent issues and support needs that involve, e.g., trouble with scientific analysis,
+\textbf{Support use cases} are designed to provide workflow examples of Rubin's issue resolution
+process, to illuminate how users report problems and get support, and how staff from across
+Rubin participate in user support.
+These use cases include, e.g., trouble with scientific analysis,
 instrumentation faults, software bugs.
 
-These seven use cases are subject to change.
-
 \clearpage
-\section{User Profiles for Science Validation of the Rubin Science Platform}
+\section{User Profiles}
 
-\clearpage
-\subsection{Portal Aspect}
-\input{up_portal.tex}
+\input{user_profiles.tex}
 
-\clearpage
-\subsection{Notebook Aspect}
-\input{up_notebook.tex}
 
 \clearpage
-\subsection{Web Application Programming Interface (API) Aspect}
-\input{up_web_api.tex}
+\section{Analysis Use Cases}
 
-\clearpage
-\subsection{Integrated Aspects}
-\input{up_integrated_aspects.tex}
+These analysis use cases represent general things that users want to do in the Rubin Science Platform,
+using specific scientific examples.
+They are not 1:1 related to the user profiles above, although some of the more advanced analysis
+use cases, for example, would be much more likely to be done by a heavy user.
 
-\clearpage
-\subsection{Accessibility}
-\input{up_accesibility.tex}
+The analysis use cases progress from ``simple" to ``more complicated".
+These cases, along with those in \citeds{dmtn-086}, will serve as the basis for usability tests and the CST's process for science validation of
+the RSP.
+
+\subsection{Rubin Science Platform}
+
+\input{analysis_rsp.tex}
+
+\subsection{Independent Data Access Centers (IDACs) and other computational facilities}
+
+\input{analysis_idacs.tex}
+
+\subsection{Alert Brokers}
+
+\input{analysis_brokers.tex}
 
 \clearpage
-\section{Use Cases}
+\section{Support Use Cases}
 
 \clearpage
-\subsection{Use Cases Template}
 \input{uc000.tex}
 
 \clearpage
-\subsection{Generic Community Self-Help Using CST Resources}
 \input{uc001.tex}
 
 \clearpage
-\subsection{Generic Sequence of Events from a CST Perspective}
 \input{uc002.tex}
 
 \clearpage
@@ -139,6 +145,8 @@ \subsection{Survey Strategy Alteration for Photometric Redshifts}
 \subsection{A Science-Driven Modification to a Data Product}
 \input{uc007.tex}
 
+
+
 \appendix
 % Include all the relevant bib files.
 % https://lsst-texmf.lsst.io/lsstdoc.html#bibliographies

analysis_brokers.tex

@@ -0,0 +1,8 @@
+\textbf{B-201} Identify Tidal Disruption Events (TDEs) and extreme AGN by constructing long-term light curves from difference images using forced photometry.
+Combine host galaxy properties (e.g., mass, Start Formation Rate, etc) with TDE candidates to build a comprehensive classification.
+
+\textbf{B-202} Detect and characterize microlensing events by analyzing DIA images.
+Model light curves and assess crowding effects to estimate baseline magnitudes and object properties.
+
+\textbf{B-203} Classify periodic variable stars using a neural network.
+Improve classification algorithms with DR1 data and develop labeled datasets for future analyses.

analysis_idacs.tex

@@ -0,0 +1,8 @@
+\textbf{I-201} Retrieve PSF cutouts from coadd images (e.g., using the Notebook Aspect of the RSP) to derive parameters of PSF models.
+Use these parameters to train machine-learning-based algorithms for galaxy classification using GPU resources at IDACs.
+
+\textbf{I-202}: Find galaxy clusters in LSST data using Rubin APIs and notebooks for exploratory analysis.
+Download data to NERSC and run cluster-finding algorithms developed by LSST DESC, incorporating data from precursor surveys such as DECam and HSC.
+
+\textbf{I-301} Using direct access to a copy of the coadd and shear catalogs at NERSC or other supercomputing facilities (which will require many iterations through the entire coadd catalogs with high I/O demands), measure weak lensing and galaxy clustering two-point correlation functions to estimate cosmological parameters and constrain cosmological models.
+Analyze tomographic data, perform photo-z calibrations, and compute covariances using parallelized HPC pipelines and Rubin APIs.

analysis_rsp.tex

@@ -0,0 +1,54 @@
+
+\subsubsection{Portal}
+
+\textbf{P-101}: User interface cone search on the object catalog, review the results overplotted on the Hierarchical Progressive Survey (HiPS) map.
+
+\textbf{P-102}: Astronomical Data Query Language (ADQL) search with column constraints, followed by the creation of plots based on the retrieved data in the results view.
+
+\textbf{P-103}: Data discoverability via e.g., schema and HiPS browsing. 
+
+\textbf{P-201}: Extraction of image cutouts from LSST image archives based on user-defined coordinates, spatial dimensions, and filters.
+
+\textbf{P-202}:  View, rerun, and refine previous ADQL queries. Save, organize, and share queries for reproducibility and collaboration.
+
+\textbf{P-203}: Upload user-defined tables.
+
+\textbf{P-301}: Identification of LSST images that spatially and/or temporally overlap user-defined coordinates and timeframes. Enable visualization and interaction with these images, including zooming, panning, rescaling, performing pixel statistics, and adding markup. Generate and save publication-quality plots and images for analysis and presentation.
+
+\subsubsection{Notebook}
+
+\textbf{N-101}: Analyze light curves and create periodograms for selected targets. 
+
+\textbf{N-201}: Query Point Spread Function (PSF) surface brightness moments to compute residuals in sizes and shapes, enabling, for example, null tests for systematic errors in weak lensing analyses 
+
+\textbf{N-202}: Detect and characterize microlensing events across the sky, using advanced algorithms and external packages for computational support.
+
+\textbf{N-203}: Assess the impact of megaconstellations on Solar System science by identifying compromised observations of Solar System objects due to satellite streaks.
+Develop statistical models and optimize survey strategies based on these constraints.
+
+\textbf{N-301}: Run and compare different cluster-finding algorithms, assessing their performance using custom-defined metrics. Utilize batch processing resources for computationally intensive tasks.
+
+\textbf{N-302}: Characterize the orbital dynamics of solar system small bodies using LSST catalogs and Minor Planet Center data.
+Refine orbits, provide orbital characterizations, and develop scalable computational tools for efficient analysis.
+
+\textbf{N-303}: Query survey property maps and cross-correlate them with shape and photo-z catalogs to investigate cosmic shear. Perform advanced filtering to optimize query performance.
+
+\subsubsection{API}
+
+\textbf{A-101}: Explore Rubin catalogs and metadata using basic API queries, and cross-match catalogs from Rubin and other surveys to study low-redshift Type Ia supernovae.
+
+\textbf{A-102}: Monitor known strong lensing systems and discover new ones by performing API-based queries to measure time delays of multiply-imaged transients and quasars.
+
+\textbf{A-201}: Verify the consistency of cosmological observables across dark energy probes by querying Rubin Web API data and performing basic statistical consistency checks.
+
+\textbf{A-202}: Characterize AGNs in LSST data by extracting photometry for extended sources. Use Rubin APIs to run deblending algorithms, estimate variability parameters, and analyze AGN activity.
+
+\textbf{A-301}: Query galaxy shapes, positions, and photo-z catalogs to measure two-point correlation functions with external packages (\emph{e.g.}, Treecorr).
+Use the results for modeling and parameter sampling with access to batch resources.
+
+\textbf{A-302}: Cross-match LSST coadd data from multiple data releases with ancillary datasets, potentially incorporating single-alert matches via broker streams.
+Publish all cross-match results as User Generated Data Products (UGDPs) for broad community use, including applications in Solar System science, cosmology, transients, and more.
+
+\subsubsection{Multi-aspect}
+
+\textbf{M-201}: Create exploratory plots and diagrams in the Portal to identify trends or interesting subsets, then use the Notebook Aspect for computationally intensive analyses and catalog queries.

uc000.tex

@@ -1,44 +1,27 @@
-{\bf Use Case:} {\it unique identifier number} {\it name (short active verb phrase)} \\
-
-{\bf Characteristic Information} \\
-Trigger: {\it the action upon the system that starts the use case, may be time event} \\
-Goal in Context: {\it a longer statement of the goal, if needed} \\
-Primary Actor: {\it a role name for the primary actor, or description} \\
-Scope: {\it what system is being considered black-box under design} \\
-Level: {\it one of: Summary, Primary task, Subfunction} \\
-Preconditions: {\it what we expect is already the state of the world} \\
-Success End Condition: {\it the state of the world upon successful completion} \\
-Failed End Condition: {\it the state of the world if goal abandoned} \\
-
-{\bf Main Success Scenario} \\
-{\it put here the steps of the scenario from trigger to goal delivery, and any cleanup after} \\
-Step 1: {\it action description} \\
-Step 2: {\it action description} \\
-Etc.
-
-{\bf Extensions} \\
-{\it put here there extensions, one at a time, each referring to the step of the main scenario} \\
-Alteration 1: IF {\it condition} THEN {\it action or sub-use case} \\
-Alteration 2: IF {\it condition} THEN {\it action or sub-use case} \\
-Etc.
-
-{\bf Sub-Variations} \\
-{\it put here the sub-variations that will cause eventual bifurcation in the scenario} \\
-Variation 1: IF {\it condition} THEN {\it list of sub-variations} \\
-Variation 2: IF {\it condition} THEN {\it list of sub-variations} \\
-
-{\bf Related Information} (Optional) \\
-Priority: {\it how critical to your system / organization} \\
-Performance Target: {\it the amount of time this use case should take} \\
-Frequency: {\it how often it is expected to happen} \\
-Superordinate Use Case: {\it optional, name of use case that includes this one} \\
-Subordinate Use Cases: {\it optional, depending on tools, links to sub.use cases} \\
-Channel to primary actor: {\it e.g. interactive, static files, database} \\
-Secondary Actors: {\it list of other systems needed to accomplish use case} \\
-Channel to Secondary Actors: {\it e.g. interactive, static, file, database, timeout} \\
-
-{\bf Open Issues} (Optional) \\
-{\it list of issues about this use cases awaiting decisions} \\
-
-{\bf Schedule} \\
-Due Date: \\
+\subsection{S-000: Template}
+
+{\bf Issue origin and description} \\
+{\it A brief description of the support use-case.} \\
+
+{\bf Success scenario(s)} \\
+{\it For this use-case, what would be considered a successful resolution.} \\
+
+{\bf Success workflow} \\
+{\it The series of steps, and who takes, to resolve the issue. In sequential order.} \\
+
+{\bf Alternative success scenario} \\
+{\it If another acceptable outcome could be considered a success, even if it is not the main use case.} \\
+
+{\bf Failure scenario(s)} \\
+{\it The unacceptable outcome that would be a failure to support the user or resolve the issue.} \\
+
+{\bf Risks}\\
+{\it The risks to community science.} \\
+
+{\bf Related information (optional) \\
+Priority: {\it The relative priority of this support use case.} \\
+Timeframe: {\it The targeted timescale from trigger to success.} \\
+Frequency: {\it The anticipated frequency of occurrence.} \\
+
+{\bf Preconditions for success}\\
+{\it List the components of user support that had to exist for success.} \\

uc001.tex

@@ -1,38 +1,32 @@
-{\bf Use Case:} 001 Generic Community Self-Help Using CST Resources \\
-
-{\bf Characteristic Information} \\
-Trigger: Community member posts a request for help in the forum hosted by the CST. \\
-Goal in Context: {\it a longer statement of the goal, if needed} \\
-Primary Actor: Community member. \\
-Scope: CST Online Resources \\
-Level: {\it one of: Summary, Primary task, Subfunction} \\
-Preconditions: An online web forum (or open help desk) with active participation from experienced community members exists. \\
-Success End Condition: The community helps itself and solves the issue. \\
-Failed End Condition: The community cannot solve the issue and the SP-CST must ingest it. \\
-
-{\bf Main Success Scenario} \\
-Step 1. A user makes a post on the LSST Community web forum detailing their issue.\\
+\subsection{S-001: Generic community self-help using CST resources}
+
+{\bf Issue origin and description} \\
+User posts a request for help in the Forum's Support category. \\
+
+{\bf Success scenario(s)} \\
+The community helps the user with no intervention from Rubin staff. \\
+
+{\bf Success workflow} \\
+Step 1. A user makes a post in the Forum detailing their issue.\\
 Step 2. Community members reply with suggestions.\\
 Step 3. The user solves their issue and marks their post solved.\\
 
-{\bf Extensions} \\
-{\it none} \\
+{\bf Alternative success scenario} \\
+The community cannot solve the issue and the SP-CST addresses it instead. \\
 
-{\bf Sub-Variations} \\
-Variation 1. IF the community does not have the relevant expertise THEN the issue is submitted to the SP-CST. \\
+{\bf Failure scenario(s)} \\
+The reported issue remains unresolved. \\
 
-{\bf Related Information} (Optional) \\
-Priority: High (the system must be able to accommodate this activity) \\
-Performance Target: Days \\
-Frequency: High \\
-Superordinate Use Case:  \\
-Subordinate Use Cases:  \\
-Channel to primary actor:  \\
-Secondary Actors:  \\
-Channel to Secondary Actors:  \\
+{\bf Risks}\\
+If the community cannot solve the issue, this creates more work for the CST. \\
+If the CST misses or cannot solve the issue, use of the Forum will decrease. \\
+If users are unable to get support for science, LSST will be less impactful. \\
 
-{\bf Open Issues} (Optional) \\
-{\it list of issues about this use cases awaiting decisions} \\
+{\bf Related information (optional)} \\
+Priority: High \\
+Timeframe: Days \\
+Frequency: High \\
 
-{\bf Schedule} \\
-Due Date: \\
+{\bf Preconditions for success}\\
+The Rubin Community Forum had to exist. \\
+The Forum has to be used by community members. \\