NIRSpec MOS Demo Notebook

[kglidic]

This notebook checklist has been made available to us by the the Notebooks For All team.
Its purpose is to serve as a guide for both the notebook author and the technical reviewer highlighting critical aspects to consider when striving to develop an accessible and effective notebook.

The First Cell

• The title of the notebook in a first-level heading (eg. <h1> or # in markdown).
• A brief description of the notebook.
• A table of contents in an ordered list (1., 2., etc. in Markdown).
• The author(s) and affiliation(s) (if relevant).
• The date first published.
• The date last edited (if relevant).
• A link to the notebook's source(s) (if relevant).

The Rest of the Cells

• There is only one H1 (# in Markdown) used in the notebook.
• The notebook uses other heading tags in order (meaning it does not skip numbers).

Text

• All link text is descriptive. It tells users where they will be taken if they open the link.
• All acronyms are defined at least the first time they are used.
• Field-specific/specialized terms are used when needed, but not excessively.

Code

• Code sections are introduced and explained before they appear in the notebook. This can be fulfilled with a heading in a prior Markdown cell, a sentence preceding it, or a code comment in the code section.
• Code has explanatory comments (if relevant). This is most important for long sections of code.
• If the author has control over the syntax highlighting theme in the notebook, that theme has enough color contrast to be legible.
• Code and code explanations focus on one task at a time. Unless comparison is the point of the notebook, only one method for completing the task is described at a time.

Images

• All images (jpg, png, svgs) have an image description. This could be

  • Alt text (an alt property)
  • Empty alt text for decorative images/images meant to be skipped (an alt attribute with no value)
  • Captions
  • If no other options will work, the image is decribed in surrounding paragraphs.

• Any text present in images exists in a text form outside of the image (this can be alt text, captions, or surrounding text.)

Visualizations

• All visualizations have an image description. Review the previous section, Images, for more information on how to add it.

• Visualization descriptions include

  • The type of visualization (like bar chart, scatter plot, etc.)
  • Title
  • Axis labels and range
  • Key or legend
  • An explanation of the visualization's significance to the notebook (like the trend, an outlier in the data, what the author learned from it, etc.)

• All visualizations and their parts have enough color contrast (color contrast checker) to be legible. Remember that transparent colors have lower contrast than their opaque versions.

• All visualizations convey information with more visual cues than color coding. Use text labels, patterns, or icons alongside color to achieve this.

• All visualizations have an additional way for notebook readers to access the information. Linking to the original data, including a table of the data in the same notebook, or sonifying the plot are all options.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[rizeladiaz]

Started review of this notebook

[rizeladiaz]

I just finished with the last review of the notebook. All changes are documented in JDAT-5047.

[kglidic]

I have completed my review of the changes made to the notebook. During my check, I identified a few areas in the notebook that required minor fixes or updates. I have already addressed these in this update and have summarized the changes I made:

MOS Notebook Edits:

• I added an example of creating an association file for spec3. This change reflects what is in the IFU-reviewed notebook but is adapted slightly for MOS.
• Updated the recent changes section.
• Fixed the titles and the linkage in the TOC to the spec2 and spec3 association sections.
• Added 6.3 MSA Metadata File in TOC. Shifted Run Spec2Pipeline down to 6.4.
• Fixed some links that pointed to the wrong page or broken link.
• I added some additional comments on background subtraction in the configuration cell.
• As a check, I added two print lines in the import section to print out the default context for the build and the one currently set.
• When downloading the MSA metadata files, I added a filter to grab the MSA for the science observation.
• The docstring format for unique_files_to_array was updated to align with the styles of the other function docstrings.
• Fixed logging in the update_asn_paths function.
• I copied the security note about the MAST token from the FS notebook review to this one.
• I added a clarifying note in the galaxies of interest section highlighting that we list source IDs.
• Added the missing header for Directory Set Up.
• Updated timestamps to return minutes. Removed redundant timestamps and formatted them all the same way.
• Added clean_flicker_noise to the det1dict.
• I separated the code into a few cells in the stage dictionaries to associate the blue comments more easily with the referenced step. I have also kept the blue comments in the notebook. These can be updated or removed when we update the notebooks to newer build versions.
• When printing the ASN files, I added the check for string vs list output so there were no crashes.
• I removed the copying of the spec2dict.
• I updated the spec3dict so as not to skip outlier detection.
• Added short descriptions to any function that was missing one.
• Removed the information about source names in MAST since we no longer compare them to MAST products.
• Removed this text: "Recent improvements in resampling might exacerbate some off-centered cases, and in certain situations, the source may be completely missed."
• Added a flag to check if doing master background subtraction, not to restrict processing only one slit in spec2 with slit_name.

[drlaw1558]

A few quick comments:

• For consistency with the other instruments/modes, please change the mos/ directory to MOS/ instead
• As with the other notebooks, can we always construct the asn internally to the notebook rather than relying on downloading it? In this case spec2 looks simple (every dither position is a background for the other dither positions), but can that be more complicated? Likewise, for the spec3 asn can we just always construct this on the fly rather than doing one thing in demo mode and another thing for all other cases?

[drlaw1558]

Just checking in on the status of this? @kglidic @rizeladiaz @hayescr

[rizeladiaz]

I should have the update version later today.

[rizeladiaz]

• I updated the directory to be MOS instead of mos.
• I added a function to create the Spec 2 association to include the nodes as background observations. This applies fro all cases (demo_mode True or False).
• I added code to create the Stage3 association for the noded case.
• Removed unused functions
• Corrected PEP8 issues
• Changed to use jwst 1.17.1
• Removed from requirements astropy and numpu tomatch what David has done in the other cases.

[rizeladiaz]

@sosey and @law I am having issues with this notebook that keeps failing when pushing it to the repo. I ran pytest in Mac and Linux and it passes without problem. I need to figure out what could be the issue here.

[drlaw1558]

@rizeladiaz It might be failing the execution test due to memory; I'll look at it locally. Are you happy with it otherwise now?

[rizeladiaz]

@drlaw1558 Yes, it could be a memory issue. Yes, besides that I am fine with it.

[drlaw1558]

I've made a couple changes, the most relevant being a fix to the loop over uncal files (which was being limited to only processing the first file in the list). Also cleaned up a few other minor things.

One request that probably best for @kglidic to address: at the moment the plot routine fails out if there are spectra from multiple sources in the output directory. E.g., run the notebook through, then change the source to extract and the plots will start returning errors because prods_1d is finding spectra from multiple sources that aren't getting filtered like for the 'spectra' vector. Please modify this routine so that it's robust against multiple sources in the same directory.

[hayescr]

@drlaw1558 @kglidic I think I found the issue and opened up a PR to Kayli's version here: kglidic#2 (sorry I don't have direct access to update the branch being pulled here). I was finding that when I ran a second source through the notebook that the spec3 plotting was returning an extra empty plot, was that what you were running into, David? If so this should fix it, and even if it was an actual error being thrown because prods_1d was finding multiple sources, I think this should still fix it. This reorganizes the filtering by source ID for stage 3 products before they are sorted into the "ftypes" dictionary where the x1d files are pulled from.

[drlaw1558]

Thanks @hayescr ; looks like that fixes the issue. I merged your changes into @kglidic 's version, which I think should take care of the last issue we'd been discussing for this notebook. Merging into main.