Firefly-2058: HIPS handling underlying FITS, with extract and readout

[robyww]

Firefly-2058: HIPS handling underlying FITS, with extract and readout

• Fits file extraction
• Fits readout
• Fits cube integration
• Wcs lock will support syncing cube planes
• Some refactor related to FitsReadUtil.java
• Some retactor related to HiPSListUtil.java
• refactor names of cube related function in PlotViewUtil.js and add support for hips in those functions
• Mouse readout direct fits access data no longer sent back and forth in the readout call
• A file download call can connect to one if it is active

Files with big changes

• HiPSRetrieve.java - much more advance to support fits retrieve. Major cleanup.
• FluxValueUtil.java - move all the flux calls into one file.
• URIFileRetriever.java
• ExtractHiPSTileTool.js - drawing layer for extracting HiPS
• HiPSTileExtractionDialog.jsx - dialog for extracting HiPS
• HiPSUtil.js - to setup and make the FITS extraction call
• MouseReadoutWatch.js - to handle fits flux readout with HiPS
• FitsCacher - improve to also cache small DirectFileAccess objects

Testing

https://firefly-2058-hips.irsakubedev.ipac.caltech.edu/firefly

Test HiPS, there have underlying FITS images as well

1. HiPS tab -> uncheck IRSA featured -> search for PanSTARRS -> choose PansSTARRS DR1 z
2. HiPS tab -> choose URL -> enter url below
   • https://irsadev.ipac.caltech.edu/data/hips/SPHEREx/spherex_allsky_cube_alpha1

Testing after the hips is loaded

• zoom in a little
• Test 1
  • move around on the HiPS, there should now the a Flux readout, it might save retrieving... at the beginning
• Test 2
  • From the image tools menu -> go to the extract rows -> choose
  • You should see a popup, click anywhere on the HiPS and choose Extract Tile
  • The underlying fits should be shown matched to the HiPS
• For the SPHEREx cube
  • the plane should change with the HiPS cube plane
  • the wavelength should show up on the plane control

[lrebull]

With the overall caveat that I have the same heebie-jeebies that Vandana has (who will use this? who will trust this? how will people find this? how will people know what (data sets, visualizations, readouts, files) to trust (or initiate this behavior with) and what not to trust?)...

• you click on the hips image, click extract tile, it attempts to do so, and then right away you get an error "Not a HiPS image: HiPS tile extraction only available for a HiPS display." I know why it does this -- you're in 'extraction' mode and now the currently selected image is fits not hips. But I'm, like, three clicks into this interaction and i've done exactly what the tool is telling me to do and now i have an error? every time? annoying. vaguely rude. There is another pop-up that the tool shows me that i can never see because the error is on top, and if I try to move the error, it goes away.
• I find this confusing (see screenshot): i've zoomed, i clicked on a tile, and hit extract. you see the error. why is there a black cutout? why is the blue tile not matching one of the presumably hips-shaped tiles? (heebie-jeebies: why is the tile a diamond polygon and the FITS a square?) MOST importantly -- why is this nowhere near where I clicked? The center of the blue polygon and the FITS image is approx 4h52m52.25s, +49d55m17.5s EQ_J2000. The place on the sky i was looking when i clicked was 18h18m48.20s, -13d48m25.9s EQ_J2000. that is superduper different. like "not even the same hemisphere" different.

• i zoomed out to see how far away i was from M16 (where'd been initially), and the tile i had clicked on briefly matched the magenta tiles. I zoomed away by using the target pulldown to get back to M16, and then used the 'recenter' button in the FITS pane. so that means i clicked away and then came back. now i have this (see screenshot). heebie-jeebies: how can i understand what part of the sky it is when the polygons keep changing? is it or is it not matched to the magenta tiles on the sky, distortions not withstanding? if it really is the square that the FITS file is, why is it having such a hard time with the blue polygon on the hips image? something isn't right somewhere, and this behavior makes me mistrust what it's doing with the coordinates. (I remember that we had problems with the PanSTARRS wcs, but i'm working on the assumption that we've run that to ground, because I think I remember doing that, but this series of screenshots has made me remember that whole thing.)

• the readout at the top of each extracted tile is telling me that there are large rotation angles and a flip applied. If I undo that by telling it unflip and north-up, as far as i can tell, it changes the FITS but not the hips which i think is weird. I'm confused about what exactly it's doing and this just further makes me suspicious of the fits files it's showing me.

• upload tab -> the hips URL tells me "could not recognize file type." is that expected?

• when i use the hips tab to upload the spherex cube, it's empty for me; i never get a readout ("retrieving value", no flux or wavelength shown at the bottom; honestly it took me a while to find the wavelength at the top, but i should have known better), and the image is just black.

further heebie-jeebies questions include but are not limited to: what will the readout give me in a 3-color hips image? where is it pulling the fits files from (truly original fits files or the pieces that were used to construct the hips image)? why are the point sources often but not always distorted in the pan starrs tiles? how will we convey to users what this is, when it's available, how and when to trust it?

(i'm still an advocate for getting this tool's functionality: https://firefly-2058-hips.irsakubedev.ipac.caltech.edu/firefly/onlinehelp/#id=visualization.selectregion
treated as another kind of 'extract' tool and placed here in the menu.)

[lrebull]

ok, updating now that i know how to see the data that appear in the SPHEREx hips image:

• i like that when you change the wavelength plane in one (hips or tile), everyone changes, this is good.
• if you extract 1 tile, everything is NOT wcs linked. if you extract 2 tiles, everything IS wcs linked. if you close that second tile, then everything NOT wcs linked again. i found this behavior odd. either it's all WCS linked or not.
• the artifacts (and satellite streaks) show me that i'm looking at the same data but the distortions (square vs. diamond) still throw me for a loop (heebie-jeebies)
• i still don't understand the "flip y" that is necessary for the FITS to match the HiPS.

copying from slack conversation so that i don't lose it - more heebie-jeebies from working with PanSTARRS z: Trey says that it works better if you pull tiles from a further zoom in rather than zoom out. this.. doesn't make any sense to me.