ddid fixes

[smasoka]

Fixes the indexing discovered by #77

[ratt-priv-ci]

Can one of the admins verify this patch?

[smasoka]

added the computing meta fix you did @sjperkins

The real test with MS from @IanHeywood 15A-310.sb30704857.eb31073319.57261.07368590278_spw24.ms is still running

[bennahugo]

ok to test

[bennahugo]

Looks good to me thanks

[bennahugo]

Feel free to post before and after plots of the VLA data here for discussion

[sjperkins]

Feel free to post before and after plots of the VLA data here for discussion

I'm not sure if this is possible! #77 is caused by is caused by two wrongs making a right for MeerKAT data, but failing on other multi-SPW datasets.

[sjperkins]

Looks good to me, just need to remove commented out code.

[sjperkins]

Feel free to post before and after plots of the VLA data here for discussion

I'm not sure if this is possible! #77 is caused by is caused by two wrongs making a right for MeerKAT data, but failing on other multi-SPW datasets.

Thankfully @IanHeywood was intrepid enough to try it on something with interesting SPW's.

[bennahugo]

Feel free to post before and after plots of the VLA data here for discussion

I'm not sure if this is possible! #77 is caused by is caused by two wrongs making a right for MeerKAT data, but failing on other multi-SPW datasets.

Thankfully @IanHeywood was intrepid enough to try it on something with interesting SPW's.

Not sure I follow. One of the datasets has all the original DDIDs but no rows in the MS aside from one. The other has only one SPW and associated rows. Either way it should now be able to flag multiple SPW. Could we plot data without flags (add the flags in casa plotms, should display in red) and with flags, just to be sure it does something logical here?

[sjperkins]

Feel free to post before and after plots of the VLA data here for discussion

I'm not sure if this is possible! #77 is caused by is caused by two wrongs making a right for MeerKAT data, but failing on other multi-SPW datasets.

Thankfully @IanHeywood was intrepid enough to try it on something with interesting SPW's.

Not sure I follow. One of the datasets has all the original DDIDs but no rows in the MS aside from one. The other has only one SPW and associated rows. Either way it should now be able to flag multiple SPW. Could we plot data without flags (add the flags in casa plotms, should display in red) and with flags, just to be sure it does something logical here?

Ah I see. To reiterate: MeerKAT data only has a single SPW and the code prior to this code only worked for a single SPW. @IanHeywood tried it on an MS with multiple SPW's, exposing the logic error.

Thus, I didn't think it would be possible to get "before the bug" plots. However, I understand that you're asking for plots with and without flagging for the VLA MS on this branch? Got a handy shadems command for @smasoka to try?

[bennahugo]

Haven't switched over to shadems yet. If you point me to the data (copy before flagging) and a copy after flagging I can do it

…

On Mon, Jan 25, 2021 at 8:42 PM Simon Perkins ***@***.***> wrote: Feel free to post before and after plots of the VLA data here for discussion I'm not sure if this is possible! #77 <#77> is caused by is caused by two wrongs making a right for MeerKAT data, but failing on other multi-SPW datasets. Thankfully @IanHeywood <https://github.com/IanHeywood> was intrepid enough to try it on something with interesting SPW's. Not sure I follow. One of the datasets has all the original DDIDs but no rows in the MS aside from one. The other has only one SPW and associated rows. Either way it should now be able to flag multiple SPW. Could we plot data without flags (add the flags in casa plotms, should display in red) and with flags, just to be sure it does something logical here? Ah I see. To reiterate: MeerKAT data only has a single SPW and the code prior to this code only worked for a single SPW. @IanHeywood <https://github.com/IanHeywood> tried it on an MS with multiple SPW's, exposing the logic error. Thus, I didn't think it would be possible to get "before the bug" plots. However, I understand that you're asking for plots with and without flagging for the VLA MS on this branch? Got a handy shadems command for @smasoka <https://github.com/smasoka> to try? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#79 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4RE6RQXNRXIE5OKDFD4ULS3W3SVANCNFSM4WSAZ2LQ> .

-- -- Benjamin Hugo PhD. student, Centre for Radio Astronomy Techniques and Technologies Department of Physics and Electronics Rhodes University Junior software developer Radio Astronomy Research Group South African Radio Astronomy Observatory Black River Business Park Observatory Cape Town

[bennahugo]

I want to compare flags prior and post flagging with tricolour. Just a glance to see if it is catching the right stuff in all the SPWs. I will colour them by SPW

…

On Mon, Jan 25, 2021 at 8:53 PM Benna Hugo ***@***.***> wrote: Haven't switched over to shadems yet. If you point me to the data (copy before flagging) and a copy after flagging I can do it On Mon, Jan 25, 2021 at 8:42 PM Simon Perkins ***@***.***> wrote: > Feel free to post before and after plots of the VLA data here for > discussion > > I'm not sure if this is possible! #77 > <#77> is caused by is caused > by two wrongs making a right for MeerKAT data, but failing on other > multi-SPW datasets. > > Thankfully @IanHeywood <https://github.com/IanHeywood> was intrepid > enough to try it on something with interesting SPW's. > > Not sure I follow. One of the datasets has all the original DDIDs but no > rows in the MS aside from one. The other has only one SPW and associated > rows. Either way it should now be able to flag multiple SPW. Could we plot > data without flags (add the flags in casa plotms, should display in red) > and with flags, just to be sure it does something logical here? > > Ah I see. To reiterate: MeerKAT data only has a single SPW and the code > prior to this code only worked for a single SPW. @IanHeywood > <https://github.com/IanHeywood> tried it on an MS with multiple SPW's, > exposing the logic error. > > Thus, I didn't think it would be possible to get "before the bug" plots. > However, I understand that you're asking for plots with and without > flagging for the VLA MS on this branch? Got a handy shadems command for > @smasoka <https://github.com/smasoka> to try? > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#79 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AB4RE6RQXNRXIE5OKDFD4ULS3W3SVANCNFSM4WSAZ2LQ> > . > -- -- Benjamin Hugo PhD. student, Centre for Radio Astronomy Techniques and Technologies Department of Physics and Electronics Rhodes University Junior software developer Radio Astronomy Research Group South African Radio Astronomy Observatory Black River Business Park Observatory Cape Town

-- -- Benjamin Hugo PhD. student, Centre for Radio Astronomy Techniques and Technologies Department of Physics and Electronics Rhodes University Junior software developer Radio Astronomy Research Group South African Radio Astronomy Observatory Black River Business Park Observatory Cape Town

[sjperkins]

I guess there's also the question of whether our default MeerKAT channel masks apply to other instruments...

I would guess not?

[bennahugo]

Only satellite (maybe -- not all constellations may be visible). I would remove the request to mask in the strategy.

…

On Mon, Jan 25, 2021 at 9:07 PM Simon Perkins ***@***.***> wrote: I guess there's also the question of whether our default MeerKAT channel masks apply to other instruments... I would guess not? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#79 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4RE6VSBB5BOWSVEL475GLS3W6QBANCNFSM4WSAZ2LQ> .

-- -- Benjamin Hugo PhD. student, Centre for Radio Astronomy Techniques and Technologies Department of Physics and Electronics Rhodes University Junior software developer Radio Astronomy Research Group South African Radio Astronomy Observatory Black River Business Park Observatory Cape Town

[bennahugo]

A followup on this. Where is the VLA dataset so that I can test whether this branch does the right thing in the case of multi-spw?

[bennahugo]

I do have VLA archival data on my NAS that I can use to test, but it is not P-band as with @IanHeywood 's case.

[bennahugo]

I'm going to deploy my spidery powers and merge this. I actually need the dask bug fix in order to do some urgent commissioning. I will get to looking at VLA data later - that is a lower priority item

[bennahugo]

Thanks @smasoka