Mailman 3 এপ্রিল 2008 - AstroPy

Streaming binary data format for spectra
by G Jones 25 এপ্রিল, 2008

25 এপ্রিল, 2008

Hello, I am working on radio astronomy instrumentation. In particular I have a spectrometer which dumps spectra very rapidly. I am using Python to capture this data and plot this data. In evaluating the equipment, I would like to make long running Allan variance measurements, and thus I would like to save every spectrum to disk for post processing. I realize I could dump the data to disk as just raw binary data, but if possible I would like to use an existing data format to avoid reinventing the wheel for reading and writing operations. I prefer to not aggregate everything into RAM and then dump it all do disk at once; it would be better to stream it to disk. I looked into using FITS, but it doesn't seem suited to dynamically increasing the size of the file. I tried using the pyfits streaming HDU construct, but it seemed to require specifying an initial dataset size first, then filling that dataset. NetCDF seemed like a good solution since it allows for an unlimited dimension to your data set, but I have not had any luck getting any of the python wrappers to work with this library. Does anyone have any suggestions of a data format that would fit my needs? Thanks, Glenn

2 1

New version of telarchive + fetchsdss
by Peter Erwin 13 এপ্রিল, 2008

13 এপ্রিল, 2008

Hi, This is an announcement of an updated version (1.4) of my quick-and- dirty, Python-based telescope archive search tool, available here: http://www.mpe.mpg.de/~erwin/code/ (A short summary of what it does is appended below.) New features (and fixes): - Searches of Gemini and CFHT archives now include a short summary of instruments and # exposures. - SDSS search now uses footprint server instead of main catalog; this ensure that image fields which have no sources in the catalog (e.g., due to the presence of very large galaxies) are still found. - Restored access to SMOKA (Japanese telescope archive), which had been disabled due to a change in the archive URL. Note that the package includes a separate script called "fetchsdss", which allows for command-line retrieval of FITS (or JPEG) images from the SDSS archive via object name, coordinates, or SDSS field specification. [previous summary, updated:] "Telarchive" is a command-line program which simplifies searching various public telescope archives to see if they might have data on a particular astronomical object or part of the sky. This includes the HST and general MAST archive, as well as the ESO, Gemini, UKIRT, CFHT, SMOKA (includes Subaru), and AAT ground-based archives, and imaging data from Data Release 6 (DR6) of the Sloan Digital Sky Survey. The NOAO Science Archive is also included, though it has very little publically available data at this point. The Isaac Newton Group Archive is also searched, but only for data from 2001 or earlier (newer data is handled by the new archive interface, which doesn't use standard HTTP forms and so is not accessible by this program). The program won't *get* the data for you, of course (but see "fetchsdss" for SDSS images), or even tell you very much about it -- for that, you still need to visit the individual archive web pages. But it will save you lots of clicking and typing in web-page forms if you just want to find out if there is *any* data available. An example (searching within a 2-arcminute box centered on the planetary nebula NGC 7027; searching on coordinates directly is also possible): $ telarchive "ngc 7027" 2.0 SIMBAD (Simbad 4, France): Found object coordinates: RA = 21 07 01.593, Dec = +42 14 10.18 Searching archives for ngc 7027 (RA = 21 07 01.593, dec = +42 14 10.18), with search box = 2.0 arcmin... ESO Archive: No data found. AAT Archive: No data found. NOAO Science Archive: No data found. HST Archive (at ESO): Data exists! (84 observations found) -- 12 WFPC2, 24 WF, 1 PC, 13 NICMOS, 3 FOC, 24 STIS, 13 FOS UKIRT Archive: Data exists! (943 observations found) Multimission Archive at STScI (MAST): Data exists! (66 observations found) COPERNICUS (1); FUSE (2); GALEX (29); IUE (34) ING Archive (old interface): Data exists! (577 observations found) 143 images, 391 spectra, 43 unclassified JKT -- RBS (1), FWHL (2), AGBX (102); INT -- WFC (9), PFCU (30), IDS (66), MES (17), FOS_1 (10); WHT -- TAURUS_2 (5), unknown (43), ISIS_BLUE_ARM (159), UES (58), ISIS_RED_ARM (75) Sloan Digital Sky Survey (DR6) Footprint Server: No data found. SMOKA (Subaru Mitaka Okayama Kiso Archive): Data exists! (921 observations found) 475 images and 446 spectra Subaru -- OHS/CISCO (9), Subaru -- IRCS (132), Subaru -- COMICS (331), Kiso -- 1k CCD (3), Subaru -- OHS/CISCO (29), Subaru -- IRCS (102), Subaru -- COMICS (281), Okayama -- SNG (3), Okayama -- HIDES (31) Gemini Science Archive: Data exists! (49 observations found) 4 imaging, 45 IFU NIRI (4), GMOS-N (45) CFHT Archive: Data exists! (213 observations found) BEAR (200), AOBIR (6), AOBVIS (3), GECKO (4) It's available as a gzipped tar file here: http://www.mpe.mpg.de/~erwin/code/ There are installation instructions/suggestions in the README file. (And feel free to email with questions or suggestions about it!) cheers, Peter ============================================================= Peter Erwin Max-Planck-Insitute for Extraterrestrial erwin(a)mpe.mpg.de Physics, Giessenbachstrasse tel. +49 (0)89 30000 3695 85748 Garching, Germany fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin

1 0

AstroGrid Python
by Rick Wagner 11 এপ্রিল, 2008

11 এপ্রিল, 2008

Hi, I wanted to point AstroGrid's Python package, which uses the Virtual Observatory for searching web services for data, querying catalogs, and handling data (amongst other things). Other than Peter Erwin's code, I haven't seen much talk about VO tools, so I wanted to get this out there. If there's interest in using the VO, I could write up a summary page on the wiki. And I'm betting some of the Space Telescope developers are up on the VO, too. There's an introduction on the AstroGrid site; at the bottom of the page you'll find more complete examples. http://www.astrogrid.org/wiki/Help/IntroScripting/AstrogridPython I recommend following Getting Started instructions closely, especially the statement "you need the AstroGrid VO Desktop running". VO Desktop is another client tool built by the AstroGrid folks, and they're designed to work together. It's pretty cool, unless you forget to start VO Desktop first. In fact, you may want to start by looking at the AstroGrid Desktop page. http://www.astrogrid.org/wiki/Home/AboutAGDesktop --Rick ------------------------------------------------------------------------ - Rick Wagner, Graduate Student Researcher UCSD Physics 9500 Gilman Drive La Jolla, CA 92093-0424 Email: rwagner(a)physics.ucsd.edu WWW: http://lca.ucsd.edu/projects/rpwagner (858) 822-4784 Phone ------------------------------------------------------------------------ - Measuring programming progress by lines of code is like measuring aircraft building progress by weight. --Bill Gates ------------------------------------------------------------------------ -

1 0

Re: [AstroPy] [SciPy-user] FITS images with header-supplied axes?
by humufr＠yahoo.fr 2 এপ্রিল, 2008

2 এপ্রিল, 2008

Hello, I just want to discuss about a problem that our python astronomer have and the answer from Perry is a very good example of it. There are too many projects which are doing exactly the same thing and essentially because nothing is centralized or worst, in the case of pywcs, not advertised. For example Perry told us about pywcs developped by STSCI (it was the first time I saw any reference to this project) but Adam spoke about astlib ( http://astlib.sourceforge.net/ ) which are different package with exactly the same goal and I did myself something similar (even if it was fast and dirtier). I think it's time to try to identified the most important task that the astronomer are needing and try to centralized all the effort at the same place. The astropy mail list is probably a good start as the astropy.scipy.org website. Perhaps we can start to identified the need and desiderata. We will have too many but that some most important or urgent must be identified. As example, we clearly needed the pywcs and thanks to STSCI we have it now. We also need a package to plot our data, images etc. Matplotlib is very good but does have a major problem (at least for my point of view), it's slow, very slow for big array or can't even produce the image if the image is too big. ( If I remember a precedent discussion, the problem is mainly due to Agg.) After we need to know, if possible, for which project if someone is doing something, who and how to contact him/her. So interested people can eventually help him/her to do it. It's seems that STSCI is doing most of the work but I'm pretty sure that other people are willingly to help them to extend python to be the ideal tool for astronomer. Just my 2 cents, Nicolas Le Sunday 30 March 2008 10:54:04 Perry Greenfield, vous avez écrit : > On Mar 28, 2008, at 8:28 PM, Keflavich wrote: > > Is there any plotting routine in scipy / matplotlib that can plot a > > fits image with correct WCS coordinates on the axes? I know pyfits > > can load fits files, astLib has routines to interpret header > > coordinates, and I think you can make the axes different using > > > matplotlib transforms, but is there anything that puts all three > > together currently available? > > > > Thanks, > > Adam > > Well, we (STScI) recently wrapped WCSLIB to obtain a mapping function > between pixel and sky coordinates for python (you can find it as pywcs in > astrolib on scipy; that may have been what you were referring to). > > But I'm not sure you understand what you are asking for with regard to > matplotlib. The new transforms stuff should make it much easier to display > the sky coordinates in the interactive display. The axis labeling is a > different matter. Suppose your image (let's say it's 1Kx1K for the sake of > discussion) is rotated 45 degrees with regard to north (either way, it > doesn't really matter). What would you expect to see for axis labels? I > don't think it is at all obvious how people would want labeling to be done > along the edges of the image. I can imagine someone wanting axes or grids > superimposed on the image itself, but that's not quite the same thing. Do > you want the image rotated so that it is resampled on to RA and Dec and > displayed that way? > > In any event, no we haven't yet done anything to try to integrate all three > things. Among other things we wanted to make sure that the api for the wcs > info was suitable before doing a lot with it (and in the meantime, Mike is > working on rewriting drizzle which is taking a lot of his time). > > Perry > _______________________________________________ > SciPy-user mailing list > SciPy-user(a)scipy.org > http://projects.scipy.org/mailman/listinfo/scipy-user

6 6