Nearing the Close

Hello folks.

While admittedly there has not been too much happening on the online front, a lot has been happening on the offline front around here!

There has been talk of introducing some more FrameAttributes to support the HeliographicStonyhurst to Heliocentric and vice-versa transformations. The previous part with the introduction of the new get_hpc_distance() method paid off handsomely – my mentor’s calculations got the HGS to HP transform to work properly. The next step is to fix some buggies and introduce the new FrameAttributes.

I have been learning Flask from Miguel Grinberg’s Mega Tutorial on the advice of my mentor and it has been a very satisfying journey so far. I have created a repo on GitHub to learn while committing. Flask is a microframework for Python based on Werkzeug and Jinja2. It is less complex in comparison with Django, and learning it really is fun. There are also Flask extensions to help one with the job of interfacing with other apps such as SQLAlchemy.

I have appeared for a couple of job interviews for some good startups in these two weeks. The GSoC project definitely has helped me gain some traction. It gets me noticed where I would earlier not have had a chance.

That’s all for now! My apologies if this post is too small, peeps.

Break – Fix – Repeat!

Cadair commented 9 hours ago

@VaticanCameos I broke it

This quote seems to sum up all that we’ve done in the past few weeks. So far, the Coordinates Framework PR has 80 commits. We are to have a meeting on Tuesday to gauge progress and set the path for the rest of the month.

It has been quite an experience. Having complete ownership of a module isn’t easy. One has to think of design and implementation, testing and experimentation, etc. all in quick succession.  On top of that, the repository does not stay the same all the time – one is always on the bleeding-edge of things. Having said that, my mentor has been really supportive. I have migrated to a new laptop, and on his insistence, tried to install ArchLinux but failed due to some problems with the Radeon graphics card. Nevertheless, I am on Linux Mint for the time being!

A summary of things -:

1. Some coordinate frames have undergone several changes – Heliographic and Helioprojective, more precisely. Helioprojective ‘Cartesian’ is in fact a spherical representation masquerading as Cartesian, so it was decided that it would be better to change the representation to spherical, but keep the attribute names as Tx and Ty.
2. Astropy have introduced the RepresentationMapping class, which is a useful encapsulation of the mapping of frame representation attributes to the names that the designer wishes to specify, as well as their units. RM objects take a 3-tuple, thus.
3. Helioprojective frames now have a new property – ‘zeta’ – which was earlier being mapped to the distance attribute. On reviewing the code, we found that distance should actually be ‘d’ (which was a FrameAttribute then) and it is better to have zeta as a property which returns the value of ‘D0 – d’. ‘zeta’ can be used in the constructor to generate a value for ‘d’. This ultimately fixed most of the transformation issues we had from Helioprojective to other class, except for a few corner cases which we are now working on.
4. A lot has been learned about how to default attributes in the frame constructor. Careful consideration of the various input cases was necessary. Heliographic frames now default ‘rad’ and a FrameAttribute called ‘dateobs’. The empty frame case was the biggest thorn – breaking a lot of things. Since Astropy’s code purposefully generates an empty frame of the same class to check certain things, it was necessary to keep empty frame design clear in mind. Helioprojective frames can default ‘D0’ and ‘d’.
5. SphericalRepresentation was subclassed to create Spherical180WrapRepresentation. This was done after Stuart noticed that the ‘hlon’ attribute actually spans a range of -180 to +180 degrees. Creating a negative valued longitude was not possible with ordinary Longitude objects. Thus, a new representation had to be created. All frames which were earlier on spherical representations have been shifted to this new frame.
6. Currently, there are a few corner cases that have to be fixed. These include transformations from Helioprojective to Heliographic when all three attributes are specified as zero and transformation from Heliographic Stonyhurst to Heliographic Carrington which is breaking, presumably because of the dateobs parameter. The dateobs parameter will also be used to generate the Carrington offset for transformation.
7. Lastly, the coordinates framework has, for the most part, successfully been integrated into the Ginga project! It is now used to model solar coordinates and visualize SunPy maps. This is great news, and it will also push us to work harder on keeping the framework bug free.

In other news, I have started delving into other projects, such as working on an IRC bot, some competitive programming (really lagging on this) and – of course – looking for jobs! The main thing though, is the side project that I have acquired from my mentor to build #sunpy’s irc logs website in Flask. It’s going to take a while as I am completely nil in web dev, but it will be quite worth it!

Oh, and here, have a gif which signifies what Germany have been like this World Cup . Over and out!

Germanic Slaughter

The Midterms Approach

It has been a long, long two months for me, folks. I have finally completed my last set of semester exams, and graduated from college. Celebratory dances and jigs apart, this also meant that I could devote full time to the work, and that is all that matters!

I had not been sitting idle for that matter during my exams, either. Following on from the issue that Stuart and I posted on Astropy, regarding SkyCoord not recognizing frames in Cartesian representation, a lot has changed since then. Tom Aldcroft, a collaborator with Astropy, began working on adding the requisite changes to the SkyCoord class, with guidance from other Astropy members such as Erik Tollerud (one of the main collaborators who worked on APE5). I summarize some of the changes here as follows -:

1. The concept of frame_attr_names was removed in favour of FrameAttribute and TimeFrameAttribute objects. For example, if I am to specify an attribute ‘d’ which defaults to 1AU (Astronomical Unit, for the uninitiated), I would write -:

   d = FrameAttribute(default=(1*u.au).to(u.km))

   Not only does this make the code easier to read, it is an example of good design principles in object-oriented programming. Additionally, such attributes are specifiable in the constructor during frame initialization.

2. Getters and setters were added for the representation property of a frame class. One may now change the representation of a frame coordinate simply by setting the representation property.
3. _frame_specific_representation_info was added as a nested dict which stores ‘overrides’ of representation information. For example, if I want a frame’s Cartesian representation units to be in degrees, I would specify it in this dict. I could then access the overriden representation by writing frame_coord.cartesian, for example.

I contributed to this pull request by adding tests for the builtin_frames.py and sky_coordinate.py files. It was an exercise in git rebasing, and I learned that the git push that follows a rebase must be done with the -f (forced) parameter. Once the SkyCoord PR was merged to Astropy-master, SkyCoord was SunPy ready. It was rather unfortunate that I fell sick in between, and I could not contribute as many tests as I would’ve liked.

The following week, my Map Tests PR was finally merged into SunPy-master. It underwent a lot of scrutiny before Stuart finally decided to merge it because it was out unmerged too long! It feels nice to have contributed something substantial, but I knew that the real work still remained: the coordinate frames!

We began by specifying some Coordinates API tests for SunPy, as a roadmap to the features we wanted to achieve. They can be found here. Initially, of course, these tests will fail, but as features are added to the codebase, each one of those tests is supposed to pass. They were modelled after the API tests in the Astropy codebase. These will also help in completing the SunPy Enhancement Proposal which I’d filed as a PR earlier, to the sunpy-SEP repository.

Then, we began work on the coordinate frames. The PR here captures the work done so far on the coordinates API. Four frames have been defined for use with SunPy – Heliographic (Stonyhurst and Carrington realizations), Heliocentric and Helioprojective. Each of their representations and frame attributes have been defined as given in Thompson. Additionally, a transformation framework based on Astropy’s frame_transform_graph has also been created. Since each frame subclasses astropy.coordinates.baseframe.BaseCoordinateFrame, each of them gets registered into Astropy’s frame registry. The transformation framework makes use of the mathemagic given in sunpy.wcs. This was a re-implementation project after all, and this is how we plan to phase out sunpy.wcs in favour of sunpy.coordinates! I feel like a lot of things are finally coming into place.

Once I’m done with the low-level frame classes, I will look to possibly patching the high-level SkyCoord object to work properly with SunPy frames.

I’d like to add a closing thanks to my mentors – Stuart, David, Steven, Nabil and the SunPy-Astropy combine for helping me come this far. It’s been fun so far, I’m looking forward to more!

Two Weeks In

A lot has transpired, two weeks into the coding period. I have been rather handicapped by the inappropriate timing of my semester exams, and yet, I have gained a few insights into work ahead.

To begin with, my map tests branch is finally passing on Travis CI. With a few cosmetic changes, I assume it will be merged on Monday during the developer meeting. Next, the Astropy community have merged their APE5 PR and are in the final stages of setting it up for the 0.4 release, which is slated for 3rd June. The APE5 is an important design document for the coordinates subpackage and introduces a clear separation of concerns – which class does what, and how to use them.

Astropy has subdivided the notion of a coordinate object into the following components -:

1. Coordinate Representation: It is a particular way of describing a unique point in a vector space (usually 3D space). Some examples of significance to SunPy are the Cartesian and Cylindrical representations.
2. Reference System: It is a scheme used to orient points in a space. It is used to describe how a point is transformed from system to system. Examples from the Astropy codebase are the ICRS and equitorial coordinates (mean equinox) systems.
3. Coordinate Frame: This is a specific realization of a reference system. Some systems may have only one meaningful frame while yet others may have several different frames. A tentative example in the proposed SunPy subpackage would be that of the helioprojective coordinates, with Stonyhurst and Carrington realizations.

For separation of concerns, there are three types of classes involved -:

1. Data Representation: These classes are to be used to represent the data in a form meaningful to the Astropy/SunPy user. They are subclassed from the `CoordinateRepresentation` abstract class. They are made to store the actual spatial information of a coordinate as a set of `Quantity` subclasses. They also provide methods to invoke transformations between systems.
2. Low-Level: These classes are to serve as descriptions of both coordinate frames and act as containers around the data. They subclass from `CoordinateFrame`. Note that a pure frame is one that does not have any coordinate data, while a frame that does have such data becomes a coordinate.
3. High-Level: These classes wrap around the low-level classes and use their functionality. They provide additional functionality to make the low-level classes easier to use. Astropy uses the `SkyCoord` class to this end.

These descriptions have been copied verbatim from my SunPy Enhancement Proposal., which is still a work in progress (here). Such proposals are put forward whenever SunPy undergoes a major feature change, or some major modifications. 

There has been some debate on whether Astropy’s SkyCoord class should be able to support representations such as the Cartesian and Cylindrical, out of the box. Such functionality is important to SunPy. This issue thread and consequent PoC seeks to address that concern. I will be adding tests and documentation so that the PR is complete and it is merged into the Astropy codebase.

I created a bunch of Interactive Python notebooks for the purpose of getting everyone up to speed on how Astropy’s coordinate subpackage works and how we will be using it. They can be found at -:

• The astropy.coordinates Subpackage
• Playing With Frames

For the duration of my exams, I’ll try my best to add new stuff to SunPy, after Astropy v0.4 is put out of development stage.

More later!

Bring On The Coding Period!

The Community Bonding period will finally come to a close, come May 19. It’s been an interesting journey thus far.

I’ve been busy with my internal exams this week, but not so much as to miss out on the latest happenings in and around SunPy. I’ve also migrated to a Ubuntu-only laptop after fighting through kernel panics, grub problems, filesystems not being recognized, etc. Ultimately, I found that the problem was a corrupt LiveUSB!

Perhaps the most important news in the context of my project is that the Astropy community have merged the APE5 pull request into their master branch. They will continue to test thoroughly and bring out the documentation for the same next week. My job is to chalk out relevant use-cases from this branch that will be handy for the project, and collate them into an IPython Notebook.

I have been working on my Map Sources PR for a while now. I haven’t had the opportunity to finish it when I would’ve liked, but I’ll be finishing it latest by tomorrow. In this PR, I have added a series of tests for each class in sunpy.map.sources.

Stuart taught me how to use meld, a 3-way merge conflict fixing tool, in a one-on-one session. This was to rebase the map sources branch on a bug-fix branch which involved modifying values in sunpy.database.tests.test_tables.py – merge conflicts had to be removed. It was really informative and interesting as well.

Also, in the midst of testing, I discovered an issue with the Virtual Solar Observatory that was preventing one of the database tests from passing. Downloads from particular time periods and of certain instrument types (such as HMI) would yield empty lists. I opened an issue for this in SunPy.

I’ve been present in developer meetings that have taken place so far. Of special note was a conference between Stuart, David, Steven and I, which took place on May 1st. This meeting was held to discuss my project and how I am to go about it. The summary of what was discussed on the meeting is as follows, posted verbatim from the SunPy GSoC 2014 mailing list -:

1. Filing in an SEP, as the project will introduce a major feature into the SunPy codebase. It will help stimulate design discussion and finalize the API that we wish to incorporate.
2. Astropy APE5 – It is still in review and will be introducing features in the Astropy codebase by means of a PR (here). Classes like CoordinateRepresentation and CoordinateFrame are central to our agenda.
3. API Testing – Stuart’s idea of creating the bare-bones API and writing tests for it as we would normally write tests for a repository. Initially all tests would fail but with time and code, the tests will be made to pass.
4. Creating a ‘notebook’ with relevant use cases from Astropy which will help the team understand how this project is going to be implemented.
5. Beginning the coding period, 19th May – and how my exams start soon after! The only exam schedule I know of right now is of my second set of internal exams – 13th, 15th and 17th. Presumably my external practical exams will be the week after. Semester exams begin in the final week of May. The entire university will be done with semester exams by June 30th, but I am not aware of my schedule yet.

Bring on the coding period, I say!

Back in hardware(software?) mode

Welcome to GSoC!

Ladies and Gentlemen,

Presenting, Google’s welcome package!

 

Look at these beauties.

Sexiest pen ever!

Not pictured here are the wonderful GSoC stickers. I’ve already popped one out of the notebook and stuck it on the fridge. 😀

Thanks for the goodies, Google!

 

So It Begins

I haven’t had a nerve wracking wait for something in a long time. I got exactly that in the final hours of April 21st, 2014.

I had to pass time until the GSoC accepted students list came out, at 00:30 IST (17:00 UTC), and I did so by listening to motivational music and talking to people close to me. SunPy’s mentors were away during the whole ordeal, and so, I didn’t get to pick their brains (which is decidedly a good thing). And then came the big moment. Carol had muted #gsoc and no one could announce their acceptance or lack thereof straightaway. I had heard from my friend, who did his GSoC last year, that selected students receive a congratulatory e-mail.

I opened my Gmail, and to my pleasant surprise, there it was. After a round of congratulations from my parents and close friends, I finally settled down. Getting accepted into GSoC ranks as the best thing I’ve done while still in college (opportunities of this kind and magnitude are hard to come by here), and it’s come to me really late. I get to relish it only once, but I’m not complaining!

My project with SunPy is regarding the re-implementation of sunpy.wcs as sunpy.coordinates using the Astropy coordinates framework, according to APE5. The mentors assigned to me are Stuart Mumford, Nabil Freij and David Perez-Suarez (SunPy is full of really nice people – couldn’t have asked for better mentors). Now Astropy has proposed to build astropy.coordinates, with classes such as CoordinateRepresentation and CoordinateFrame. The main task for me is to recreate sunpy.wcs while wrapping over astropy.coordinates functionality. It is always better to avoid re-inventing the wheel.

Some basic things in the pipeline for this project are as follows -:

1. sunpy.coordinates will have classes such as HelioProjective, HelioCentric and HelioGraphic, which will derive from CoordinateFrame.
2. Further, Carrington and Stonyhurst heliographic coordinates will be accommodated.
3. Different coordinate representations will also be worked upon, such as the cylindrical and spherical representations.
4. An inter-conversion framework for different coordinates will be created using the bi-directional transformation registry in astropy.coordinates.
5. Astropy’s Quantity class will be absorbed for use by sunpy.

I’ve been asked to submit a SunPy Enhancement Proposal (SEP), similar to APEs, since the project I am to work on is to be a major feature of SunPy. The community bonding period will be utilized in brainstorming over design and creating the bare-bones API. I like this workflow – it is quite organized and there is a process for everything.

All in all, I’m looking forward to this. The learning curve will be steep but extremely worth it!

Intro to GSoC

This will be my first relevant post chronicling my journey so far.

I was on the lookout for a relevant internship and experience a few months ago, when my friend Ankur (GitHub) suggested that I look into Google Summer of Code. Ankur had completed his SoC with an organization called SciRuby a year ago. SciRuby maintains an eponymous library in Ruby whose main user demographic is the scientific community, and by extension, the mathematician community as well. It is a FOSS organization, meaning it develops open source software for the benefit of one and all.

I have to say, I really liked the concept. I am also aware of how development experience can bring clarity in the understanding of a programming language. Since I had just begun on Python, I decided to find an organization and communicate with them. A month and a couple of pull requests later, I’ve begun collaborating with SunPy. SunPy is a FOSS organization that maintains the Python library for solar physics. This library includes functionality to view, parse and extrapolate solar image data as is required by the scientific community. It began as a concentrated effort to build an open-source alternative to SolarSoft, which is commercial software written in IDL.

I was introduced to a bunch of amiable and enthusiastic folks on #sunpy (irc), some of whom are Stuart Mumford, David Perez-Suarez, Jack Ireland, Simon Liedtke, etc. They sure take their development work seriously, and I have personally experienced this on one of their Hangouts meetings, where they dissected what they had accomplished so far and how they wanted to proceed. I had never been on a live conference call with people of nationalities different to mine before, so it was quite the experience. I was the silent spectator to their hustle and bustle!

I’m looking forward to getting a chance to working with them, and contributing something of note to their organization. My childhood fascination of astronomy would finally have paid off then!