The Final Push

All good things must come to an end.

~ Some Dude on the Internet

And so, we have reached the close of the code-fest that is Google Summer of Code. It has been an enlightening, thoroughly enjoyable and skill-building journey. One that has laid the foundation of my future career and one that I will never forget.

In any program like GSoC, the mentors matter the most – they are the ones who enable a student to reach their potential and deliver something of note. I would like to thank all my SunPy mentors – Stuart, David, Steven, Nabil – for ensuring that I never had any uncleared doubts. Stuart, especially, has been available throughout the duration of the program, and was always around for a screenshare session, or to explain something that I didn’t understand. He was open to conversation even when he was out of town. I am very thankful for their active support! I would also like to thank Google and Melange for ensuring a smooth and wonderful experience. 

The last few weeks have gone mostly in testing and documentation. Stuart and I introduced some analytical tests to check on the accuracy of the coordinate transforms. The tests were mostly passing, except for one parameter which seems to be giving us the wrong values, i.e., expected values don’t match the output. Those interested in the mathematics behind this kind of testing may access the relevant document here, and those who wish to see the test code in action may check it out here. 

The final (in GSoC terms) code can be accessed here and the PR is on GitHub, of course! Over the next few days, I will be completing the SunPy Enhancement Proposal to get this pull request fully accepted. I also gave a presentation on Hangouts to explain the purpose of my module, whose IPython notebook can be accessed here.

Besides all this, I have been learning a bunch of web frameworks to aid in my quest for a job. It was Stuart’s suggestion to start learning Flask, and now that I am learning Django as well, I can see why he suggested it. Everything is so…simple in Flask. But then, it would be to my benefit to learn both to some extent. Miguel Grinberg’s Mega Tutorial on Flask has been extremely helpful, and I would recommend it to those who wish to learn web development in Python. Hopefully, with my Flask knowledge, and a bit of Bootstrap, I will be able to make a nice web framework for SunPy in the near future, or atleast I plan to!

I am signing off for now, but I may post some more stuff in the future. Stay tuned!

Nearing the Close

Hello folks.

While admittedly there has not been too much happening on the online front, a lot has been happening on the offline front around here!

There has been talk of introducing some more FrameAttributes to support the HeliographicStonyhurst to Heliocentric and vice-versa transformations. The previous part with the introduction of the new get_hpc_distance() method paid off handsomely – my mentor’s calculations got the HGS to HP transform to work properly. The next step is to fix some buggies and introduce the new FrameAttributes.

I have been learning Flask from Miguel Grinberg’s Mega Tutorial on the advice of my mentor and it has been a very satisfying journey so far. I have created a repo on GitHub to learn while committing. Flask is a microframework for Python based on Werkzeug and Jinja2. It is less complex in comparison with Django, and learning it really is fun. There are also Flask extensions to help one with the job of interfacing with other apps such as SQLAlchemy.

I have appeared for a couple of job interviews for some good startups in these two weeks. The GSoC project definitely has helped me gain some traction. It gets me noticed where I would earlier not have had a chance.

That’s all for now! My apologies if this post is too small, peeps.

Break – Fix – Repeat!

Cadair commented 9 hours ago

@VaticanCameos I broke it

This quote seems to sum up all that we’ve done in the past few weeks. So far, the Coordinates Framework PR has 80 commits. We are to have a meeting on Tuesday to gauge progress and set the path for the rest of the month.

It has been quite an experience. Having complete ownership of a module isn’t easy. One has to think of design and implementation, testing and experimentation, etc. all in quick succession.  On top of that, the repository does not stay the same all the time – one is always on the bleeding-edge of things. Having said that, my mentor has been really supportive. I have migrated to a new laptop, and on his insistence, tried to install ArchLinux but failed due to some problems with the Radeon graphics card. Nevertheless, I am on Linux Mint for the time being!

A summary of things -:

1. Some coordinate frames have undergone several changes – Heliographic and Helioprojective, more precisely. Helioprojective ‘Cartesian’ is in fact a spherical representation masquerading as Cartesian, so it was decided that it would be better to change the representation to spherical, but keep the attribute names as Tx and Ty.
2. Astropy have introduced the RepresentationMapping class, which is a useful encapsulation of the mapping of frame representation attributes to the names that the designer wishes to specify, as well as their units. RM objects take a 3-tuple, thus.
3. Helioprojective frames now have a new property – ‘zeta’ – which was earlier being mapped to the distance attribute. On reviewing the code, we found that distance should actually be ‘d’ (which was a FrameAttribute then) and it is better to have zeta as a property which returns the value of ‘D0 – d’. ‘zeta’ can be used in the constructor to generate a value for ‘d’. This ultimately fixed most of the transformation issues we had from Helioprojective to other class, except for a few corner cases which we are now working on.
4. A lot has been learned about how to default attributes in the frame constructor. Careful consideration of the various input cases was necessary. Heliographic frames now default ‘rad’ and a FrameAttribute called ‘dateobs’. The empty frame case was the biggest thorn – breaking a lot of things. Since Astropy’s code purposefully generates an empty frame of the same class to check certain things, it was necessary to keep empty frame design clear in mind. Helioprojective frames can default ‘D0’ and ‘d’.
5. SphericalRepresentation was subclassed to create Spherical180WrapRepresentation. This was done after Stuart noticed that the ‘hlon’ attribute actually spans a range of -180 to +180 degrees. Creating a negative valued longitude was not possible with ordinary Longitude objects. Thus, a new representation had to be created. All frames which were earlier on spherical representations have been shifted to this new frame.
6. Currently, there are a few corner cases that have to be fixed. These include transformations from Helioprojective to Heliographic when all three attributes are specified as zero and transformation from Heliographic Stonyhurst to Heliographic Carrington which is breaking, presumably because of the dateobs parameter. The dateobs parameter will also be used to generate the Carrington offset for transformation.
7. Lastly, the coordinates framework has, for the most part, successfully been integrated into the Ginga project! It is now used to model solar coordinates and visualize SunPy maps. This is great news, and it will also push us to work harder on keeping the framework bug free.

In other news, I have started delving into other projects, such as working on an IRC bot, some competitive programming (really lagging on this) and – of course – looking for jobs! The main thing though, is the side project that I have acquired from my mentor to build #sunpy’s irc logs website in Flask. It’s going to take a while as I am completely nil in web dev, but it will be quite worth it!

Oh, and here, have a gif which signifies what Germany have been like this World Cup . Over and out!

Germanic Slaughter

The Midterms Approach

It has been a long, long two months for me, folks. I have finally completed my last set of semester exams, and graduated from college. Celebratory dances and jigs apart, this also meant that I could devote full time to the work, and that is all that matters!

I had not been sitting idle for that matter during my exams, either. Following on from the issue that Stuart and I posted on Astropy, regarding SkyCoord not recognizing frames in Cartesian representation, a lot has changed since then. Tom Aldcroft, a collaborator with Astropy, began working on adding the requisite changes to the SkyCoord class, with guidance from other Astropy members such as Erik Tollerud (one of the main collaborators who worked on APE5). I summarize some of the changes here as follows -:

1. The concept of frame_attr_names was removed in favour of FrameAttribute and TimeFrameAttribute objects. For example, if I am to specify an attribute ‘d’ which defaults to 1AU (Astronomical Unit, for the uninitiated), I would write -:

   d = FrameAttribute(default=(1*u.au).to(u.km))

   Not only does this make the code easier to read, it is an example of good design principles in object-oriented programming. Additionally, such attributes are specifiable in the constructor during frame initialization.

2. Getters and setters were added for the representation property of a frame class. One may now change the representation of a frame coordinate simply by setting the representation property.
3. _frame_specific_representation_info was added as a nested dict which stores ‘overrides’ of representation information. For example, if I want a frame’s Cartesian representation units to be in degrees, I would specify it in this dict. I could then access the overriden representation by writing frame_coord.cartesian, for example.

I contributed to this pull request by adding tests for the builtin_frames.py and sky_coordinate.py files. It was an exercise in git rebasing, and I learned that the git push that follows a rebase must be done with the -f (forced) parameter. Once the SkyCoord PR was merged to Astropy-master, SkyCoord was SunPy ready. It was rather unfortunate that I fell sick in between, and I could not contribute as many tests as I would’ve liked.

The following week, my Map Tests PR was finally merged into SunPy-master. It underwent a lot of scrutiny before Stuart finally decided to merge it because it was out unmerged too long! It feels nice to have contributed something substantial, but I knew that the real work still remained: the coordinate frames!

We began by specifying some Coordinates API tests for SunPy, as a roadmap to the features we wanted to achieve. They can be found here. Initially, of course, these tests will fail, but as features are added to the codebase, each one of those tests is supposed to pass. They were modelled after the API tests in the Astropy codebase. These will also help in completing the SunPy Enhancement Proposal which I’d filed as a PR earlier, to the sunpy-SEP repository.

Then, we began work on the coordinate frames. The PR here captures the work done so far on the coordinates API. Four frames have been defined for use with SunPy – Heliographic (Stonyhurst and Carrington realizations), Heliocentric and Helioprojective. Each of their representations and frame attributes have been defined as given in Thompson. Additionally, a transformation framework based on Astropy’s frame_transform_graph has also been created. Since each frame subclasses astropy.coordinates.baseframe.BaseCoordinateFrame, each of them gets registered into Astropy’s frame registry. The transformation framework makes use of the mathemagic given in sunpy.wcs. This was a re-implementation project after all, and this is how we plan to phase out sunpy.wcs in favour of sunpy.coordinates! I feel like a lot of things are finally coming into place.

Once I’m done with the low-level frame classes, I will look to possibly patching the high-level SkyCoord object to work properly with SunPy frames.

I’d like to add a closing thanks to my mentors – Stuart, David, Steven, Nabil and the SunPy-Astropy combine for helping me come this far. It’s been fun so far, I’m looking forward to more!

Two Weeks In

A lot has transpired, two weeks into the coding period. I have been rather handicapped by the inappropriate timing of my semester exams, and yet, I have gained a few insights into work ahead.

To begin with, my map tests branch is finally passing on Travis CI. With a few cosmetic changes, I assume it will be merged on Monday during the developer meeting. Next, the Astropy community have merged their APE5 PR and are in the final stages of setting it up for the 0.4 release, which is slated for 3rd June. The APE5 is an important design document for the coordinates subpackage and introduces a clear separation of concerns – which class does what, and how to use them.

Astropy has subdivided the notion of a coordinate object into the following components -:

1. Coordinate Representation: It is a particular way of describing a unique point in a vector space (usually 3D space). Some examples of significance to SunPy are the Cartesian and Cylindrical representations.
2. Reference System: It is a scheme used to orient points in a space. It is used to describe how a point is transformed from system to system. Examples from the Astropy codebase are the ICRS and equitorial coordinates (mean equinox) systems.
3. Coordinate Frame: This is a specific realization of a reference system. Some systems may have only one meaningful frame while yet others may have several different frames. A tentative example in the proposed SunPy subpackage would be that of the helioprojective coordinates, with Stonyhurst and Carrington realizations.

For separation of concerns, there are three types of classes involved -:

1. Data Representation: These classes are to be used to represent the data in a form meaningful to the Astropy/SunPy user. They are subclassed from the `CoordinateRepresentation` abstract class. They are made to store the actual spatial information of a coordinate as a set of `Quantity` subclasses. They also provide methods to invoke transformations between systems.
2. Low-Level: These classes are to serve as descriptions of both coordinate frames and act as containers around the data. They subclass from `CoordinateFrame`. Note that a pure frame is one that does not have any coordinate data, while a frame that does have such data becomes a coordinate.
3. High-Level: These classes wrap around the low-level classes and use their functionality. They provide additional functionality to make the low-level classes easier to use. Astropy uses the `SkyCoord` class to this end.

These descriptions have been copied verbatim from my SunPy Enhancement Proposal., which is still a work in progress (here). Such proposals are put forward whenever SunPy undergoes a major feature change, or some major modifications. 

There has been some debate on whether Astropy’s SkyCoord class should be able to support representations such as the Cartesian and Cylindrical, out of the box. Such functionality is important to SunPy. This issue thread and consequent PoC seeks to address that concern. I will be adding tests and documentation so that the PR is complete and it is merged into the Astropy codebase.

I created a bunch of Interactive Python notebooks for the purpose of getting everyone up to speed on how Astropy’s coordinate subpackage works and how we will be using it. They can be found at -:

• The astropy.coordinates Subpackage
• Playing With Frames

For the duration of my exams, I’ll try my best to add new stuff to SunPy, after Astropy v0.4 is put out of development stage.

More later!

Bring On The Coding Period!

The Community Bonding period will finally come to a close, come May 19. It’s been an interesting journey thus far.

I’ve been busy with my internal exams this week, but not so much as to miss out on the latest happenings in and around SunPy. I’ve also migrated to a Ubuntu-only laptop after fighting through kernel panics, grub problems, filesystems not being recognized, etc. Ultimately, I found that the problem was a corrupt LiveUSB!

Perhaps the most important news in the context of my project is that the Astropy community have merged the APE5 pull request into their master branch. They will continue to test thoroughly and bring out the documentation for the same next week. My job is to chalk out relevant use-cases from this branch that will be handy for the project, and collate them into an IPython Notebook.

I have been working on my Map Sources PR for a while now. I haven’t had the opportunity to finish it when I would’ve liked, but I’ll be finishing it latest by tomorrow. In this PR, I have added a series of tests for each class in sunpy.map.sources.

Stuart taught me how to use meld, a 3-way merge conflict fixing tool, in a one-on-one session. This was to rebase the map sources branch on a bug-fix branch which involved modifying values in sunpy.database.tests.test_tables.py – merge conflicts had to be removed. It was really informative and interesting as well.

Also, in the midst of testing, I discovered an issue with the Virtual Solar Observatory that was preventing one of the database tests from passing. Downloads from particular time periods and of certain instrument types (such as HMI) would yield empty lists. I opened an issue for this in SunPy.

I’ve been present in developer meetings that have taken place so far. Of special note was a conference between Stuart, David, Steven and I, which took place on May 1st. This meeting was held to discuss my project and how I am to go about it. The summary of what was discussed on the meeting is as follows, posted verbatim from the SunPy GSoC 2014 mailing list -:

1. Filing in an SEP, as the project will introduce a major feature into the SunPy codebase. It will help stimulate design discussion and finalize the API that we wish to incorporate.
2. Astropy APE5 – It is still in review and will be introducing features in the Astropy codebase by means of a PR (here). Classes like CoordinateRepresentation and CoordinateFrame are central to our agenda.
3. API Testing – Stuart’s idea of creating the bare-bones API and writing tests for it as we would normally write tests for a repository. Initially all tests would fail but with time and code, the tests will be made to pass.
4. Creating a ‘notebook’ with relevant use cases from Astropy which will help the team understand how this project is going to be implemented.
5. Beginning the coding period, 19th May – and how my exams start soon after! The only exam schedule I know of right now is of my second set of internal exams – 13th, 15th and 17th. Presumably my external practical exams will be the week after. Semester exams begin in the final week of May. The entire university will be done with semester exams by June 30th, but I am not aware of my schedule yet.

Bring on the coding period, I say!

Back in hardware(software?) mode

Welcome to GSoC!

Ladies and Gentlemen,

Presenting, Google’s welcome package!

 

Look at these beauties.

Sexiest pen ever!

Not pictured here are the wonderful GSoC stickers. I’ve already popped one out of the notebook and stuck it on the fridge. 😀

Thanks for the goodies, Google!

 

So It Begins

I haven’t had a nerve wracking wait for something in a long time. I got exactly that in the final hours of April 21st, 2014.

I had to pass time until the GSoC accepted students list came out, at 00:30 IST (17:00 UTC), and I did so by listening to motivational music and talking to people close to me. SunPy’s mentors were away during the whole ordeal, and so, I didn’t get to pick their brains (which is decidedly a good thing). And then came the big moment. Carol had muted #gsoc and no one could announce their acceptance or lack thereof straightaway. I had heard from my friend, who did his GSoC last year, that selected students receive a congratulatory e-mail.

I opened my Gmail, and to my pleasant surprise, there it was. After a round of congratulations from my parents and close friends, I finally settled down. Getting accepted into GSoC ranks as the best thing I’ve done while still in college (opportunities of this kind and magnitude are hard to come by here), and it’s come to me really late. I get to relish it only once, but I’m not complaining!

My project with SunPy is regarding the re-implementation of sunpy.wcs as sunpy.coordinates using the Astropy coordinates framework, according to APE5. The mentors assigned to me are Stuart Mumford, Nabil Freij and David Perez-Suarez (SunPy is full of really nice people – couldn’t have asked for better mentors). Now Astropy has proposed to build astropy.coordinates, with classes such as CoordinateRepresentation and CoordinateFrame. The main task for me is to recreate sunpy.wcs while wrapping over astropy.coordinates functionality. It is always better to avoid re-inventing the wheel.

Some basic things in the pipeline for this project are as follows -:

1. sunpy.coordinates will have classes such as HelioProjective, HelioCentric and HelioGraphic, which will derive from CoordinateFrame.
2. Further, Carrington and Stonyhurst heliographic coordinates will be accommodated.
3. Different coordinate representations will also be worked upon, such as the cylindrical and spherical representations.
4. An inter-conversion framework for different coordinates will be created using the bi-directional transformation registry in astropy.coordinates.
5. Astropy’s Quantity class will be absorbed for use by sunpy.

I’ve been asked to submit a SunPy Enhancement Proposal (SEP), similar to APEs, since the project I am to work on is to be a major feature of SunPy. The community bonding period will be utilized in brainstorming over design and creating the bare-bones API. I like this workflow – it is quite organized and there is a process for everything.

All in all, I’m looking forward to this. The learning curve will be steep but extremely worth it!

Regarding Payoneer, taxes for Indian students

MAJOR NEWS

Skip down to the last parts of this post if you’ve read it before.

 

Excerpts from the GSoC mailing list (before we were warned for off-topic discussion) -:

• Payoneer does not issue prepaid cards in India because of a RBI directive.
• The bank transfer facility allows for transfer as USD or INR.
• For transfer as INR, the charges are $3 + 3% of the transaction amount per transaction – this amounts to ~$150 for three transactions.
• For transfer as USD, the charges are flat $15 per transaction – amounting to $45. One needs to check with their bank regarding any extra charges, forex rate charges, service tax, etc.

Clearly, transfer as USD is the best option at this time.

Apparently, there is some confusion regarding the transfer fees. The FAQ and pricing page at Payoneer says that the fees are $0. If someone could clarify this, please post a comment!

EDIT: For those who have questions, please refer to the comments thread of this post. Nissim, head of community at Payoneer, has decided to lend us a helping hand.

SECOND EDIT: This chat with the Payoneer representative that I had a while ago should clear everything -:

Bella: Hello and welcome to Payoneer Customer Support.

Pritish: Hello!

Bella: Before we proceed, please provide me with the answer to your security question:

Bella: What was the name of your first pet?

Pritish:

Bella: Thank you.

Bella: You wish to switch to a USD transfer?

Bella: The fee for the USD transfer is $15 per transfer.

Pritish: Yes, I’d mistakenly selected INR

Pritish: Wait, I had a doubt regarding this

Pritish: Sending a page link from Payoneer to you, gimme a min

Pritish:https://payouts.payoneer.com/webapps/methodregistration/fees.aspx?country=IN&pid=DJYxRUStsMq3EKWpggmvRQ%3d%3d&langid=1&sessionid=5qkt01fiyysakfdwkoubi14k&processor_type=1&ProcessorTypeId=1

Pritish: This page says that the fee for USD transfer is $0

Pritish: I live in India and this is regarding my stipend for Google summer of code

Bella: Let me look into this for you.

Pritish: Thanks!

Bella: Please hold one moment.

Bella: I see, you are affiliated with Google Summer of Code and therefore the fee setup is different than usual. There is no flat fee for the transfers apart from the conversion rates that apply for certain transfers.

Bella: You wish to use a USD transfer instead?

Pritish: So if I transfer as USD, the only charges I will have are those relating to exchange rates and other associated fees at my bank, correct? Nothing from Payoneer’s side?

Bella: Yes, correct.

Pritish: Please do set my transfer mode to USD. Thank you!

Bella: We will transfer your request to the relevant department and keep you updated via email.

Pritish: Right, thank you.

Bella: Are you still to use the same bank account?

Pritish: Indeed, I am.

Bella: Okay. Thank you. We will keep you updated and let you know once this is done.

Bella: Is there anything else I can help you with?

Pritish: This was it, thank you so much!

So, in conclusion, I have an account at HDFC bank – to clarify, they charge the following -:

1. The forex rate, let us call this the base rate.
2. Some percentage of the base rate as conversion fees.
3. Now from the money that is deducted by 1 and 2, 12.5% is charged on this difference as service tax.

Transfer as USD is *still* the winner.

I am posting a link to a blog regarding the tax issue. Apparently, stipends are exempt from taxes, and we need to declare exemption.

Link: Click Here

 

Conclusion

Payoneer recently conducted an experiment with me as their recipient to see which transfer mode would be more beneficial to students residing in India. They sent money through two transactions, one in SWIFT/USD mode and one in INR mode. Long story short, INR mode yielded around Rs 200 extra for the same transaction amount. For larger amounts, such as those distributed by Google, the difference could be larger. It may or may not depend on the bank your account is linked to, but so far, I’ve found that very few banks have been favourable in USD mode (such as IDBI Bank). Most banks, SBI in particular, and surprisingly, even HDFC (experiment confirms for HDFC, since I’m an HDFC account holder) is better off with INR. So whatever you do, choose carefully!

GitHub Wiki RST Solution

After some deliberation with Cadair and DavidPS, we finally found out what was going wrong on the SunPy Wiki page for my GSoC application.

<Cadair> It seems that it throws it’s toys out of the pram if the title gets too long
<Cadair> and then wont come back
<Cadair> though changing the underline from = to # seems to have worked

The application page, stored in RST(restructuredText) format, would sometimes not render properly. Garbled text would be the outcome of saving the page, and yet, the preview worked perfectly fine. The solution to this is to not use “=” as a header/H1 marker, and to use “#” instead.