Case study: Agile development at Srijan

Information may not be reliable

In my earlier post, I touched upon the processes we go through while designing and planning for a project. This entry is a more detailed look at the initial phases of such a project. The application under discussion is highly focused and yet multifaceted, therefore it would seem surprising that much of the concept was crystallized during the development process. Together, the client and the project team at Srijan cut and polished the initial rough concept into what it is today, a mobile based coupon delivery and redemption system which is supported and enhanced by methods both online (i.e. a web application) and offline (i.e. printed catalogs). We had spent months answering fundamental questions like what the application is supposed to do and who were our target audience. At the end of this process, we had a clear aesthetic and business vision. However, it was also understood that features had to be implemented for real to see how they work with the overall vision. Agile development was more of a necessity here than just one of the options we had.

Our version of agile development

Some of us were already practicing agile development for more than a year. Though these were home grown methods inspired by multiple sources, they seemed to work well for us. Rather than following established methodologies like Scrum or XP, we decided to formalize our own processes and adopt appropriate tools. Our method revolves around iterations. All iterations must yield tangible results. This way, the stakeholders can use the software and provide us with feedback which can take the form of change requests, bug reports and sometimes wholesale scrapping of features. It is not necessary to implement change requests and new features in the next iteration, but bug fixes should be immediately scheduled. The focus of the iteration is on completing allied features; however we aim for 1 or 2 week durations. Once work on the iteration has been started, any change in plan has to be deferred until the next iteration. One or more iterations can lead up to a release, which is more public in nature and expected to be used by a larger group. To ground these practices and allow easier collaboration, we decided to use Mingle for transforming the requirements into user stories. In addition to the user stories, we create an iteration plan document which tells the reader exactly what the iteration is supposed to achieve and each participant’s part in it. We prefer Dokuwiki, a free wiki which is simple to use, yet offer adequate access control mechanisms. We have created a PDF generator plug-in for Dokuwiki, which allows us to generate printer friendly documents for distribution. The client had some deadlines by when they would like to see certain features. The stories were accordingly separated into iterations so that features were grouped by dependency and priority. We collected iterations into releases and then started on the iteration plan.

Design decisions and platform choice

Since we knew we would have to plug in and rip out components often, the primary design goal was to keep functionally distinct components loosely coupled. The database would be used to connect components and preserve state as much as possible. The following things were in our mind when we decided on the platform:

1. Features could be implemented with minimal code so turnaround time is low. However, flexibility is equally important and granular customizations should be possible as well.
2. Practical considerations like availability of developers, our efficiency with the chosen platform, etc.
3. Modular architecture, so that developers can be responsible for a functional area and modules can be replaced with minimal fuss.
4. Enough built-in tools and components to get us up and running fast. An example would be some out-of-the-box admin interface which would suffice until we decide to build a custom one.

Despite a plethora of choices, only Symfony and Django web frameworks seemed to satisfy all criteria. We had worked with both and preferred Django. But we chose Symfony since it was built on PHP and we had immediate access to a larger pool of PHP developers in our organization. It was decided that we would continue to use Python for components not connected with the web application such as sending and receiving of SMS, etc. The rest of the platform more or less fell into place; PostgreSQL as the database since we envisaged using stored procedures and other similar features, Apache 2 as the web server and so on. We decided to use Sphinx as an indexing engine to reduce load on database for search and also to improve response times in general. Sphinx was chosen over other alternatives since it was fast and had good bindings available for both PHP and Python, which would turn out to be very important later.

Initial hiccups

The first iteration was primarily about getting a structure up and running which would tie the different components together. Though this was accomplished without difficulty, we foresaw problems if we continued with Symfony. Apart from minor irritants like configuration duplicated in multiple places, there were major issues as well. Propel, the default ORM would not reliably generate foreign key relationships. Since good parts of our application were linked through the database and it would be responsible for data integrity, this flaw was unacceptable. We knew that there was an alternative ORM called Doctrine, but we were not willing to spend significant development time to learn its quirks. We could have written custom SQL every time we changed something in database, but that would slow us down. After much deliberation, we decided to switch to Django since it was free from these issues and had all the advantages of Symfony. Most of our aforementioned PHP developers were occupied at that point and thus the sole advantage of Symfony was nullified. Since the application was loosely joined, it did not take more than a couple of days for the switch and large portions of our work could be reused.

Current status and future outlook

After many iterations and a couple of releases, the application is now almost ready to be rolled out. Of course, there were slips during development. At certain points, we had to change plans in the middle of iterations. The later iterations were not as brief or clear as we would have liked. In some cases, tests were written after major work was done. However, in the end this was one of the smoother projects we have done thanks to agile methods and our initial design decisions. We are confident that it is a good platform for future improvements and additions as well.