Interview with Nirmal Bhagabati & PyGeoProcessing
Nirmal Bhagabati is a Senior Program Officer with the World Wildlife Fund (WWF-US) and a critical part of NatCap’s team, mainstreaming approaches and tools in WWF applications.
You’re among the few who have been at The Natural Capital Project for nearly 10 years. What did you do before you started working for NatCap?
My background is kind of eclectic. I did a dual major in computer science and biology. I was interested in conservation, but I couldn’t figure out how to marry the two. I ended up working in the field of bioinformatics. Though not conservation, I was working with open source software developed for mining large data sets, more in the biomedical realm and genomics.
What’s the difference between genomics and genetics?
Genomics is genetics on steroids. A lot of people used to do genetics experiments to understand gene pathways, anything from the different processes of cancer, or studying a gene in a plant—to figure out what makes rice more drought tolerant, for example. People rapidly started realizing that you need to understand how thousands of genes interact within an organism. It’s not enough to study individual genes in the human body. We need to study all the genes in the body and how they work together, using computers. That’s where genetics became genomics.
How did the field of genomics prepare you to do conservation with NatCap?
Our goal was to take all of these mathematical and statistical techniques and make user-friendly software. Most of these techniques were previously published, but not easily accessible. It was rather hard for a biologist doing hands-on experiments in the lab to take that mathematical knowledge and apply it to what he or she was doing. We wanted to make these techniques available without needing to hire 50 statisticians to work for you.
This is a lot of what NatCap is doing. There’s a lot of awesome science out there, but a lot of NGOs don’t have close ties with academic institutions like Stanford and the University of Minnesota to make that science accessible. It’s really interesting to see how NatCap has evolved. I have a strong sense of deja vu when I see what I was doing eight to ten years ago in the world of genomics. We’ve had the same successes for the same reason—for being open source and collaborative.
How did you become interested in conservation?
I grew up in the northeastern part of India, surrounded by all this nature and biodiversity and rare and beautiful landscapes. It’s called Assam and is very close to Myanmar and Bhutan. That’s a biodiversity hotspot, it’s under a lot of threats and is very complicated, politically. We did a lot of field trips. My dad would take me and my sister out birding. We’d get up at the crack of dawn and go walking through neighboring tea plantations, bamboo groves and villages on winter mornings, when the air was crisp and often foggy to start, but as the sun rose and the fog parted, the place would come alive with birdsong. That’s how I got bit by the conservation bug.
How did your father become interested in birding?
I don’t know how it started, but he refers to his academic mentor, the anthropologist Nirmal Kumar Bose (after whom I was named), being a birder—I think that had something to do with it.
You mentioned Assam is a biodiversity hotspot, but what did that mean in your daily life? Were you seeing rhinos wandering on the outskirts of town?
Yes. Even in urban areas, there’s some pretty impressive birdlife and other biodiversity, although sadly not as much in the bigger cities. But to this day, there are leopard sightings and encounters in urban neighborhoods. And there’s a national park with rhinos 30 miles from my city.
What did your parents think of you choosing to pursue a career in conservation?
I didn’t have very many non-traditional career paths open to me. It was expected that I would become a doctor or a lawyer, or something that would pay the bills. I found it hard to connect my interests to a career. My dad and mom were understandably concerned in the sense of—’You know, it’s great, but did you think about the practical aspects?’ But when I said, ‘This is very much what I want to do,’ and when I got into a PhD program in the US, they were supportive. My dad is a professor of anthropology, and that’s not a standard career choice in India either. He did his Ph.D. outside of India. In the 60s he went to New Zealand, and worked in the Maori community. That was pretty unusual.
You did something else unusual. After your Ph.D. and working for several years, you went back to school to get a master’s degree?
Genomics was a good career, but it’s still not conservation. I was trying to find ways to get back to my interests, and I thought maybe I should do a policy post doc, but I couldn’t find anything that really fit, so I went for a two-year masters, at the University of Maryland. They had a program called Sustainable Development and Conservation. It was a mix of hard science, ecology, and they also had policy courses and economics. I also took some internships, as many as I could cram in. My whole idea was not do the things I had done before, but to round out my background. I worked with the National Wildlife Federation on biofuels, and did some advocacy on the hill, which was very different for me. I worked for the International Union for the Conservation of Nature for a bit, doing policy work on deforestation, and for WWF and Conservation International, doing spatial analysis. That’s what landed me at WWF.
Now you travel the world, training people to use NatCap’s approach and tools. What have you seen recently that inspired you?
What makes me passionate is being out there in the field. Everywhere we go, we see how useful these tools are. Just recently, we heard from a colleague in Thailand. He works for their national park authority and they’re using InVEST for some of their modeling, these analyses helped halt a decision to put up a dam in the park. I understand the battle still continues, however [see http://www.nationmultimedia.com/national/Irrigation-chief-still-wants-to-build-Mae-Wong-dam-30253726.html].
In preparing land cover data to use with InVEST, colleagues in Cambodia updated the forest cover map of their study area, highlighting extensive recent deforestation not reflected in official records, and projecting forest cover loss in the future. Although the government forestry administration contested their assessment, their study is shedding new light on high rates of ongoing deforestation in the country, also borne out by analyses done by other Cambodian researchers. Time and again we see that these are not just cool tools that people run on their computers—but have practical implications for informing policies and decisions.
You’ve been at NatCap since 2008. What are your thoughts on where NatCap is headed?
I’m really excited about the fact that NatCap is starting to coalesce around these big ideas and initiatives: infrastructure, national development planning, and supply chains—the whole idea that it’s not just about doing the same approaches over and over, but here are some big over-arching thematic issues. That’s the way WWF is evolving too, as you work in places, you’re working in places with big issues.
PYGEOPROCESSING AND THE SONG IT INSPIRED: OPEN SOURCE LIBRARY FREES DEVELOPERS FROM ARCGIS
Since March of this year, when NatCap released PyGeoProcessing, a free open source programming library that eliminates the need for ArcGIS, it has been downloaded over 5000 times, outpacing the download rate of all of NatCap’s other tools.
When The Natural Capital Project released the first version of its ecosystem services modeling software InVEST in 2008 (on CD’s!), it relied upon ArcGIS to process all the underlying information.
This meant that anyone who wanted to do the analyses that InVEST allowed had to first purchase ArcGIS, and anyone who couldn’t afford an ArcGIS license was out of luck.
“What we had done is directly tied our science and our flagship tool to a proprietary product,” said Rich Sharp, NatCap’s Chief Software Architect.
“We looked at a lot of alternatives,” Sharp said. “One was, ‘What if we implement what ArcGIS provides for us? What if we just did it ourselves?'”
“Now, InVEST is dependent on PyGeoProcessing just like it was dependent on ArcGIS, but it’s free and open source.”
It took the software team three years.
“We went through InVEST and ripped out all these core features,” Sharp said. They then re-implemented them into a new library, called PyGeoProcessing.
“Now, InVEST is dependent on PyGeoProcessing just like it was dependent on ArcGIS, but it’s free and open source.”
The “Py-” in the name is short for Python, the programming language they chose to use, based on its concise readability compared to other languages. Python itself, according to its FAQ page, is named after Monty Python’s Flying Circus, which was a favorite show of the language’s author, Guido van Rossum, who has been affectionately nick-named BDFL, or “Benevolent Dictator for Life” by the programming community. (Python is the underlying language for many popular web applications, such as Dropbox, where Van Rossum went to work after leaving Google.)
For the non-GIS experts: Geoprocessing takes information about specific places, embedded in maps, and allows the user to manipulate the data, using mathematical equations, to find out things like—how much loose soil will wind up in a particular stream after a storm.
“There’s a well-known equation in hydrology for it, the universal soil loss equation,” Sharp said. The equation requires certain data inputs, such as rates of precipitation, and the “erosivity” of the soil in that region.
“You might go on a NASA website and get a map of precipitation, and somewhere else to get erosivity,” Sharp said, but then you might run into problems. “The pixels on your two maps might be different sizes. If you had only the universal soil loss equation, and some programming skills, you might spend weeks figuring out how much sedimentation to expect in your stream. If you had PyGeoProcessing, it would take a couple of minutes.”
“This is a tool for developers. There are no limits on the size of the input data, or the resolution.”
Again for the non-GIS initiated, Sharp described pixels and the difference between information encoded in pixels and rasters. “Pixels are the smallest piece of a two dimensional image. As atoms are the fundamental building blocks of molecules, pixels are the fundamental building blocks of digital imagery. The term “raster” is often used when a two dimensional digital image represents something other than a photograph, such as a digital elevation or precipitation map. In the latter case, a pixel in the precipitation raster could represent the amount of rainfall in a year centered at that location.”
Though central to InVEST, PyGeoProcessing is flexible and can be used for applications that have nothing to do with ecosystem services.
In its first month, it was downloaded 2000 times, and this month over 3400 times.
“It’s best in class,” said NatCap’s Software Lead James Douglass. It’s faster and works better than any comparable tool available.
“This is a tool for developers. There are no limits on the size of the input data, or the resolution,” Douglass said. “It’s a geoprocessing toolset designed with the needs of real world developers and data in mind.”
Rich Sharp hopes to see PyGeoProcessing eventually included into the Open Source Geospatial Foundation’s list of projects. The foundation’s mission is to “support the collaborative development of open source geospatial software, and promote its widespread use.” It supports several open source projects, including GDAL (Geospatial Data Abstraction Library), a highly popular programming library the NatCap team drew upon to build InVEST.
“It’s a pretty fundamental library in the world of GIS stuff,” Sharp said. “Almost everybody uses it.”
“I would like to see PyGeoProcessing become part of that project. I would love to see it go into there,” Sharp said.
“There may be urban planners that need maps of where water is going to flow, to plan streets so they don’t flood. It solves a lot of people’s geoprocessing problems.”
Inclusion would raise the profile of PyGeoProcessing among the GIS community, making it more likely to be taken up and used by anyone who uses two-dimensional mapping in their work. “There may be urban planners that need maps of where water is going to flow, to plan streets so they don’t flood,” Sharp said. “It solves a lot of people’s geoprocessing problems.”
PyGeoProcessing has also distinguished itself by being the first of our tools to inspire its own song.
“The PyGeoProcessing Song”
By Henry Borrebach
Project Manager for Outreach & Training
O PyGeoProcessing,you’ve been so good to me, open source and high performance, I can’t believe you’re free!
The other platforms I’ve left behind ook over in jealousy as we transform time and space together, a model of efficiency.
To show I mean it, as a matter of fact: you can calc-u-late my raster stacks, handle my features when they overlap, integrate flows up- and downstream—a single program where I used to need a team. You can process vectors in the wink of an eye—distance transforms go flying by, aggregated shapefiles wav-ing goodbye, when you route my hydrological routines my outputted data are squeaky clean.
O PyGeoProcessing, you’ve been so good to me, open source and high performance, I can’t believe you’re free! The other platforms I’ve left behind look over in jealousy as we transform time and space together, hand in hand ’til d-infinity.
Stacey Solie is the Communications Lead at The Natural Capital Project.