Sustainability in Open Source Software, According to Creators of PhantomJS and Scrapy
Ask not what open source can do for you, but what you can do for open source
Would people willingly give away their work for free? Surprisingly, yes—and this generosity powers much of what we rely on daily.
This paradox underpins the open-source software (OSS) ecosystem, where individuals create and maintain tools that power industries worldwide—with little recognition and monetary rewards.
A 2024 Harvard Business School (HBS) study estimated that without OSS, businesses would need to spend 3.5 times more on software than they currently do. Yet, while the systems, workflow, and businesses built atop these tools are highly valued, the open-source tools themselves remain undervalued and underfunded.
What drives someone to build tools that benefit millions, often with little to no financial gain? What changes need to happen to ensure sustainability in the open-source ecosystem? And most importantly, what role can we play to help sustain this ecosystem?
These questions lingered in my mind as I prepared for a fireside chat with Ariya Hidayat, creator of PhantomJS—one of the first open-source headless browser projects to gain wide adoption after its release in 2011—and Shane Evans, CEO of Zyte and co-creator of Scrapy, the popular Python-based open-source web-crawling framework introduced in 2008.
Both open-source software and web scraping share a common ethos: empowering people by democratizing access. While open source breaks down barriers to software solutions, allowing individuals to build upon each other’s work, web scraping enables the free flow of public information, turning scattered, unstructured web data into actionable insights.
Be sure to catch the full conversation here and learn about the inception and significance of these projects, how their motivations evolved over time, and how they manage engagement and contribution from the community.
But for now, this article will focus on unpacking those questions through Shane’s and Ariya’s stories, serving as our entry point into understanding sustainability in the OSS world.
Setting Up the Sustainability Problem
First we need to recognize that sustainability holds different meanings and challenges for different stakeholders.
Sustainability in open-source software (OSS) treads two interwoven paths:
For creators, maintainers, and contributors: What keeps those who build and sustain these projects motivated and able to continue?
For users—both individuals and businesses: How can these projects remain dependable, secure, and adaptable for those who rely on them?
While these two groups definitely overlap, examining them separately helps clarify the challenges and opportunities in maintaining a healthy OSS ecosystem.
Sustainability for Creators and Maintainers: Balancing Intangible Rewards with the Tangible Demands
For many open-source creators and maintainers, sustainability has little to do with financial rewards. Instead, they often find fulfillment in the intellectual challenge of solving problems—usually their own, to begin with, driving technological progress, and the joy of knowing their work positively impacts others.
Ariya illustrated this perspective: "I looked at PhantomJS primarily as a hobby. I had a full-time job, and I didn’t need to worry about getting some financial benefit out of my open-source project. It was about solving a problem and sharing my work. I would even spend money to fund my hobby, the way we all have expensive hobbies."
He acknowledged that while financial support is essential for sustainability, it can even threaten the very freedom and joy that make hobbies enjoyable in the first place.
Ariya continued: "I might not enjoy it if I got paid to fix some bugs. I created this tool because I wanted to solve a particular problem and share the work. I want to work on the project at my own time without external pressure. It's better if there's no money involved in this manner."
This sentiment echoes a common tension in OSS: creators risk losing the autonomy that initially drew them to the work. In these situations, financial compensation can erode the sense of purpose derived from work—and bring with it the pressures of deadlines and expectations.
This is not to say that burnout is not real. Ariya reflects: "I started to lose some motivation because the contribution rate is not necessarily the rate where it can sustain the project with the overwhelming demand. You work on one feature, but people ask for five, and then you feel guilty because you can't deliver the other four.”
And this story is nothing new.
For creators and maintainers grappling with sustainability, Shane and Ariya advise starting with getting clear about their own intentions and what they want to achieve with their project. Based on these goals, there are several potential paths to sustain their work:
Build a business around your project: For instance, monetizing a scraping framework like Scrapy by offering services built on top of it.
Join a project and use it to get paid: Working part-time on the project or using it as a portfolio to secure consulting jobs.
Explore alternative business models: Infrastructure-type projects might thrive under a Platform as a Service (PaaS) approach, while tools like React or Flutter could explore premium features, support services, or commercial versions.
Each project’s nature will dictate the best approach, but the key is balancing intangible rewards—such as autonomy and purpose—with tangible needs like financial stability and scalability.
Creators could preserve the joy and freedom that inspired their work while adapting to the realities of sustaining it over time.
Sustainability for Businesses and Individual Users
For many businesses and individuals, OSS sustainability is mainly about keeping access to cost-effective, reliable, highly adaptive solutions that power their operations and growth.
The HBS study also reveals the stark imbalance in value contributions: just 5% of programmers generate 95% of the demand-side value. While OSS is a shared resource, its sustainability disproportionately depends on the efforts of this small, dedicated group.
While creators build open source out of passion, users derive immense value from their work. This raised questions about the responsibilities of those who depend on OSS.
If OSS delivers such substantial value to the economy, why isn’t its sustainability treated as a collective priority? Could the issue be that the value OSS creates isn’t clearly quantified, and therefore appreciated?
Without tools like Scrapy or PhantomJS, web scraping and automation would be far less accessible. These tools have enabled data-driven products, streamlined operations, and more efficient testing—creating significant business value.
The HBS study estimated the supply-side value of OSS—the cost to recreate the most widely used tools—at $4.15 billion. While the demand-side value, which reflects the replacement cost for businesses relying on OSS, reached an astonishing $8.8 trillion.
So, how should businesses approach their relationship with open-source software? Is it merely a resource to extract value from, or does it require a more thoughtful engagement and sense of responsibility?
Taking and Giving in Open Source: Altruism, Responsibility, or Strategy?
Shane Evans frames Zyte’s approach pragmatically: “When we release tools and projects as open source, we apply a very permissive license where there’s no expectation for contribution. Somebody can just use it and not contribute back—that’s totally fine.”
“Maybe they’re not supporting your project directly, but they’re definitely giving back in different ways,” Shane added. He points to examples such as companies that create valuable open-source tools that are used at Zyte, as well as firms that provide services around Scrapy. “Other companies have provided consulting and support around our framework and might not necessarily always contribute back. But we never really minded that. And we've been open about adding them to our Scrapy.org website. We just expect everyone to do what they can.”
Ariya added, “I think it's almost impossible to start a business these days without using open source. Supporting them isn’t altruism or charity—it’s a business strategy. If a certain open source project becomes unmaintained or worse, has security holes, then it's going to affect the business and therefore less investment.”
“Even if you don't contribute code, there are a lot of ways you can help, like answering questions or showing appreciation,” Shane continued.
For businesses, the most impactful way to contribute is by addressing maintainability and submitting fixes upstream. “By not doing this, you're placing a maintenance burden on your own engineers and your own company,” Shane added.
Businesses can start by identifying the OSS projects critical to their operations and finding ways to give back. Supporting OSS doesn’t always require large-scale commitments. Smaller actions such as sponsoring meetups, talks, or swags for contributors, can have a meaningful impact in fostering community growth.
During our conversation, we ran a survey among attendees—44% self-reported as OSS users, 41% as contributors, and 15% as maintainers—asking if their organizations had a policy for open-source contributions. The results highlighted varying approaches: 57% reported that contribution was encouraged, while 17% said it was mandatory. An equal 17% stated their organizations had no policy, and 9% revealed contribution was discouraged or only allowed outside work hours.
These findings underscore the uneven landscape of organizational engagement with open source. While some companies actively promote participation, others fail to provide structured support—or even create barriers. Establishing policies that encourage or mandate open-source contributions can help businesses align their practices with the sustainability of the OSS ecosystem. Whether through direct code contributions or indirect support, thoughtful engagement ensures the tools businesses rely on continue to thrive.
Conclusion: Sustain the Human Behind Essential Tools
While these answers may not seem definitive, we hope to highlight a paradox: modern business value is built on the freely given work of a few, yet the sustainability of these contributions is often overlooked.
The systems we rely on daily are built on acts of generosity. Open-source software (OSS) thrives on a gift economy where creators freely share their work, enabling businesses and users to derive immense value. Sustaining this ecosystem is a shared responsibility that demands reflection and deliberate action.
For businesses, the question is not what open source can do for you, but what you can do for open source. Supporting OSS doesn’t require sweeping changes; even small actions, like sponsoring events, recognizing community contributions, or addressing upstream issues, can make a meaningful impact. It’s not charity—it’s a practical investment in the stability of the tools you depend on.
For creators, the challenge lies in balancing autonomy with sustainability. Some may succeed through monetization or consulting models, while others may lean on indirect contributions or community-driven support. There’s no universal solution, but preserving the joy and freedom that inspires this work is essential.
By understanding the dynamics of OSS—its tensions, dependencies, and contributions—we can better appreciate the human effort that sustains the tools we use every day.