Recently, dbt Labs announced a price increase for their hosted dbt Cloud service. The change impacts two of the three pricing tiers, Developer and Team.
It applies to both new and existing accounts and is already in place for new customers. Beginning February 1, 2023, existing customers will also pay more.
Who Will This Impact?
If you use the team-based pricing for dbt Cloud, you will see an increase in cost soon.
This timing isn’t ideal since many organizations tie their budget to the calendar year and may have finalized spending for 2023. If you are an analytics manager or lead, you could have a tough decision:
Should you ask for more money to meet this higher price point? Or should you replace dbt Cloud and allocate time to implement a self-managed solution? (A less appealing option is to reduce the number of people who can use dbt Cloud in your organization).
This article will present some considerations to help you decide.
Understanding the Changes
Developer plan
The only change to the free developer plan is that it now has a one-project limit. If you already have a developer account (pre-December 15, 2022), the new limitation does not apply. If you have multiple projects today, you won’t lose them.
Team plan
On the team plan, the monthly cost per seat increases from $50 to $100, plus the one-project limit also applies. Aside from the price doubling, it’s essential to note the team plan now has a maximum of eight developer seats.
The most significant change in the announcement: If your organization has more than eight people working in dbt Teams, you’ll need to upgrade to an enterprise plan.
Reaction to the Change
After the announcement, a few data practitioners posted about migrating to a self-managed solution. Nobody likes a sudden price bump, and the mid-December timing forces people to react fast.
They quickly pointed out that dbt is an open-source tool that can run from a command-line interface without recurring costs. Plus there isn’t any new functionality to justify this sudden increase.
You can build an infrastructure to manage it in-house if you want!
If you believe them, it sounds easy to replace dbt Cloud. Perhaps your manager saw one of these posts and even sent you someone’s step-by-step instructions to implement a different solution.
Smart people on Twitter are doing it! We should too!
As a long-time user, I started using dbt before the hosted solution. Even now, I have maintained more CLI implementations than dbt Cloud. When dbt Cloud launched in 2020, my team decided not to be an early adopter and continue to manage it internally.
For a little while...
Having experienced both methods, I would not replace dbt Cloud. Here’s why:
There’s No Guarantee You Will Save Money
If there’s a discussion about replacing dbt Cloud, it’s easy to predict everyone’s first question.
How much money will we save?
Well, that’s tricky.
While a hosted product has a fixed cost, you need to consider the value of your time before building a custom solution. Plus, you need to build an infrastructure to run the jobs, which will continue to incur usage costs.
Since the entire analytics team will interact with the new workflow, you should consider each person’s learning curve and response to the changing process.
You need to estimate the cost of the internal infrastructure + the team's time (not just yours) to implement and adopt a new solution.
Now the real question becomes: WILL we save money?
Are There Cheaper Ways to Schedule Jobs?
Some posts suggest replacing dbt Cloud with a different scheduling tool. That idea seems simple enough, but dbt Cloud is more than just a scheduler. It’s essential to consider every element of the analytics workflow that dbt Cloud performs.
dbt Cloud provides:
- Job scheduling
- Notification and logging to help troubleshoot errors
- Source control integration
- Easy facilitation of CI/CD (continuous integration, continuous deployment)
- An IDE that removes complexity from the analytics experience by providing a UI wrapper for the command-line interface
- Support if something goes wrong
As an analytics engineer, I used to support our in-house CLI implementation. We created tons of documentation and defined processes for analysts to follow, but I often found myself on the receiving end of time-sensitive support requests.
My engineering productivity wavered, and my manager often asked: why can’t you make this process easier for our growing team?
User Experience
When my team didn’t have the option to use dbt Cloud, we dealt with the complexity of our in-house infrastructure. Our analysts selected a free IDE (most chose VS Code) and had to become proficient with command-line workflows.
Analysts got frustrated when they couldn’t complete their tasks because of some vague command line error or a failure in the CI/CD checks.
I turned into a sounding board for people who hated the process.
But the problems disappeared when we implemented dbt Cloud. Productivity and job satisfaction hit an all-time high. dbt Cloud was created and designed to provide a straightforward user experience. Its first goal was to make dbt accessible to analysts at every level of technical agility.
I was surprised by how easily every person on my team could create models and work independently. One contributing factor is how dbt Cloud seamlessly integrates with source control. Using its UI is much easier than interacting with source control on the command line.
The learning curve for dbt Cloud is also small, which is valuable if your team will be growing or changing.
I can’t imagine telling a team that’s grown accustomed to the efficiency of dbt Cloud to start using multiple new applications AND the command line.
To do the exact same tasks.
Let’s revisit the question about how much money you might save and add a few more considerations to the equation:
- Are your data consumers easy-going when something goes wrong with your ELT process?
- Can critical business decisions be made if the data is stale?
- Can you afford to put your project work on hold when you need to support problems in the custom workflow or user onboarding?
- Does your team enjoy changes that make existing processes more complex?
If the answer to all four questions is yes, then replacing dbt Cloud might be a fun project to start the new year!
Proceed with Caution
When evaluating the benefits of dbt Cloud holistically, the recent price increase is insignificant. In fact, I would argue that dbt Cloud was priced too low from the start.
A few users have pointed out that no new functionality has been added.
To this, I’d argue that dbt Labs has grown their internal infrastructure to support enterprise-quality hosting for a much larger customer base than they likely anticipated in 2020 when the product launched. The price had to go up eventually.
Would you prefer a reliable product or a lower price point?
As an engineer, I love building things and solving problems. If I didn’t have as much experience with custom solutions before working in dbt Cloud, I might consider implementing an in-house workflow too.
But after supporting dozens of analysts on custom platforms for way too long, I do not want to build and maintain something that smarter people than me have already built and are better positioned to support.
I have far more important problems to solve at work.
I'm sure that you do too.
