Three Years with the WordPress REST API

In December WordPress 4.7 shipped with a built-in REST API, giving every WordPress site out-of-the-box REST endpoints for the core WordPress data types such as posts, comments and categories. This release is the culmination of almost four years of work by a globally-distributed contributor team, and I’m proud to say that here at Bocoup we’ve been involved in the project for over three years now.

I’m proud to have been named as the Design Lead for the WordPress REST API, and as we look forward to where the project goes next I wanted to take a moment to share some of the history of the project (from my point of view). While I’ve spoken about this story in the past, I’ve never written it down—so here we go!

Finding a Content Management System

In January 2014, Bocoup was in the early stages of building a complex web application for one of our clients. We’d already selected Node.js as our primary server language, and then the scope grew to include a blog. One does not simply build a content management system, so we began to look for an off-the shelf product we could use.

There are a number of Node.js content management systems out there, many of which are quite good—however, even today a lot of those systems are heavily biased toward technical users. It would not have made sense for us to force the site’s editorial team to learn Markdown or to figure out how to use a static site generator; we needed something more user-friendly, with writer-oriented capabilities. Moreover, we didn’t want to have to maintain two front-end projects, so we wanted our CMS to provide an API that we could use to pull out the content into our existing application.

Nothing in the Node ecosystem at the time fit the bill. So we chose WordPress.

Now, Bocoup loves WordPress; but three years ago it wasn’t a major part of our toolset. At the time we were building API-driven front-end applications, and without built-in API endpoints WordPress hadn’t been relevant. However as a contributor to the WordPress project I’d been following the development of a plugin, started at the end of 2012 and continued as Ryan McCue’s Google Summer of Code project, implementing a JSON-based REST API intended to replace WordPress’s aging and obstinate XMLRPC infrastructure. By that February, this new API was looking promising. I took a week to spike an Express.js front-end to browse through the posts from a WordPress install, and pitched it to the team. It worked, the writers loved it, and we crossed our fingers and moved forward.

As our project grew alongside theirs I began contributing to the API plugin, and released the code we wrote to talk to WordPress from Node as a query-builder library that is now available as wpapi on NPM. Tracking against the WordPress REST API as it matured wasn’t entirely smooth sailing, but after our client’s site launched in October I had the pleasure of presenting about the work at WordCamp San Francisco, the largest WordPress developer event of that year, as a success story.

Building the REST API

Fast-forward a year. That REST API plugin hits 1.0, then the base API infrastructure is added to WordPress 4.4 in December 2015. Now WordPress came with all the plumbing necessary for plugin and theme developers to build their own API endpoints! But there were still no endpoints shipped with WordPress out of the box. Most plugins have some relation to editorial content, so any plugin that wanted an AJAX front-end was stuck rolling their own duplicative post endpoint—they could have just told their users to install our plugin, but WordPress has no plugin dependency resolution and the onus of managing that dependency tree should not be on the end user. The API infrastructure was a good start, but we still wanted to provide a shared foundation of endpoints for the core content resources.

To that end, the project leads (Ryan McCue, Rachel Baker, Joe Hoyle & Daniel Bachhuber) branched off a new v2.0 beta plugin where we began to refine those content endpoints enough to be able to merge them into WordPress itself. I continued to maintain our wpapi module, and over time I joined the project leaders in more and more planning discussions and meetings.

At Bocoup we found ourselves part of an ever-growing community of developers building modern API-driven applications on top of WordPress, a community that came together in January 2016 at the inaugural Day of REST WordPress REST API conference in London. But last year the endpoints still had too many rough edges to be considered for inclusion in WordPress Core. WordPress is over 13 years old now, and as a project it’s fairly conservative, with a strong commitment to backwards compatibility, security, and usability. We couldn’t risk compromising the integrity of a platform that runs over a quarter of the top sites on the internet; we had to be sure that our REST API was going to be a positive addition to the most widespread CMS in the world.

Towards Merge

The project wasn’t evolving fast enough to guarantee that we’d be ready to merge the API into WordPress core by the end of 2016. So in the late summer of last year Joe & I called each other to discuss a final sprint to make the fall release cycle deadline. I requested some dedicated time from Bocoup to join the project leadership; we resumed our weekly meetings, we brought in new contributors, and we ruthlessly worked through our issue backlog. There was a three-week window in October to make our case that the API was finally ready for WordPress core, and by that time we felt ready. At the weekly WordPress developer meeting on Slack, we proposed that the endpoints for core WordPress content types—posts, comments, pages, media attachments, tags, categories, authors, and taxonomies—be included in WordPress 4.7.

But before the feature was approved, we were asked one final question: who is the REST API for?

Defining Our Audience

This was a tricky one. The most vocal users of the REST API had been consultancies like us and agencies like Human Made or 10up, who were all using it already for custom development projects. Why did it need to be a part of WordPress core if it was already being used just fine as it was?

The flexibility of plugins and themes are critical to WordPress’s past growth and success, and one answer we heard a lot is that the API endpoints significantly benefit plugin developers. By leveraging, extending and re-using our code, plugin authors can provide the endpoints they need to drive a new generation of powerful interfaces without continuously reinventing the REST API wheel. But no plugin or custom project has as many users as the core WordPress software itself.

WordPress is software for writers and publishers. They shouldn’t know or care about a developer-facing feature like a REST API. But if it doesn’t benefit the users directly, why should a feature this large be included in the core software product?

In that context, the answer was clear: there’s a lot of good features in WordPress, but the code foundation upon which they are built is uneven. The REST API endpoints provide a new foundation for future core feature development, a facade layer that abstracts the inconsistencies of the past. We won’t need to build one-off APIs for each new front-end feature, and both front-end JavaScript and back-end PHP can consume the same data structures when needed. Faster, easier development will allow the WordPress admin to be improved in ways that will tangibly benefit the writers who depend on the software.

Conditional Approval

So we got the go-ahead to merge the API into WordPress 4.7. With one caveat: we were asked to migrate an existing administrative feature from our legacy AJAX APIs to consume the new REST API, as a proof-of-concept. We also defined some success metrics that we’ll be using to evaluate whether the REST API is having a positive impact on the WordPress ecosystem. Recent WordPress projects haven’t been subjected to these kinds of criteria, but prior WordPress projects didn’t necessarily have the scope of impact that we’re looking for!

We actually missed our goal of shipping 4.7 with code that actually leverages the REST API; it turns out that switching code from one paradigm to another takes time. But the API stayed in, and in the State of the Word session at WordCamp US in December project founder Matt Mullenweg named the REST API as one of the three main focus areas for WordPress development in 2017.

Looking Forward

I had the pleasure of speaking at LoopConf, a conference focused exclusively on WordPress development, to reflect on this process and to share my thoughts about where we go from here.

As design lead for the project my role will be to oversee improvements to the consistency and flexibility of the existing endpoints so that they can be effectively used by client applications running both inside and external to WordPress. I’ll be partnering with REST API technical lead Ryan McCue, as well as the other focus leads and lead developers, to ensure that our endpoints meet their needs for WordPress core development. I’m also working to identify ways we can improve our REST API documentation to better on-board new developers to the REST API’s capabilities—and to find new contributors to work on improving the REST API itself, because we’ve got a lot of work we want to do!

Here at Bocoup I will continue to maintain the wpapi library, and I’m looking forward to seeing other tools evolve which use it—such as kasia, a set of Redux reducers and decorators for connecting components to remote WordPress resources. I also hope to see other client libraries grow and evolve, such as Adam Silverstein‘s Backbone.js client for the WordPress REST API, and hopefully other utility libraries written specifically for Angular, Ember, etcetera. API-driven JavaScript development is still a new thing for much of the WordPress community, and the more tools we can provide to make that easier, the better.

And that last point is what excites me most. The WordPress REST API is for improving WordPress core, and for expanding what plugins can do; but it is also for the learners, for the future generation of WordPress developers. Today, a new generation of developers around the world is making WordPress sites, learning CSS, muddling their way through PHP, discovering the power of JavaScript; and now they can get exposure to REST APIs through WP in a way none of us could.

We came to WordPress and wanted it to do more. Now it does; but those who have yet to join our community will push it still further. This is what excites me; this is why I will continue to be involved in the project. I hope that this work we’ve done will keep WordPress moving forward and relevant for a long time.

In The Toolbox

At Bocoup we’ve built projects on top of backends written in Node.js, Go, Django, Java, .NET, Ruby on Rails, you name it—that framework agnosticism isn’t going to change. But I’m both excited & quite proud to welcome WordPress to the list of tools upon which I would be excited to build a complex API-driven application.

Each tool has its strengths and weaknesses. Some WordPress REST API projects (like our first foray, back in 2014) will bring it in for the robustness of its current user-facing admin. Others will replace that admin experience entirely, like WordPress.com’s new Calypso interface. There’s even potential for applications that implement their own reading and writing experience, and use WordPress only as a shared database to store app content alongside your existing site data.

As more people build applications on top of the API we’re finding room for improvement, such as providing a first-party authentication solution for external clients, improving the ability to traverse hierarchical data types, and effectively modeling complex resources. But the WordPress REST API is open for business, and I’m excited to add it to the list of powerful Open Source tools I can use here at Bocoup.

Learn More

Next week we’re partnering with Human Made to host the A Day of REST conference in Boston, MA. It’s a full-day, single-track conference focused entirely on understanding and building with the WordPress REST API, bracketed by two great days of front-end development workshops (including one from us on using Webpack!) If you’re interested in attending, we still have tickets available: use code BostonLovesRESTWP for 20% off the conference. Hope to see you there!