Blog

Guest blog post by Donald Atha, Software Architect, BetterCloud

One of the biggest challenges facing IT teams today is the increase in tedious and repetitive tasks driven by the explosive growth in SaaS applications over the last 10 years. Companies are using more and more SaaS products, which presents challenges when it comes to managing employee onboarding and offboarding, managing licenses and permissions, and resolving help desk tickets. At BetterCloud, we give IT teams the tools to reduce the amount of manual work on their plate with automations and configurable workflows. As the number of SaaS applications grows, our customers rely on our library of SaaS integrations to also grow.

Testing OpenAPI Generators for Rapid Integration

Recently the Innovation Team within BetterCloud, a team tasked with discovering and testing new capabilities that accelerate innovations, took on the challenge of prototyping a solution that would allow us to more rapidly integrate with various SaaS applications and scale that solution to onboard hundreds of API providers. The team agreed early on in order to reach the ambitions of the business, we would need to automate much of the process. The team also agreed that this would be possible because much of adding a new integration is the same and that the OpenAPI Specification would allow us to automate pieces of this by using existing community generators. 

To fully understand the problem, it helps to understand what it means to add an integration at Bettercloud and a bit about our product. BetterCloud’s workflow engine automates processes like employee onboarding by detecting when a new employee is hired and subsequently granting access to the various SaaS applications required for their role. This onboarding scenario is just one of many use cases we support. All these use cases invariably necessitate ingesting and storing data from the 3rd party provider and updating the provider’s state. For instance, in the onboarding use case, the ingestion process fetches employee data, detects a change in status, and triggers the onboarding workflow. This workflow calls the provider’s APIs to create accounts or grant access to shared resources for the employee—functionality we internally refer to as “actions.”

When embarking on adding a new integration, BetterCloud begins by validating that the provider has the necessary APIs to support our use cases. This crucial step sets the stage for efficiently accessing required data and assessing whether the provider’s APIs are compatible with our action use cases. The OpenAPI Specification provides a consolidated view of this information.

Reducing the Amount of Code Written by Using the OpenAPI Specification

Once the API is understood, the next phase is implementing the ingestion logic. This phase involves calling the provider’s APIs to fetch and store all necessary data within BetterCloud’s infrastructure. The team has explored two methods to coordinate this, both leveraging the OpenAPI Specification. The intent with both approaches is to reduce the amount of code that a developer must write when adding a new integration. The first involves code-generating an HTTP client in Java using the OpenAPI Java code generator, while the other approach uses extensions to markup the spec, enabling the configuration of the ingestion logic. This marked-up spec feeds into a general-purpose engine that implements the logic based on the extensions. This covers the ingestion of data from the APIs, but once acquired, it must be stored to serve our applications. 

BetterCloud utilizes a SQL database to store the entities we ingest. To create the schemas, the team employs the OpenAPI MySQL generator, Flyway, and Terraform. These tools facilitate the creation and configuration of a database for the new provider. The team was able to slightly modify the existing mustache templates of the MySQL generator in order to add our internal fields such as customer id. Since both the ingestion data models and the MySQL data schemas derive from the OpenAPI Specification’s entities, automation of this process becomes straightforward as the data models are aligned. With just an OpenAPI definition, we can generate a new database and its schemas at the click of a button in all of our environments!

Conclusion – Better Scaling the OpenAPI Specification, Better Value

We anticipate that this approach will allow us to bootstrap the ingestion portion of adding a new integration. By automating and code-genning common elements, the team can concentrate on the unique aspects of each new integration. Much like how we at BetterCloud automate workflows for IT teams to allow them to scale the impact of their work, the OpenAPI Specification has given us the tools to build our own automations that scale the value we can provide to our customers.

Guest Author: Tony Blank

Tony is an experienced software developer with an interest in web design, web marketing, customer integration, and software IT including networking, data center management, and sysadmin. He has attended many of the GlueCon events in the past years because of its highly curated content and focus on cutting-edge technology transforming the web, including API Standards. GlueCon was sponsored by the OpenAPI Initiative.

“Gluecon 2023 was an incredibly enjoyable conference. I extend my thanks to the OpenAPI Initiative for providing me with a complimentary ticket. If anyone is considering attending a future Gluecon and has any questions, please feel free to reach out.”

Email: me@tonyblank.com

Twitter: @thetonyblank

I can’t believe it’s been ten years since I first started attending GlueCon. This event has always been on my “must-attend conference list” because of its highly curated content, focusing on cutting-edge technology that is poised to transform the web. Throughout the years, I’ve had the opportunity to represent a few different companies at Gluecon leading community teams. However, this year is different. I was unfortunately laid off in March, and now I find myself in search of my next exciting challenge.

A conference like Gluecon is the perfect place for me to network and connect with innovative companies.

Back in 2013, when I first attended Gluecon, the conference was abuzz with discussions about web services and API standards. As a developer evangelist for Context.io, an email data API, I found Gluecon to be the ideal gathering to meet fellow developers and individuals keen on exploring new APIs.

This year, the main topic of conversation was WebAssembly.

For those unfamiliar with WebAssembly, it is a binary instruction format specifically designed for the web. Its purpose is to enable high-performance code execution within web browsers. These low-level languages execute on the web with near-native performance, revolutionizing the way applications are built.

It reminds me a little of when Docker was first released and it was apparent to everyone how useful containers are. In fact, Solomon Hykes, the co-founder of Docker, famously tweeted that if WASM had existed in 2008, Docker would never have been necessary (https://twitter.com/solomonstre/status/1111004913222324225).

The second time Solomon presented Docker publicly was 10 years ago at Gluecon, along with Jeff Lindsey. Jeff referred to Solomon’s quote during his talk at Gluecon titled “Futuristic Architectures: Session Backends and WebAssembly Components.” In his talk, Jeff brilliantly illustrated how standard WASM components will lead to innovative app architectures. It was a delight to catch up with Jeff, as the last time we crossed paths was in Paris at the dotScale conference back in 2014. He was showing dotScale Dokku, which began… as Jeff’s Docker demo for his Gluecon talk the previous year.

Another remarkable session was the fireside chat between Alex Williams, from The New Stack, and Matt Butcher, founder and CEO of Fermyon. Alex has been covering the most intriguing emerging technologies since his days at Programmable Web (RIP), and his conversation with Matt was no exception. Hearing about how the benefits of WASM, like cross-language compatibility and fast execution times, are being implemented by Fermyon in their new cloud offering was very exciting.

Not every talk at Gluecon was about WebAssembly. Any developer would have enjoyed Jason Harmon’s talk focused on internal API best practices. Treating APIs like products vs. a feature of a product leads to good developer experience, faster production cycles, and fewer headaches. Jason is currently the CTO of Stoplight.io, who went through the Techstars Cloud accelerator in 2015. Editor’s note: Jason was also a key figure in the early days of the OpenAPI Initiative!

It was fun bumping into Samy Fodil, who I met when his startup Sage Hero was in Techstars Cloud in 2016. Samy was at Gluecon this year talking about his current startup Taubyte and, yep, WebAssembly!

Gluecon always closes with some amazing keynotes. Unfortunately, I had to miss this year to get set up for Mile High Startups & Music, the monthly tech event I promote featuring Denver’s best local musicians. (Big shout out to Elastic for supporting both Gluecon and Mile High Startups & Music!)

I hope to see you there next year!