Platform Engineering: A Conceptual Framework for Scalability

Platform engineering is a term that has become increasingly popular over the last several years. When new terms are introduced in the IT industry, the loudest voices often attempt to assign their own definitions to them, but the discrepancies usually boil down to the scope of the platform being referenced. For example, one organization may use the term to describe the provisioning of one or two public cloud providers, such as Azure and AWS. Another may use the term to refer to its company’s entire technology landscape.

In the broadest sense, and returning to the core definition of platform, we can see that platforms underpin everything a business does, crossing the technical and business domains. With the rising importance of FinOps, businesses are learning that technology choices must be strategic. If not, they risk purchasing duplicate or excessive functionality and overpaying for unused capabilities. Avoiding these sunk costs is only possible if leaders fully understand what platforms have been procured and why.

Building a good team and using the right tools are key first steps to successful platform engineering. Many teams have evolved while adopting cloud services, likely including hiring for SRE and DevOps capabilities, and are now trying to understand who is responsible for building, maintaining, and monitoring platforms.

This article aims to explain why platform engineering is important, how your existing teams can influence platform engineering, and some tools that can facilitate the process.

Summary of key platform engineering concepts

The following concepts will be discussed throughout the article:

Why platform engineering is important

The introduction of public, private, hybrid, and SaaS cloud platforms has created a vast technology landscape that organizations must now observe and manage; consider that Amazon introduced 119 AWS services at re:Invent in 2022 alone. Many of these new services promote quick time to value and ease of adoption, particular in the SaaS space. However, when eager developers activate these new services, they may unintentionally introduce new risks to the organization, including operational overhead and unintended licensing or billing commitments. In the worst-case scenario, they give up on existing tools and introduce entirely new technology to the mix because they couldn’t accomplish their goals easily enough.

Effective platform engineering results in well-defined technology choices, simple adoption cycles, and “paved roads” for users to leverage new technology, and tools such as CloudBolt include this natively.

As much as software salespeople would like you to believe that there’s a magic platform that can do everything, unfortunately, there is no such thing. Businesses must purchase many separate capabilities that can be combined to create competitive advantages in their respective markets.

Centralizing these technology decisions, evaluating tradeoffs with the input of cross-functional teams, and defining what technology should ultimately be purchased are the key responsibilities of an effective platform engineering team.

Measuring the benefits of platform engineering can be difficult, but the most common benchmarks that can be tracked are the following:

• Total technology expenses as a percentage of revenue
• Money spent or saved on recurring or variable bills
• Engineering onboarding time and DORA metrics
• Time between technology procurement and deployment
• System stability and outages (often combined with SLAs)

Platform engineering in the context of SRE and DevOps

If a company has already adopted public cloud technologies, it may also have existing DevOps or SRE employees managing the environments. Some organizations have even gone so far as to create separate DevOps teams, when DevOps was simply intended to be a cultural shift applied to existing development and operations groups.

Every company defines SRE and DevOps differently, so it’s important to understand your existing definitions before attempting collaboration or a reorganization. Platform engineering is a paradigm shift. Existing teams should not simply be renamed, or the essential mindset adjustments, retooling, and intended benefits will not be realized.

That being said, if a DevOps team has already been segmented to design and build upon technology platforms, its members are likely great early adopters of a platform engineering mindset. The following section on building a platform engineering team will explore this more deeply.

Similarly, if site reliability engineering (SRE) employees are partitioned from the rest of the organization, they must be aware of platform engineering decisions and able to contact platform engineering teams during an outage. At a high level, platform engineers should be responsible for building platforms, while SREs should keep the platforms running.

Building a platform engineering team

It’s important to outline the various roles on a platform engineering team in order to properly assemble a team. Platform engineering is a discipline, not just a group of engineers, so the team should have cross-functional abilities. The final structure of a platform engineering team will ultimately depend on an organization’s size, whether technology is an enabler or core competency for the firm, and the number of other teams that the platform team must support.

Platform teams are often made up of a combination of the following roles:

• Platform architects: Senior, top-level influencers who have a breadth of knowledge about making technology decisions, evaluating tradeoffs, and understand system design principles. Platform architects should collaborate with the firm’s senior-level leadership, meet across silos, and be given the authority to make decisions.
• Platform engineers: These engineers are potentially former sysadmins or DevOps professionals who have the ability to leverage purpose-built tools such as CloudBolt, Terraform, and public cloud provider interfaces to build and manage platforms. If your company leverages containerization or Kubernetes, platform engineers must also understand networking abstractions and virtualization.
• FinOps analysts: FinOps professionals may be involved in procurement or vendor negotiations, but their primary contribution to the team should be a finance-first approach to spending money on platforms. They should be able to measure ROI along with the metrics and KPIs highlighted in the section above.
• Product managers: When an organization forms a platform engineering team, it shifts its focus so that internal users become customers. A product manager with a user-first or strong UX mindset can be a force multiplier to identify pain points and design platforms with end-users in mind.

The best platform engineering teams are made up of people with a combination of broad and deep experience in technology, such as former solutions or enterprise architects. Many platform professionals have successfully worked on both the vendor and customer sides of software companies or consultancies and understand why efficiency is important for a business.

Why developers are not platform engineers

Developers are often asked to expand the scope of their work into managing the infrastructure upon which they build. Engineering interviews have gone beyond coding challenges and system design to managing cloud infrastructure and deploying CI/CD pipelines.

However, while learning and expanding capabilities is important, each new technology introduces vast learning requirements and additional cognitive load. Cognitive load is a concept from the late 1980s that theorizes that humans hit a limit in the amount of information that they can retain and process at once. Once this limit is reached, they struggle to complete the tasks they’re working on, especially if the tasks are complex or technical. 

Simply put, developers cannot scale to learn every technology that a company introduces. If they are asked to do so, organizations end up with a lot of generalists who can only manage the basics of each platform and miss out on the benefits of their specialized knowledge.

Successful platform engineering should enable developers and other internal users to be productive in a safe and purpose-built walled garden. A common example of early platform engineering success is generalizing DevOps toolkits across an organization. If deployment pipelines are prebuilt and follow an organization’s best practices for security, networking, and service selection, then developers should be able to visit an internal developer portal or service catalog and quickly start pushing new project code.

By focusing on developers as platform engineering customers instead of platform contributors, you will ultimately minimize shadow IT risks while giving them access to the best tools they need to do their jobs effectively.

The platform engineering tooling landscape

The average IT estate involves hundreds of disparate tools, platforms, and systems. Without an accessible and intelligent framework for centralizing services, platform teams will quickly fall back on silos of process and technology patterns.

There are several out-of-the-box tools and services that make managing platforms easier for quick wins. The primary technology goal of platform engineering leadership is to create a “single pane of glass” experience for users by deploying a cohesive model for the creation, delivery, and oversight of IT projects, services, and actions.

It can be difficult to determine which platform engineering tools to adopt first. Some teams jump to Terraform or other infrastructure-as-code tools. However, Terraform is just one tool in a large toolbox, and it should be used in conjunction with other platform tooling. 

A good rule of thumb before adding a new tool or capability is to compare the initial complexity introduced to the engineering time saved in the long run. Overengineering a platform solution can become a net drain on resources and energy if the tools introduce unnecessary features or functionality. An early example of this was when organizations rushed to adopt Kubernetes without first evaluating whether containerization was actually the best abstraction for their workloads. Deploying platform engineering solutions may look easy, but deploying them correctly can be hard.

Tools such as CloudBolt make things easier by integrating your technology portfolio into a unified control plane and providing a public-cloud-like experience for your end-users. This brings a modern experience and innovative capabilities to all clouds, public or private, and allows organizations to centralize upon a self-hosted platform.

Level Up Your Data Center With CloudBolt

Learn More

Integrate with best of breed tools like Terraform and Ansible

Create a self-service UI-based catalog of controlled automation for standard IT requests

Improve security posture and regulatory compliance with preset policies

Summary of key concepts

With the expansion of technologies across organizations over the past several years, effective platform engineering has introduced a paradigm shift and is an important discipline that ensures strategic technology decision-making.

Assembling a platform engineering team does not have to involve massive changes to current organizational structures. Existing employees are likely performing platform engineering duties in addition to their day-to-day jobs and can be brought together to evolve in a new direction.

Centralizing platform engineering knowledge and tooling helps create best-in-class experiences for internal users, including developers, to work efficiently within the bounds of an organization’s technology decisions. Procuring and implementing the best platform engineering tools reduces the overhead on platform engineering teams, so they can focus on improving end-user experiences.