Adopting a Structured RFC Process at Diamond
This article explains why and how we came to adopt a Structured RFC Process at Diamond. We found that this process streamlined the planning and execution of our software development projects. Additionally, we found the format flexible enough to be applicable across our organization in areas other than software development.
As a digital agency partnering with clients that range in scale across various industries, not every project that Diamond takes on requires a consistent roster of skills or follows the same organizational structure or methodologies. However, we pursue reusing and standardizing processes/tools across teams whenever possible.
Project/Task Oriented Team Need
We were contracted to build the tech (a subscription-based audio-on-demand service) for the largest podcast distributor in the US. The initial scope of work included greenfield iOS and Android Apps, cloud services, re-development of an existing website, and account migration to support cross-platform payment. Diamond partnered with this Client through the build phase into stabilizing, maintaining, and enhancing their technology platform, eventually facilitating their acquisition by Amazon. Within the first year, as the platform started scaling, the size of the team grew and subsequently split into several teams. To stay organizationally agile as we grew, we were thoughtful in retrospectives, identified challenges, and took actions to improve our process as we advanced.
One challenge we identified, was that we often received high-level (market-focused) requirements from external product owners/stakeholders that necessitated an engineer perform some technical discovery and we needed to present our discovery - aka "SPIKE" - research in a format that welcomed and encouraged feedback. We wanted discovery feedback cycles to be as asynchronous and efficient as possible (vs. waiting until the next sprint planning or sprint review to present), but we lacked consistency. Various teams were already writing down their plans and designs in documents. However, these varied drastically in format, contents, and objectives without shared and clear guidance or process. There was also variance in how we advertised/distributed these docs to other individuals who should be feedback givers. Additionally, we needed transparency as the project matured and the knowledge base grew. Both within and across the work streams, there was no easy way for an engineer to know what topics people were discussing at a given time or how to provide input on critical architectural decisions or proposed changes to flow.
Engineering Department Level Need
As this client initiative was scaling up, Diamond's Engineering Department was also growing to staff other projects. We had similar needs at the Engineering Department level. Our engineers needed visibility into the topics colleagues were discussing in order to provide input and peer feedback during the decision-making process. We needed to encourage people to expose their ideas and have them debated ethically. We needed a clear and straightforward process that allowed people to share and receive feedback to inform decision-making.
The approach we alighted on was to adopt an RFC (request for comments) process.
What is a RFC?
The term RFC has a long history and is almost as old as the internet itself.
Steve Crocker invented the RFC system in 1969 to help record unofficial notes on ARPANET's development while at UCLA. RFCs have since become official documents of Internet specifications, communications protocols, procedures, and events.
Outside formal standard bodies, people typically use the term RFC to refer to a document describing an idea written by someone who expects feedback on it from their peers.
I stumbled upon Phil Calçado's post, 'A Structured RFC Process,' and his template licensed under Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) was the most suitable format to solve our needs. We have slightly modified the template in our implementation (our changes were limited to metadata fields - we tag commenters in our docs). I highly recommend reading Phil's post because it contains much guidance on using the format, which I will not delve into in this article.
Phil Calçado is a software engineer (and leader) who has worked at high-profile technology companies (ThoughtWorks, SoundCloud, DigitalOcean, Meetup.org), where he implemented his structured RFC process.
Phil's format has a title and a header table for capturing metadata. The remainder of the outline borrows from the SRI's (Stanford Research Institute) NABC value proposition format. An NABC comprises the four fundamentals that define a project's value proposition:
The conciseness of the NABC framework is part of its power.
One of Phil's goals when designing this format was to make the tool useful across an organization in areas other than software development. He also emphasizes that RFCs are not a decision-making process in and of themselves but exist to elicit proposal feedback.
Every RFC has a lifecycle and moves through the following phases:
- Draft: The author(s) are working on the RFC before asking for broader feedback.
- Feedback Requested: The author(s) notify commenters that the RFC is ready for peer review and await feedback.
- Active: The deadline for comments on this RFC has passed, and the author(s) have decided to go ahead.
- Abandoned: The authors have decided not to move forward with the changes proposed in this RFC.
- Retired: The changes proposed on this RFC are no longer in effect; keep the document for historical purposes.
Templatizing within Confluence
Without delving too deep into implementation details, I think it is worth mentioning a little bit about how we integrated this new process into our existing tools. On most of our agile software development projects, Diamond uses the Atlassian Suite for work management. We also use Google Drive as a company, but we favor Confluence as our knowledge base software across projects and departments. I chose to use Confluence as the repository for RFCs because documents also have version control and analytics in addition to Confluence's template and reporting features.
I created a custom RFC Space Template in Confluence, which we used in our Client's spaces and within the Engineering Department space. The space template includes an "RFC" label, which enables the creation of an RFC rollup page per space - a Page Properties Report Macro - that aggregates each space's RFCs and presents metadata for each RFC in a table layout.
Artifacts for Accountability in Decision-Making
The first significant benefit of the Structured RFC approach is stronger distinction between decision-making and feedback gathering. We can create a disagree and commit culture with an appointed accountable team. We will carefully hear all positions and reckons from everyone, but ultimately a decision will be made by a specific person or team. After the person/team makes the decision, everybody will commit to implementing and championing the decision irrespective of differences in opinion during the RFC process. If it turns out that the decision wasn't a good one, the revisit date on the RFC is there to ensure a follow-up discussion occurs in the near future.
A Culture of Transparency and Openness
Another significant benefit of the proposed RFC process is openness. We have fantastic people, and we need to use our collective knowledge as leverage. None of us is as smart as all of us, and it's beneficial for everyone to see what proposals are seeking feedback to facilitate collaboration. RFC should be a safe space for collaboration where comments focus on tangible benefits and tradeoffs. RFC participation is motivating; literally capturing and preserving peoples perspectives, experiences, and contributions.
A Focus on ‘What’ More than ‘How’
The NABC format is an industry tool for making structured "pitches", and using this tool will likely lead us to discuss the what without losing ourselves in an expanse of technical detail.
Option 1: Do Nothing
Any engineer worth their salt will tell you, "laziness is a virtue". We considered the option of not making any change and keeping the ad-hoc model we currently had. Unless something changed, the problems described in this document's "Need" section would remain.
Option 2: Adopt the Official RFC Series Model As-is
We could have followed the official RFC Series model used by the IETF and maybe even required authors to use terms like MUST, SHOULD, and MAY as specified by RFC2119 to avoid ambiguity. However, this approach felt too process-heavy and formal for our use cases.
On the other hand, the Structured RFC Process is about putting forward an idea as early as possible and receiving feedback from a broad audience. With this goal in mind, we should prefer a less formal process.
More Details on the ‘Official’ RFC Series
- Four streams produce RFCs: IETF, IRTF, IAB, and Independent Submissions.
- www.rfc-editor.org is the host location for the RFC Series on the web.
- Given that RFC documents may impact the whole industry, they warrant a complex publishing workflow.
- Not every RFC is a standard. Concerning status, the Internet standardization process assigns a designation to each RFC. This status is one of the following: Informational, Experimental, Best Current Practice, Standards Track, Historic.
- Each RFC is static; if the document is changed, it is submitted again and assigned a new RFC number.
Selected Examples of Official RFCs
Historical standards track RFCs
- Hypertext Transfer Protocol -- HTTP/1.1
- Uniform Resource Identifier (URI): Generic Syntax
- JSON Web Token (JWT)
- Hypertext Transfer Protocol Version 2 (HTTP/2)
Some Fun “Informational” RFCs
- Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0)
- The Arte of ASCII: Or, An True and Accurate Representation of an Menagerie of Thynges Fabulous and Wonderful in Ye Forme of Character
Option 3: Use the Architecture Decision Record (ADR) Format
ADRs are well suited for small changes and implementation details.
An engineer named Michael Nygard published a model to document and manage change in software architecture called Architecture Decision Record (ADR). Its motivation, format, and lifecycle are similar to the Structured RFC approach but are more specialized for software architecture work. ADRs can be used together with the Structured RFC process when developing software systems.
RFC vs. ADR
When it comes time to make a technical decision, many engineers are familiar with documenting it - including the options evaluated, their tradeoffs, and costs - in a decision record format.
At Spotify, their engineering teams typically arrive at decisions through discussion in Request for Comments (RFCs) or during engineering meetings. Spotify outlined its process in a blog post with a decision tree diagram: if there is a blessed solution or if the change is small, write an ADR; otherwise, start with an RFC.
Iterate and Expand
The revisit date in the header table of the RFC serves a function: it encourages reflection.
Project/Task Oriented Case Study
After implementing the Structured RFC process across a significant client initiative, the different teams (mobile, web, platform) used it in a variety of situations:
- Gather feedback on the design for a new service
- Propose a strategy for upgrading shared libraries
- Propose changes to git workflows
- Identify gaps in documentation and propose new artifacts
- Plot enhancements and refactors
- Plan performance testing
The project teams and the Client were happy with this format, which allowed us to streamline backlog refinement and sprint execution.
Department Level Case Study
The structured RFC format was also adopted internally by the Engineering Department to propose big changes to existing processes, capture ad hoc (undocumented) processes, and present new processes for consideration, such as:
- Technical Hiring
- Scalable Resource Sharing
- Contributing Technical Writing to our Blog
It worked again and proved helpful in areas other than applied software development.
After iterating twice with success, we decided to expand the process org-wide. I created an RFC Global Template in Confluence, which is now available for use across all our Confluence Spaces by EVERYONE.
When a proposal has the potential to impact the entire company or at least requires cross-functional visibility, an RFC gets created within our company-wide confluence space. I constructed an RFC rollup report that aggregated all the RFCs within the company-wide space. Then, I wrote the first company-wide RFC, a self-referencing pitch for adopting RFCs org-wide called "A Structured RFC Process at Diamond" (a copy-and-extend version of Phil Calçado's open source text).
When a task-oriented team uses the Structured RFC process for software development, sometimes scoping and rough order of magnitude (ROM) estimates are essential for stakeholders to make an actual decision. To get to those estimates, engineers often need to analyze the "how" (to build), and the technical detail of the document increases.
RFCs that receive the most engagement include visualizations and have metrics.
RFCs generally spend several rounds cycling between the draft and feedback requested states before they are ready to be included in the decision-making process. As an author of an RFC, you should ask for feedback early and often (likely more than once).