Best Practices for Short-term and Permanent Flags

899
LaunchDarkly
Serving over 200 billion feature flags daily to help software teams build better software, faster. LaunchDarkly helps eliminate risk for developers and operations teams from the software development cycle.

This post is by Dawn Parzych of LaunchDarkly

What are best practices when it comes to feature flags? The answer to that as with most questions is “it depends.”

  • It depends on whether the flag is a short-term or permanent flag.
  • It depends on the purpose of the flag.
  • It depends on your specific business needs. What works for one company may not work for you.

This blog series will dive into best practices for feature flags, starting with best practices pertaining to short-term and permanent flags.

Determine whether you need a short-term or permanent flag

The first recommendation is to determine whether a flag will be a short-term or permanent flag as that will influence future decisions and best practices.

Short-term flags

A short-term flag has a limited lifespan. Remove the flag once it has fulfilled its business purpose. When thinking of feature flags, most people think of short-term flags. Types of short-term flags include:

Release: Slowly exposing a feature to new users—moving from internal users to beta and/or canary users until 100% of users are receiving the feature. When you reach 100% remove the flag (unless it is needed as a circuit-breaker as described below).

A/B testing: Segmenting your population to determine a preference for one option over another. Once testing ends, remove the flag, and 100% of users should receive the preferred variation.

Operational interaction testing: When rolling out a new microservice, infrastructure component, or third-party tag, a flag can be used to determine the impact on systems. If the CPU spikes or there is a memory leak, or unexpected errors occur, disable the new element while further troubleshooting takes place.

Kill switches: A toggle to disable a single feature when things start to go sideways during a release.

After reading those descriptions, you may be thinking that a release flag sounds a lot like an operational interaction testing flag. And they are similar. The primary difference is in who is the controller. An operational interaction flag is controlled by the ops team to protect the systems where the release flag is controlled by the product or business owner to control how user adoption progresses.

Permanent flags

A permanent flag is designed to provide control for an extended time after the release of a given feature. In some cases, the flag will be in existence for the life of a feature. Types of permanent flags include:

Entitlements: Giving users access to certain features based on a subscription level or authentication status.

Circuit breakers/Load shedding: Having the ability to quickly turn a feature off or terminate a connection when problems arise prevents problems from impacting all users. These flags are often activated based on an event. For example, a monitoring tool generates an alert when orders fail to complete. When the alert is triggered, a flag is toggled setting the site to ‘read-only.’

White labeling: Configuring the look and feel of an application for each client for a white-labeled solution.

Accessibility: Allowing users to select which accessibility options and refinements they prefer.

Feature flag best practices for all flags

Whether you have a permanent or short-term flag, consider these best practices.

Make flag planning a part of feature design

Feature flags shouldn’t be an afterthought. If you think about feature flags during the design process, you will be setting yourself up for success. Part of the planning includes whether the flag will be a short-term or permanent flag. This decision will then impact other areas such as a naming convention, configuration settings, review and removal processes, and access control and safety checks. We suggest proper planning up front for all flags.

Standardize naming

You may have a style guide for code with conventions on how to write code for your application that includes things like when and where to use camelCase or the proper use of indentation. These style conventions make it easier to read and understand the code.

Before creating your first flag, come up with a naming convention to be used. Our first recommendation is for verbose flag names, don’t try and be brief. You want people to know what the flag does. Verbose flag names can be helpful for others to understand what the flag is doing.

Things to consider when writing the style guide or naming convention.

  • Be descriptive about the flag’s behavior.
  • Include a prefix with a project name or team name.
  • Indicate whether the flag is temporary or permanent.
  • Include a creation date for the flag. (This will be helpful when cleaning up flags, more on this below).
  • Whether or not to use flag in the name. If using a service like LaunchDarkly using flag in the name is redundant. If using a home-grown solution using flag in the name may help clarify the purpose of the code.

For example, you are creating a flag to progressively roll out and test a new chatbox widget of your UI. This will be a short-term flag. Without a naming convention in place, you may end up with a flag called “brand-new-flag” or “new-UI-widget.”

These names don’t tell us a whole lot about this flag. But with a standard naming convention in place that addresses all of the above, you can create a more descriptive flag name like “aTeam-chatbox-widget-temp-030619.” We know from the name, this is a temporary flag for a chatbox widget created by the “a team” on June 3rd, much better.

Minimize the reach of a flag

The focus of a flag should be small. Having a flag that controls more than one feature action at a time can be confusing and will make troubleshooting issues harder. Think about the smallest unit of logic needed for the most responsive flag. If there are multiple parts to a feature that have to work together, you can create a master flag as a dependency.

For example, say you’re launching a new dashboard in your application. The dashboard has three widgets. You should create a total of four flags: one flag for each widget with a dependency on a fourth flag for the main dashboard. With this scenario, if one widget causes problems the dashboard with two widgets can still be served.

Review use at regular intervals

Whether you are creating a short-term or permanent flag you need to review flag use at regular intervals. The frequency at which you review the flags may vary based on business requirements and the type of flag. To avoid accrual of technical debt, review both permanent and short-term flags at a regular cadence.

For short-term flags, look to see if the flag has rolled out to 100% of users, or if a flag is served to no users. For permanent flags, examine whether the flag is still needed—was a feature once part of a premium bundle and is now available for all users.

We will cover specific criteria related to removing short term flags below.

Best Practices for permanent flags

Establish access control and safety checks

If you have regularly scheduled flag clean-up events, you may worry about the accidental removal of permanent flags. Minimize the risk by implementing access control and safety-checks.

Within LaunchDarkly a flag cannot be deleted without confirmation, but that is a partial solution. There are two additional ways to implement access controls for added peace of mind.

  • Use tags and custom roles to assign permissions to flags within LaunchDarkly quickly.
  • Set role-based access control (RBAC) to specify who can delete flags in a given environment.

Best Practices for Short-term flags

Create a process for removing flags before you create one

Coding a flag is a two-part process. The act of removing a flag should not be a separate process from the act of creating a flag. As mentioned above, you should plan for flags during the feature design process. This includes the removal of short-term flags. An easy way to handle the removal process is to write a pull request to remove the flag at the time you create it. Schedule a Github reminder for after the feature is deployed to review and determine if the PR to remove the flag should be committed.

Conduct regular clean-up and review cycles

Avoiding and eliminating technical debt is necessary. If you don’t pay attention to your flags and conduct regular clean-up and review cycles, you can quickly end up with hundreds or thousands of unused feature flags. Here are some ways to schedule flag clean-up.

  • Schedule time at the end of every sprint to review existing flags.
  • Schedule a clean-up/refactoring sprint at a regular cadence (quarterly, semi-annually, whatever works for your business) to pull out old flags & tags.
  • Make it a competition. Hold a “Capture the Flag” day. The individual or team that removes the most flags wins. (Caution: make sure teams aren’t gaming the system and creating fake flags just to delete them.)

Within LaunchDarkly, we make it easier to identify which flags for removal.

You can:

  • Filter tags by create date to view the oldest flags.
  • View which variations of a flag were recently served. Are 100% of users receiving the same variation?
  • Filter on the last evaluation date of a flag.

Once you have identified a flag for removal, use code references to find all the instances of that flag in your code base.

If not using LaunchDarkly a consistent naming strategy can help you grep the code for instances of flags.

Conclusion

Although I refer to these as best practices, other practices may be better. These are recommendations that we follow and have heard from other customers. Over time these recommendations may change as we learn more. These should be seen as recommendations and suggestions, feel free to modify and alter based on your specific needs. What works well for one company or team may not work well for another team. Future blogs in this series will cover best practices and recommendations around specific use cases. If there is a best practice you would like to share please drop us a line.

Look for more best practices blogs coming soon:

Operational flags best practices

Release management flags best practices

Experimentation flags best practices

Entitlement flags best practices

LaunchDarkly
Serving over 200 billion feature flags daily to help software teams build better software, faster. LaunchDarkly helps eliminate risk for developers and operations teams from the software development cycle.
Tools mentioned in article
Open jobs at LaunchDarkly
Application Engineer
- US

As a SaaSOps Engineer, you are passionate about learning and automation technologies. You will work closely with your IT engineering peers to identify bottlenecks, improve system efficiency by reducing manual tasks through automation and ensure our core systems are designed to scale as our team grows. 

The ideal candidate is an excellent communicator and fast learner who is looking for a supportive environment to continue to grow their technical skills and expertise.

You will be a stakeholder in the discovery, procurement, implementation and management of our growing corporate infrastructure. You have a passion for innovation and creative problem solving, as well as the ability to be adaptable to the constantly changing technology.

LaunchDarkly is a rapidly growing software company with a strong mission and vision carried out by a talented and diverse team of employees. Our goal is to help teams build better software, faster. You'll join a small team from companies like Atlassian, Intercom, and GitHub, and you'll have an immediate impact with our product and customers.

What you'll do:

  • Perform day-to-day administrative tasks on SaaS applications within our stack.
  • Handle technical escalations, provide timely resolution of problems or engage lead staff for assistance.
  • Support SSO implementations using Okta.
  • Identify ways to improve efficiency and scale services leveraging tools and automation.
  • Build workflows to enable seamless and secure access to applications that support our internal customers.
  • Help build, scale, and maintain system monitoring and alerting tools
  • Compose scripts in an administrative language (Python, Ruby, or Shell)
  • Customize, create, and package software, scripts, and applications for deployment. 
  • Collaborate with cross functional teams to solve business problems.
  • Create IT systems’ compliance policies and provide compliance proof for audits.
  • Knowledge of SSH, keystores, security certificates, user and password management, Single-Sign-On integration, and authorization tokens.
  • Document and update processes and procedures for supporting applications across the org.
  • Drive tool adoption and proficiency through training and documentation.
  • Support initiatives to continuously improve, integrate, and consolidate systems company-wide

Who you are:

  • Experience with SaaS application management including: Okta, GSuite, Slack and AWS
  • Project management, system implementation, and system integration experience
  • Scripting expertise with GAM, Python, TerraForm, JSON, Perl, Powershell or equivalent tools
  • Proficient with API integrations
  • JAMF administration, package creation, policy scripting
  • Strong analytical and problem-solving skills; ability to work creatively in a problem-solving environment
  • Ability to balance technical skills with business savvy – ability to transform end-user needs into technical and functional requirements

You may also know: (Bonus Skills)

  • Experience with Terraform, Docker, Container Management, and all things AWS

About LaunchDarkly:

LaunchDarkly is a Feature Management Platform that serves hundreds of billions of feature flags daily to help software teams build better software, faster. Feature flagging is an industry standard methodology of wrapping a new or risky section of code or infrastructure change with a flag. Each flag can easily be turned off independent of code deployment (aka "dark launching"). LaunchDarkly has SDKs for all major web and mobile platforms. We are building a diverse team so that we can offer robust products and services. Our team culture is dynamic, friendly, and supportive. Our headquarters are in Oakland.

At LaunchDarkly, we believe in the power of teams. We're building a team that is humble, open, collaborative, respectful and kind. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity, sexual orientation, age, marital status, veteran status, or disability status.

Don't let the confidence gap get in the way of applying! We'd love to hear from you.

We've partnered with KeyValues to help demonstrate the amazing culture we've built here at LaunchDarkly, find more info at https://www.keyvalues.com/launchdarkly

Backend Engineer
- US
As a Backend Engineer, you will help us build features, design and implement API methods, and improve the performance and reliability of our systems. We're looking for someone who knows what it takes to deliver value to customers and takes pride in the quality of their work.

Our platform serves over twenty billion feature flags daily. The core technologies we use daily include Golang, MongoDB, ElasticSearch, Redis, and NATS. As part of our you-build-it-you-run-it culture, all developers may be responsible of support applications in production, including on-call. On call is compensated in addition.

What you'll get to do: 

  • Build and expand our APIs and services, written in Go
  • Collaborate with frontend engineers to deliver user-facing features
  • Monitor and improve server-side performance
  • Write unit, integration, and load tests as necessary
  • Actively participate in code reviews
  • Write and review technical proposals
  • Improve engineering standards, tooling, and processes
On day one, you should have:
  • Proven experience and fluency with server-side web development (e.g. in Java / Scala, Ruby, Python, Golang, Node.js)
  • Experience building RESTful APIs
  • Strong computer science fundamentals: data structures, distributed systems, concurrency, and threading
  • Strong communication skills, a positive attitude, and empathy
  • You write code that can be easily understood by others, with an eye towards maintainability
  • You hold yourself and others to a high bar when working with production systems
  • You value high code quality, automated testing, and other engineering best practices

Bonus Points:

  • Experience with NoSQL databases (MongoDB, ElasticSearch)
  • A deep understanding of networking technologies (TCP, HTTP, websockets, server-sent events, etc.)

About LaunchDarkly:

LaunchDarkly is a Feature Management Platform that serves hundreds of billions of feature flags daily to help software teams build better software, faster. Feature flagging is an industry standard methodology of wrapping a new or risky section of code or infrastructure change with a flag. Each flag can easily be turned off independent of code deployment (aka "dark launching"). LaunchDarkly has SDKs for all major web and mobile platforms. We are building a diverse team so that we can offer robust products and services. Our team culture is dynamic, friendly, and supportive. Our headquarters are in Oakland.

At LaunchDarkly, we believe in the power of teams. We're building a team that is humble, open, collaborative, respectful and kind. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity, sexual orientation, age, marital status, veteran status, or disability status.

Don't let the confidence gap get in the way of applying! We'd love to hear from you.

We've partnered with KeyValues to help demonstrate the amazing culture we've built here at LaunchDarkly, find more info at https://www.keyvalues.com/launchdarkly

 

Demo Engineer
Oakland, California, United States

As a Demo Engineer, you will be in a unique position to build the foundation and strategy for how we visually articulate our technology to our prospects and customers. You will be responsible for building and maintaining the demo environments that the Revenue organization uses during a deal cycle and will directly impact every single opportunity the team works on. You will be working with the latest technologies in the space, allowing you to use your creativity to craft compelling storylines that speak to the value proposition of our solution. Collaboration will be key in this role and will require you to work closely with our amazing SEs, PMs, Engineers, and Marketing to ensure that we are building a strong and relevant message.

LaunchDarkly is a rapidly growing software company with a strong mission and vision carried out by a talented and diverse team of employees. Our goal is to help teams build better software, faster. 

Software powers the world and LaunchDarkly empowers all teams to deliver and control their software.

Responsibilities:

  • Solicit feedback from SEs, Product, Marketing, and our customers to initially understand the current state of our demo product & process, and continuously drive this feedback loop to ensure we’re constantly adapting to our changing product and market
  • Own developing the strategy for all things demo-related and driving priorities within that strategy to align to the highest impact gaps
  • Become a LaunchDarkly expert understanding our SDKs and platform in details, as well as the growing number of integrations to tell a rich end to end story around feature management
  • Collaborate closely with Product Managers as we release new product features to guarantee that our demo experience highlights new functionality in LaunchDarkly's platform as it is available
  • Write and maintain documentation on how the demo environments work in-depth and how to use it

Basic Qualifications:

  • 2+ years of experience in a demo org, technically oriented, or pre-sales role: Demo Engineer, Solutions Engineer, Solutions Consultant, or similar experience in an engineering role
  • Programming experience required, preferably in Java, .NET. GO and/or python
  • Ability to learn and synthesize large amounts of information with little context
  • A self‐starter and problem solver, willing to take on hard problems and work independently when necessary.
  • Superior communications skills (presentation, written, and verbal) and demonstrated ability to communicate/present/motivate effectively across technical and selling teams
  • Technical proficiency creating and maintaining demo environment

Preferred Qualifications:

  • Experience working with teams that underwent development process transformation
  • Experience with data persistence technologies like Varnish or Redis
  • Full-stack and mobile development experience

About LaunchDarkly:

LaunchDarkly is a Feature Management Platform that serves hundreds of billions of feature flags daily to help software teams build better software, faster. Feature flagging is an industry standard methodology of wrapping a new or risky section of code or infrastructure change with a flag. Each flag can easily be turned off independent of code deployment (aka "dark launching"). LaunchDarkly has SDKs for all major web and mobile platforms. We are building a diverse team so that we can offer robust products and services. Our team culture is dynamic, friendly, and supportive. Our headquarters are in Oakland.

At LaunchDarkly, we believe in the power of teams. We're building a team that is humble, open, collaborative, respectful and kind. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity, sexual orientation, age, marital status, veteran status, or disability status.

Don't let the confidence gap get in the way of applying! We'd love to hear from you.

We've partnered with KeyValues to help demonstrate the amazing culture we've built here at LaunchDarkly, find more info at https://www.keyvalues.com/launchdarkly.

Developer Advocate
Oakland, California, United States

As a Developer Advocate, you will join our Developer Marketing team, writing, speaking, and creating other forms of content to help expand the awareness of Feature Management. Our team helps make developers more successful by improving their software development practices. The content we provide covers both technical as well as human-centered topics. We are looking to expand our team with a Developer Advocate whose primary focus is writing but does not hesitate to take the stage (virtually or eventually in-person).

Responsibilities:

  • Write, speak, and create content about technology trends with the goal of engaging our developers, developer managers, and senior technical leaders.
  • You will drive awareness about best practices for feature management and modern application architecture with the goal of influencing customer success.
  • Develop a persona as a trusted advisor in Feature Management and modern application development trends.

About You:

  • You have passion, curiosity, technical depth, and extraordinary written and verbal communication skills.
  • You are a creative and skilled storyteller.
  • You are able to converse with a broad range of programming language communities (Java, .NET, Node.js, Python, Ruby, iOS, Android, etc.), and have a real passion for modern application development trends at the intersection of development and operations.
  • You understand that the hardest parts about DevOps practices revolve around the people, not the process or technology.
  • You manage multiple deadlines and projects independently.
  • You have publicly available writing samples and/or recordings of presentations.

Bonus points if you also:

  • Experience in the DevOps space
  • Interest in producing short videos for a technically-savvy audience.

Specific Requirement:

  • 25% travel required (when travel is a thing again)
  • Must be authorized to work in the country you reside in (no visa sponsorship / relocation)
  • Must have published writing samples
  • Minimum 5 years work experience

About LaunchDarkly:

LaunchDarkly is a Feature Management Platform that serves hundreds of billions of feature flags daily to help software teams build better software, faster. Feature flagging is an industry standard methodology of wrapping a new or risky section of code or infrastructure change with a flag. Each flag can easily be turned off independent of code deployment (aka "dark launching"). LaunchDarkly has SDKs for all major web and mobile platforms. We are building a diverse team so that we can offer robust products and services. Our team culture is dynamic, friendly, and supportive. Our headquarters are in Oakland.

At LaunchDarkly, we believe in the power of teams. We're building a team that is humble, open, collaborative, respectful and kind. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity, sexual orientation, age, marital status, veteran status, or disability status.

Don't let the confidence gap get in the way of applying! We'd love to hear from you.

We've partnered with KeyValues to help demonstrate the amazing culture we've built here at LaunchDarkly, find more info at https://www.keyvalues.com/launchdarkly.

Verified by
Special Circumstances
Head of Ecosystem Partnerships
Demand Program Manager
Engineering Lead
Director Marketing
VP of Product and Engineering
You may also like