Best Practices for Short-term and Permanent Flags

407
LaunchDarkly
Serving over 200 billion feature flags daily to help software teams build better software, faster. LaunchDarkly helps eliminate risk for developers and operations teams from the software development cycle.

This post is by Dawn Parzych of LaunchDarkly

What are best practices when it comes to feature flags? The answer to that as with most questions is “it depends.”

  • It depends on whether the flag is a short-term or permanent flag.
  • It depends on the purpose of the flag.
  • It depends on your specific business needs. What works for one company may not work for you.

This blog series will dive into best practices for feature flags, starting with best practices pertaining to short-term and permanent flags.

Determine whether you need a short-term or permanent flag

The first recommendation is to determine whether a flag will be a short-term or permanent flag as that will influence future decisions and best practices.

Short-term flags

A short-term flag has a limited lifespan. Remove the flag once it has fulfilled its business purpose. When thinking of feature flags, most people think of short-term flags. Types of short-term flags include:

Release: Slowly exposing a feature to new users—moving from internal users to beta and/or canary users until 100% of users are receiving the feature. When you reach 100% remove the flag (unless it is needed as a circuit-breaker as described below).

A/B testing: Segmenting your population to determine a preference for one option over another. Once testing ends, remove the flag, and 100% of users should receive the preferred variation.

Operational interaction testing: When rolling out a new microservice, infrastructure component, or third-party tag, a flag can be used to determine the impact on systems. If the CPU spikes or there is a memory leak, or unexpected errors occur, disable the new element while further troubleshooting takes place.

Kill switches: A toggle to disable a single feature when things start to go sideways during a release.

After reading those descriptions, you may be thinking that a release flag sounds a lot like an operational interaction testing flag. And they are similar. The primary difference is in who is the controller. An operational interaction flag is controlled by the ops team to protect the systems where the release flag is controlled by the product or business owner to control how user adoption progresses.

Permanent flags

A permanent flag is designed to provide control for an extended time after the release of a given feature. In some cases, the flag will be in existence for the life of a feature. Types of permanent flags include:

Entitlements: Giving users access to certain features based on a subscription level or authentication status.

Circuit breakers/Load shedding: Having the ability to quickly turn a feature off or terminate a connection when problems arise prevents problems from impacting all users. These flags are often activated based on an event. For example, a monitoring tool generates an alert when orders fail to complete. When the alert is triggered, a flag is toggled setting the site to ‘read-only.’

White labeling: Configuring the look and feel of an application for each client for a white-labeled solution.

Accessibility: Allowing users to select which accessibility options and refinements they prefer.

Feature flag best practices for all flags

Whether you have a permanent or short-term flag, consider these best practices.

Make flag planning a part of feature design

Feature flags shouldn’t be an afterthought. If you think about feature flags during the design process, you will be setting yourself up for success. Part of the planning includes whether the flag will be a short-term or permanent flag. This decision will then impact other areas such as a naming convention, configuration settings, review and removal processes, and access control and safety checks. We suggest proper planning up front for all flags.

Standardize naming

You may have a style guide for code with conventions on how to write code for your application that includes things like when and where to use camelCase or the proper use of indentation. These style conventions make it easier to read and understand the code.

Before creating your first flag, come up with a naming convention to be used. Our first recommendation is for verbose flag names, don’t try and be brief. You want people to know what the flag does. Verbose flag names can be helpful for others to understand what the flag is doing.

Things to consider when writing the style guide or naming convention.

  • Be descriptive about the flag’s behavior.
  • Include a prefix with a project name or team name.
  • Indicate whether the flag is temporary or permanent.
  • Include a creation date for the flag. (This will be helpful when cleaning up flags, more on this below).
  • Whether or not to use flag in the name. If using a service like LaunchDarkly using flag in the name is redundant. If using a home-grown solution using flag in the name may help clarify the purpose of the code.

For example, you are creating a flag to progressively roll out and test a new chatbox widget of your UI. This will be a short-term flag. Without a naming convention in place, you may end up with a flag called “brand-new-flag” or “new-UI-widget.”

These names don’t tell us a whole lot about this flag. But with a standard naming convention in place that addresses all of the above, you can create a more descriptive flag name like “aTeam-chatbox-widget-temp-030619.” We know from the name, this is a temporary flag for a chatbox widget created by the “a team” on June 3rd, much better.

Minimize the reach of a flag

The focus of a flag should be small. Having a flag that controls more than one feature action at a time can be confusing and will make troubleshooting issues harder. Think about the smallest unit of logic needed for the most responsive flag. If there are multiple parts to a feature that have to work together, you can create a master flag as a dependency.

For example, say you’re launching a new dashboard in your application. The dashboard has three widgets. You should create a total of four flags: one flag for each widget with a dependency on a fourth flag for the main dashboard. With this scenario, if one widget causes problems the dashboard with two widgets can still be served.

Review use at regular intervals

Whether you are creating a short-term or permanent flag you need to review flag use at regular intervals. The frequency at which you review the flags may vary based on business requirements and the type of flag. To avoid accrual of technical debt, review both permanent and short-term flags at a regular cadence.

For short-term flags, look to see if the flag has rolled out to 100% of users, or if a flag is served to no users. For permanent flags, examine whether the flag is still needed—was a feature once part of a premium bundle and is now available for all users.

We will cover specific criteria related to removing short term flags below.

Best Practices for permanent flags

Establish access control and safety checks

If you have regularly scheduled flag clean-up events, you may worry about the accidental removal of permanent flags. Minimize the risk by implementing access control and safety-checks.

Within LaunchDarkly a flag cannot be deleted without confirmation, but that is a partial solution. There are two additional ways to implement access controls for added peace of mind.

  • Use tags and custom roles to assign permissions to flags within LaunchDarkly quickly.
  • Set role-based access control (RBAC) to specify who can delete flags in a given environment.

Best Practices for Short-term flags

Create a process for removing flags before you create one

Coding a flag is a two-part process. The act of removing a flag should not be a separate process from the act of creating a flag. As mentioned above, you should plan for flags during the feature design process. This includes the removal of short-term flags. An easy way to handle the removal process is to write a pull request to remove the flag at the time you create it. Schedule a Github reminder for after the feature is deployed to review and determine if the PR to remove the flag should be committed.

Conduct regular clean-up and review cycles

Avoiding and eliminating technical debt is necessary. If you don’t pay attention to your flags and conduct regular clean-up and review cycles, you can quickly end up with hundreds or thousands of unused feature flags. Here are some ways to schedule flag clean-up.

  • Schedule time at the end of every sprint to review existing flags.
  • Schedule a clean-up/refactoring sprint at a regular cadence (quarterly, semi-annually, whatever works for your business) to pull out old flags & tags.
  • Make it a competition. Hold a “Capture the Flag” day. The individual or team that removes the most flags wins. (Caution: make sure teams aren’t gaming the system and creating fake flags just to delete them.)

Within LaunchDarkly, we make it easier to identify which flags for removal.

You can:

  • Filter tags by create date to view the oldest flags.
  • View which variations of a flag were recently served. Are 100% of users receiving the same variation?
  • Filter on the last evaluation date of a flag.

Once you have identified a flag for removal, use code references to find all the instances of that flag in your code base.

If not using LaunchDarkly a consistent naming strategy can help you grep the code for instances of flags.

Conclusion

Although I refer to these as best practices, other practices may be better. These are recommendations that we follow and have heard from other customers. Over time these recommendations may change as we learn more. These should be seen as recommendations and suggestions, feel free to modify and alter based on your specific needs. What works well for one company or team may not work well for another team. Future blogs in this series will cover best practices and recommendations around specific use cases. If there is a best practice you would like to share please drop us a line.

Look for more best practices blogs coming soon:

Operational flags best practices

Release management flags best practices

Experimentation flags best practices

Entitlement flags best practices

LaunchDarkly
Serving over 200 billion feature flags daily to help software teams build better software, faster. LaunchDarkly helps eliminate risk for developers and operations teams from the software development cycle.
Tools mentioned in article
Open jobs at LaunchDarkly
IT Support Engineer
Oakland, CA
LaunchDarkly is a rapidly growing software company with a strong mission and vision carried out by a talented and diverse team of employees. Our goal is to help teams build better software, faster. You'll join a small team from companies like Atlassian, Intercom, and GitHub, and you'll have an immediate impact on our product and customers. We are looking for an IT Service Desk / Support Engineer to be a first line of defense for our growing IT support team. We currently have over 150 employees globally and will be adding additional headcount over the next year. This role will help to enable the scale of the company by being the face of the technical service desk and IT team. This position will be responsible for maintaining the service level of our service desk and will also be the first point of contact for all technical matters. This role will report directly to the IT Lead.
  • Be a primary responder for LaunchDarkly's service desk
  • Support a primarily Mac-based environment with some Windows / Linux devices
  • Install, configure and troubleshoot computer hardware
  • Assist with on-boarding and off-boarding LaunchDarkly employees
  • Support company AV suites and All Hands meetings
  • 1-3 years of experience as a technical support expert supporting users in a corporate environment
  • Excellent interpersonal and customer service skills. Interact well at all levels within the organization.
  • Well-organized and highly efficient
  • You bring a curiosity for technology trends and implementing cutting edge technical solutions
  • Strong documentation and communication skills
  • Okta
  • JAMF
  • Asset management
  • On-boarding/Off-boarding Automation
  • AV room and video conference platforms like Zoom, WebEx, BlueJeans
  • Supported a user base using SAAS based ticketing system
  • Supported Wireless 802.11X installations in corporate settings
  • Ability to troubleshoot complex systemic problems
  • Ability to troubleshoot Apple hardware
  • Systems administration for mac and Linux systems.
  • General understanding of ITIL, OSI Model, ISO 27001, SOC II
  • Scripting or Automating SaaS applications
  • GSuite administration
  • Atlassian administration
  • A+ Certification
  • Okta helpdesk certification
  • Solutions Engineer - Sydney, Australi...
    Sydney
    As a Solutions Engineer, you will educate and guide prospects on the proper implementation of LaunchDarkly's SaaS product and Private Instances. You will be the technical voice during our sale and ensure our customers are comfortable with the way our systems work. You are passionate about the developer tools space and helping development teams eliminate risk and deliver value. This role will support the Eastern timezone, as such candidates who are in New York or the East Coast are preferred. LaunchDarkly is a rapidly growing software company with a strong mission and vision carried out by a talented and diverse team of employees. Our goal is to help teams build better software, faster. You'll join a small team from companies like Atlassian, Intercom, and GitHub, and you'll have an immediate impact with our product and customers.
  • Evangelize and advise customers on the importance and different uses of feature flags and how to administer them
  • Create solutions to customer's challenges implementing feature flags across large monolith and microservice applications, large organizations, and different technology stacks
  • Become a domain expert on LaunchDarkly architecture
  • Demo LaunchDarkly product to technical and business audiences
  • Become a subject matter expert on LaunchDarkly and communicate our value and features to potential customers
  • Be the voice of the customer by translating, aggregating, and representing customer feedback to the Product and Engineering teams
  • You learn and synthesize large amounts of information with little context
  • You are an effective communicator and you can simplify complex technical concepts
  • You are a self‐starter and problem solver, willing to take on hard problems and work independently when necessary.
  • You have 4+ years experience consulting with enterprise customers and large development teams
  • You led successful technical proof of concepts and built strong customer relationships
  • You are passionate about trends and technologies involved in modern application development
  • You have worked with teams that underwent development process transformation
  • You are familiar with at least one of our supported languages: Java, .NET, GO, JS, Python, PHP, node, Ruby, Rails, iOS, or Android
  • You have worked with data persistence technologies like Varnish or Redis
  • Director, Infrastructure Delivery
    Oakland, CA
    Software powers the world, and LaunchDarkly empowers all teams to deliver and control their software. We serve hundreds of billions of feature flags daily, to help teams ship better software, faster and eliminate risk for companies big and small. We're based in downtown Oakland and growing quickly.  Infrastructure Delivery at LaunchDarkly is an engineering discipline responsible for building and delivering technology solutions for LaunchDarkly employees. The team's mission is to use technology to help make LaunchDarkly employees efficient and happy. It does this by delivering IT services and infrastructure (e.g. help desk, networking, physical security), building internal tools and services and delivering BI and data tools.  The ideal fit is someone who can dig into business challenges and ask core questions, ensure the effective use of data, and build convincing plans for action. The Director of Infrastructure Delivery will build and manage the department and report directly to the CTO. 
  • Hire and manage IT, Tools, and Data teams
  • Run the team with the same level of discipline, process, and rigor that we use on our product-focused engineering teams 
  • Help your team understand their mission and how they fit in with the overall business goals
  • Run a software team charged with crafting and maintaining internal LaunchDarkly tools using modern software development processes and stacks (React, Go, TypeScript, AWS, Terraform etc.)
  • Define and measure success criteria for your functional teams
  • Lead IT operations for all LaunchDarkly’s  locations, ensuring employee experience as measured by user satisfaction surveys and internal SLAs.
  • Manage the inventory of LaunchDarkly's IT assets, ensuring budget compliance, and best practices.
  • Determine business requirements for IT systems and coordinate activities to ensure Availability.
  • Effectively communicate with stakeholders regarding the effectiveness and progress of business systems and enterprise applications, as well as IT projects and performance.
  • Efficiently manage LaunchDarkly's global IT budget, anticipating for company growth as required.
  • Help the data team build an operate a scalable, reliable data pipeline and data warehouse.
  • Help identify and answer the business' most pressing questions.
  • Build tools and teach people how to find answers to their own questions using trustworthy data.
  • Establish and maintain strong relationships with vendors, MSPs, suppliers, and key consultants.
  • Three or more years’ experience leading IT or Engineering teams: hiring, motivating, growing, empowering, and performance managing.
  • A history of creating or maintaining great team cultures.
  • Skills and knowledge that will establish credibility with your team.
  • You understand build vs. buy dynamics and know when each is appropriate
  • Demonstrated analytical, problem-solving, organizational, interpersonal and communication skills.
  • Organized, thorough and diligent with an acute attention to detail.
  • Knowledge of middleware integration tools like (Bettercloud, Zapier, Celigo) 
  • Verified by
    Director Marketing
    VP of Product and Engineering
    You may also like