Autopsy of DevOps Foundation - Part 1

Explaining the term - DevOps

"The practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support"

Again we can say that ->

"DevOps is also characterized by operation staff, making use of many of the same techniques as developers for their systems work."

The Five Levels of DevOps ~

1. Values
2. Principles
3. Methods
4. Practices
5. Tools

Why should we practice DevOps?

• It’s has been shown to be effective in improving in business outcomes.
• Teams using DevOps Practices
  • Deployed changes 30 times more frequently.
  • Had 200 times shorter lead times.
  • Experienced 60 times fewer failures.
  • Recovered from error 168 times faster.
• Its makes the daily life easier as high tech is a very interrupt driven high pressure exercise.
• DevOps approach reduces the unplanned work and increases friendly relationships between coworkers and reduces stress too.

Extra Notes

DevOps is a combination of two words :

Development + Operations

Where Development includes every one on code side like ->

• Developers
• Front end
• QA

Where Operations includes every one on system side like ->

• System admins
• Network admins
• Database admins

Collaboration among everyone participating and delivering software is a key DevOps tenant.

What DevOps is Not

• A new name for an operations team
• A job title
• A new tool category

DevOps core Values : CAMS

CAMS become the model set of values used by many DevOps practitioners, where ->

• C -> Culture
• A -> Automation
• M -> Measurement
• S -> Sharing

Insights

DevOps is a human problem. Often DevOps is thought of as a technical problem.

In reality it's a cultural and business problem

Breaking each terms of CAMS ~

• C -> Culture

  • Culture is driven by behavior. It exists among people with a mutual understanding of each other when they are coming from.

• A -> Automation

  • Once the culture is understood, one can create a fabric of automation that allows to control the system and the application. It acts as an accelerator which gives all the benefits of DevOps.

• M -> Measurement

  • One of the keys to rational approach to the systems is the ability to measure them all. DevOps strongly advices to Measure Key Metrics across the Organization.

• S -> Sharing

  • Sharing ideas and problems is the heart of collaboration. DevOps expects to see a high premium placed on openness and transparency.

  DevOps Principles

  Three Ways - It proposes

    - System thinking
    - amplifying feedback loops
    - a culture of continuous experimentation and learning

  A. System Thinking -

  

  • It explains that we should make sure to focus on the overall outcome of the entire pipeline or value chain.
  • Its easy to make mistakes of optimizing one part of that chain at the expense of overall results.

  Concept to Cash

  1. The “Concept to Cash” (C2C) process in DevOps refers to the entire software development and delivery lifecycle, encompassing the journey from the initial concept for a software product or feature to its eventual delivery and monetization.

  2. It is a holistic view of the end-to-end process of bringing software from ideation to production and, ultimately, generating value or revenue from it.

  3. Concept to Cash is an extension of the DevOps principles and emphasizes the importance of optimizing the entire software delivery pipeline.

  B. Amplifying Feedback loops -

  

  • It is all about creating, shortening and amplifying feedback loops between the parts of the organization that re in the flow of that value chain.
  • Short, effective feedback loops are the key to productive development, software development and operations.
  • Effective feedback is what drives any control loop designed to improve a system.

  

  C. Continuous Experiment -

  

  • It reminds to create a work culture that allows for both continuous experimentation and learning.
  • It also means engaging in the continuous practice required to master the skill and tools.

  Use the Three ways

  • To create team processes
  • To create team standards
  • As part of management style

Five DevOps Methodology

• First Methodology ~ People over Process Over Tools ->

  • In short, it recommends identifying who’s responsible for a job function first, then defining the process that needs to happen around them, and then selecting and implementing the tool to perform that process.

• Second Methodology ~ Continuous Delivery ->

  • In short, it’s the practice of coding, testing and releasing software frequently and really small batches so that one can improve the overall quality and velocity

• Third Methodology ~ Lean Management ->

  • It consists of using small batches of work, work in progress limits, feedback loops and visualization.

• Fourth Methodology ~ Visible Ops-Style Change Control ->

  • It focuses on an emphasis of eliminating fragile artifacts, creating a repeatable build process, managing dependencies, and create an environment of continuous improvement.

• Fifth Methodology ~ Infrastructure as code ->

  • One of the major realizations of modern operations is that systems can and should be treated like code. System specifications should be checked into source control, go through a code review, whether a build an automated test, and then we can automatically create real systems from the spec and manage them programmatically.

  With this kind of programmatic system, we can compile and run and kill and run systems again, instead of creating handcrafted permanent fixtures that we maintain manually over time.

Ten practices for DevOps Success

• Practice. Number 10 -> Incident Command System

  • used primarily in emergency management and response, including scenarios such as natural disasters, wildfires, and public safety incidents. It provides a clear organizational structure, roles, and processes for managing and responding to incidents effectively.
  • While this is not a concept directly associated with DevOps, some organizations have adapted its principles to handle incidents and outages in their DevOps and IT operations environments. This adaptation is sometimes referred to as “DevOps Incident Command.”

• Practice. Number 9 -> Developers on Call

  • ”Developer on Call” (DoC) is a practice commonly implemented in DevOps and IT operations teams to ensure that there is a designated developer or engineer available to respond to incidents and urgent issues outside of regular working hours.
  • The DoC concept is a key component of an organization’s incident response and on-call procedures.

• Practice. Number 8 -> Public Status Pages

  • Public status pages in DevOps are web pages or portals that provide real-time information about the operational status and health of an organization’s IT systems, services, and applications.
  • These status pages are publicly accessible and serve as a transparency and communication tool to keep customers, users, and stakeholders informed about service availability and any ongoing incidents or outages.
  • Public status pages play a crucial role in incident management, customer trust, and service reliability.

• Practice. Number 7 -> Blameless Postmortems

  • The primary goal of a blameless postmortem is to understand what went wrong, why it happened, and how to prevent similar incidents from occurring in the future.

• Practice. Number 6 -> Embedded Teams

  • Refers to cross-functional teams where members from different disciplines, such as development, operations, quality assurance, security, and other relevant areas, work closely together on a specific project or product.
  • The idea behind embedded teams is to break down traditional organizational silos and promote collaboration and shared responsibility throughout the software development and delivery lifecycle.

• Practice. Number 5 -> The Cloud

  • The cloud plays a central and transformative role in DevOps practices and principles.
  • It provides the infrastructure, services, and resources needed to enable agility, scalability, automation, and collaboration throughout the software development and delivery lifecycle.
  • Organizations that embrace cloud-native DevOps practices can gain a competitive edge by delivering innovative solutions more rapidly and efficiently.

• Practice. Number 4 -> Andon Cords

  • Andon cords,that have been adapted and applied in some DevOps environments to improve collaboration, communication, and problem-solving during software development and operations processes.
  • In DevOps, Andon cords are used as a visual and audible signal to indicate the need for immediate attention to an issue, similar to how they are used in manufacturing to signal a production line stoppage due to a problem.

• Practice. Number 3 -> Dependency Injection

  • In the context of DevOps and the automation of software delivery pipelines, there are ways in which the concept of dependency injection can be relevant.
  • Dependency Injection is a technique used to achieve loose coupling between classes and their dependencies. This approach promotes modularity, testability, and flexibility in software design.
  • In DevOps and automation, similar principles of dependency management can be applied, although the context is different.

• Practice. Number 2 -> Blue Green Deployment

  • Blue-Green Deployment is a DevOps deployment strategy that focuses on minimizing downtime and risk when releasing new versions of an application or service.
  • This approach involves maintaining two separate environments, the “blue” and the “green,” and carefully managing the transition from one version to another.
  • The goal is to ensure a smooth and controlled deployment process while providing a quick rollback option if issues arise.

• Practice. Number 1 -> Chaos Monkey

  • Chaos Monkey is a tool and concept, is designed to help organizations improve the reliability and resilience of their applications and services by intentionally introducing controlled failures and disruptions into their production environments.
  • The underlying idea is to proactively identify weaknesses and vulnerabilities in the infrastructure and applications, allowing teams to address them before they lead to unplanned outages.

DevOps Tools Criteria

1. Programmable - Developers want the tool to play well with others in the tool chain. It should be able to perform in an automated manner.
2. Verifiable - Tools need to be verified as the best kind of DevOps tool exposes clearly what it is doing and provide some manner of validating that it did, what it was supposed to do correctly. Events and metrics from the tools are an important source of feed.
3. Well behaved - The tool should be well behaved both from the developer and an operation point of view. One should check the tools configuration into Source control.It should come with tests that can be used to verify its behavior and able to deploy it in an automated manner.

What is Blameless Postmortems in DevOps

• Do it within 24 or 48 hours after a customer is affected with a major outrage
• Have the team collaborate and build up a timeline.But assign one person to run the meeting.
• Have a third party run meeting

A “Blameless Postmortem” is a key practice in DevOps and incident management that emphasizes a culture of learning, improvement, and accountability without assigning blame or fault to individuals when addressing incidents, outages, or other operational failures. Blameless postmortems are an essential practice for building resilient and reliable systems and fostering a culture of continuous improvement in DevOps and SRE environments.

Run Blameless Postmortems

1. Description of the Incident
2. Description of the root cause
3. How the incident was stabilized or fixed
4. Review the entire Timeline of events and actions taken
5. Discuss about how customers were affected
6. At the end ask how can we detect the issue sooner i.e Remediation and corrections
7. Create tickets or action items.

Record these above items and get everyone in the meeting to publicly agree on a deadline.

What is Transparent UpTime in DevOps

Transparent uptime in DevOps refers to the practice of -

• providing real-time visibility and open communication about the availability and performance of applications, services, and systems to stakeholders, including users, customers, and internal teams.
• It involves making uptime and downtime information easily accessible, often through public status pages, dashboards, and alerts, to ensure transparency and build trust with users and stakeholders.

Rules for Postmortem Communication

1. Admit Failure
2. Talk real to the real customers and avoid using any doublespeak when apologizing
3. Have a communication channel
4. Above all else, be authentic - take one individual out of the team to have them manage communication during the outrage

Management Best practices in DevOps implementations

1. Independent, cross-functional teams
2. Help people through change
3. Use agile, lean processes

Kaizen : Continuous improvement

• Kaizen, which translates to “continuous improvement” in Japanese, is a philosophy and practice that can be highly beneficial in the context of DevOps.
• DevOps itself is centered around principles like automation, collaboration, and continuous feedback, which align well with the principles of Kaizen.
• This approach enables teams to adapt to changing requirements and market conditions while consistently delivering high-quality software.

Kaizen’s Guiding Principles

• Good processes bring good results
• Go see for yourself
• Speak with data and manage by facts
• Take action to contain and correct root causes
• Work as a team
• Kaizen is everybody’s business

Kaizen emphasizes going to look at the actual place where the value is created, or where the problem is. Not reports about it, not metrics about it, not processes discussing it, not documentation about it. It’s actually going to look at it

How Kaizen and continuous improvement can be applied in DevOps

1. Kaizen encourages organizations to make small, continuous improvements to their processes over time. In DevOps, this means regularly reviewing and refining the software development and delivery pipelines.
2. DevOps promotes collaboration between development, operations, and other teams. Kaizen extends this collaboration to include continuous improvement efforts.
3. Kaizen emphasizes the importance of feedback. In DevOps, this translates to gathering feedback from various stages of the software development lifecycle, including development, testing, deployment, and monitoring.
4. When problems occur in the DevOps process, Kaizen encourages a thorough analysis of the root causes. This helps prevent the recurrence of similar issues in the future.
5. CI/CD pipelines are at the core of DevOps practices. Kaizen principles can be applied to continuously enhance and optimize these pipelines for faster and more reliable software delivery.
6. Kaizen encourages teams to identify manual and repetitive tasks that can be automated, thereby reducing human error and increasing efficiency.
7. Kaizen emphasizes the importance of real-time data and metrics. DevOps teams can apply this by continuously monitoring applications.
8. Kaizen promotes a culture of continuous improvement. In a DevOps culture, this mindset becomes ingrained in the team’s DNA, driving them to seek better ways of working and delivering value to users.

Five Whys of Kazien Tool

The “Five Whys” is a problem-solving technique that is a part of the Lean manufacturing and Kaizen philosophy. In DevOps, the Five Whys technique is used to identify the root causes of issues, incidents, or problems that occur during the software development and delivery process.

It helps teams dig deeper into problems to understand why they happened and how to prevent their recurrence.

While understanding the Five Whys - below pointers are important to keep in mind ~

• Focus on underlying causes, not symptoms
• Don’t accept answers like “not enough time”
• Track the forks in the five Whys
• Don’t accept human error as a root cause

Summary of Part 1

DevOps is a set of practices, principles, and cultural philosophies that aim to improve collaboration and communication between software development (Dev) and IT operations (Ops) teams. The goal of DevOps is to streamline and automate the software delivery process, allowing organizations to develop, test, and deploy software more quickly and reliably. It seeks to break down traditional silos between development and operations, leading to improved software quality, faster time-to-market, and increased agility.

Please stay tuned for Part 2 of DevOps Foundation.

Happy Learning!!