SIOS SANless clusters High-availability Machine Learning monitoring
Patch management solutions help IT teams apply updates, test patches, and maintain security without planned downtime. This video explains how SIOS DataKeeper and LifeKeeper enable near-zero downtime patching through high availability clustering, standby node updates, and automated failover, helping organizations stay secure, compliant, and uninterrupted.
In the first part of this series, we explored how approximation can help guide business decisions when there isn’t a perfect answer. This article builds on that idea by examining how observation and experience shape better judgment over time, helping professionals make more informed decisions in future situations.
There’s a lot to learn in the way ancient mathematicians would draw inspiration from the world around them for their practice and study. It makes sense since back then, math was more driven to ease the pain of practical everyday problems: representing large quantities succinctly, how to divide up irregularly shaped land equally, calculating interest on loans, and so forth. The answers to these problems have many wider theoretical implications, but they were driven by real-world observations and helped to inform practices going forward.
You can’t always conjure up or recall some perfect formula or behavior to predict or explain everything. Especially when you don’t have the blessing of an additional 2 thousand years of mathematical progress and rigor to stand on. In our current age, we tend to scoff at anecdotal findings, and in many cases (mathematical, scientific, medical), that is absolutely true. But we’re not talking about that right now! Observation is a very powerful tool, perhaps the most powerful by a mile, in interpersonal relationships. These can be relationships within your company or external relationships with users, customers, or partners.
However, the big problem with observation is that it is inherently experiential. There is no way that you can try to observe something before it actually happens, and sometimes waiting until something actually happens to figure out what to do is too late.
This is where proactive calculation comes into play. You use your knowledge of similar situations to build an expectation of what you expect to happen, and you can prepare yourself ahead of time to address the situation and keep it guided towards your desired outcome. It’s no Pythagorean Theorem, but in the context of social and business relations, it is nonetheless still a very calculating approach.
The two make a cycle. You observe something, you learn from it, that observation informs your future calculations, those calculations help you better prepare for future situations, and it goes on and on. It’s like a juggling act.
With that in mind, you can start looking at how to apply it to your business and your relationships. Where do your observations come from, what are they observations of, and where are they recorded?
Most likely, they come from your meetings and engagements, so the observations will lie in your memories and (ideally) in recorded notes or minutes. As such, when you are structuring your notes and minutes, you should consider how to make the most of them as an archive of what you observed and experienced in the meeting.
Important items that you may want to record your observations of can include:
Then, when you have to do it again, you can look back at the observations and start asking questions like:
Those are your calculations.
The best part is that for all the mind work, the hypothesizing, and the calculating, what you’re figuring out is still an approach to concrete real-world situations that you are going to experience. That makes it feel much more approachable, and it makes it very easy to see when and how it pays off as you start applying these lessons. Plus, maybe you’ll feel some comfort or even pride knowing you’re employing practices used and honed since antiquity! Just like the ancients, you’ll be pioneering your own theory of practice driven entirely by your experiences and observations of the world around you.
Author: Matthew Pollard
Reproduced with permission from SIOS
High availability and disaster recovery were once viewed mainly as IT responsibilities. They were important, but often treated as technical safeguards managed behind the scenes.
That mindset is changing.
In today’s digital economy, uptime is directly tied to revenue, productivity, customer experience, and brand trust. When critical systems go down, the impact extends far beyond IT. Transactions stop, employees lose access to essential tools, customers become frustrated, and the organization’s confidence can erode quickly.
High availability (HA) and disaster recovery (DR) are no longer just technical checkboxes. They are essential parts of business continuity, risk management, and long-term resilience.
The cost of downtime continues to rise as organizations rely more heavily on digital systems. A single outage can create financial losses, operational delays, compliance concerns, and reputational damage.
For healthcare organizations, downtime can delay access to patient information or disrupt care. For manufacturers, it can stop production lines. For financial services firms, it can interrupt transactions and damage customer confidence. Even short outages can create lasting consequences.
Public outages also attract attention quickly. The 2024 CrowdStrike incident demonstrated how a single technology disruption could affect airlines, banks, and healthcare providers worldwide. But today, organizations face an even more deliberate threat: targeted cyberattacks. Ransomware operators now actively target backup repositories to prevent organizations from restoring their systems. Because of this, disaster recovery is merging with cybersecurity. IT leaders are shifting their focus to cyber resilience by ensuring they have air-gapped, immutable backups that cannot be encrypted. This approach allows them to restore a “known-clean” environment without paying a ransom.
Today’s IT environments are more distributed and complex than ever. Organizations are moving past traditional virtual machines, shifting critical applications across multi-cloud platforms, hybrid environments, and containerized infrastructure like Kubernetes. Each layer introduces dependencies that must be understood and protected. When a modern, cloud-native application goes down, teams cannot simply restore a server. They must restore the orchestration platforms, cloud configurations, and infrastructure-as-code (IaC) that make the application run.
At the same time, IT teams are expected to keep systems available while managing patches, upgrades, configuration changes, security requirements, and evolving business needs. Many teams are also operating with limited resources or facing turnover that creates knowledge gaps.
This complexity makes resilience harder to achieve through technology alone. Organizations need clear processes, trained teams, documented procedures, and tools that simplify availability across environments.
Strong HA and DR strategies help reduce this burden. By improving visibility, automating recovery actions, and simplifying management, organizations can help IT teams respond faster and with greater confidence.
High availability and disaster recovery were once treated as separate disciplines. HA focused on keeping systems running during local failures, while DR focused on recovery from larger disruptions such as data center outages, regional events, or natural disasters.
Today, organizations need a more unified approach.
HA and DR should be part of everyday IT operations, including routine maintenance, patching, system updates, and configuration changes. Instead of treating these activities only as risks to availability, teams can use them to validate failover processes and confirm recovery readiness.
Regular testing is especially important. Recovery plans that are reviewed only once or twice a year may not reflect current infrastructure, application dependencies, or staffing realities. Modern HA and DR approaches enable more frequent testing, often without disrupting production workloads.
This shifts resilience from a reactive effort to a proactive practice.
Every organization will eventually face disruption. Failures may come from hardware issues, software bugs, human error, cyber incidents, cloud service interruptions, or unexpected external events. What matters most is how quickly and effectively the organization can respond.
Controlled resilience testing, including practices such as chaos engineering, can help.
Chaos engineering involves introducing controlled failures into a system to understand how it responds under stress. The goal is to uncover weaknesses before they cause real outages. These tests help teams identify hidden dependencies, improve recovery procedures, and clarify roles during an incident.
The concept is similar to an emergency drill. Teams that practice under controlled conditions are better prepared when a real disruption occurs.
With the right tools, IT teams can validate configurations, confirm failover readiness, and train staff without taking production systems offline. This builds operational confidence while reducing the risk of unexpected failure.
As infrastructure grows, manual recovery processes become harder to manage. Human-led responses can be slow, inconsistent, and error-prone, especially during high-pressure incidents.
Automation is now essential to effective HA and DR, and it is rapidly evolving into AI-powered resilience. Automated, defensive AI can monitor systems to detect anomalies and trigger intelligent failovers before a total crash occurs. Predictive analytics help identify patterns that signal future hardware failures or traffic spikes. When teams can act on these early warning signs, they can resolve problems before users are ever affected.
Ease of use matters too. HA and DR solutions should not require deep specialist knowledge for every task. Clear interfaces, simplified configuration, and strong visibility help generalist IT teams manage resilience more effectively. This reduces operational burden and lowers the chance of mistakes.
Not every application requires the same level of protection. Some systems can tolerate short delays or limited data loss. Others must remain available with minimal interruption.
That is why HA and DR planning should begin with business impact.
Organizations need to identify which applications are most critical, how downtime would affect operations, and what level of recovery is required. Recovery Time Objectives (RTOs) and Recovery Point Objectives (RPOs) should reflect real business needs, not assumptions.
This helps avoid two common problems: overprotecting less critical workloads and underprotecting essential systems.
This alignment is no longer just a best practice, as in many cases, it is a legal requirement. Governments and regulatory bodies are turning operational resilience into a strict mandate. Regulations like the Digital Operational Resilience Act (DORA) in Europe and stricter SEC disclosure rules in the US are forcing boards of directors to prove their recovery capabilities, not just document them.
When HA and DR strategies align with business priorities, leaders can more easily demonstrate compliance to auditors and make better decisions about infrastructure investment. Resilience becomes easier to justify when tied directly to business outcomes.
Executive involvement is also critical. Availability should be discussed alongside financial risk, compliance, customer experience, and operational performance. When leadership understands uptime as a shared responsibility, resilience becomes part of the organization’s culture.
Recent years have shown that disruption can come from many directions. Software failures, supply chain issues, cyber events, staffing changes, infrastructure problems, and cloud outages can all affect business continuity.
The most resilient organizations do more than build redundant systems. They create a culture of readiness.
That means documenting recovery plans, testing them regularly, updating procedures as environments change, and making resilience part of everyday IT decision-making. It also means ensuring that critical knowledge does not reside with a single person or team.
Preparedness is not a one-time project. It is an ongoing discipline.
By embedding HA and DR into daily operations, organizations can reduce uncertainty and improve their ability to deliver reliable service even in the face of unexpected events.
High availability and disaster recovery have moved beyond technical checkboxes. They are now core components of business resilience.
Organizations depend on critical applications to serve customers, generate revenue, support employees, and maintain trust. When those applications are unavailable, the business feels the impact immediately.
As IT environments become more complex, resilience requires the right combination of people, processes, and technology. Organizations that make HA and DR part of broader business planning will be better positioned to manage disruption, protect uptime, and maintain confidence in an unpredictable world.
The goal is no longer simply to recover after failure. The goal is to keep the business moving!
Author: Benjamin Roy, Marketing Specialist at SIOS
Reproduced with permission from SIOS
A warning appears, services stop responding, tickets begin to pile up, and someone says, “We need to do something!” That instinct to begin recovery activities immediately is understandable, as during an incident, action feels productive, while waiting can feel irresponsible. Under this kind of pressure, doing almost anything can feel better than doing nothing.
However, some of the most damaging decisions in disaster recovery are made because nobody acted, but because someone acted before understanding what had happened. The philosopher, Marcus Aurelius, often wrote about the importance of separating an event from the judgment we form about it. We first receive an impression of what has happened, and then our minds quickly form an explanation. If we do not pause to examine that explanation and the actions leading to the circumstance, we can begin reacting to an assumption as though it were a fact.
Suppose a server becomes unreachable, for example. The immediate conclusion may be that it has failed, but all we actually know is that we cannot communicate with it. It might still be running while a network problem prevents us from seeing it.
That difference matters in a highly available environment, as moving an application to another server manually might restore service, but it could also create a situation where both servers believe they should be active. In high-availability environments, this kind of condition is often referred to as a split-brain scenario: two systems each acting as though they have ownership of the same application or resource. A response intended to improve availability for end users can introduce a risk to the application or its data.
We can observe the same problem occurring during ordinary troubleshooting of non-critical components. Restarting a service may clear an issue, but it also changes the conditions we were trying to understand. Once the restart is complete, useful evidence about the original problem may be gone. We may have restored service without learning what happened or whether it is likely to happen again.
None of this indicates that teams should stand still during an outage, as Aurelius was not advocating indecision, and restraint should not become an excuse for delay or inaction. The point is to act based on what we know rather than what we fear might be happening. I think that distinction is easy to lose when an incident becomes stressful. People want updates, alerts continue to appear, and periods of silence on a conference call can feel longer than they really are. Someone may suggest a reboot because it worked last time a critical incident of this nature was observed.
So, the suggestion begins to sound like the plan, even though the current problem may have a different cause, yet present similar symptoms as the previous issue. Experience can help, but it can also create shortcuts in our thinking. Recognizing a familiar symptom is useful; assuming it must have the same cause as the last incident is not. Similar symptoms can come from very different problems.
A more disciplined response starts by stating only what has been confirmed. Instead of saying, “The server is down,” a better approach may be, “The server is not responding from this location.” That wording may seem like a minor detail, but it keeps the team from treating a conclusion as an observation. It also leaves room for another person to report that the server is reachable from somewhere else.
This is where incident response becomes more than technical knowledge. It requires control over the urge to solve the problem before the problem is understood. Sometimes, one additional check is enough to change the direction of an investigation and the next steps.
A second monitoring location might show that the application is still available internally, and a local console might confirm that a supposedly failed server is healthy but isolated. That information can prevent an unnecessary recovery action and point the team toward the actual problem.
Well-designed automation follows a similar principle. Automation is valuable because it can respond consistently and does not need to wait for an administrator to wake up or join a call. However, speed alone does not make an automated response correct.
An automated system should act when the conditions for recovery are clear. When the available information is incomplete or contradictory, the safer behavior may be to stop and seek more evidence. Highly available systems account for this in several ways. Independent communication paths can help distinguish the failure of one connection from the loss of an entire server. Quorum or witness mechanisms can provide another perspective when systems can no longer communicate with each other. These controls are important because a system’s view of the environment may be accurate but incomplete.
LifeKeeper can support this decision-making through resource monitoring, defined dependencies, and recovery policies. The technology helps carry out an established recovery plan, but it cannot decide what level of risk is acceptable for a particular business. That judgment has to be made by people while the environment is stable, not improvised after an incident begins.
Clear procedures make restraint easier. A good incident runbook should help the team establish what is known before making a consequential change. It should explain how to confirm whether an application is already active elsewhere and identify who has the authority to initiate recovery.
The goal is not to remove human judgment. It is to give that judgment a reliable foundation when time is limited. We cannot eliminate uncertainty from technology; hardware will fail, networks will behave unpredictably, and applications will occasionally surprise the people who know them best. What we can do is prepare ourselves to recognize the difference between an event and our first explanation of it.
Aurelius returned to that idea because it applies most when circumstances are difficult. Clear judgment is easy when nothing is at stake. Its value becomes relevant when pressure makes the fastest answer feel like the only answer. During an incident, the calmest person in the room is not necessarily doing nothing. They may be making sure that the next action solves the problem that is actually happening. In incident response, discipline is not the absence of action. It is the refusal to let pressure choose the action for you.
Protect critical applications with high availability and disaster recovery solutions built for complex IT environments. Request a demo to see how SIOS LifeKeeper can help your team reduce downtime and recover with confidence.
Written by Aidan Macklen (Associate Product Support Specialist)
Reproduced with permission from SIOS
Within this blog, there is also the assumed familiarity with the LifeKeeper Resource Hierarchy framework and LifeKeeper Clustering in general. For background on these topics, the blogs listed below provide fantastic context. Additionally, this blog builds upon a previous blog regarding one means to close the gap between possible use cases and supported protection mechanisms via the use of the “Quick Service Protection Application Recovery Kit” (QSP ARK) within LifeKeeper, linked below.
This blog, however, will explore the options available when the QSP ARK cannot meet the demands for High Availability and Disaster Recovery for a particular application or use case.
The previous part of this blog addressed the means to think about an application for the purpose of creating a Generic Application Recovery Kit to protect that application with LifeKeeper. In this section, the foundations for understanding an application presented in part one will be contextualized for the application to LifeKeeper’s Generic Application Recovery Kit framework.
As this installment into the blog series is best considered in tandem with the previous installment, here is a short refresher of the concepts presented in part 1.
In determining how broad questions will be answered, break them down into smaller questions with simpler answers. Use those smaller questions to iteratively rebuild the answer to the original “broad” question.
Understand the utilities available and how these utilities convey information. Knowing the information needed to answer the “smallest questions”, determine how the information presented via an Application’s API can be used to determine the answers to the “smallest questions”.
With the refresher out of the way, LifeKeeper can finally be brought into the picture. Though High Availability and Disaster recovery are often associated with complexity, significant effort has been made to ensure that understanding of a particular application is easily ported into the Generic Application Recovery Kit framework. With the previous foundation in mind, it is time to use that as the base for thinking like LifeKeeper.
Imagine you are in a “Freaky Friday” scenario, such as the movie where a girl and her mother switch bodies and fumble with the lack of familiarity with each other’s daily responsibilities. The person with whom you have switched is attending a go-live activity in your stead and needs to know how to start key applications, make sure they are running correctly, and then stop the application. How would you explain these things to them if you only had a 15-minute phone call to prepare them? What are the details you would have to specify so they can complete the job?
This scenario, while contrived, is a great way to break down the key resource protection actions. LifeKeeper manages applications to ensure they are only running on a single system, and the LifeKeeper Resource Hierarchy ensures that pre-requisite applications and system resources are processed in the correct order whenever resources are restored or removed. In turn, LifeKeeper enables a developer to think about the actions on an application protected with a Generic Application resource in the context of a single system. Distilling the process to perform a start, stop, or query upon an application to the bare essentials is the first step to defining what the Generic Application action scripts need to accomplish. When start, stop, and query are defined according to the above strategy, the resource actions relate one-to-one, like so:
Sometimes, LifeKeeper will need to know certain details about a running application in order to perform the actions listed above. All LifeKeeper Resources, including the Generic Application Resource, have what is called a “Resource Information Field”, its purpose being to provide this information to the action scripts for use during resource actions. The resource information contained within the information field can be configured independently for each system at the time of resource extension, allowing for resource actions to make use of information specific to the system on which the action is performed. LifeKeeper also provides command-line utilities to easily get or set the resource information.
Calling back to the “Freaky Friday” example, what are the details that you would have to tell the person with whom you switched? Think of things such as key paths for application files, specific settings/values for command arguments, and similar details. The information field is a great place to put information that must be known to determine other details about the application. The information field is also a great place to insert values for settings, command arguments, or values that otherwise could not be derived. It is worth considering, by LifeKeeper convention, that the information field is not frequently (if ever) changed over the lifetime of a resource. Information that varies over the lifetime of a resource is best kept out of the resource information to avoid corruption of this field, and instead obtained programmatically in a LifeKeeper Resource’s action script(s) or via a “helper” script that the action scripts can then invoke.
As an extra assistance, LifeKeeper is delivered with template scripts for developing a generic application. These are a fantastic starting point for a Generic Application’s action scripts, as they come pre-prepared to receive the input arguments LifeKeeper will use when invoking actions for a particular resource. In turn, this also makes that information available for use within resource action scripts.
LifeKeeper provides a myriad of ways to protect applications. Still, some applications have requirements beyond what is offered in the LifeKeeper Application Recovery Kits. In such cases, High Availability and Disaster Recovery protection is still possible, and may be simpler to achieve than previously thought. Generic applications are not something from which an organization should shy away; instead, they are one of the many powerful tools offered by LifeKeeper to uplift an environment’s High Availability and Disaster Recovery capabilities. The Generic Application framework was created to be accessible and versatile. Still, if your organization does not have the resources to spare for writing a Generic Application Recovery kit in-house, SIOS offers Professional Services offerings wherein SIOS Engineers will coordinate requirements and develop a Generic Application Recovery Kit on behalf of your organization. If ongoing support is a requirement, SIOS Professional Services also provides offerings that expand normal product support to include Generic Applications developed by SIOS Professional Services. The barriers to entry for protecting your organization’s business-critical applications are ever-shrinking, and SIOS Protection Suite for Linux or Windows aims to be at the forefront of the charge to render unprotected applications a thing of the past.
Not every application fits a standard high availability model. SIOS can help you design and implement the right LifeKeeper solution for your business critical workloads. Request a demo today.
Author: Philip Merry Support Engineer at SIOS Technology Corp.
Reproduced with permission from SIOS