Migrating to the cloud? Here’s how your DevOps priorities should change when you move to Amazon EC2

A majority of companies migrating to the cloud, or creating “cloud-native” applications, are doing so with Amazon Web Services (AWS).  AWS offers a lot of cost and functionality advantages.  Companies who have adopted industry-standard developer operations (“DevOps”) best practices for monitoring and managing their on-premise environments often ask themselves how they adapt to their new cloud environments and applications.

How will DevOps priorities change when you move from on-premise applications to Amazon EC2?  Here’s an explanation of the differences between the two and what you should keep in mind.

DevOps priorities in the cloud?  The same. But different.

We often hear customers say that operations will be easier when they move to AWS. We caution them that moving to the cloud (or even AWS) does not mean that they no longer need to monitor and manage their applications.

Companies moving to Amazon AWS can take advantage of lower costs and manpower resources when it comes to hardware procurement, provisioning, and maintenance.  But you need to take into account that when you decide to host applications on Amazon EC2 that anything above the Operating System layer is your responsibility.

When it comes to backup/availability guarantee/security measures, etc. for your Amazon EC2 environments, the priorities are the same as if they were on-premise applications. And Amazon provides some native tools and functionality.  But you need to decide if they are the right fit for requirements.

Security, Backup… What do you need to know when managing Amazon AWS environments?

So what are some of the AWS-specific considerations you need to keep in mind as you move to Amazon EC2?  And what are the right tools for you?  The time you invest upfront in designing your applications and how you will deploy and manage them will pay off.

Your first consideration should be how you will secure your Amazon EC2 applications.  Network design, such as “which ports to open” and “from where to allow access” must be considered in the same way as for your on-premise applications.  These can be configured in AWS using security groups and network ACLs (access control lists).

You can use the AWS Trusted Advisor functionality*, which automatically examines your AWS environment and points out whether or not it is set to the recommended settings, making it possible to check your company’s AWS environment for security issues.  We recommend checking with the AWS Trusted Advisor both at the time of implementation and periodically.

Another essential aspect of security is the management of authentication and access privileges.  AWS consolidates all of these into AWS Identity and Access Management (AWS IAM).  In addition to controlling who can access which EC2 instances, you can also use AWS IAM to set up access permissions from EC2 instances to other resources (such as DBs), etc.  Once you have migrated to AWS, the first thing you need to do is to set up the accounts and access restrictions properly in AWS IAM.

The next consideration is “how will I backup my applications on Amazon EC2?”  Amazon EC2 provides the ability to take snapshots, which allows you to do so.  In addition, using “Amazon Data Lifecycle Manager” makes it easy to set up periodic snapshots, as well as incremental backups.  Snapshot files are stored on the Amazon S3 storage service.  You are charged according to their capacity, so you need to be aware of the amount of data you have and set up such settings as “reduce the capacity by incremental backups” and “delete from old data.”

“Availability” needs to be considered in advance. The key is to operate the system in accordance with the priority level of the system.

The last consideration is availability.  With Amazon EC2 applications, as well as those that are on-premise, you should consider the level of availability required based on cost and system importance. However, if you use Amazon’s Multi-AZ deployment functionality, you can specify a redundant configuration between different data centers.  However, using Multi-AZ costs more than using a single-AZ configuration (although not as much as if you had redundant on-premise systems).  When designing your applications you need to consider whether Multi-AZ is required and how much you should invest in availability.

If you aren’t investing in failover, then you should at least be monitoring your applications and planning how to recover them when downtime is experienced.  You can use Amazon CloudWatch to easily monitor general items such as CPU, memory, and disks, and you can also program the Amazon EC2 Auto Recovery function to automatically recover instances when an error occurs in the EC2.

If your application is mission-critical, then you will want to invest more in its availability.  You should consider many of the excellent third-party solutions that offer valuable functionality to the AWS community.  One choice is SIOS AppKeeper, an easy to configure and use solution that monitors your Amazon EC2 instances and automatically restarts services or reboots instances if they experience system failures.  Here’s a quick video of how AppKeeper works

While moving to the cloud for your applications makes a lot of sense, you cannot abandon DevOps best practices.  Amazon AWS provides you with a rich set of functionalities and tools, but you still need to take primary responsibility for the security, backup and availability of your applications.  How you do this depends on your skills and the importance of the applications themselves.

We invite you to join the hundreds of customers who have been taking advantage of AppKeeper to reduce their Amazon EC2 downtime by signing up for a free 14-day trial of the service.

* Note:  To use AWS Trusted Advisor, a contract for business support or higher is required.

Reproduced with permission from SIOS

Expand Your High Availability Metrics

In the technology field, we love data. We love data about data and all the metrics and measures that our tools can bring. We’ve created industries around analytics, products that capture every detail from thousands of connected devices. We love metrics and measures. In many instances within the higher availability space, we love the high availability metrics that tell us how quickly a system recovered from the failure. We calculate and track the time between detection and remediation, and we obsess over knowing and measuring how much transactional data would be lost in a disaster, system failure, or disk crash.

Ironically, in high availability and disaster recovery (HA/DR) systems, there are some metrics that don’t get enough attention.

Here are eight other high availability metrics you should be watching to manage your environment:

1.  Security alerts

Availability isn’t just about application monitoring and recovery.  Systems that are publicly available are always under attack.  If you aren’t monitoring security alerts and warnings, your applications may be running flawlessly, while your intellectual property is being funneled flawlessly out the door.

2. Idle connections

Idle connections sound harmless, but they are about as harmless as the green leafy kudzu on a southern lawn.  Idle connections take up resources and threaten to fill database pools, congest networks, and stifle performance.  Furthermore, idle connections can indicate a problem in the application layer or database configuration.

3. Long-running queries, commands, or jobs

This applies not just to database queries or jobs, but also to commands and backups.  Long running queries, commands and jobs can be an indicator of poor system health, slow disk speeds, CPU or other resource contention, or deeper systematic, application compatibility or OS problems.

4. Disk IO

Disk IO typically refers to the input/output operations of the system related to disk activity. Measuring disk I/O can help identify bottlenecks, poor hardware configurations, improperly sized disk or poorly tuned disk layouts for a given workload.  Monitoring disk I/O can help tell you if the long running queries are a function of poor sql syntax, poorly coded applications, or latency and access problems.

5. Memory

We all think about how much memory is being used, but memory monitoring goes beyond measuring and looking at free versus used.  Monitoring memory helps you look into bottlenecks, leaks, identify improperly sized systems, understand load, load average, and spikes.  In addition, knowing about memory intensive patterns can help you tune your availability suite to avoid false failures.

6. Disk Space

As VP of Customer Experience I once had the unfortunate experience of waking up early in the morning for an emergency call.  The customer was facing a down production system after a power outage.  When they tried to restart their system their protected applications failed to start.  After a quick check of the error logs it was clear that the root drive was 100% full.  The application could not write to any of the file systems.  Disk space monitoring is available in many forms and ways and having it as a metric can prevent unnecessary problems and costly last-minute scrambles to add more. .

7. Errors and alerts

Errors, alerts, and recovery messages in the logs are another good metric to consider.  Your availability solution may be keeping your clients online and happy, but it may also be masking an issue that will need your attention soon.  Adding log monitoring for FATAL, PANIC, and key ERROR messages can help you identify issues that your availability solution is frequently recovering from, such as database crashes, application panics or core dumps, or fatal errors requiring a cold restart.

8. Recovery numbers

Similar to monitoring errors and alerts, the recovery numbers can tell you a lot about the health of your system’s availability.  If you are averaging more than one application recovery per week, you’re likely experiencing something more than your normal availability protection.  And while the recovery was successful in restarting your application or system, too many of these false or even real recoveries isn’t healthy.

The list of HA/DR metrics that we can monitor and the tools to monitor them are growing by leaps and bounds.  Be sure that you and your team consider expanding your current data capture and analysis to include those that make for the best higher availability system possible.

— Cassius Rhue, VP, Customer Experience