Don’t Depend on Maintenance Windows

Lee Atchison: 00:00:00

What happens when a web application you depend on

Lee Atchison: 00:00:03

goes offline? It can be inconvenient, disappointing and

Lee Atchison: 00:00:08

potentially cause serious problems for you.

Lee Atchison: 00:00:11

But what if you find out that the web application wasn't

Lee Atchison: 00:00:14

working by design, because it was down for maintenance? That

Lee Atchison: 00:00:20

only makes the situation worse. Your customers expect your

Lee Atchison: 00:00:25

application to be operational when they need it. Not when you

Lee Atchison: 00:00:29

find a convenient maintenance windows impose your schedule on

Lee Atchison: 00:00:34

your customers, and can negatively impact their

Lee Atchison: 00:00:37

experience. And there is no reason for it. A well designed,

Lee Atchison: 00:00:43

well operated modern web application never ever has to be

Lee Atchison: 00:00:48

down for scheduled maintenance. In this episode, we'll discuss

Lee Atchison: 00:00:52

why maintenance windows are a thing of the past. Are you

Lee Atchison: 00:00:55

ready? Let's go.

Voiceover: 00:00:59

This is the modern digital business podcast, the

Voiceover: 00:01:02

technical Leaders Guide to modernizing your applications

Voiceover: 00:01:05

and digital business. Whether you're a business technology

Voiceover: 00:01:08

leader, or a small business innovator, keeping up with the

Voiceover: 00:01:11

digital business revelation is a must here to help make it easier

Voiceover: 00:01:15

with actionable insights and recommendations, as well as

Voiceover: 00:01:18

thoughtful interviews with industry experts, Lee Atchison.

Lee Atchison: 00:01:24

Several years ago, I purchased a digital smart

Lee Atchison: 00:01:27

thermostat for my home, I wanted to be able to set the

Lee Atchison: 00:01:30

temperature remotely and check on it when I wasn't at home.

Lee Atchison: 00:01:34

Basic requirements of a smart thermostat nowadays.

Lee Atchison: 00:01:39

I set it up and connected it to the manufacturers cloud back

Lee Atchison: 00:01:42

end. All was fine. Or so I thought.

Lee Atchison: 00:01:46

A couple of weeks later, I received an email from the

Lee Atchison: 00:01:49

manufacturer about the upcoming upgrade of their back end cloud

Lee Atchison: 00:01:53

service.

Lee Atchison: 00:01:55

During the time of the upgrade, which was going to last for

Lee Atchison: 00:01:58

several months, the company would be bringing down its

Lee Atchison: 00:02:02

applications for quote many hours at a time and would do so

Lee Atchison: 00:02:08

at quote various times during the day.

Lee Atchison: 00:02:12

They wouldn't commit to specific days and times, just that they

Lee Atchison: 00:02:16

would be bringing the system down for a while, seemingly

Lee Atchison: 00:02:20

randomly, many times over the next several months. The

Lee Atchison: 00:02:25

company, of course, apologized in advance for the

Lee Atchison: 00:02:29

inconvenience.

Lee Atchison: 00:02:31

Yeah, right.

Lee Atchison: 00:02:34

What was that all about?

Lee Atchison: 00:02:37

at seemingly random times, my thermostat would stop working

Lee Atchison: 00:02:41

for many hours at a time. And this would go on for months. I

Lee Atchison: 00:02:45

don't think so. The very next day, I replaced the thermostat

Lee Atchison: 00:02:49

with one from another company throughout the old one and wrote

Lee Atchison: 00:02:54

them a scathing negative review, there is no way I would deal

Lee Atchison: 00:02:58

with that level of bad service.

Lee Atchison: 00:03:01

Available availability matters.

Lee Atchison: 00:03:05

Let's look at another example. In order to receive certain

Lee Atchison: 00:03:08

disability benefits that he's eligible for, my son has to

Lee Atchison: 00:03:12

report his income to the US government. To do that he uses

Lee Atchison: 00:03:16

an application on his cell phone. Once a month, he logs

Lee Atchison: 00:03:21

into the application to report his income for the previous

Lee Atchison: 00:03:24

month.

Lee Atchison: 00:03:26

This iPhone application, however, has a major problem

Lee Atchison: 00:03:29

with it.

Lee Atchison: 00:03:31

When you launch the application at the wrong time of the day, it

Lee Atchison: 00:03:35

shows you a message. The message says this application only works

Lee Atchison: 00:03:40

between the hours of 8am to 5pm. Eastern Time, Monday through

Lee Atchison: 00:03:44

Friday.

Lee Atchison: 00:03:46

Hmm, what is that? That's right, this online SAS based web

Lee Atchison: 00:03:53

application only operates during, quote normal business

Lee Atchison: 00:03:57

hours. This obviously makes the application very hard to use.

Lee Atchison: 00:04:03

Why would they restrict the hours that you'd use an

Lee Atchison: 00:04:06

application like this?

Lee Atchison: 00:04:08

As a government institution, they undoubtedly figured they

Lee Atchison: 00:04:11

didn't want to let the application run when nobody was

Lee Atchison: 00:04:15

in the office to support it. After all, how could they

Lee Atchison: 00:04:18

possibly fix anything that went wrong if they weren't in the

Lee Atchison: 00:04:21

office?

Lee Atchison: 00:04:24

Availability matters.

Lee Atchison: 00:04:29

These two stories are extreme examples, but they represent

Lee Atchison: 00:04:33

examples of a common problem in many online applications. The

Lee Atchison: 00:04:37

company's operating the applications create maintenance

Lee Atchison: 00:04:41

windows, periods of time where they regularly bring the

Lee Atchison: 00:04:45

application offline in order to perform routine maintenance and

Lee Atchison: 00:04:49

upgrades. Wants to find they treat these windows like they

Lee Atchison: 00:04:53

are free downtime periods. They feel they are free to bring

Lee Atchison: 00:04:58

their applications down and work on

Lee Atchison: 00:05:00

Mmm. Without it counting is downtime.

Lee Atchison: 00:05:04

Well, nothing could be further from the truth. Downtime is

Lee Atchison: 00:05:10

downtime, whether it's planned, unexpected, or unplanned and

Lee Atchison: 00:05:14

unexpected if your customers want to use your application,

Lee Atchison: 00:05:19

and the application is not available for any reason, it's

Lee Atchison: 00:05:23

downtime.

Lee Atchison: 00:05:25

You cannot operate a modern digital online application or

Lee Atchison: 00:05:29

service without maintaining a high level of availability.

Lee Atchison: 00:05:34

When your customers want to use your service, they expect your

Lee Atchison: 00:05:38

service will be operational. They do not care about

Lee Atchison: 00:05:41

maintenance schedules, they do not tolerate downtime.

Lee Atchison: 00:05:46

They use your application when it's convenient for them, not

Lee Atchison: 00:05:50

when it's convenient for you. It's bad enough when a real

Lee Atchison: 00:05:54

application failure causes your availability to suffer. But

Lee Atchison: 00:05:58

planning and having downtime in the form of a maintenance window

Lee Atchison: 00:06:03

is just formalizing customer dissatisfaction.

Lee Atchison: 00:06:07

In these modern times, with the tools and services that are

Lee Atchison: 00:06:11

available for modern cloud native application development,

Lee Atchison: 00:06:15

there is simply no reason why a digital application should

Lee Atchison: 00:06:19

require any sort of downtime at all, for any maintenance or

Lee Atchison: 00:06:23

upgrade. In today's world, it's unnecessary. From a customer's

Lee Atchison: 00:06:29

point of view, it is unacceptable.

Lee Atchison: 00:06:33

Almost any upgrade can be made live without any downtime. Even

Lee Atchison: 00:06:39

upgrades are required database schema changes and other data

Lee Atchison: 00:06:43

migration tasks can be implemented without requiring

Lee Atchison: 00:06:47

any downtime. maintenance tasks can be performed while the

Lee Atchison: 00:06:51

application continues to operate. There is no longer any

Lee Atchison: 00:06:55

valid reason for you to plan on bringing your modern application

Lee Atchison: 00:06:58

down.

Lee Atchison: 00:07:00

And if your application does require maintenance windows, due

Lee Atchison: 00:07:04

to some historical architectural issue with your application,

Lee Atchison: 00:07:08

they used to treat this as what it is a serious problem. This is

Lee Atchison: 00:07:15

technical debt imposed on your application that is costing your

Lee Atchison: 00:07:19

company money. Your customers don't care why your application

Lee Atchison: 00:07:23

is down. They just care that it is down. as your application

Lee Atchison: 00:07:28

grows and expands it will be harder to justify having a

Lee Atchison: 00:07:32

regular downtime window. customer usage patterns expand

Lee Atchison: 00:07:37

and customers expect the application to operate at all

Lee Atchison: 00:07:40

times of the day and night.

Lee Atchison: 00:07:43

Additionally, building systems and processes for your

Lee Atchison: 00:07:46

development organization that don't require the use of

Lee Atchison: 00:07:49

maintenance windows encourages them to adhere to deployment and

Lee Atchison: 00:07:55

operational best practices. You see, we developers tend to get

Lee Atchison: 00:08:00

lazy when we act like we know we have maintenance windows

Lee Atchison: 00:08:03

available for our years. designing and implementing

Lee Atchison: 00:08:07

changes that don't require a maintenance window requires

Lee Atchison: 00:08:11

additional time and thought, which encourages better

Lee Atchison: 00:08:14

attention to detail and to best practices. When developers are

Lee Atchison: 00:08:20

required to think about the operational impact of a change,

Lee Atchison: 00:08:24

they tend to produce fewer operational problems than when

Lee Atchison: 00:08:28

they simply throw it into production and not consider the

Lee Atchison: 00:08:31

ramifications when you depend on maintenance windows. Ultimately,

Lee Atchison: 00:08:37

overall quality and availability suffers. Even if you currently

Lee Atchison: 00:08:42

have easily identifiable low usage times during which you

Lee Atchison: 00:08:46

feel you can afford to bring your application down. That

Lee Atchison: 00:08:49

doesn't mean that those same low usage times will be available to

Lee Atchison: 00:08:54

you as you expand and grow. International expansion products

Lee Atchison: 00:08:59

that expansion, customer base expansion can all contribute to

Lee Atchison: 00:09:03

increased need for 24 by seven availability. A previous client

Lee Atchison: 00:09:09

of mine regularly scheduled a two hour maintenance window each

Lee Atchison: 00:09:13

week, so they can perform upgrades and data adjustments

Lee Atchison: 00:09:17

while allowing them to keep operating normally the rest of

Lee Atchison: 00:09:21

the time. The problem is the maintenance window is by itself

Lee Atchison: 00:09:26

a major hit to their availability. a two hour

Lee Atchison: 00:09:29

maintenance window means that the greatest availability they

Lee Atchison: 00:09:34

can offer their customers was 98.8%. By definition, you will

Lee Atchison: 00:09:41

not be able to operate greater than 98.8% of the time by

Lee Atchison: 00:09:46

comparison to other online applications 98.8% is a horrible

Lee Atchison: 00:09:52

statistic. For example, the Amazon s3 service guarantees 99

Lee Atchison: 00:09:59

Point 99% service availability, their data integrity SLA is even

Lee Atchison: 00:10:05

higher than that. This guarantee amounts to a maximum of 61

Lee Atchison: 00:10:10

seconds of downtime each week. In order for s3 to make this SLA

Lee Atchison: 00:10:16

consistently, they can never planned to have any downtime for

Lee Atchison: 00:10:20

any maintenance ever. Any outage at all will cause them to fail

Lee Atchison: 00:10:25

their contracted SLA. And they back up their SLA policy with

Lee Atchison: 00:10:31

money. If Amazon s3 is down just a mere 4.3 minutes in any given

Lee Atchison: 00:10:37

month, then AWS will refund 10% of everyone's storage costs for

Lee Atchison: 00:10:44

the entire month. As you can imagine, this would be a

Lee Atchison: 00:10:48

significant amount of revenue loss for Amazon. And it's not

Lee Atchison: 00:10:53

just s3. It's a mindset across all of AWS and across all of

Lee Atchison: 00:10:59

Amazon. This commitment is ingrained in the minds of every

Lee Atchison: 00:11:03

engineer that works at Amazon. You build everything so that no

Lee Atchison: 00:11:08

downtime is ever needed. No matter what the change to the

Lee Atchison: 00:11:12

system involves no downtime ever.

Lee Atchison: 00:11:17

Yes, 99.99% is a high level of availability to guarantee and

Lee Atchison: 00:11:23

not every company needs that level for their business to

Lee Atchison: 00:11:27

thrive. But even at a lower percentage of availability,

Lee Atchison: 00:11:31

there's no room for planned maintenance windows. 99%

Lee Atchison: 00:11:37

availability means 1.6 hours per week of maximum downtime. 99.9%

Lee Atchison: 00:11:45

availability, which is pretty reasonable, means 10 minutes per

Lee Atchison: 00:11:50

week of maximum downtime, and 99.99% availability means less

Lee Atchison: 00:11:56

than 61 seconds per week of maximum downtime. Even at these

Lee Atchison: 00:12:01

lower availability levels. A planned two hour maintenance

Lee Atchison: 00:12:05

window each week means your application will always always

Lee Atchison: 00:12:11

fail its SLA. Now some companies don't count planned downtime as

Lee Atchison: 00:12:16

official downtime, but you can bet that your customers do And

Lee Atchison: 00:12:21

isn't that what really matters. Thank you for tuning in to

Lee Atchison: 00:12:26

modern digital business. This podcast exists because of the

Lee Atchison: 00:12:30

support of you my listeners. If you enjoy what you hear, please

Lee Atchison: 00:12:34

leave a review on Apple podcasts or directly on our website at

Lee Atchison: 00:12:38

mdb.fm/reviews. And if you'd like to record a quick question

Lee Atchison: 00:12:43

or comment, click the Microphone icon in the lower right hand

Lee Atchison: 00:12:56

corner of our website. Your recording might be featured on a

Lee Atchison: 00:13:00

future episode. Make sure you get every new episode when they

Lee Atchison: 00:13:04

become available. Click subscribe and your favorite

Lee Atchison: 00:13:07

podcast player or check out our website@mdb.fm If you want to

Lee Atchison: 00:13:12

learn more from me to check out one of my books, courses or

Lee Atchison: 00:13:15

articles by going to Lee atchison.com. And all of these

Lee Atchison: 00:13:19

links are included in the show notes. Thank you for listening

Lee Atchison: 00:13:23

and welcome to the world of the modern digital business.