Month: July 2020

Categories
About Learning Better Life improvement unintended consequences

Unintended Consequences BL019

I’ve mentioned unintended consequences. This is not as complicated as it may sound, so I’ll illustrate with a few examples.

In the first case, the result is obvious once you think it through, but it depends on framing the problem properly.
Suppose the roads in your town are very busy. So the town looks at the current traffic volume, projects some future growth, and adds a lane to the major roads to increase capacity. 
Because it’s faster to get around, people decide to take more trips. So the traffic volume immediately exceeds the growth projection, and the roads get just as busy as before.
Often we make decisions based on “all other things being equal”. But all other things are rarely equal. Because this system has intelligent agents (people) who make decisions based on the changing characteristics of the system (road capacity as reflected by travel time), any assumptions about a constant traffic load are simply incorrect. Systems can react in this way without intelligent agents, through simple physical, chemical, or operational processes.
What isn’t obvious is that it’s travel time, not road capacity, that drives the behaviour. People don’t care how big the roads are. They only care whether it’s worth the time it takes to get somewhere compared to other options such as public transit.
In general this is called a feedback loop, where a change in one factor leads to a change in another factor in a way that either magnifies or diminishes the first change. More about this later.
The second case is more personal. I had the privilege of fulfilling a lifelong dream to trek in Nepal. The weather was fine and the mountains stupendous. But I felt like I had to pee. Really bad. Nearly all the time. It was uncomfortable, embarrassing, and inconvenient in an area with few toilets. I’m not super shy but getting just a bit of privacy is preferable when going outdoors. Are you getting the picture?
This had happened to me before. I had gone on an awesome bus tour of Buddhist sites in Sri Lanka, and had essentially the same problem. It did happen on occasion at home, but not as bad as on trips. So I assumed perhaps I was getting older and it was my plumbing; or I was not getting enough of the right kinds of fluids; or maybe I had some kind of infection. I had discussed it at some point with a doctor and probably had run a course of antibiotics and tried a couple of other things, but because it wasn’t ahem, pressing, when I was at home, it didn’t get a lot of attention.
So on both trips I just did the best I could, suffered my plight and embarrassment, and still mostly enjoyed myself.
Where does the unintended consequence come in? It was many years later that an internet search revealed that vitamin supplements such as vitamin C, or drinking juice that is acidic such as orange juice, can cause an urge to urinate. Guess what I was doing to stay healthy under demanding conditions or in hot weather? Lots of fluids including juice, and vitamin C supplements.
Since that revelation I am able to observe a direct and immediate correlation between the two. Needless to say, vitamin C supplements are saved for when I’m at home, and my juice consumption is down (also reduces my sugar intake, but that’s a different story).
Generalizing, it’s often the case that many of our problems may be aggravated by their own cures, or by the attempted cures of other problems: a plug for the structured approach to improvement presented earlier.
References
Business Dynamics: Systems Thinking and Modeling for a Complex World by John D. Sterman
Categories
About Learning Better Life improvement

Improvement Cycle 6 – An Alternate Approach BL018

I’ve described a rigorous method for improvement, fairly well-known as plan-do-check-act, with my own embellishments. As described it takes a scientific approach in the face of many unknown factors, to narrow down cause and effect relationships.

Sometimes a different approach can be taken to improvement. In the sleep example, I could say, there are a lot of factors that are well-established by solid research to be beneficial to better sleep, for most people. Unless I have specific doubts, I don’t need to prove that a specific method is effective.
I could measure my baseline, select several aspects to change simultaneously, keep track of them as I go, and measure the results. If I get positive results, then I’m happy. Nothing wrong with this method in terms of getting results. I don’t care which factors in particular worked, and as long as I keep all of the changes in place, then I should be able to maintain the new improved state.
The limitations are that I don’t know which of the factors helped the most, which could be discontinued, and I can’t make statements about just one of the factors. I really need to say “I changed a bunch of things, and I don’t know what worked in particular, but it’s improved so I’m happy.“
This method doesn’t apply if I throw into the mix unproven methods. If I do that, I can end up with a false positive: a belief that the unproven method had a positive effect when in fact is was some or all of the proven methods. This can also happen if I don’t track my baseline or my behaviour across the proven factors, and just add one unproven method. I see positive results and I think it was the unproven method. But it was actually something that changed in one or more other factors, but I wasn’t keeping track.
This is what’s referred to as anecdotal evidence: it’s a valid personal observation that hasn’t been conducted with any rigour or controls. In one sense there isn’t harm for some situations, as long as the benefits are achieved and there aren’t any unintended consequences. But a person can’t be justified in stating that the unproven method is now supported, even for themselves, because they haven’t really approached it in any kind of structured way.
Similar effects occur in organizations when several initiatives are undertaken in parallel. Everyone takes credit for any benefits achieved, but in fact some initiatives may have been very helpful and others quite unhelpful, or positive results may be from external factors unrelated to the initiatives. In politics, taking credit for results achieved in the presence of global economic conditions over which no single body has much control, or policies which happened to be in place rather than intentionally designed for a situation, is everyday behaviour.
About Learning
What is the shape/path of progress in learning? In school you might have experienced a fairly steady progression through each grade, a relatively straight line from knowing little to knowing enough to do the assignment, pass the test, get a passing grade. There might have been some bumps: a missed assignment, a failed test, a poor or failing grade, a change of courses.
Learning a sport, at least at the beginning, is a similar process. You observe, try, practice, and improve your skills in spurts and plateaus.
Advanced study and research at a mastery level, and exploration of complex problems and situations, often proceeds differently. Major setbacks, long plateaus, backsliding, abandoning a line of inquiry or action completely, starting from the beginning, making major changes in methods, dealing with injury, insult, or rejection, and straight-up failures and crashes, are part of the possible territory.
Recognizing that the path will often not be easy or straightforward, that you can feel lost or like a failure for long periods, will help you be persistent in the face of difficulties: you’ll expect them as being normal, rather than something to be feared.
Categories
About Learning Better Life improvement

Improvement Cycle – Review Step 5 BL017

Without getting into the details of data analysis and statistics, I have introduced just a few things that you might watch out for as you interpret the data collected after performing the selected actions for a significant period of time – at a minimum, 6 to 10 data points.

At this time it’s appropriate to do a review and make any adjustments. Based on your measurement system you will repeat this review periodically until you have achieved the benefits you set out to achieve, or have decided to abandon the initiative.
  • What has changed since you set up the improvement process?
  • Is there anything that is obviously not working or is causing problems?
  • What is obviously working or beneficial?
  • Does the data analysis support what seems to be obvious?
  • Are there any clear trends emerging compared to the baseline?
  • Is there anything about the measurement system that needs to be adjusted?
  • Do you need to collect information for a longer period of time to establish a new baseline before making further adjustments?
  • What do you need to try to make sure that the effects you are observing are a result of the cause you believe is at work?
  • What are some alternate explanations for what you are observing?
  • Have you performed several improvement cycles and reached the end of your exploration such that it’s time to end this initiative and shift your focus to something different?
Exercise
1. Answer the above questions for your improvement initiative.
2. What questions would you add to the review?
About Learning
Learning can be unpopular with those around you. As you dig deeper and get into the habit of asking questions, you might:
a) quite simply know more about the world than you used to
b) start to question and understand the underlying reasons why things work or don’t, how they came to be the way they are
c) be able to define problems in new ways and find alternative solutions
d) begin to develop a better sense of the different possibilities about how events might unfold
e) develop a new sense of the probabilities of those possibilities
f) identify positive and negative, intended and unintended, consequences, in a broader and deeper way
g) start shifting your priorities and choices about how you spend your time, effort, money
These shifts in habits and beliefs may reduce your compatibility with, and appreciation for, people in your social and professional circles. This is a natural process of learning and maturation, and some relationships may need to be transformed or let go.
Categories
About Learning Better Life improvement sleep

Improvement Cycle 4 – Pitfalls of Analysis BL016

Every step of the improvement cycle has its own pitfalls, but perhaps the most blatant and preventable errors occur in the collection and analysis of results to determine the effects of actions taken.


It’s in this step that certain specialized skills are necessary, and unfortunately, far from universal. Even among the highly trained, logical and statistical mistakes are occasionally made.

So what can you do? First, know what you don’t know. I’ll mention here and in future posts to some of the areas that you need to be aware of. 

Second, develop those skills. To improve in any area means that you need to develop your improvement skills in addition to the core functional skills. An athlete who wants to improve performance learns how to train. A writer learns how to edit. An actor learns how to rehearse. These are all process skills – how to get better – not just hitting the ball, writing a paragraph, or acting a scene.

Third, get a second opinion (or third, or… .) In high stakes situations, organizations use “red” teams, a group of reviewers who is given permission to be critical about the thinking and actions of the team being reviewed.

What are a few of the pitfalls?
  • inconsistent collection of data
  • excluding or dismissing some data points that don’t seem to fit
  • interpreting data to confirm what is already expected or believed
  • reaching conclusions before sufficient data has been collected 
  • presenting results that aren’t statistically significant – there is a certain degree of variation that is expected (“noise” in the system) that can’t be said to mean anything
  • delays in effects – any action that has a delay in its effect can be difficult to follow
  • confusing correlation with causation – events may align in time, but may not have any cause and effect relationship – to propose a causation, a plausible mechanism for that cause to produce that effect needs to be put forward
  • ignoring multiple interconnected variables – to rigorously demonstrate an effect, we want to keep all other things constant as we change one factor. In practice this is extremely difficult, even in a laboratory setting, and it makes us susceptible to seeing what we want to see.
In the sleep example, how do the multiple variables come into play? I chose to focus on temperature as an action, out of a long list of possible factors. How could I keep all of the other factors constant? 
Short answer: I couldn’t. If I collect data for a long enough period of time, there might be a case to say that natural variation in the other factors that are not consciously being changed are less likely to be contributing to any trend I observe, than the specifically tracked temperature.
Let’s consider the short list of factors from the last post. I do have control over my screen time, and can attempt to keep some consistency over it. I could track sleep medication, how much and when. I can perhaps get away with assuming that sleep apnea is relatively constant. I can track my alcohol and exercise. Even the temperature itself is subject to the ambient outside temperature with the window open, whether I used the oven or fireplace in the evening, etc. 

But some of these weren’t in my measurement system! Am I now going to start tracking a lot more factors? In any case it gets interesting because my tracking of other factors may result in changes to my behaviours in those areas. 

I might see that my alcohol use fluctuates, and because I want to make the effect of temperature change more clear, I reduce my variation in alcohol use, whether consciously or more incidentally. This in itself could have a larger impact on sleep than the temperature change I chose as my conscious action. 
Almost every improvement initiative has these sorts of effects – focusing on an area for improvement; the selection of actions, the measurement of data; the process of analysis and interpretation; the communication and integration of results; learning about the domain and about the improvement process. All of these can affect the system, even for this personal example.
When I see positive results, that’s great! But I might think it was because I changed the temperature, when actually it was changing my pattern of alcohol use. This type of conflation of actions and results is common in every area of life and business.
Exercise
1. What don’t you know about your improvement area?
2. What formal learning have you had about improvement?
3. Who could you consult as a red team?
4. How might your data collection and interpretation be affected by the pitfalls mentioned above?
5. How might your system be changing unintentionally as you go along?
About Learning
We need to understand what is at the edge of knowledge and what is well-integrated. We don’t need to blindly accept the mainstream, but if we choose to question or reject it, we need to invest more into learning and understanding deeply before following another path. (And here lies the failure of education systems to teach the basic techniques for evaluating knowledge and for identifying the interaction between fact and belief.) It’s for some reason common for some people to reject equally what’s well-established and what is more provisional.
There are many frontiers in our understanding of how things work. And sometimes things change. But by itself this fact doesn’t provide a basis for rejecting current knowledge. Changing one piece requires that change to make sense in the context of everything else we know. It’s our responsibility to dig deeper into how things are interconnected as part of the questioning process.
Where we have detailed, reliable, and proven knowledge that can be used to design new processes, materials, drugs, treatments, products, etc. then it’s an extremely bold claim to reject one small piece of this interconnected knowledge system. Such extraordinary claims require a careful approach of study, experiment, demonstrations, and repetition to be considered valid.
An example is the rejection of vaccines. To say one doesn’t believe in the effectiveness of vaccines to prevent life-threatening illnesses is similar to saying that antibiotics don’t work. I can only assume that people would not refuse antibiotics when they show up very sick at the hospital. Yet both have well-understood, and hidden, mechanisms. Both have short- and long-term trade-offs. Neither area is at the edge of understanding medicine.
Psychoactive drugs are a different story. The mechanisms, consistency of effects, side-effects, and other aspects of many of these drugs is not well-known. This is a leading edge of medical treatment and a healthy skepticism with regard to their use seems well-justified.