If You Fail to Plan….

Posted on by
4

Benjamin Franklin is credited with saying, “If you fail to plan, you are planning to fail.” Dog training is perhaps a very unusual exception.

Dogs are so amazingly adaptive, forgiving of the mistakes we make.  Dogs learn despite our errors.  Present them with the most convoluted, confusing training system and somehow they get it.  Had Ben Franklin observed dog training, he may have said, “If you fail to plan, plan to struggle.”

Planning is an essential skill.  I have as part of my Awesome Dogs C.O.R.E. skills. C.O.R.E. is not a protocol – it just stands for “Components Of Reliable Execution.”  It is a reminder that there are certain basic things that are the bedrock of effective, efficient training. Good trainers do these things.  Pet owners need to know their value so they can hire professionals that do them well.

Each time someone faces problem behaviours, they want a solution.  Exercises should address the problem by providing a path to a clear end goal.  A plan is not a soundbite, a suggestion or flippant piece of advice.  Protocols are not plans.  Cycling through ideas is not a plan.

People all over the world offer suggestions.   They range from someone saying they read an article on the “Top Ten Amazing Ways To Get Your Dog To STOP JUMPING!!!!”  It’s the person who stops you and says, “You know what you ought to do with your dog?”  It’s the person who watched that television show and feels compelled to share the latest miracle solution.  It’s also the person who reads a lot of books and jumps from one new thing to another, never stopped to fully assess if any have merit or risks.  Throwing stuff out there as a suggestion is not the sign of a plan.  It’s simply regurgitating information which may or may not be appropriate – even if it may come from a place of good intentions.

plan on chalkboard.jpg

A plan is goal, expressed in a series of steps that suits a specific dog and their family. A plan is the roadmap.  It shows the route and plots the steps to be taken to reach a specific destination.  It’s the pencil and the piece of paper, working out the BEST way to get there while avoiding detours and determining the best means of transportation.  Planning a trip to Toronto should be more detailed than saying, “Go east.”  You could fly, drive or take the train.  Or if you happen to be starting in Montreal, you might actually have to “Go west.”

Similarly, plans allow us to work through the dog’s training journey.  All dogs learn in the same manner.  It is a myth that different dogs learn differently. Not all dogs need to learn the same thing and not all families want the same thing.  A plan allows for preferences such as these.

Planning is in my opinion an ethical responsibility.  It indicates that we care to hear what a family wants to achieve.  It considers whether training suggestions might flare up other existing problems.  It values a client’s time and money.  We expect families to do right by their dog.  So we must do right by the families.

What are the attributes of a good plan?   First, they address the problem.  A roadmap to New York does little good if you want to go to Toronto.  Similarly, a solid plan for a leash reactive dog isn’t the best route for an anxious one.

Good training plans for success by setting a clear achievable goal or destination.  That goal is then broken down into steps – or criteria.  Criteria are the levels or steps throughout the training plan.  Good training splits these steps into tiny splinters of progression.  Each step needs to be clear and measured.  A trainer who says, “Reward the dog for being close,” has not created clear criteria.  It would be like saying, “Turn somewhere ahead.” Saying, “Reinforce the dog when it is within 30 cm of your legs,” is concrete.  How big is 30 cm?  Look at a ruler and you’ll know exactly.

Good planning addresses details.  Types of reinforcers are chosen with care.  For example, a toy might be ideal for teaching a dog to go over a jump.  You can toss it into the distance, into the grass, and not have the headache of waiting while the dog searches out crumbs of food.  Food is excellent for reinforcing a dog for walking in heel.  It allows for high rate of reinforcement drills, feeding the dog by your leg.  Plans also consider where the reinforcement ought to be given.  It can support and enhance the process.  Done poorly, it can slow you down.

Plans create the basis for clear instruction and reduce chaos.  Eyeballing and emotionally charged reactions are reduced.  We really cannot complain that clients do not follow instructions when the plan is sloppy.  No one can follow poor, arbitrary, loose instructions – not the dog and not the person either.

Let’s look at the usual suggestion given to families with dogs that jump up on people.  A common suggestion is this:



“Teach the dog an incompatible behaviour. If he learns to sit, then he cannot be jumping when he is sitting.”

While that seems very sound, it isn’t a plan.  It’s a wish, and not a very thoughtful one at that.  Different families will interpret and execute these instructions in different ways.

One family will tell the dog to sit each time it jumps. The end result will be a dog that jumps – a lot – because each time they jump the person tells them to sit and then hands out a cookie.  Others will ask for the sit when the dog is out of control. They will wind up sit nagging.

Another family might hunker down and get to work.  Diligently they will dispense cookies to all friends and family.  People will be instructed to, “Give the dog a cookie each time he sits.”  The dog will wind up charging every person it sees.  To sit for strangers, you have to get to strangers.  Perhaps that deals with the jumping, but it also leads to a lot of lunging, darting and pulling on lead.

Finally, some might successfully teach the dog to sit for every stranger. They will be happy until the dog keeps sitting for the veterinarian trying to get a rectal temperature. Hopefully that client has a good sense of humour.

All of these are plausible interpretations of “teach a dog to sit instead of jumping.”  The problems are ensue are avoidable had there been a plan.

Discuss with the family what they actually want.  Many people do not want the dog at the door when guests arrive.  Teaching the dog to go to a mat, when cued by the presence of someone at the door, and to stay on that mat until released  That goal might have been more appropriate.

On the street, most families want their dog to ignore strangers and keep walking, politely, on leash.  The goal may be, “When cued by the sight of a person, move to within 20 cm of the owner’s leg.  Continue to stay in that location until released.”

During free time when the dog can socialize, few families want a sit.  What they want is the absence of jumping.  Standing, sitting and lying down are all appropriate.  A training plan can work that into the mix.  It will involve working through time based criteria, building duration.

Once the destination is set, the work of breaking down the steps begins.  Decide on which cues will be used.  Think through type and position of reinforcement.  Plan how to proof skills, adding in layers of distractions.  A good plan will keep in mind that every training sequence will also create various associations.  Pavlov is a monkey on your back. He can mess with the best laid out plan.  You can also nudge him so that the associations created are the ones you want.

That’s a lot of planning.  It does not even begin to touch additional skills such as chaining or sequencing behaviours.  Teaching takes skill.  I’m not sure when we started thinking that throwing a pile of suggestions at someone was good teaching.  Amassing a pile of strategies and cycling through them shows very little skill.  Good trainers have method to their madness – a plan.

The beauty of a plan is that it addresses the individual needs of the family.  Roadmaps are created. Do clients need to know how to write a training plan?  Of course not.  But they do need to know that one is important.  They should be free to ask for an overview at any time.  The steps need to be expressed in such a way that they can easily understand what to do.

Ask Your Dog Questions – Overshadowing 2

Posted on by
5

How to ask your dog questions…overshadowing part two.

My last blog introduced the concept of overshadowing by offering a simple example to illustrate the point. For an explanation of the concept, you can read that blog here. But that basic understanding doesn’t go far enough in my opinion. Many unusual variations exist and can interfere with training.

To recap, animals are more likely to attach meaning to information that is more important and noticeable – the more “salient.” Abstract variations can be hard to spot. It’s a bit like the video of a gorilla dancing through a crowded room. Few people notice the gorilla until it’s pointed out. Once you see it, that gorilla is obvious.

Spatial properties are one such example. Dogs don’t just see and hear things. They also notice that cues are in specific locations. Where can be a very beneficial piece of information.

Hungry feral dogs hear and see small prey. Does it really matter if the squirrel is running through leaves? Over twigs? Makes chattering noises? Is digging? It’s more important that the dog pay attention to where the sound is coming. Knowing where allows them to find dinner.

Similarly, a dog is likely less concerned whether a scary sound is a bang, boom or grunt. Where allows the dog to run in the opposite direction – to safety. Where can be a matter of life or death.

Researchers have designed elegant experiments to test spatial factors. Dogs were taught to lift their right paw when they heard a metronome that was placed in front of them. They learned to lift their left paw when a buzzer sounded behind them.

Overshadowing experiment one

Then the researchers swapped the sounds. They wondered if the dog would lift the paw based on the type of sound, or the place from which it came from.

Overshadowing experiment

The dogs had not picked up on the type of sound.  They honed in on the location of the cue, not the sound quality. The full extent of the experiment was a bit more complex. The conclusion the researchers came to was this: Responses differentiated by location are most likely to come under control of the spatial features. (Dobrzecka, Szwejkowska, Konorski 1966)

Many dog training skills involve spatial components. Discrimination tasks where the dog chooses between things to the left, right or middle potentially fall into this category. If we’re not careful, the dog figures out that we move our hand just a little bit to the left, middle or right, giving the right answer by where our hand happens to be. It might seem that the dog has learned a complex task. In reality, they’re watching for a hint we might not even realize we are giving.

Sports dogs that learn to turn left or right are doing a skill with spatial features. If you’re not hyper vigilant in how you train, the dog might be paying attention to where you’re standing, not what you’re saying or doing.

Left and right paw is a skill many pet owners teach. Most dogs don’t actually connect the words to the skill. They learn, “Give the paw that is closest to the extended hand.” Or as you’ll see in the video blow, a foot can cue the behaviour.  The foot swaps for the hand just like the metronome and buzzer swapped. Where matters more than what.

Left and right paw is still a cute trick. Owners can effectively wipe their dog’s paws without polishing this skill. If you don’t mind issues with the cue, it does not matter.

It does start to matter if a family does not like it when their dog swats and scratches their arms – arms that are reaching out to scratch the dog’s chest. It matters if you’re the type of person who becomes frustrated or annoyed when the dog is not doing what you think they ought to.  If you well up with irritation, it matters.  It most certainly matters if the consequence for pawing at the wrong time results in punishment or discipline. This is a predictable effect. It is a dog doing exactly what they have been taught to do. It is not a stubborn dog.  It isn’t the dog’s fault or error.

There are three choices on the table. Don’t teach behaviours that might become obnoxious if you have no intention of polishing them. The second option is to accept what you have taught, happily. The third option is to finish the behaviour. That includes assessing and incorporating the correct cue.

How can we know if location is overshadowing other cues? Ask the dog. Manipulate little details and see how the dog responds. In the following video, I work through variations of Karma’s paw shake behaviour to see how she’s interpreting the lessons. It also shows how important where is to Karma, more than any other feature of the cue.  If you want to see spatial elements overshadowing, click right here to watch footage of it!

We cannot ask our dogs to repeat information. We can listen with our eyes and test assumptions. Based on the dog’s response, we can see if they heard us clearly. Learning about factors such as overshadowing, well that allows us to become better at knowing what to ask.  More importantly, it eliminates stress that doesn’t need to be present in our training because the dog is not being defiant, stubborn or anything malicious at all.  They’re just being normal.

Why Your Dog Doesn’t Know Sit

Posted on by
12

When I was a young girl, my grandmother would send gifts of books from Czechoslovakia.  The books were filled with stunning moving pop-up illustrations.  I learned a lot from those books.  I learned how those illustrations popped up.  I learned how one moving part operated another moving part.  What I failed to learn was how to read Czech.  My attention was so fixated on the illustrations that I memorized the words.  I recited the story based on the illustration.  I never focused on the letters.  Illustrations overshadowed the letters.

fox with cheese

Overshadowing is a well-researched part of dog training.  One place it applies involves adding cues.  (Commands for those who still use that term)  Animals, when simultaneously given two or more cues are likely going to learn about the most salient – to the detriment of the others.  Which facet carries the most weight depends on many factors.

Lured downs offer an example many can envision.  Dogs see humans bending at the waist during the lure.  If owners simultaneously say, “down” while luring or pointing, the word is unlikely to register.  Despite dozens or hundreds of repetitions, the dog stays focused on the owner’s gestures.  The word, “down,” becomes irrelevant background noise.  It predicts nothing of importance because with each repetition, the movement gives the dog all the information it needs.  Touching the ground comes to mean “lie down” rather than the word.  Pointing is perfectly acceptable if one wants to touch their toes in order to ask the dog to lie down.  I just don’t know many people who put it on their wish list.

Overshadowing affects many areas of training.  This is not a luring problem.  It can affect targeting, shaping and capturing.  It is not a positive reinforcement based problem.  Overshadowing has far reaching tendrils that reach pretty much everywhere.

Gestures do not need to be grandiose.  It can be as subtle as the wiggle of a finger or a raised eyebrow.  We all have dog training tells much like poker players have tells.  It takes self-awareness to compensate.  Often it takes a second set of eyes to spot them.

did you say something copy

This does not mean hand signals are bad.  Rather that few people want ridiculous signals.  No one wants to touch their toes to get a down.  It’s annoying.  Holding an arm extended for “stay” isn’t useful.  You can’t hold your hand up to sign stay and tie your shoe at the same time.  If you train with a prolonged gesture, that’s what the dog will need in order to understand.

Good hand signals are functional.  It takes effort to teach a great verbal cue.  Teach both but don’t mash them together and expect the dog to recognize them individually.  Don’t blame a dog for misunderstanding cues that were never made clear.

This might seem persnickety.  Who cares if signals are mashed so long as the method is “positive?”  Frankly, it is a problem when many trainers tell clients to mash signals and then encourage “balance” to clean things up.  Many repetitions become justification that the dog ought to know better and do better without recognizing that the repetitions were flawed.

Clients may feel that positive reinforcement failed because the dog is not responding to verbal cues.  This perceived failure again leads to “discipline.”  Positive reinforcement didn’t fail.  It worked exactly how science said it would.  In the absence of another reasonable explanation “dominant, stubborn, willful” can sound reasonable to a frustrated owner.

That’s a real shame because overshadowing is pretty easy to deal with.  Cues need to precede the behaviour without interference.  Fade lures.  Remove training aides during shaping or targeting.  Get a clean behaviour.  Then purposely add in the cue, and only the cue.  Spend a little time highlighting that these words or signs that are important and have meaning.  Do this in a thoughtful and planned way.

If we dismiss science, then we don’t learn about factors such as overshadowing.  I suppose it must be easier to justify corrections and sleep at night if one remains willfully oblivious to such things.  It is a choice to ridicule science and then justify corrections under the pretense of “balance.”

Learning the science is hard.  It demands that we regularly rip off the bandage that protects our fragile egos.  We need to find the thing we have yet to learn.  It goes beyond lip service.  We can’t just say, “We all have more to learn.”  You have to do something about it.

Learn better.  Do better.  Get better results.  Often those who suggest this path went down it before.  At least I certainly have.  There are walls you hit where it seems like something isn’t working right.  Finding out why is worthwhile.

Individually we should all lose a little sleep wondering if there isn’t some variation of overshadowing playing havoc with our cues (probably).  We should wonder if it impacts other areas of training (it does).  We should wonder if there are other effects to learn about (there are.)

Owners absolutely don’t need to learn decades of research in order to train their dog.  The trainers they hire should.

The takeaway for owners is twofold.  First, don’t mash your cues.  Second, science often does have all the answers….and it makes training a lot less frustrating when you work with it rather than against it.

Reaching the Holy Grail of Training

Posted on by
7

Years ago, I taught our Kiki a formal recall using targeting.  Systematically I proceeded to work through the exercise.  Much to my delight, Kiki developed the most fantastic competition recall.  People gasped at her speed and enthusiasm.  Her formal recall never failed us over the years.  You could say that it had behavioural momentum.

Domjan, in The Principles of Learning and Behavior describes behavioural momentum as, “response persistence in extinction.”  In non-technical language, behaviours with momentum are enthusiastic, despite distractions.  They are highly resistant to extinction.  Behavioural momentum is the Holy Grail of dog training.

behavioural momentum copy

Behaviours with momentum are like boulders rolling down a hill.  Considerable effort is required to stop them.  By contrast, weak behaviours are like pebbles.  The slightest bump in the road and they get derailed.

Sluggish responses are an indication of poor behavioural momentum.  Dogs that are easily distracted have poor momentum.  When sniffing a blade of grass is more appealing than coming when called, recall behavioural momentum is low.  If you wonder if your dog enjoys doing what you ask, then you might have a behavioural momentum issue.

When owners ask for reliable manners, they are asking for behaviour momentum.  They want a dog that walks politely and ignores squirrels.  They want a dog that keeps four paws on the ground, even when visitors approach.  They want fast, immediate recalls.

People want solid, reliable, strong behaviours they can count on.  There is no magic.  Behavioural momentum combines operant and classical conditioning.  Pavlov might always be on your shoulder.  For too many, he’s snoozing, periodically waking to create an unexpected association here or there.  We forget about him and he dozes off again.  Wake him up.  Intentionally create behavioural momentum by using a fast rate of reinforcement.

“Behavioral momentum is directly related to the rate of reinforcement (see Nevin & Grace, 2000).  A higher rate of reinforcement produces behavior that has greater momentum and is less susceptible to disruption.”  The principles of learning and behavior – Domjan

Crazy fast reinforcement triggers two types of learning.  The dog learns the skill through positive reinforcement.  The fast reinforcement creates a positive association to the behaviour.  When dogs love executing learned skills, distractions are less tempting.  That is crazy powerful stuff.

b momentum copy

Food is a natural fit for creating momentum.  Food offers ease of use, speed of delivery and speed of consumption.   Anything that slows the rate of reinforcement can interfere with creating momentum.  Slow reinforcement leads to frustration, boredom and wanderitis.  It is the road to “I’d rather be doing something else.”

How fast is fast?  The following video shows my Karma working with a high speed of reinforcement.  A reasonable goal for initial training is ten reinforcements per minute (or about a third of the reps in the video.)  Reinforcements should come quickly enough that the space between repetitions is devoid of wanderitis.

Slow reinforcement tanks momentum.  The temptation to prove that we can quickly wean away from “treats” can lead to slow reinforcements.  We employ a litany of protocols, “real life” reinforcements, games, variable reinforcement schedules and Premack.  While there is nothing wrong with some of these things some of the time, they are slow reinforcements.

Sluggish, sloppy outcomes tied to poor momentum convince people that positive reinforcement is not reliable or effective.  There’s a real tragic irony in there.  Tactics that we use to convince people to use positive reinforcement may slow reinforcement, sabotaging reliability and enthusiasm.  Weak results convince people that positive reinforcement did not work well enough.  Aversives can trickle back into the dog’s training.

Behavioural momentum is within anyone’s reach.  Both are possible when operant and classical conditioning neatly combine.  Enthusiastic, persistent responses tell us there is no place the dog would rather be.  There is no behaviour they’d rather be doing.  If we say, “sit,” the dog responses with “I thought you’d never ask.”

Creating behavioural momentum is the trainer’s choice to use food to its full potential.  We can choose to decide, today, that we will not only teach behaviours, we will teach so the dog loves doing them.  We owe the dogs that.  If we create behavioural momentum, there will be no doubt in our mind that the dog wants to do the things we ask of them.

There is no greater feeling than knowing you have reliable behaviour from a happy dog.  It really is a Holy Grail worth pursuing.  Wake Pavlov up and get him to work.  He is not just a tag-a-long.

Reading Research: Does Size Matter?

Posted on by
1

Following up from Reading Research part one where I review key aspects of the book Experimental Psychology by Myers and Hansen, I thought it would be interesting to tackle the question, “Does size matter?”  Of course, by size, I mean the sample size in a research study – often referred to as “n.”

Many introductory books make the point that small sample sizes are a red flag.  Small samples might provide flawed information.  A small group might be comprised of unique or unusual individuals – subjects who do not reflect the majority of the population.

Yes, Skinner, undoubtedly one of the most well known names in psychology did experiments with only a few subjects.  His work is held in high regard for being tightly controlled.  Much of his work has held up over many decades.

skinner

That presents quite a discrepancy to resolve.  Large samples are purportedly good, yet Skinner’s exceptionally controlled research used small samples.  Assuming that both premises are true, an explanation must exist.

Generally, large samples are beneficial.  One reason is that large samples are more likely to reflect the whole of the population.  There is another reason illustrated by the following example.

Let us pretend that we want to know if a dog training technique is better than another is.  We randomly divide the dogs into two groups.  Group A learns a task with our technique.  Group B learns with a different technique.  We train each group of dogs and compare the results.  This research design is called a two independent group design.

Statistics analyze the data. A standard t-test is a probable choice for this study.  Each statistical test makes assumptions in its calculations.  Standard t-tests assume that the data we are collecting creates a normal curve (bell curve.)  Without enough participants, there isn’t enough data to make a solid, fully formed bell curve.  As the diagram below shows, without a fully formed curve, it is impossible to compare if the curves are similar or different from one another.  Standard t-tests generally require at least twenty subjects in each group – but thirty is better.

normal curve

There are other forms of research.  Some researchers prefer to focus on details.  These details can be lost when data is pooled or averaged.  Instead of blending the results of many subjects, “small n” researchers focus intently at the individual responses of a few.

Such an approach can offer key insights.  For example, if we measured how dogs learn new skills, blended results might create a gentle sloping curve.  Individual results could paint a jagged process – breakthroughs and setbacks.

average graph

There are a number of low n experiments including ABA; Multiple Baseline Design; Changing Criterion Designs and Discrete Trial Designs.

Here are a couple examples to show how some of these processes work.

In an ABA design, the subject acts as both the experimental and control group.  Assume that we want to test a new anxiety treatment.  A baseline is measured during phase one (The first A in ABA).  Treatment is then given during the second phase (B).  Finally, treatment is discontinued (A).  ABA design allows us to see if the treatment has an effect.  We can also see if results disappeared when treatment stopped.  There are many variations of the ABA design such as ABABA, ABACADA and so on.  The reversals allow researchers to see if the order of treatment is having an impact rather than the actual treatment.

Discrete trials are common in conditioning experiments.  For example, we might want to know if dogs discriminate sound better with one ear versus the other.  In other words, we want to know if dogs are left or right “eared.”  Dogs learn to discriminate a tone.  Probe tones are presented to the left or right ear.  The dog’s responses – how quickly they discriminate on either ear is measured.  Comparisons are made.  A response is measured over many treatment conditions.  In this case, hearing is measured across a number of manipulations.  Humans who participated in a similar experiment each performed over 2000 trials.  The sample might be small, but the volume of data is massive.  It requires meticulous record keeping and data analysis.

The question should not be “does size matter?”  That is an overly simplistic question.

Of course, size matters.  Bigger is not always better when it comes to sample sizes.  What matters is whether the size of the sample works with the type of study and the statistical analysis used.

The various types of research are like tools.  A hammer is no better or worse than a screwdriver.  Using a hammer to drive a screw is fraught with problems.  It is similar with studies.  Different types of research serve a different purpose – they need to be used correctly.  Keep looking at sample sizes.  Also, look to see if that sample matches the type of research.  It can be helpful to grab a few studies, look up the sample size and look up the type of study.  Start becoming familiar with the jargon.

I would highly recommend Experimental Psychology to anyone wanting a deeper understanding.  My blogs are just highlighting a few small sections.  Well worth the investment.

Part one on reading research:  Internal Validity can be found here.
Part three – coming soon.

Reading research – 8 classic red flags

Posted on by
11

Ten years ago, few trainers had access to research studies.  These days with Google University, we have moved into the era of research wars.  It is a battle of quantity, novelty and link littering.  Unfortunately, few seem to be reading past the abstract soundbites to see if the study in question is any good.  Even more problematic are lure of pop psychology magazines with sexy titles, articles that probably misinform more than educate.

Every professor and textbook on the subject of research sends a consistent message.  Read research with a critical mind.  Not all studies are well executed.  Peer reviewed academic papers are no exception.  Sometimes journals will publish research with poor design to inspire further research.  Looking at study design is scientific, not sacrilegious.

As readers of studies, we can take steps to improve our research reading abilities.  We can face our own biases directly.    Do we militantly tear apart research that goes against our point of view, while offering leniency to findings that feel warm and fuzzy?  More importantly, do we know how to read an analyze research?

I won’t pretend to be a research expert.  Rather, over the next series of blogs I will be highlighting what I have learned from Experimental Psychology by Myers and Hansen.  It is a worthwhile investment for anyone wanting to flex his or her mental muscles.

One core concept of research is internal validity.  As the name implies, we need to assess if a study is valid on the inside.  External validity, by contrast, would look at whether results apply to “the real world.”  Lesson number one is that internal validity should not be sacrificed for external validity.  If a study is not valid on the inside, there is nothing of substance to apply to the “real world.”

Campbell identified eight “Classic Threats to Internal Validity.”  They apply to research involving experimentation.  This includes true experiments and quasi-experiments.  True experiments have strict parameters or rules.  Quasi-experiments are not bad, just different.  In both researchers manipulate a variable and then measure the effect it has on another variable.

IV isolation

For example, we might want to know if training method A is faster than training method B.  We divide a number of dogs into two groups and compare results of those two methods.  The type of training is the variable being manipulated.  We call this the independent variable (IV).  The goal of experimentation is to isolate the independent variable, to ensure that no other factor is interfering or confounding the results.

Revisiting our dog-training example, let’s say that group A tested on a Monday and group B tested on Tuesday.  If Monday was sunny and Tuesday was stormy, any claim that treatment A was better is highly suspect.  Stormy weather could have agitated the dogs in group B.  The independent variable was not adequately isolated.  The study would not have internal validity.

The following itemizes Campbell’s Classic Threats to Internal Validity and provides examples.  One step we can take toward understanding research is to understand how these threaten validity.

Our thunderstorm example above is a history threat.  Dogs in group B had a shared history during the experiment that differed from dogs in group A.  Training methods varied.  However, so did weather.  No one can say for sure which training method was faster because the weather interfered.  History threats can be subtle.  Another example would be if one group receives an orientation while the other does not.  Orientation can prime one group, giving them a head start.  It would also be a history threat.

Maturation threat reflects internal changes.  An obvious example might be age.  Behaviour can change as puppies mature.  Maturation can also mean the maturation of knowledge.  Students handling dogs during experiments will have gained knowledge throughout the term. It would not be wise to test group A with new students and group B at the end of term.  Increased knowledge by the end of term can mean that students guess the hypothesis or influence results.

Subjects rarely get the same test results when re-tested.  Practice leads to improvement, even without treatment of any kind.  Suppose we take a group of anxious dogs and test their heart rate.  Heart rates can drop simply because the dog habituates and becomes more comfortable.  A second round of testing should show habituation.  It is not enough to ask if a dog improved, we need to know if the dog improved more so than dogs that did not receive any treatment.  Otherwise, we have a testing threat.

Measuring results is not without potential pitfalls.  Instrumentation threats involve data collection problems.  Equipment can fail or be unreliable.  Scales used to score results need to be set correctly.  Assume we want to know if dogs become anxious at the dog park.  Imagine if the measurement options are:  highly anxious; moderately anxious; mildly anxious and fully relaxed.  Answers obviously will be weighted toward the “anxious” side of the scale.  Unless a dog is fully relaxed, it is by default labelled as anxious. Had moderately relaxed and slightly relaxed been offered as choices, an entirely different picture may have emerged.

Random selection between groups is important.  This process helps balance the characteristics between groups.  When groups are not random by design or chance, this is a selection threat.  Assume that wrandom balancinge want to know which training technique obtains a faster recall.  Group A dogs are mostly short hounds and toy breeds.  Group B has mostly large dogs with a smattering of Border Collies and Whippets.  Under those conditions, we could not claim that group B training produced faster recalls.  To avoid accidental selection threats, random selection and balancing offers an even comparison between groups.  Researcher choice is not random.

Mortality should be listed in an experiment.  It is the dropout rate.  When large numbers drop out of an experiment, it’s a red flag.  According to the text, “Often it means that the treatment is frightening, painful, or distressing.  If the dropout rate is very high, it can mean that the treatment is sufficiently obnoxious that typical subjects would choose to leave, and the ones who remain could be unusual in some respect.”  Assume we are testing a protocol to help reactive dogs.  Many drop out.  Those who remain seem to improve.  The obvious question is whether those who left were distressed or deteriorated so much so they did not return.  That is critical information.

The seventh threat comes with a big word:  Statistical Regression.  Extreme test results are unreliable.  Think back to grade school I.Q. tests.  Scoring low could mean you had the flu.  If an experiment uses subjects with extreme characteristics, we can expect some of that to level out on its own.  Testing a new anxiety treatment on highly anxious dogs can appear to work.  That result looks similar to statistical regression.  As with a testing threat, it is not enough to ask if an animal improved.  We need to ensure that improvement happened because of the treatment.

Finally, we come to selection interaction threats.  It’s the one-two punch of threats.  It happens when a selection threat combines with another threat.  Returning to our experiment that asks which dog training method is faster, suppose we non-randomly select dogs from two training schools.  Immediately, that creates a selection threat.  Now suppose school A has a trick team.  Students at this school are motivated to join the team.  The second training school does not offer tricks sessions.  That creates a history threat.  Trick dogs would have a wide array of skills to draw on – to guess the right answer instead of learning it via the training method tested.  Selection threat combines in this case with a history threat to create one hot mess.

classic threats to validity copy

Campbell’s Classic Threats are the tip of the iceberg in terms of red flags.  It can make it seem no research can hold up to its standard.  Following a defined process for evaluating research is a far sight better than pointing to number of subjects and chanting “the n value is too low.”  It may not be possible to control for every bump in the road.  Experimental Psychology states, “Control as many variables as possible.  If a variable can be held constant in an experiment, it makes sense to do so even when any impact of that variable on the results may be doubtful.”

Knowing the threats to internal validity are only useful if you start using them to read studies more carefully.  It might be tempting to annihilate an experiment you dislike.  Perhaps a more interesting exercise would be review an experiment you love and have shared.  Challenge your bias.  Look at the design and the various threats to internal validity.  Did you find any?

(Part 2, 3 and more….about those n values, non-experimental research and more.)

 

 

Shock Collars are so NOT like T.E.N.S.

Posted on by
3

I had a more practical blog lined up.  Humor me, as we look at one more expression.  This one keeps coming across my news feed and it’s interesting.  Some trainers claim that, “Shock collars are like T.E.N.S. machines.”

I have used a T.E.N.S. machine many times under the guidance of various therapists.  For those unfamiliar with this treatment, mild electric current flows to pads secured to the patient’s skin.  It causes muscles to contract.  Dr. Ho’s Pain Therapy System seen on infomercials is a T.E.N.S. machine.

Here is one of the machines that I use on a regular basis.  At the bottom are two dials for “intensity.”  There are two intensity dials because the machine regulates the left and right pads separately.

tens

Comfort levels change between individuals.  They also change from moment to moment for the same person.  Intensity levels vary from one area of the body to another, hence the two dials.  There is no “consistent” setting.

If you asked a patient using a T.E.N.S. “Are you okay?” the answer should be “Yes.”  Discomfort results in a reduction in intensity.  If asked, “Do you want me to turn it off?” the answer should be, “No.”  Stimulation should be neutral or pleasant.

With shock collars, it is this underlying premise that is fundamentally different.  Bark collars shock dogs to suppress barking.  Barking reduces because the dog does not like the shock.  If the shock were neutral, nothing would change.  If it were pleasurable like a T.E.N.S. machine, barking would increase.  Barking would increase because the dog would be learning that barking leads to massages.

The same goes for electronic training collars used as negative reinforcement.  Negative reinforcement is like the game “Uncle.”  Children pinch one another.  They keep pinching until the other child says, “Uncle.”  The behaviour of saying uncle leads to cessation of pinching or relief.  Similarly, dogs turn the shock off by obeying a command.  Reinforcement is the cessation of the electronic stimulation – relief.  Dogs that respond quickly can “beat the buzzer”, avoiding shock altogether.

“Pleasant” shock, under the level of aversiveness wouldn’t be effective here either.  Ask a dog, “Do you want the electrical current to stop?”  The answer needs to be, “yes” or the collar will not work as designed.

Both may involve electrical current but the intensity is on a gradient.  Many things are on gradients.  Music is nice, unless it is too loud.  Massage feels good, unless Olga the Horrible is hurting you.  (My apologies to anyone named Olga.  Olga was the name of the massage therapist who hurt me.)  Light helps us see unless it is glaring.  Flowers smell pretty unless it is an overdose of noxious perfume.  Cool water makes a refreshing swim.  Cold water is icy and painful.  Food is usually fabulous, unless you have eaten excessively and feel nauseated.

No one thing fits neatly into a naughty or nice category.  Dogs decide what they find pleasant, neutral and aversive.  We infer how the dog feels by observing their change in behaviour.

I would have gladly paid Olga the Horrible to stop the massage.  Massage therapist Mike was different.  He had magic hands.  I would pay money for longer massages.  Olga’s massages were aversive.  Brad’s were appetitive.  I was not screaming and thrashing in pain with Olga.  I just wanted it to stop and it changed my behaviour.  Contrary to popular belief, it’s not the dog’s attitude that will tell you if the training is aversive.  It is the dog’s responses.

There is a second key difference between these two devices.  Shock collars are training.  With T.E.N.S., the goal is long-term if not permanent pain elimination.  It is not a teaching tool.

Olga may have been brutal, but her intent was to offer long-term relief.  Magic Hands Mike offered the same goal under threshold.  Suzy who does scalp massages at the hair salon….she just gives nice feeling massages.  Training cannot be compared to these scenarios.

In training, the trainer creates the plan with the goal of creating a change in behaviour.  We determine which behaviour we want to teach.  Therapies for pain relief have no skills development.  Olga never taught me to dance a jig in return for pain relief.  She did not use my discomfort as a tool.

I never went back to Olga.  I had the choice to leave.  More importantly, I had a choice to never return.  I could refuse to participate.  Our dogs do not have that choice during training because repetition is part of the gig.  Once the aversive ceases, the aversive is back in play for more training.  We can’t see all the differences unless we look at the specifics of how each of these work.

Olga vs Mike

If you are ready to jump in and say, “You don’t need batteries to use aversives,” then you won’t get any argument from me.  There are many forms of aversives in dog training.  Some are easier to swallow than others.  You do not need batteries to use aversives.

But, let me broaden the statement.  Using aversives in dog training is not the same as using a T.E.N.S. machine.  It’s not even close.  It does not matter if the aversive is added to the dog, the dog is added to an aversive scenario or whether we use a dog’s discomfort for our training purposes.  Using the aversive is trainer’s choice.  If the dog is put into a position where they want to leave, they have been put between a rock and a hard place.  It’s not free will.  The trainer has free will.

Training paradigms are not the same as T.E.N.S. machines.  In dog training, the choices to use an aversive are ours and ours alone.