New products and practices are being developed and introduced to the marketplace constantly. The question is: Will they work in your herd?
“The final decision on whether to implement (or not) a practice or purchase a product needs to be based on facts and not on a great sales pitch,” says Donna Amaral-Phillips, an Extension dairy specialist with the University of Kentucky.
First and foremost, ask for research that has been done on the new product or practice. To be considered valid and unbiased, research should be conducted that looks at the new product or practice as the sole variable in the trial with both animals treated and untreated (the control group) managed identically within the same time period. “With experiments utilizing lactating dairy cows, selected cows are similar in number of days in milk or days till expected calving and each treatment group contains equal numbers of first lactation and mature cows,” she says.
Animals must also be assigned randomly. “For example, if a study preselected all of the low producing cows for the control condition and then high producing cows for the treatment, the study is considered biased,” Amaral-Phillips says.
Under ideal conditions, the individuals taking measurement will also be unaware of which animals are in the treated and control groups. “’Blinded’ studies are especially important when an important measured response has a subjective component, such as body scoring or lameness,” she says.
To determine if a practice or treatment is effective, researchers average the response rate (referred to as the mean) with and without the product. But that’s not the end of it. Even if the mean response to the product being tested is greater than that of the control group, the two responses may actually be no different depending on the amount of variation in response seen within the groups of animals.
“The more variation between animals on each treatment, the bigger the difference needed between treatment means for one to consider that the treatments are not the same,” Amaral-Phillips explains. “In studies where we expect a large amount of variation in a response, more animals are needed on each treatment and in the control treatment…. Experiments measuring reproductive performance usually use hundreds of cows versus nutritional studies that use less than 100 cows total.”
To account for variation and to test if the treatment means are the same or different, researchers will calculate the probability that the means are the same. This probability is known as the P value. “When the P value is less than or equal to 0.05, essentially we conclude that the treatments are not the same and that they are different from one another,” she says.
Even if they are different, you’re still not done deciding whether a product will work in your herd. You next need to look for experiments that were conducted under conditions similar to where your farm and herd are located. “For example, products/practices that are effective in Florida in summer may not translate to winter conditions in Kentucky,” she says.
For more information on evaluating product research trials, click here.