Should You Consider Using The Newest Product?

New products and changes within management practices are continuously being developed and introduced in the market place. Some of these products or practices are truly cost effective and should receive consideration. But, how do you decide which products or management practices to purchase or implement? Many factors, such as economic benefits, improvements in labor efficiency, or ease of incorporation of a practice or product into your operation, should be considered. The final decision on whether to implement (or not) a practice or purchase a product needs to be based on facts and not on a great sales pitch. The bottom line is, what type of response is expected from said product/practice and does good research data support its use? Over and over again, you have heard UK Extension specialists and agents make the statement, “before you purchase a product you need to ask for the research data supporting the use of this product and then determine if the research was conducted in an unbiased way”. So, what are we referring to when we talk about unbiased research?

What is a “Good” Research Study?

When we use the term “research study or trial”, we usually are referring to a specifically-designed experiment(s) where a product/practice is tested on individual animals or crops. To be considered an unbiased research study, animals or crop-rows receiving the new treatment product or practice must be managed identically within the same time frame as animals/crops not receiving the product or practice. Those not receiving the new treatment are called the controls or 0 treatment group. The only difference between the control group and the treatment group is the treatment itself. For field crop experiments, all rows in the comparison would be planted at the same time, under similar soil characteristics, and would receive the same amount of rainfall. With experiments utilizing lactating dairy cows, selected cows often are similar in number of days in milk or days till expected calving and each treatment group contains equal numbers of first lactation versus mature cows.

Each treatment contains multiple replications of animals, crop rows, or pens of animals. An equal number of animals or crop-rows receive each of the treatments being tested. In these experiments, similar animals, i.e. cows with the same lactation number (first calf heifers vs. mature cows) or rows in a field plot, are RANDOMLY assigned to receive either the treatment or no treatment or product. The most important point here is that the treatment group should be exactly the same as the controls. For example, if a study preselected all of the low producing cows for the control condition and then high producing cows for the treatment, the study is considered biased. That is, the “treatment” group already has an advantage over the control group. This random replication allows one not to favor one treatment over another.

Under ideal conditions the individuals taking measurements on the animals will be unaware of the treatments. This type of study is sometimes referred to as a “blinded” study. “Blinded” studies are especially important when an important measured response has a subjective component; such as scoring body condition or lameness.

To ensure an adequate number of cows receive each of the treatments, cows could be placed on the treatments over time as long as they enter as pairs where one cow of the pair (selected randomly) gets the control and another the treatment. Treatments applied to a pen of animals (i.e. a feed additive included in the TMR and then fed to the pen of cows) or herd and not individual animals only gets counted as if the treatment was given to one cow. Multiple pens of cows are needed to test the effects of a product in this situation. Remember that the “treated” cows and the “control” cows need to be managed the same and during the same time period. Thus, the control period for all animals cannot occur before or just after applying the treatment or product being tested. 

Depending on the experiment, more than one treatment may be tested along with the control treatment. Seldom are products from different companies tested in the same experiment, since companies do not want to fund research for their competitors.

Different Means = Difference???

Once the data from the experiment is collected, researchers summarize the data for each treatment and complete a statistical analysis using specialized math. First, the average response (referred to as the mean) with and without the product is calculated. For example, the average amount of milk produced by cows receiving or not receiving the tested product X is calculated.

If the animals’ or crop’s average response (or mean) when receiving the treatment is greater than those not receiving the treatment, does that mean that the product or treatment resulted in a positive response? The answer is—maybe, but not necessarily. The answer depends on the amount of variation in response seen within groups of animals or crop rows either receiving or not receiving the tested product or management practice. The more variation between animals or crop rows on each treatment, the bigger the difference needed between treatment means for one to consider that the treatments are not the same and, thus, are different from one another. Therefore, we need to look at more than the means or average response on each treatment.

In studies where we expect a large amount of variation in a response, more animals are needed on each treatment and in the control treatment. One example are responses related to reproductive performance; these studies require more animals per treatment than those just measuring a milk production response. Experiments measuring reproductive performance usually use hundreds of cows versus nutritional studies that use less than 100-cows total.

To account for this variation and to test if the treatment averages or means are the same or different, researchers calculate the probability that the means are the same. This probability is commonly denoted as the P value. When the P value is less than or equal to 0.05 (P≤ 0.05), essentially we conclude that the treatments are not the same and that they are different from one another.

Will I see the same response if I feed or use the product or technology?

Realize that the measured response was seen under the described conditions and different responses can occur under different conditions. The more controlled trials showing a favorable response, the higher the probability a positive response will be noted under differing conditions. If possible, look for experiments that are conducted under conditions that are similar to your situation. For example, products/practices that are effective in Florida in the summer may not translate to winter conditions in Kentucky. If you are considering using a product be sure to take note of the inclusion rates (or dosages) used in the research studies versus the manufacturer’s recommendations. Some research studies will test multiple dosage rates and conclude that, while one rate was beneficial, another dosage rate was not.

Use Sound Research Experiments When Considering New Products

Sound research experiments involve comparing the response between animals or crops not receiving the tested treatment versus those receiving the treatment. Adequate numbers of animals (or pens of animals) or crop rows managed under identical conditions and randomly assigned to each treatment are used in these experiments. Then, the average response for the product being tested and whether the average response is the same or different than those not receiving the product being studied are calculated. From this information, one can determine the probability this response was not due to chance and is different from those animals or crops not receiving the new product. These steps allow one to make decisions based on facts and not the opinions of fellow farmers or sales people.

Author:  Donna M. Amaral-Phillips