Five Tips for Evaluating Clinical Studies

Click to View

Evaluating the results of clinical studies can be daunting, but some simple tools can make it a fun experience. I consider drug literature evaluation to be a journey down a somewhat winding road that ultimately leads to a better understanding of a clinical study. 

Each study or clinical trial of a drug tells a story, and it is up to us as pharmacists to use our skills just like a detective to critically evaluate and understand the behind-the-scenes statistics. Statistics are tools that evaluate data to provide important study results. Essentially, clinical trials are types of studies that are the basis of the drug approval process, which is a take-home point that I always emphasized when teaching pharmacy students.  

Here are five things to know about statistics from clinical studies:

1. Read Beyond the Abstract

The study abstract is like a restaurant menu in that it gives you a taste of the study and includes descriptions of the most important points. However, the abstract does not go into detail about all the statistical methods used in the study. Reading the full study enables you to take an in-depth look at the results and statistical methods. 

Trending: The Future of Rebates: The Black Swan Scenario

Pharmacy conferences can be a great way to get a glimpse at ongoing or completed research projects through poster presentations, which are basically abstracts or summaries of the study’s results. If the researchers who presented the poster or abstract decide to submit their manuscript to a journal for publication, then the information could change from what was presented. A study by Saldanha and colleagues in the journal Trials looked at discrepancies between conference abstracts and the later, full publications and found differences in the main outcome results that were reported. The take-home point is to never base clinical decisions on what is in an abstract or poster. 

2. Determine Whether All Results Were Included

When evaluating clinical trials, it is always important to see how the data were analyzed. 

Intention-to-treat is considered the gold standard analysis since it accounts for data from all patients who were randomized or initially assigned to treatment groups. Even if some of the patients failed to complete the entire study, their information is still analyzed and reported in the results. Intention-to-treat is designed to mimic what happens in clinical practice. Patients will not always be compliant with their medications or may experience side effects that cause them to discontinue treatment. 

Per-protocol is another type of analysis, but it should be viewed cautiously since it excludes the results of patients who did not adhere to their protocol treatment. If a patient misses one or two doses, their valuable information may never be revealed in the results. This could make the treatment look better than it really is and lead to biased results.  

Continue reading on page 2...3. Know the Most Common Types of Statistics 

It’s important for pharmacists to have a basic understanding of the different types of variables involved in a study in order to determine what types of statistics should be used. 

When it comes to statistics, there are two basic types: descriptive and inferential. Descriptive statistics use data to provide descriptions of a population, which may include the mean (average), median (middle data point), or mode (value occurring most often) calculations. If you are reviewing a study that involves looking at the average number of pharmacists who report adverse drug reactions to the FDA, then this involves descriptive statistics. 

Inferential statistics make inferences and predictions about a patient population based on a sample of data taken from the population. Inferential statistics are important for clinical trials that are evaluating medications in a treatment versus placebo group for the FDA drug approval process. 

When evaluating clinical studies, you should recognize some inferential statistics that are commonly used. Variables are divided into the three main classes: categorical, ordinal, and continuous. Nominal data are also referred to as categorical and includes variables such as sex, presence of disease, and smoking status. Ordinal data involve rankings, such as if you were rating a movie, restaurant, or hotel. In clinical studies, this may include a pain scale that measures a patient’s pain intensity. Continuous data consist of variables that can be measured and take on any possible value, which may include height, weight, blood glucose, and cholesterol. 

Read More: Dealing with Difficult Colleagues

Nominal or categorical variables measured from two independent or unrelated samples are usually analyzed using a chi-square test. The chi-square test is used a lot to evaluate baseline characteristics of patients in randomized trials to show there is no significant difference between the groups in terms of age, sex, use of other medications, or other factors. 

The Mantel-Haenszel test is generally used to control for any confounding variables between groups that could introduce bias. For example, if a study is evaluating a weight loss medication, it may be difficult to control the diet of the study participants. 

Ordinal data are usually evaluated using the Mann-Whitney U and the Wilcoxon rank sum tests. 

Continuous data will generally use T tests, analysis of variance, or analysis of covariance to evaluate these data. 

The goal of any clinical trial is to determine whether the results are statistically significant, which is usually designated as (p < 0.05). This means that the difference observed between the two treatment groups would occur by chance fewer than five out of 100 times. 

As pharmacists, we should also ask the important question of whether the results are clinically significant or applicable to practice. If a study showed that a new antihypertensive medication lowered blood pressure by 3 mm Hg (p < 0.05), what would you say about its impact on patients? This may not be clinically significant, but we need to see more outcomes data such as whether it decreases the risk of cardiovascular adverse events and mortality. 

4. Observational Study versus Randomized Controlled Trial

Randomized controlled trials are considered the gold standard type of study to assess the efficacy and safety of medications because they have defined endpoints and a set protocol. Observational studies can provide information about associations, but cannot assess cause and effect. They should be looked at cautiously when it comes to applying the results to the clinical practice setting. 

Trending: Shingles Complications in the Elderly

Pharmacists should look for key terms that identify whether a study is observational; terms like cross-sectional, cohort, and retrospective. Many observational studies analyze data from national databases.

5. Odds Ratios and Confidence Intervals

Studies generally use the 95% confidence interval (CI), which is the range of values within which the researchers are 95% confident that the true value lies. For example, a study reports that a diabetes medication taken over six months lowered the A1c by 3% with a 95% confidence interval of 1% to 5%. This means that the researchers are 95% confident that anyone who takes the medication, even outside of this study, for six months will have an A1c lowered by 1% to 5%. 

This is a great concept to teach pharmacy students during clinical rotations when evaluating studies for a journal club assignment. If the range of the CI includes zero, there is no difference in efficacy between the two treatment groups. This means the results are not statistically significant.

Odds ratios are used to determine risk of an adverse effect in case control studies. An odds ratio of one means there is no difference between groups in regard to adverse effects. An odds ratio greater than one shows an increased risk of adverse effects, while less than one indicates a lower risk.