Depression GWAS Study, 2018

This will be my first critique of a depression GWAS study (Link Here):

Genome-wide association study of depression phenotypes in UK Biobank identifies variants in excitatory synaptic pathway (Howard, et al.)

Let me start by discussing the first sentence of this study:

"Depression is a polygenic trait that causes extensive periods of disability. Previous genetic studies have identified common risk variants which have progressively increased in number with increasing sample sizes of the respective studies."

First, the statement that "Depression is a polygenic trait" is an assumption, based largely on the fact that no specific genes have been found and that it is heritable.  Secondly, (spoiler) none of the previous "common risk variants" were matched in this study.  I believe this is because these studies are simply producing false positive results.

Let me also point out, in fairness, that I am making some assumptions related to this study.  I attempted to contact the author to clarify a few things, but received no response.  I welcome input from any of the authors in the comment section and I will address any clarifications.

Getting into the meat of this study, this is a genome-wide association study (GWAS), which uses the UK Biobank on a database of 300,000 participants.   (If you haven't read it, please check my post about the preponderance of false positive results from GWAS studies here. )  The study divides the participants into 3 categories related to depression: "Broad depression, probable major depressive disorder (MDD), and International Classification of Diseases (ICD, version 9 or 10)-coded MDD."  For a few reasons, I take some issue with these divisions.  The first is the use of the term "Broad Depression," which is not a DSM or ICD diagnosis , is ambiguously defined (self-reported past help-seeking for problems with “nerves, anxiety, tension or depression”) and mixes several marginally related symptoms together.  I would prefer if such a study confined to a single, DSM diagnosis, particularly when you have a group that have apparently received this diagnosis specifically available to you, although I can understand the desire to use a larger database.  

There is a numbers disparity amongst the 3 categories being used, with the largest category for Broad Depression (113K), followed by Probably MDD (30K), with the fewest designated as ICD MDD (8K).  This fact is important in my view, because the larger the database, the more false positives you are likely to find (you can probably guess where I'm going with this).  The results of the GWAS over the 3 categories of depression were as follows:

"A total of 17 independent variants were genome-wide significant (P < 5 × 10−8) across the three depression phenotypes analysed (Table 1), of which 14 were associated with broad depression, two were associated with probable MDD and one was associated with ICD-coded MDD."

There are a few things I'd like to say about this.  The first is that, this seems entirely consistent in ratio to what you would expect from randomly generated false positives, with the larger datasets producing by far the most positive results.  This is actually probably confirmed by the fact that, when looking at non-significant, positive variants (P < 5 × 10−6 ), we continued to have a similar ratios. There were 3,690 variants with P < 10−6 for an association with broad depression, 189 for probable MDD, and 107 ICD MDD.  That is fairly consistent with the 14, 2 and 1 numbers for "significant" variants.  
So how many false positives might we actually expect in a study of this size?  Again, there is a fairly simple technique that I continue to advocate for, and I hope that at some point these studies will use.  That is is to divide up the datasets at random, in this case randomly assigning 113 K to one group, 30 K to another and 8 K to the third.  At that point, we can recheck the study for "significant" variants, which we would know are false positives and would have a basis for comparison.  My guess is that the false positives would come close to a 14 to 2 to 1 ratio and maybe tip us off that we are working with false positives.  (Addendum:  I have now quantified this suggestion with a suggestion for a specific test that I call The Pittelli Test).
Something else that is worth pointing out here is the fact that none of the 3 categories produced a significant variant that was also found in the other 2.  In fact, the p values of the other two categories when we had a significant variant in the other category, were mostly nowhere near significant.  
One could argue that "broad depression" might involve different variants, but this also applied to the probable MDD, with 2 variants that were not the same as the one found with ICD MDD.  If we were only finding valid associations, I would also expect the most well defined dataset to have the most positive correlations (I'm open to a counter argument here), and we in fact got the opposite scenario, with the most vaguely defined multi-causal diagnosis giving us the highest number of positive results by far.  All of this, in my view is a good indication that we have random false positives.
One of the more interesting parts of this study is the "replication" phase.  Again, I object to the use of the term replication being used especially intra-study, unless we are truly replicating findings.  I am not aware of a standard for replication of a GWAS, but I think a valid replication would involve picking a single significant variable and replicating it in a follow-up study (really, probably two).  
That said, the stated replication results are actually the most impressive and somewhat puzzling part of this study.  As I mentioned, I tried to get some clarification from the author, but was unsuccessful, so I will make my observations and I'm open to clarification and correction.
The authors compared their 17 variants to the results of a previously performed study of Major Depression via 23andMe.  Interestingly, they state that 16 of the 17 variants in their study, had correlates in the 23andMe study that had an "effect in the same direction."  I believe the point here is that, even though they weren't significant in the 23andMe study,  the ones that were positively correlated and negatively correlated remained consistent.  On it's face, this seems like an interesting finding, but certainly poses a question for me, since the "replication" study from 23andMe is for Major Depression and the current study only found 3 potential Major Depression variants, the rest being for broad depression, which were already not matched in the current study.  16/17 is hard to attribute to luck, admittedly, but it makes little sense to me that the results for the broad depression variants would be much more in line with the "replication" study for Major Depression than they were in their own study.  
In any case, the next part of the paper is a bit confusing and as I said, I was not able to get clarification, so I'll quote it here:

"There were 16 variants that had an effect in the same direction as the 23andMe analysis, with seven variants shown to be significant (P < 0.0029) in the 23andMe cohort after applying a Bonferroni correction (Table 1). All 17 variants remained significant (P < 5 × 10−8) in the meta-analysis. "

I am unsure here why someone would use a Bonferroni correction (I am honestly just learning about this) in this instance.  When the 23andMe study was conducted, these 7 variants were not found to be significant, so I don't know why there would be a reason to do a "correction."  They are what they are.  I'm assuming in the last sentence above, by "meta-analysis" they have combined their study with the 23andMe study and continued to have significant results.  Again, 17/17 is a good number, but you have to take into account that we are double dipping, since we are adding our positive results from the current study to results of the 23andMe study (I do not know the size of the 23andMe dataset), so this artificially bolsters the data.  It is also unclear whether this comparison was made only on the positive part of the results.  In other words, did they only add the positive results of the Broad Depression with the full dataset to arrive at their significant results?
The authors then note that 14 of the 17 variants they found were novel (seen for the first time).  I find it interesting that these GWAS studies continue to note "novel" findings and it doesn't raise any suspicion that perhaps we are getting random data.  Why were none of these 14 noted before in previous studies?
Of course, 14 out of 17 implies that there were 3 repeats, which still might be interesting, but let's take a look at what they say in this regard:

"Fourteen out of the 17 significant variants identified within our analysis of UK Biobank were novel, i.e. not reported within ±500 Kb of a significant variant (P < 5 × 10−8) reported by either Hyde et al., Okbay et al., or the Converge Consortium. The three variants (rs6699744, rs6424532 and rs1021363) that were unlikely to be novel were close to variants identified by Hyde et al."

 So I am certainly not convinced that these are the same 3 variants noted in the other studies.
To recap:  We have 17 variants, none of which were definitively found in previous depression studies, that appear to be consistent with newly created, random false positives.

I won't go into too much detail in the next part of the study, which attempts to correlate their Depression findings with other diagnoses, because I am of the belief that there are way too many factors to really draw such conclusions.  To give one example:

"Broad depression and probable MDD were significantly genetically correlated with bipolar disorder"

First ICD MDD would be the most likely to be correlated with Bipolar Disorder since that is a specific aspect of Bipolar Disorder, so that is the opposite of what one should expect.

Finally, the authors look at potential neuropathways related to Depression based on their findings.  I note the overall result here:

"The summary statistics from the three phenotypes were examined for enrichment in 209 different tissues using DEPICT21. None of the tissues showed significant enrichment across the three depression-related phenotypes, after applying the default false discovery rate correction (Supplementary Data 13). However, within broad depression the tissues that showed greatest enrichment were those involved in the brain and nervous system." Can I simplify this by saying that no significant results were found?

To conclude:
The study did not confirm previous depression studies, with the exception of a mathematical comparison which seems rather dubious on its face.  The findings within the study were not confirmatory within each of the 3 depression categories studied.  The attempts to correlate the results to other diagnoses seemed random and there were no significant, confirmed neuropathways consistent with the results of the study.