Depression and Bipolar: Looking at the Positive While Inadvertantly Demonstrating the Negative

I wanted to critique this study which I admittedly struggled to get my head around, so I needed to get help from one of the authors on Twitter. In short, it takes data from two previous studies of depression and bipolar disorder and recombines them. Here is his given explanation:

The combination of the MDD and the bipolar data (which have not been combined in this way before). That is, we are seeing some loci have statistical evidence for "MDD or bipolar" versus control individuals that we haven't seen when looking at either individually so far.

I'm not really sure if that is what they established even on its face since, as I understand it, they simply combine the data from the two studies and perform new GWAS's for both Bipolar Disorder and Depression, creating a new case vs. control for both (I welcome the authors giving a better explanation than I'm putting forth, lest I be accused of creating a straw man. I really just don't fully understand the underlying premise). In doing so, they came up with 15 new loci related to these disorders without using any new data. I believe the point here is to show that bipolar disorder and depression have some genetic commonalities that were demonstrated. They go on to assess these further, but I suggest maybe the lede was buried here and that the study demonstrated another, perhaps more plausible, conclusion: That the original significant loci were false positives, as are these. Let me explain below the fold:

It is true that they had 15 more loci that reached significance in the combined study, that did not reach significance in the original studies. However, I believe the study glosses over the fact that there were also 20 loci that were significant in the previous GWAS's, but did NOT reach significance in this study. Here is the breakdown:

We meta-analysed the PGC MDD, PGC BD and the UKB MDD cohorts  (MOOD, cases = 185,285, controls = 439,741, non-overlapping N = 609,424). 73 loci reached genome-wide significance, of which 55 were also seen in the meta-analysis  of PGC MDD and UKB MDD ... 39 of the 44  PGC MDD loci reached genome-wide significance in MOOD ... In comparison, only four of the 19 PGC BD loci  reached genome-wide significance in MOOD. 

 Anyone who reads my blog (if you're out there), knows I am going to jump on that last sentence regarding bipolar disorder (BD), but let's first take a look at the Major Depressive Disorder (MDD) results. On the surface, 39 out of 44 might not seem like a bad result when adding new data to the previous study, but in this case the old dataset dwarfs the added dataset from the BD study (MDD: 135,458  cases, 344,901 controls vs. BD:  20,352 cases, 31,358 controls). So you've taken a study with an N of 500,000 with 44 genome-wide significanct loci and added a study with 50,000, or one tenth that, and 5 of your loci bleed significance, while adding some new loci that didn't reach significance, previously.
Is that what you would expect when adding data to your dataset if your loci were legitimately related to the trait in question? When you have a headstart like that, the added data don't even need to reach full significance in order to maintain statistical significance. They just need to trend a bit in that direction. So, if you assume, as I do, that the initial 44 loci from the previous study are simply false positives (probably bolstered by some pop/strat), then I think adding a small new dataset is analogous to adding more coin flips to a skewed heads/tails result and getting your heads/tails ratio back towards 50%, which I think is what we are seeing here.
If you still aren't convinced, then let's take a look at it from the other side with Bipolar Disorder (BD). If you have a smaller dataset that has 19 significant loci and you add a much larger dataset to that, and you again assume that the 19 significant loci are false positives (which, full disclosure, is invariably my assumption), then you should expect that most of them will not be able to maintain significance. This is exactly what has happened here.
So they took two datasets, combined them, redid GWAS's for the two traits and had more loci lose significance than gain significance. I think it's fair to say that this points to false positives until proven otherwise. In effect, they have demonstrated that shuffling databases around gives you different "significant" loci, even when using the same data.
There is more to the study, particularly involving comparing the genetics of Bipolar Disorder, Major Depression, etc. with other disorders and making a case for a genetic correlation for the two disorders. I might critique that at a later date but, as they say, cut off the head and the tail dies.
That said, I think it's worth pointing out some things from the perspective a clinical psychiatrist who has plenty of experience with these disorders: The DSM (whether III, IV or V), which is used for genetic studies of mental disorders, while having a veneer of being "scientific," is not really as useful for scientific studies as one might think. It was created for commercial (and perhaps inter-political) purposes. If, for example, I see a study that finds genetic correlations between Bipolar I disorder and Bipolar II disorder, I know the correlations are bogus even if you accept the premise that these are genetically driven disorders. They are entirely different animals. Any clinician worth his salt, who has dealt with a Bipolar I disordered patients in an acute manic phase, where the person has no need for sleep, believes, say, that he is Jesus Christ, hears the voice of God, and is convinced he is a billionaire, knows that is a far cry from someone who "has a lot of mood swings," which really is good enough to get a Bipolar II diagnosis, which was largely invented (in my view) to justify giving patients with personality disorders (Borderline, Antisocial) or substance abuse problems that wouldn't normally be covered by insurance, a meatier diagnosis. The same patients will often get a diagnosis of Major Depression, probably dependent on who their psychiatrist is.  Moreover, many individuals will give "Bipolar" as their diagnosis, without designating I or II. In short, you are going to find a lot of crossover between these diagnoses, because they are the diagnoses most commonly given to the same people. They will corrupt any attempt to study the diagnosis in a pure sense, if such even exists.

Addendum: In my discussion with an author of this study (Jonathan Coleman), I suggested that he repeat this experimental design for many different behavioral traits (one with a large N and one with a smaller N). I have little doubt that the result will be the same. The small N significant loci will largely disappear and the large N will bleed a few. This would show that we are simply dealing with statistical aberrations and that this entire field is one big shell game. Doubt he will take me up on it, but if there are any young behavioral geneticists looking to make a mark in the field (by effectively toppling it), this is your chance to make history.