A Quick Review of a Dyslexia GWAS study claiming two significant loci

I will try to trim this review down to its barebones for an easier read, but welcome any comments or clarifications.  The study in question can be found here.

Genome Wide Association Scan identifies new variants associated with a cognitive predictor of dyslexia 

The study is on a much smaller scale, with only a few thousand cases, so one might expect fewer false positives.  In this case, likely we have two.

The study starts off talking about many of the previous associations found in dyslexia studies, but then notes:

Although many of the genes suggested by these GWAS studies showed interesting potential biological links to DD and underlying skills, most of these associations did not reach genome-wide significance and were not replicated in independent datasets

You might note that this is a common refrain in GWAS studies.  They try to explain this as follows:

This might have different reasons, including the low statistical power of these studies implied by the relatively small sample sizes, and the heterogeneity of recruitment criteria and phenotypic assessment of the cohorts involved. In addition, the candidate susceptibility genes identified and replicated so far explain only a minor part of the genetic variance underlying dyslexia and the related cognitive traits, and a big proportion of this heritability remains unexplained.

I'm thinking they forgot one other possibility: that these associations were nothing more than false positives and failed to replicate, because they aren't related to dyslexia.  It's odd that this possibility is not even considered, especially when (spoiler alert), they did not replicate any of them in this study.  At what point can you discard positive findings when they are repeatedly not replicated?

Moving on to their study, they combined 9 different European cohorts with N = 3,468  (it's not clear to me how many cases vs. controls they are referring to here).  They used 8 different tests: (WRead), nonword reading (NWRead), and word spelling (WSpell), and on five cognitive measures underlying reading ability and dyslexia, namely phoneme awareness (PA), digit span (DigSpan, a measure of verbal short-term memory), and rapid automatized naming of letters (RANlet), digits (RANdig), and pictures (RANpic). 

To what extent these are related and actual measures of dyslexia, or different kinds of dyslexia is not explored in any depth.  I think this creates a bit of a question as to what we are really looking at here.  Do we have 3,468 kids with dyslexia or are these 8 different things?  It's also worth noting that when you compare your dataset to 8 different tests, separately, you increase your likelihood to find a false positive by a factor of 8, as you get 8 different tries.

In any case, they came up with two significant loci for one of the tests (RANlet).  My first question, of course, is what would we see at random?  Again, we have no randomized control, and two false positives doesn't seem out of the realm of likelihood for a smaller sample of this nature.  I will encourage the authors to run the data again using a randomized control, as I have suggested and can be found here.

They then attempted to locate genes within these loci that might be related to dyslexia.  They did a lot of analysis and I found it to be a bit of a reach, but the authors can feel free to comment further on it.  I will comment on this specific paragraph, however:

Despite the biological appeal of the top association signals mentioned above, an imaging genetic follow up of these SNPs on variation in seven different subcortical volumes previously analysed in a large independent GWAS34 did not reveal any significant association. Considering the sample size of the neuroimaging genetic analysis (N~13,000), we deem it unlikely that this lack of support is caused by a lack of power of the analysis. However, this negative result does not rule out genetic effects of the RANlet-associated variants on other brain structures involved in reading networks, such as the inferior frontal gyrus and the temporal and parietal gyri. These potential associations should be tested in the future, as was previously done for other variants associated with reading-related traits

So, they couldn't really correlate their findings to the alleged subcortical volume correlation to dyslexia.  They give their explanation above.  I will offer another:  Not only were the previous studies implicating subcortical volume random false positives, but the two loci you found were also false positives.  The suggestion to look at other parts of the brain to correlate future false positives is part of an endless cycle of finding false positives, noticing any correlation to a particular part of the brain and surmising that this has something to do with the mechanism of the the trait in question, then doing another study that finds entirely different loci acting on entirely different parts of the brain.  To their credit, they did not try to take it beyond the subcortical that had been "observed" in previous studies.

To conclude:
1.  The study did not replicate any previous studies.
2. The study found 2 new loci correlations, and we cannot say whether that is beyond the expected number of false positives.
3. The loci they found did not have SNP's that correlated to previous mechanisms noted for dyslexia.
4. It is, in my opinion, highly likely that this, and all the dyslexia GWAS studies that preceded it have found nothing but false positives.