Editing Wiki language research (section)

=== 05-22-16 ===
DG: I spent some time looking at the data distributions and ran a bunch of models on the simple EN models overnight. The data for len_1 are reallllly long-tailed with very low frequencies -- this is causing the convergence issues. Below is a table of the simple model (len_1 ~ num_editors_1), run through a series of truncated data sets. The models will converge all the way up to removing the final data point out of the 4,077,819 data points we have. In other words, I was able to get convergence by dropping a single data point. Here's a quick table of the results from running the models: 

{|
! style="text-align:left;"| RESOLVES
! CUTOFF_LEN_1  
! PERCENT_DATA_COVERED  
! NUM_EDS1_COEFF  
! NUMEDS1_SE  
! ALPHA  
! FREQ_COUNTS_FOR_CUTOFF_BIN  
! NOTES
|-
|Yes
|50
|97.81%
|0.1392615
|0.0000551
|0.05119
|1,993	
|-
|Yes	
|500
|99.85%
|0.1028963
|0.0000536
|0.2065
|19	
|-
|Yes
|5000
|99.99%
|0.0913293
|0.0000536
|0.2743573
|1	
|-
|Yes
|10000
|99.99846%
|0.0903963
|0.0000536
|0.2806345
|1
|missing only 63 data points (4077756/4077819)
|-
|Yes
|50000
|99.999975%
|0.0896678
|0.0000536
|0.2858428
|1
|missing only 1 data point
|-
|No
|no cutoff
|100%
|.
|.
|.
|1
|cannot compute an improvement, discontinuous region encountered
|}

So, there are a few things we should take note of when looking at the data here: 
* len_1 ranges in this data set from: 2 - 133,529
* as the cutoff increases (i.e., more data included in the model) the coeff for editors decreases (to be expected since we have longer tail) and the SE stabilizes around .0000536. 
* alpha also increases (as expected)
* the last data point is len_1 = 133,529; but if you look at the tabs you get to 99.99% of the data at len_1 ~= 2500, and the last 5 len_1's are 28364, 30313, 34390, 44586, and then 133529. that's likely causing all kinds of havoc with numerous things (e.g., dispersion calculations, covariance and corr issues, etc.)