Difference between revisions of "F-Scores and Accuracy/zh-hans"

Revision as of 17:19, 19 October 2016 (view source) H.F. (Talk | contribs) (Created page with "在Eyewire，您将获得基于您的F分数的准确率评级。F分数是决定准确度占准确率和召回这两种统计方法。或者更简单地说，F分数是HQ根...") ← Older edit		Revision as of 16:08, 20 October 2016 (view source) H.F. (Talk | contribs) (Created page with "在计算最终的F分数时，要提前计算好玩家个人的准确率和召回。方块由玩家搭建完后，程序识别并输出4种判断的结果：正确的阳性...") Newer edit →
Line 3:		Line 3:
	[[File:F_score_calculation.png‎|center]]		[[File:F_score_calculation.png‎|center]]
−	Before we can calculate the final F-score first we must calculate your individual precision and recall. When a player does a cube there are four possible outcomes for every segment in that cube: a true positive result, a false positive result, a false negative result and a true negative result. A true positive (tp) result is when a player adds a segment that should be added. A false positive (fp) is when a player adds a segment that should not be added. A false negative (fn) is when a player misses a segment they should have added.  A true negative (tn) is when a player correctly leaves out a segment that does not belong. In the figure below you can see an example of false negative and of false positive.	+	在计算最终的F分数时，要提前计算好玩家个人的准确率和召回。方块由玩家搭建完后，程序识别并输出4种判断的结果：正确的阳性结果、错误的阳性结果、正确的阴性结果、错误的阴性结果。正确的阳性结果(tp)是你正确添加的小区块个数；错误的阳性结果(fp)是你错误添加了不该有的小区块；正确的阴性结果(tn)是你移除原来模型中错误的小区块；错误的阴性结果(fn)是你遗漏了需要添加的小区块。你可以在下图看到错误的阳性与错误的阴性的区别。

---

Revision as of 16:08, 20 October 2016

在Eyewire，您将获得基于您的F分数的准确率评级。F分数是决定准确度占准确率和召回这两种统计方法。或者更简单地说，F分数是HQ根据添加多的、缺失的模块来决定你的准确率。传统F分数的公式为：

在计算最终的F分数时，要提前计算好玩家个人的准确率和召回。方块由玩家搭建完后，程序识别并输出4种判断的结果：正确的阳性结果、错误的阳性结果、正确的阴性结果、错误的阴性结果。正确的阳性结果(tp)是你正确添加的小区块个数；错误的阳性结果(fp)是你错误添加了不该有的小区块；正确的阴性结果(tn)是你移除原来模型中错误的小区块；错误的阴性结果(fn)是你遗漏了需要添加的小区块。你可以在下图看到错误的阳性与错误的阴性的区别。

To the left is an example of a branch submitted by a player. In this example the red and the green segments are what the player submitted, while the purple segment was left out.

The red segment here is a false positive and the purple segment is a false negative. The player mistakenly added the red segment when they should have added the purple segment instead. The green segment is correct.

This brings us to precision; precision is how much of a volume was added correctly. For example if Player A has a precision 0.9221 that means about 92% of what Player A added was correct and about 8% of what Player A added should not have been added. To determine a player’s precision we use their true positive (tp) results, correctly added, and their false positive (fp) results, incorrectly added, in this formula:

Recall measures how much of the volume was missed. Let’s say Player A has a recall of 0.9409. That means that Player A missed about 6% of the correct segments in the cubes Player A worked on. To determine a player’s recall we use their true positive (tp) results, correctly added, and false negative (fn) results, incorrectly missed, in this formula:

Now we would take the results from both of those formulas and plug them into the formula above to get a player’s F-score. Another way to look at it is we take the harmonic mean of a player’s precision and recall to get their overall accuracy rating.

How Accurate are F-Scores?

One question we a get a lot is how do we know what is correct and what isn’t? What is correct is determined by combining the GrimReaper’s corrections with the EyeWirer consensus. If a cube does not have a GrimReaper correction we just use the EyeWirer consensus. EyeWire consensuses have proven to be quite accurate. However, there is still a small chance that a consensus may contain a wrong piece. This means that F-scores cannot prove user accuracy 100% of the time. However, they are accurate enough that we feel confident using them as a player guide.