[jhenrikj]

Never actually heard anywhere where a LA is 1/6. Normally it takes more than 1/5 to be considered an LA. So if you ask 5, 1 must choose it and at least 2 more or even perhaps 3 must consider (or 2 choose it) it to become a LA.

If the ruling is close (say you have 1 choosing it and 2 more considering it out of 5), you simply poll 1 or 2 more.


I think you got the 1/6 from the old 12C1© where the offending side would get the most unfavorable possible and the offending side the most favorable probable score. Worst possible was 1/6 best probable was 1/3.

I'm absolutely sure that the last option with 5 TD's deciding will by far get the most incorrect rulings.

[gordontd]

Trinidad, on 2020-February-27, 15:17, said:

we do not want to know what a player would do. We want to know whether pass is an LA.

Whether or not pass is an LA is defined in the laws as being dependant on what (like) players would do.

[Trinidad]

gordontd, on 2020-February-28, 01:30, said:

And yet the widespread perception is that rulings in general have become better since polling became mainstream, which is why appeals have become less frequent.

Yes, appeals have become less frequent. Yes, this is caused by polling. Yes, perhaps even the TD rulings have improved due to polling. (We got rid of TDs ruling against the alleged offending side, counting on an AC to rectify it.) But no, the overall process has gotten worse due to polling.

Why? ACs can only check whether proper procedure was followed. No matter how ridiculous the poll result is, it stands. My previous analysis shows that poll results are way too close to coin flips. There are 2 reasons for this:
The criterion for an LA (1 in 6) makes it like looking for a needle in a haystack. You need to look many times before you can say something reliable about low probability events. (see P.S. below)
The number of available pollees is already small if you are searching for a 1 in 2 criterion, sinc ethere simply are not that many players.

gordontd, on 2020-February-28, 01:30, said:

I find it strange that you present lots of figures to support your argument until you come to the conclusion where you say "it is hard to quantify this, but I am convinced..."

This is obviously not what I meant in my post. I assume that I wasn't clear enough. "This" referred to the fact that it is hard to quantify the accuracy of a decision by a group of people reaching a conclusion by exchanging ideas, opinions and weighting arguments. (In contrast to quantifying "that": the accuracy of a poll that can be calculated to be close to that of a coin toss.)

Rik

P.S.

Don't worry too much about this. Just last week I heard a story at work about a manufacturer who had a process where 0.2 % of the products failed. They decided to test a new production method during the weekend to see if they could improve on this. Between making the changes to the equipment and before reverting the changes, so they could produce the old way on Monday morning, they managed to make 30 pieces of this product. They all passed the quality test, which does not give an answer to the question whether the new method is better than the old one: At a failure rate of 0.2 %, the probability to find 0 failures in a set of 30 is 94 %. So, they had been working an entire weekend, without being able to draw a conclusion. When the consultant asked the engineer why he had done this experiment, he told: "I had told the boss that it was useless, but he told me to do it anyway. But maybe he will listen to you."

So, bridge bosses are not the only ones who do not understand the problems that come with sampling.

[Trinidad]

gordontd, on 2020-February-28, 06:20, said:

Whether or not pass is an LA is defined in the laws as being dependant on what (like) players would do.

Yes, and why couldn't you ask players what players would do (multiple answers possible) rather than ask these players what they would do (1 answer possible)?

Rik

[Trinidad]

jhenrikj, on 2020-February-28, 01:42, said:

I'm absolutely sure that the last option with 5 TD's deciding will by far get the most incorrect rulings.

Oh, I agree on that, but that was not my last option.

My last option was 5 experienced bridge players deciding. The big difference with a poll of 5 people is that these people are now not simply answering a question and we are counting, but they will need to exchange ideas, convince and weight the arguments. This means that what started as a minority view may be able to correct a wrong majority view when there are better arguments for the minority view.

Rik

[gordontd]

Trinidad, on 2020-February-28, 06:51, said:

Yes, and why couldn't you ask players what players would do (multiple answers possible) rather than ask these players what they would do (1 answer possible)?

Rik

Why do you expect them to know what other people would do? Asking them what they would do themselves at least has an expectation that they would know the answer.

[Zelandakh]

gordontd, on 2020-February-28, 06:20, said:

Whether or not pass is an LA is defined in the laws as being dependant on what (like) players would do.

One of the points Rik was making in his post is that the sampling does not necessarily help us to answer this question in a meaningful way as the sample size is simply too small. Typically, a TD will sample a certain number of players and then look to see whether x percent of them choose the call and/or y percent of them considered it. But TDs are not statisticians and that is not actually the right question. The questions to ask are, given that we got a sample result of (b,c), what is the probability that the underlying probability of the call being chosen, p >= x and what is the underlying probability of the underlying probability of the call being chosen, q >= y?

If the sample size is too small to answer these questions then the sampling might not be giving us useful information. That seems to me to be a fairly fundamental issue to address and might be incredibly useful in providing TDs with guidelines for collecting an adequate sample size for a given level of mathematical confidence. Surely governing bodies and their TDs would welcome that rather than seeing it as a challenge to their authority in some way?

[barmar]

I'm not really an expert in statistics, but I think you should consider it in a Bayesian way. The TD likely already has a preconception about what the LAs are, and he's using the poll to support or refute it. The player's actual action and explanation of why they took it is also an existing data point.

Does that change the equations?

[gordontd]

Zelandakh, on 2020-February-28, 08:49, said:

If the sample size is too small to answer these questions then the sampling might not be giving us useful information. That seems to me to be a fairly fundamental issue to address and might be incredibly useful in providing TDs with guidelines for collecting an adequate sample size for a given level of mathematical confidence. Surely governing bodies and their TDs would welcome that rather than seeing it as a challenge to their authority in some way?

Certainly it is helpful to increase the sample size if possible and one way of doing this that I use is to poll on Bridgewinners. But I don't think his solution was to increase the poll size, it was to revert to the subjective method of asking strong players what they thought.

[pescetom]

gordontd, on 2020-February-28, 10:05, said:

Certainly it is helpful to increase the sample size if possible and one way of doing this that I use is to poll on Bridgewinners.

I too have sometimes used discussion forum polls to increase the sample size. The main difficult of that is that Bridgewinners offers no discretion whatsoever and this forum generates a sample size no larger than what I achieve at the club (especially if some people comment but fail to actually vote, hint Zel ). Also there is little real possibility to limit responses to peer level. But it's still useful and interesting.

[gordontd]

pescetom, on 2020-February-28, 10:54, said:

I too have sometimes used discussion forum polls to increase the sample size. The main difficult of that is that Bridgewinners offers no discretion whatsoever and this forum generates a sample size no larger than what I achieve at the club (especially if some people comment but fail to actually vote, hint Zel ). Also there is little real possibility to limit responses to peer level. But it's still useful and interesting.

This is true, though the results are usually sufficiently clear, and I have a sense of the standard of a large number of the respondents, that it is quite helpful. Aside from the large numbers involved, it is also helpful that almost none of them will know anything about the hand, whereas polling players in an event, they will usually know the hand.

[Trinidad]

gordontd, on 2020-February-28, 10:05, said:

Certainly it is helpful to increase the sample size if possible and one way of doing this that I use is to poll on Bridgewinners. But I don't think his solution was to increase the poll size, it was to revert to the subjective method of asking strong players what they thought.

Indeed it was.

But this has a reason: The needed sample size is reduced significantly if you simply ask the question that comes with a 1:1 criterion. Or, the other way around, with the same sample size, a poll with a 1:1 criterion will be much more accurate than a poll with a 1:5 criterion.

Rik

[Trinidad]

barmar, on 2020-February-28, 09:22, said:

I'm not really an expert in statistics, but I think you should consider it in a Bayesian way. The TD likely already has a preconception about what the LAs are, and he's using the poll to support or refute it. The player's actual action and explanation of why they took it is also an existing data point.

Does that change the equations?

Yes. And this is exactly what I mean with the last solution: An interactive discussion with different inputs will be more accurate than n individual opinions, that are simply tallied.

If the TD discusses the case with a few players this will be more accurate than asking some of the available players individually what they would call and some others what the UI suggests.
And, of course, the same holds for the AC.

Rik

[blackshoe]

jhenrikj, on 2020-February-28, 01:42, said:

Never actually heard anywhere where a LA is 1/6. Normally it takes more than 1/5 to be considered an LA. So if you ask 5, 1 must choose it and at least 2 more or even perhaps 3 must consider (or 2 choose it) it to become a LA.

If the ruling is close (say you have 1 choosing it and 2 more considering it out of 5), you simply poll 1 or 2 more.


I think you got the 1/6 from the old 12C1© where the offending side would get the most unfavorable possible and the offending side the most favorable probable score. Worst possible was 1/6 best probable was 1/3.

I'm absolutely sure that the last option with 5 TD's deciding will by far get the most incorrect rulings.

Throwing numbers around where they aren't really applicable can lead to silly conclusions. Yet people seem to think putting a number on something has some value. I'm not so sure. Granted, "a significant proportion of the class of players in question" is a bit vague, but putting a number on it, whether it be 1 in 5, 1 in 6, or anything else, isn't a panacea. The theory behind polling is that if you poll a (very) small proportion of the set called "class of players in question" you can conclude that something is an LA if one player chooses it. When you look at it that way, the conclusion that the theory is nonsense seems obvious.

[sanst]

Rik argued correctly that polling doesn’t necessarily give a reliable or useful result. The assumption is, that the players polled

• Are of comparable strength as the players involved
  
• Have no prior knowledge of the hands and the results,
  
• Understand the methods and agreements used.

It can be quite hard to find those who meet these criteria. Ever tried to organize a poll when weak players were involved? Or players between who there was a considerable difference in strength?
Is there a better way to come to a decision? I rather doubt it, but prefer this situation to the one that we had before.

[pescetom]

sanst, on 2020-February-29, 02:48, said:

It can be quite hard to find those who meet these criteria. Ever tried to organize a poll when weak players were involved? Or players between who there was a considerable difference in strength?
Is there a better way to come to a decision?

Finding other weak players is simple enough, the problem is getting meaningful answers out of them.
For difference in strength, I was taught to consider the stronger player.
One means of improvement would be online polling, involving more players with documented choices and demonstrable results.

[Zelandakh]

sanst, on 2020-February-29, 02:48, said:

Is there a better way to come to a decision?

The answer to this as well as many other issues in bridge, such as many forms of cheating, is separating players into different rooms, thus allowing for self-alerting. Of course this has its drawbacks too, both in terms of the requirement for (at least) 4 different rooms and because many (most?) players enjoy the social aspect at least as much as the game, which would become lost in this model.

Perhaps the real solution is to accept these shortcomings and issues in club bridge but to invest in a better alternative for tournaments where money is on the line. But we are talking bridge here, so best not to expect any improvements for at least 20 years, if not longer.

[jhenrikj]

Trinidad, on 2020-February-28, 06:58, said:

Oh, I agree on that, but that was not my last option.

My last option was 5 experienced bridge players deciding. The big difference with a poll of 5 people is that these people are now not simply answering a question and we are counting, but they will need to exchange ideas, convince and weight the arguments. This means that what started as a minority view may be able to correct a wrong majority view when there are better arguments for the minority view.

Rik

But that is exactly the opposite of what we want. That's why it's so important to poll players separately so that they have no chance whatsoever to influence each other.

[Trinidad]

jhenrikj, on 2020-February-29, 06:48, said:

But that is exactly the opposite of what we want. That's why it's so important to poll players separately so that they have no chance whatsoever to influence each other.

Yes, and that is exactly what is the wrong starting point. If you keep them separate, you will not get enough good votes in the poll. If you let them work together, the probability that they will get to the right answer together will be much higher.

It is not as if I am saying something that is revolutionary: Suppose you have 5 people available to solve a complex problem (Doesn't matter what kind of problem: financial, engineering, climate, you name it). Do you know any manager who would let these 5 work separately to gather votes at the end of the process? Of course, you don't. You let them get ideas, discuss them and weight the arguments. This will lead to a much better solution then 5 separate views.

This doesnot mean that polling is useless.
Suppose that a TD decides that the UI suggested the action that was taken, but that there is no LA, so no foul. Before he finalizes his decision, he checks with 4 players. It turns out that they all would have chosen an alternative action that was not suggested by the UI. Oops! The poll just corrected the TD. There is an LA.

But in the reverse case, it doesn't work: Now, the TD decides that the UI suggested the action that was taken and that there was an LA, so guilty. He, again, checks with 4 players. Now, none of them would chose the alternative action. The TD cannot decide that there was no LA, since the sample group is too small to establish this. The reason is that the TD is now looking for the needle in the haystack. In the previous case, he was looking for the haystack around a needle.

You can not reliably determine whether there is a needle in a haystack or not by sampling some parts of the haystack. You can determine whether there is hay in the haystack by sampling some parts of it.

Rik

[blackshoe]

I see an assumption there that the fact that some number of people chose a particular action makes that action logical. I don't think that assumption is necessarily valid.