[BebopKid]

When using programs like BigDeal I expected the hands to follow the probabilities published for hand and suit probabilities.

See Hand pattern probabilities as an example.

I used BigDealX to generate 65000 deals, which is of course 260000 NS suits to look at.
65000 * 4

Of those suits, 2422 were 6322 in some combination of NS 63 EW 22.

That is less than 0.931%.

From the Hand Pattern Probabilities, the number should be 5.64% for totally random deals.

Every suit combination and hand pattern total has significant error from the 65000 deal sample.

Could someone who understands this please explain it to me?

I have received much flack for thinking that random deals should tend toward the hand pattern probabilities with large samples. If they don't, then how do I make sense of the probabilities. Are the probabilities useless?

Before using other people's generators, I wrote my own with a random shuffle algorithm that mimics a human card shuffle. Any time I generate 65000 deals with my generator, it conforms within close proximity to the probabilities.

Because of that, I have been told that my generator is not random.

I am completely confused at this point.

Please only respond if you want to help me, not insult me. I have had enough of that.

[awm]

I'm not totally sure what you're trying to measure here. The hand pattern probabilities say that the odds I have 6-3-2-2 distribution (in any order) are about 5.64%. Of course there are 12 possible 6-3-2-2 hands so any particular such pattern will be about 0.47%.

What it seems like you're trying to do is look for the probability that the cards are 6-3-2-2 in a particular suit around the table. I think this is the same odds, but you're restricting to N/S have 6 and 3 and E/W have 2 and 2, which means you're only considering two of the possible twelve ways to allocate those cards 6-3-2-2 around the table. So you'd expect a probability of 0.94%, pretty close to the 0.931% you observed.

Of course when you try very large number of samples the probabilities should be close to the expected ones. There are some deal generators out there which are buggy as well...

[Trumpace]

BebopKid, on Sep 8 2007, 12:37 AM, said:

Before using other people's generators, I wrote my own with a random shuffle algorithm that mimics a human card shuffle. Any time I generate 65000 deals with my generator, it conforms within close proximity to the probabilities.

Because of that, I have been told that my generator is not random.

I did look at the code for your shuffler you posted and there seemed to be two problems:

i) You are using Rnd, which is not reliable.
ii) You are trying to mimic hand-shuffling by clumping no more than 4 cards and you do this only 4 times.

That is the main reason for the claim that your hand shuffler is not uniformly random and that claim is not just because of the values you were getting.

In fact, if you are still reading the other thread, I posted a simple test for your generator which you can try out and let us know if it matches your expectations. (I would have run it myself, but I don't have excel).

Sorry if any of my posts came out as insulting. I didn't intend that in any way.

[hrothgar]

Cleve Moler - one of the principle authors of MATLAB - wrote a textbook on numeric computing that he placed on the MathWorks web site. Chapter 13 of this text deals with the topic of random number generators.

http://www.mathworks...oler/random.pdf

The chapter is quite short and very readable. While the examples focus on pseudo-random number generation and approximating different distributions, the basic principles can easily be extended to hand generators.

(BTW: I reviewed the thread in question. As Frances notes, Excel, manual shuffling and rnd are (probably) fine for what you are doing. The error in your implementation was a basic mistake in the sorting structure. I suspect that the reference that I provide might be overkill. However, I found it an interesting read)

[FrancesHinden]

See the other thread for where I think you were going wrong on that calculation.

By the way, while TrumpAce is right in theory about the problems with your code, I think in practice your code should work OK, in that the results you get from simulations are likely to be good enough. (although there are various ways you could speed it up and I also don't really see the point of mimicking hand shuffling).

In yet another thread you calculated the % chance of various HCP combinations in exactly the right (tedious) way. The biggest thing to be careful of is trying to do any sort of short cut in the calculations. I know to my own pain that these usually go wrong.

I hope awm's explanation has helped.
Another way to understand the same point is to look at your simulation again.

Look at the 65000 North hands you have dealt.
Count how many of them are 6=3=2=2 with 6 spades, 3 hearts, 2 clubs, 2 diamonds
Count how many of them are 3=6=2=2 (3 spades, 6 hearts, 2 clubs, 2 diamonds)

Do this for the 12 different 6322 distributions that are possible (6322, 6232, 6223, 3622, 2632, 2623, 3262, 2362, 2263, 3226, 2326, 2236)

You should find that each of those individual distributions is approximately 0.47%, but the overall percentage of the time that North has a 6322 in order is indeed 5.64%.

[jtfanclub]

BebopKid, on Sep 8 2007, 12:37 AM, said:

Of those suits, 2422 were 6322 in some combination of NS 63 EW 22.

That is less than 0.931%.

From the Hand Pattern Probabilities, the number should be 5.64% for totally random deals.

Possible 6322 combinations:

S: 6 E: 3 N: 2 W: 2
S: 6 E: 2 N: 3 W: 2 <---
S: 6 E: 2 N: 2 W: 3

E: 6 S: 3 N: 2 W: 2
E: 6 S: 2 N: 3 W: 2
E: 6 S: 2 N: 2 W: 3

N: 6 E: 3 S: 2 W: 2
N: 6 E: 2 S: 3 W: 2 <---
N: 6 E: 2 S: 2 W: 3

W: 6 S: 3 N: 2 E: 2
W: 6 S: 2 N: 3 E: 2
W: 6 S: 3 N: 2 E: 3

So, there are 12 possible 6322s. You counted two of them (the only two where NS has 63).

So you would expect there to be 5.64% * 2/12=.94%

You actually got .93%.

I don't know if the error was due to rounding of 5.64% or sample size, but I'd say being off by .01% is pretty good.

[Trumpace]

FrancesHinden, on Sep 8 2007, 06:44 AM, said:

I think in practice your code should work OK, in that the results you get from simulations are likely to be good enough. (although there are various ways you could speed it up and I also don't really see the point of mimicking hand shuffling).

Actually if Rnd is not so good, trying to mimic hand shuffling (say imperfect riffle shuffle seven times) might turn out to be a better approach than just randomizing 52 slots.

[jtfanclub]

Another way to think of it. Start with 5.64% for ALL 6322s.

EW will have the 6 card suit as often as NS will, right? So only 1/2 the time will you have NS with the 6 card suit. That makes 2.82%.

So, N/S has the 6, but you're also requiring them to have the 3, right? Well, if South has the 6, then 1/3 of the time N will have the 3, yes? After all, the 3 card suit could be with West, North, or East, and each is equally likely. And the same is true if North has the 6, 1/3 of the time South with have the 3.

So, 2.82% /3 = .94%. Which is what you got.

You don't have to use combanitorial probabilities for this. I find it easier (especially for more complex problems) but for each their own.

[mycroft]

I am a strong believer, given that you need a good (p)RNG anyway - if it is Pseudo, a statistically good one if you're doing statistics, a cryptographically good one if you're creating hand records (there are PRNGs that are statistically random but not cryptographically random, and given enough state, can be forecasted - and given that there are sponsors who will pay $1m/year for players, the chance that one of them is unethical enough to pay $.5m to break the predictor is non-ignorable) - since the incremental cost of random bits is negligeable, never do any shuffling. Grab 36x92 (IIRC) bits of data (you could get 97 bits and div and mod em down 36 times) and "look up" the hand in a mapping (see Thomas Andrews' page for an example).

All this "shuffling" business is attempting to be random. Real random is better.
Michael.

[jdeegan]

I don't know the program you are using, but I do know that Richard Pavlicek has a proper deal generator on his website which I use and which you can download for free. Also, you do need to check to make sure that the simulations you performed are constructed properly in that they do generate the same situation that your theoretical tables apply to.

That said, the main thing to remember when doing any kind of deal generating simulation is there is a right way and a wrong way.

The wrong way is to fill in the known cards in the four handed deal, and then deal the remaining cards randomly to fill in the empty slots. This only works properly if you specify one complete hand and then randomly deal the other three hands. Otherwise, the complete four hands you generate will bear little overall resemblance to what you can expect in the real world.

The right way is the deal four random hands (or three complete random hands) and then to examine each one to see if it meets the criteria you have specified (e.g. North is 6-2-2-3 with 9 to 11 HCP and two ♠ honors). If not, it is discarded, and you move on to the next deal. You may have to generate a million random deals to get 100 examples of the hands you are interested in, but those 100 hands will properly represent what you can expect in the real world. The computer makes this possible.

The big gap between theory and your results may well be because your program does things the wrong way.

P.S. By the by, if your deal generator is doing things the right way, the more hands you simulate, the closer your results should be to the theoretical answer

[BebopKid]

Sorry, I've been off line for a while. Plus I was sick, and apparently not using the same algorithm for checking the BigDeal numbers that I used for my own numbers where I did check all the hands/suits. Teaches me not to rush

Here are the results I got from the BigDeal numbers by looking through all the dealt hands for 5-4 distribution and 6-3 distribution.

9 card-fit with 5-4 had this distribution for the other hands:
3-1 split 52.2%
2-2 split 42.6%
4-0 split 5.1%
I used 64452 hands for this analysis.

9 card-fit with 6-3 had this distribution for the other hands:
3-1 split 33.3%
2-2 split 54.3%
4-0 split 12.4%
I used 26854 hands for this analysis.

This is tending to back up what I learned from my teachers, that 3-1 split occurs less often than 2-2 split with a 6-3 declarer combination.

I am going to look for Richard Pavlicek's deal generator as suggested by jdeegan. I will post the results here when I'm done.

Also I have learned quite a bit through research this week. I have implemented a Mersenne Twister for random number in my own deal generator. I am planning to write a very usable graphical interface with lots of export options and post it on my web site for people to use. But I'm mostly doing it for the fun of it. (What a geek!) It will be a while before it's ready as it is a pretty big endeavor.

I appreciate all the comments.

[BebopKid]

<double post>

[jtfanclub]

BebopKid, on Sep 12 2007, 03:27 PM, said:

I appreciate all the comments.

READ

MY

POSTS

For God's sake, an 8 year old can understand what you're doing wrong here. It's a simple math error, not hard at all to understand, and quite understandable, but your head-in-the-sand mentality is just too frustrating for words.

I swear, if it's too complicated for you, I'll simplify it. How about this:

If South has 6 cards in a suit, that North will have 3 cards in the suit 2/3 of the time when the suit breaks 6331 vs. 1/3 of the time when it breaks 6322. Two three card suits vs. one three card suit.

[hrothgar]

BebopKid, on Sep 12 2007, 11:27 PM, said:

This is tending to back up what I learned from my teachers, that 3-1 split occurs less often than 2-2 split with a 6-3 declarer combination.

Couple quick comments

1. Here, once again, its very important to remember that there are two possible 3-1 splits to worry about and only one 2-2 split. A 3-1 split is MORE common than a 2-2 split. However, either one of the two specific 3-1 splits is less common.

2. The 6-3 declarer combination is irrelevant. The odds that the side suit splits 2-2 v 3-1 v 4-0 are the same regardless of whether declarer has six opposite three, five opposite four, or seven opposite two.

[jtfanclub]

hrothgar, on Sep 12 2007, 03:48 PM, said:

2. The 6-3 declarer combination is irrelevant. The odds that the side suit splits 2-2 v 3-1 v 4-0 are the same regardless of whether declarer has six opposite three, five opposite four, or seven opposite two.

You don't get it. This is religion, not math.

The fact that several people have shown mathematically that they're exactly the same thing hasn't phased him.

The fact that I've shown exactly what he's doing wrong and why it gives him the bad results doesn't even slow him down. He even asked the right question, got exactly the right answer, and yet he STILL makes the same mistake over and over and over and over.

Trust me, simple declarative statements are pointless. I've already tried a 2X4.

[Stephen Tu]

Quote

9 card-fit with 6-3 had this distribution for the other hands:
3-1 split 33.3%
2-2 split 54.3%
4-0 split 12.4%
I used 26854 hands for this analysis.

Whatever method you are using for counting, you must be doing something wrong. Are you using some sort of program to count? Let us see the code. It is unbelievable to get these kind of numbers. jtfanclub is probably right that your algorithm is double counting something.

Quote

This is tending to back up what I learned from my teachers, that 3-1 split occurs less often than 2-2 split with a 6-3 declarer combination.

Any teacher who tells you that the odds of the opponent's split depends on how the cards split between you & dummy should be fired. That assertion is easy to prove mathematically false. Odds of suit splits can be affected only by information about the opponents hands, e.g. if one opponent or another is known to have length or shortness somewhere. How cards are distributed between you & dummy is irrelevant! North & south can trade cards all day long, there are always the exact same C(26,13) ways to deal the E-W hands, for each particular combination of N-S hands.

I find it hard to believe that a teacher would tell you this. Almost certainly you misconstrued what was said. Go back & ask them again.

There are zillions of bridge web sites & books that list the odds for various suit splits based on how many cards are missing in the suit. All of them merely say "if the total cards between you & dummy is x, the suit will split thus". None of them list separate suit splits for when dummy has 3 cds, dummy has 4 cds, etc. This is because it doesn't matter if they split 5-4, 6-3, 7-2, 9-0! Don't you think that if it mattered the sites would mention it?

http://www.durangobi...SplitStats.html
&
http://www.durangobi...SplitHowTo.html

explain a simple, direct way to calculate how suits will split. You & dummy have 26 known cards, the opps have 26. There are exactly 10,400,600 combinations of ways to give 13 cards to one opponent & 13 to the other out of their 26. To calculate how often a suit splits 2-2, one first calculates the number of ways to give 2 cards out of the 4 missing to each opponent =C(4, 2) = 4!/((2!)*(2!)) = 6. If in general you are trying to enumerate combinations of b cards out of a total, the formula is a!/(b!*(a-b )!). Then, for each combination of 2 cards, out of the 22 remaining cards you have to give 11 to each opponent. So by the formula, you have 22!/(11!)*(11!) = 705432.
In total, you have 6*705432 = 4232592 ways to deal the cards such that they split 2-2. 4232592/10400600 = 40.70%, like you'll see in every single publication of suit splits anywhere. How cards divide between the known hands never comes into it, as you see there is no place for it in the calculations! Only matters how many cards are missing.

For 3-1, it would be
C(4,1) * C(22,10)/C(26,13) = (4!/((1!)*(3!))) * (22!/((10!)*(12!)))/10400600
= (4 * 646646) / 10400600 = 2586584/10400600 = 24.87%

Of course, it can split 3-1 either way, so you double this to 49.74%, the well-known figure for 3-1 split in either direction, which is going to be more frequent than 2-2 again no matter how cards split between you & dummy.

[ralph23]

Stephen Tu, on Sep 12 2007, 06:05 PM, said:

There are zillions of bridge web sites & books that list the odds for various suit splits based on how many cards are missing in the suit. All of them merely say "if the total cards between you & dummy is x, the suit will split thus". None of them list separate suit splits for when dummy has 3 cds, dummy has 4 cds, etc. This is because it doesn't matter if they split 5-4, 6-3, 7-2, 9-0! Don't you think that if it mattered the sites would mention it?

All those sites are wrong. And all the books are wrong, too.

We are in the midst of the Copernican Revolution in bridge probabilities. Everything that everyone thought they knew for years is wrong. The new discovery has been made. Throw off the chains of the conventional wisdom!

You guys are just like those old Vatican types who persecuted Galileo. When this shows up as the new learning in the NY Times bridge column, you'll see.

[BebopKid]

If you read my first post, I asked for no insults. If you have something positive or constructive to comment, please do so.

I am happy for everyone that can keep quoting the same things to me. Please test and results from your trials as I have done.

Here are the results from Richard Pavlicek's deal generator.

Okay, using Richard Pavlicek's deal generator with this RPD config
D 65000:1000000000
123WS=6
123WS=3
01234WS=4
0123WS=2

This finds hands that meet the following requirements
1. any deal with a hand having a 6 card spade suit
2. any deal with a hand having a 3 card spade suit from set (1)
3. exclude any deal with a hand having a 4 card spade suit from set (2)
4. exclude any deal with a hand having a 2 card spade suit from set (3)

If you do not believe this is not an accurate test, please post the configuration file for a test you believe to be accurate.

Here are the results:
308912 hands with a 6 card spade suit in any hand
195462 hands with a 6-3 spade fit between any 2 hands
170625 hands without a 6-3-4-0 spade distribution
-> 24837 hand with a 6-3-4-0 spade distribution
65000 hands without a 6-3-2-2 spade distribution
-> 105625 hands with a 6-3-2-2 spade distribution
-> 65000 hands with a 6-3-3-1 spade distribution

So:
54.04% are 2-2 split
33.25% are 3-1 split
12.71% are 4-0 split

[BebopKid]

ralph23, on Sep 12 2007, 05:14 PM, said:

We are in the midst of the Copernican Revolution in bridge probabilities. Everything that everyone thought they knew for years is wrong. The new discovery has been made. Throw off the chains of the conventional wisdom!

You guys are just like those old Vatican types who persecuted Galileo. When this shows up as the new learning in the NY Times bridge column, you'll see.

I hope I didn't imply, by the way, that this was an original idea of mine.

This has been taught many times by many people before I started playing bridge.

A bridge instructor at our local club, also brought this back with him from the Instructor Training given at the ACBL in Nashville this year.

So please don't give me credit for proving something that someone else came up with.

[Trumpace]

BebopKid, on Sep 12 2007, 06:06 PM, said:

This finds hands that meet the following requirements
1. any deal with a hand having a 6 card spade suit
2. any deal with a hand having a 3 card spade suit from set (1)
3. exclude any deal with a hand having a 4 card spade suit from set (2)
4. exclude any deal with a hand having a 2 card spade suit from set (3)

Since you are the mathematical type, I suggest you try proving (mathematically, that is) that this config generates exactly the hands you are looking for (and with the same chances).

Instead of 13 cards, you can try with a lesser number of cards in a suit and try it out by hand and see if it matches.

Probability is a tricky subject and easy to go wrong with "intuition". For instance, I am pretty sure you might have heard of Monty Hall...