[mikeh]

I use Bridgemaster Pro to make my simulations when I want to assess the merits of particular bids, but it is apparent to me that several other posters use an entirely different approach, and then there is the BridgeBrowser method as well.

I have never used the BB approach, and think that it is of extremely limited assistance (close to zero) in terms of assessing the merits of actions in an expert partnership or in a good field. I have discussed some of my thinking in other posts, but was wondering if Ben and other BB users have any arguments that might show me the light.

The other common form of simulator, used by several posters, seems to generate many thousands of hands and then produces an analysis of likely trick-taking for certain actions or denominations.

I am not familiar with such analyzers, and I may be missing the point, which is, in part, the reason for this post.

I don't understand how the information generated from that process helps us evaluate or develop our bidding methods or decisions.

It seems to me that there are several problems:

1. The defence and play is, I believe, double-dummy. This immediately makes the value suspect. I am on the 'scientist' end of the bidding spectrum, but even I know that there are lots of auctions in which concealment will be effective, allowing us to make a contract that would be easy to beat if the opps had more info. And I think that double-dummy analysis will usually reward the defence more than the offence, if only because the most important trick for most contracts, where success hangs in the balance, is trick one.. the opening lead.

2. I may be wrong, but I have the impression that this process may incorporate the results of hands on which the posed auction simply couldn't happen. This is especially true when we are simulating an uncontested auction.. because we may not be constraining the opps' hands. Say we constrain only N-S.... and we give ourselves a combined 5 card heart holding. Once in a while, an opp will hold 6 or 7 or even 8 of them, and would surely enter the auction. Yet our simulator, while using these hands in the double-dummy analysis of the play, will not 'know' to make that opp bid. So we are including in our analysis of our auction the results from hands that could never occur in real life.

3. In a similar vein, I suspect that most of the small-sample simulators who use BridgeMaster or similar programmes are familiar with the tedious weeding out of hands that are inappropriate. No matter how well you program your constraints, you will inevitably generate some hands where the bidding simply doesn't fit the hands. We may specify that W holds 5+ spades, no other suit longer than 3 cards, and 7-16 hcp because we want to generate hands involving a 1♠ overcall. Then we see a hand such as 87654 KQx Qxx xx... well, maybe you overcall 1♠ but most wouldn't. That was an extreme example, and it is possible to be more restrictive in your constraints, but the point is that, no matter how much time you spend on your constraints, some generated hands won't mesh. I don't see how that is accomplished when I read that someone generated 100,000 hands and the results showed how many tricks NS could take in hearts, notrump, etc.

And this is NOT a trivial point. Bidding is a complex subject, and minor deviations can have an enormous impact on the outcome. Yesterday, we lost a grand/small swing because one player chose not to overcall 3♣, over a 2♠ opening, because he felt that his hand and suit were borderline.. he had KQ98xx in clubs.. had he had the 10 of clubs, he says he would have made the call. The preempt was raised, and altho we got to slam, we missed the laydown grand. I mention this not to argue his hand, but to show that judgment plays a huge role in bidding decisions.. in other words, we have to be painstaking in making sure that our sample of hands matches hands that would most likely be bid, in real life, as we are specifying in our problem.

So while I understand the fascination of being able to say: I ran 100,000 or 200,000 such hands, and the result was.... I don't see how that helps with the subtle problems that get posted here.

Am I missing something?

I am posting this rant because I fear that there are a lot of players who have never used a simulation and may be thinking that all simulations are similar, and that all simulation results are important. Plus, maybe I am misunderstanding the double-dummy, multi-thousand simulations. As for me, I find that my simulations have at times generated useful information... tending to make me rethink certain auctions.. but they take a LOT of time to even do 100 or 150 hands.

[FrancesHinden]

I'm not going to help much here, because overall I agree with you. I typically simulate 50-100 deals (using a dealer program I wrote myself) and then wade through them individually and see if I can draw any conclusions. In addition, depending on what I am trying to decide, I may look at the 'single dummy' result, and come up with an answer along the lines of "You want to be in slam on 75 of the hands and it is making on 60 of them, it is also making on 5 of the other hands where you don't want to be in it". As mikeh says, this tends to take quite a long time.*

I have prevously ranted at length about my perception of the useless of BB analysis.

I feel slightly open-minded about the multi-thousand hand DD analyses. As long as the parameters are set up right (which one can usually check by scanning through the first 10-20 or so hands generated), I feel I might buy into the idea that - certainly for game level contracts - the DD advantage/disadvantage may even itself out in the long run.

The one thing that does bias the results (I think) is not jsut the opening lead, it is that when you are defending, you base your defence on partnership agreements and inferences such as 'partner usually leads fourth highest of his longest and strongest', 'we would make an attacking lead on this auction therefore if partner made a passive lead he didn't have an obviously attacking lead available' so I think real life defence is 'more worse' than DD defence than we might believe. Or to put it another way: if I knew that my partner was defending double-dummy then I would make different assumptions about his hand, even if I was not defending DD.

*the last simulation I did was on my lead-to-trick 3 defensive problem from the spring fours. On the first 50 hands I dealt it made no difference what I did at trick three, so I didn't learn anything at all. Perhaps here a big DD simulation would have been more useful, assuming I can specify the play to the first two tricks.

[helene_t]

1. With GIB you can make simulations SD. (I have seen such results presented in BW. Haven't done it myself).

2. You should obviously constrain opps also, otherwise the analysis makes little sense.

3. If you let a program like GIB bid the hands and then select hands that yield the given auction, you won't have that problem. But of course that is computationally more intensive. You probably won't filter 100,000 deals in that way.

[awm]

I mostly use C code. The sort of "simulation" I can easily do involves specifying constraints on hand types and then generating many random hands satisfying these conditions. There are some interesting things one can discover this way -- for example some recent ones:

(1) After three passes, you are more likely to hold a 15-17 notrump than a 12-14 notrump. This holds even if opponents are quite conservative about opening bids (i.e. virtually never open less than 12 hcp); it is even more the case when opponents have a more free-wheeling opening style.

(2) Suppose RHO opens a standard 1♦. You hold either 4-2-4-3 or 4-3-2-4 distribution. What is partner's expected spade length? It turns out that it is virtually identical in the two cases -- holding additional diamonds in your hand doesn't make partner more likely to hold spades. Of course, this does not necessarily say much about the relative merits of overcalling 1♠ on these hands, since when partner does hold three spades the 2♠ contract may well play better when the shorter diamonds are in dummy, and you also have a reasonable alternative non-pass call on the second hand (double). But it does contradict a claim many posters were making.

Bridge Browser can be useful in some situations. The problem is that a lot of people seem to try and use it for the wrong things. It can easily answer questions about what "most people" in the field do with certain hands (maybe not that interesting since the average level of the field is not so high). I think it is also useful for stylistic things that are hard to evaluate otherwise. Here's one that I tested recently:

(3) Is it a good idea to open 4♠ in first seat, non-vulnerable, when holding 13-15 hcp and seven or more spades? Note that this is not really a "system" question (except in as much as a few people play namyats) -- the 4♠ opening has pretty much the same meaning in all systems. There are obvious problems with opening 4♠ (you might miss a slam, 4♠ might fail on a misfit when opponents didn't have anything, might be hard to work out whether to double if opponents compete). But there are also obvious advantages (keeps opponents guessing). This kind of thing is very hard to evaluate using any kind of "double dummy" analysis because it depends on your opponents. Over a large number of hands, my result was that opening 4♠ wins over 1 IMP/board (opening 1♠ is basically zero because it was so common). This remained the case when I restricted to tables where all four players had good lehman ratings (although the sample size was then small enough that it may not be statistically significant any more). Now this doesn't say "you should always open 4♠ with such a hand" since obviously there is some self-selection and people probably don't open 4♠ with seven spades to the eight. But it does suggest that opening 4♠ is fairly often a winning action, despite possibly missing a slam, and worthy of consideration.

[hotShot]

Any simulation process has consist of 4 steps, it does not matter much if it's bridge or economics or for the weather forecast.

1) Make a model for your problem
In bridge this is done by setting the constrains for the hand. Of cause you have to set constrains for all relevant hands, this includes opps.

If your model does not fit your problem, you will misunderstand the results.
At this point most people fail at least at the first try.

2) Create the input for your problem
In bridge this would be deals. If your dealer does not produce truly random deals, your results will be flawed.

3) Perform the simulation
This is executing the DD-Analysis or performing the BB query.

Up to here you should be aware of the SI => SO principle which is politically correct expanded to: Nonsense In => Nonsense Out.
So you should check if the deals created really fit your problem and you should reconsider the constrains you set. If your simulation tools does not allow you to check the deals used in some kind of log, you should get yourself a different tool.

4) Interpret the results
Ask yourself again: "What question does my model answer?"
And check if the answer is relevant. Before you install a 50.1% over 49.9% advantage in your system, find something else to optimize.

[hotShot]

mikeh, on May 16 2008, 05:50 PM, said:

1. The defence and play is, I believe, double-dummy. This immediately makes the value suspect. I am on the 'scientist' end of the bidding spectrum, but even I know that there are lots of auctions in which concealment will be effective, allowing us to make a contract that would be easy to beat if the opps had more info. And I think that double-dummy analysis will usually reward the defence more than the offence, if only because the most important trick for most contracts, where success hangs in the balance, is trick one.. the opening lead.

I have a program that parses lin-files and checks every card played, if it loses against the best double-dummy play.
Intermediates average about 1 trick per board less than double-dummy play, experts to world class about 0.25-0.5 tricks per board.

So if you are intermediate, forget about simulations and work on your cardplay and bidding.
If you are expert+ you will play 3 of 4 boards equivalent to the double dummy solver, and can rely on the double dummy result.

There have been scientific studies comparing ok-bridge data with double-dummy results, showing an average deviation of 0.25 tricks. (The deviation is larger for 1-level contracts and is shrinking the higher the level is.)

These results are relevant for BB too, because this means that if intermediate pairs play each other, they will produce the same result as world class pairs. The former because each side misses 2 tricks, the latter because both sides miss no trick.
So if there are enough scores/board the result of a BB query has a significant meaning.

[whereagles]

trouble with simulations is they just work post-mortem

[Stephen Tu]

(1) Double dummy -- well people have studied this and for most normal contracts it's close enough to single dummy for practical purposes. It tends to break down for things like bidding grand slams missing a crucial queen or two, obviously declarer knowing which way to hook is very important & it doesn't really help the defense in this case.

(2) Opp's hands -- well constrain them if it's important! Also sometimes it doesn't matter too much in the big scheme of things, if the distribution for which they would have come in is rare, it will have a negligible effect on the final tallies if you neglect to exclude those.

(3) The better dealers give more options for constraints than dealmaster. It's easy to print out a few to see if the hands generated all seem to "make sense", and adjust your constraints if they don't. And even if you miss some, if it's rare & the great bulk of the hands generated fit the problem conditions, the effect of a few outliers will again be negligible.

Quote

So while I understand the fascination of being able to say: I ran 100,000 or 200,000 such hands, and the result was.... I don't see how that helps with the subtle problems that get posted here.

I think most of the people who run simulations are capable of recognizing when a simulation is likely to be useful in answering a problem, and when it is clearly not. Simulations are great when the problem is purely statistical in nature. If the problem is stylistic, or involves far too many variables and scenarios to consider, then it's not.

[ASkolnick]

Simulators: One of my favorite simulators is the one design by Hans Van Steveren since it becomes more open source. Any GUI usually has limitations to the constraints that you can use. But this builds a file where you can create your own classes, define shapes (not just number of cards), and add cards to particular opponents. By having the flexibility, you will be able to draw the same kind of conclusions.

As for single dummy versus double dummy, I think that there will be only a slight difference in the two since both the defense on the offense always get it right. For the most part, I look at Deep Finesse on bridge hands, and the number of tricks usually seems reasonable.

[Mbodell]

As some one who has done such simulations, and continues to do them I think they can be immensely valuable. You can answer lots of questions with it from the simple, like how many ♦ do I hold on average when I open a completely artificial unbalanced 1♦ precision like opener versus a standard American 1♦, to harder to answer questions like how likely is it that when holding Tx AK K876543 Tx in 4th seat with 3 passes to me that passing is a losing matchpoint action.

Quote

1. The defence and play is, I believe, double-dummy. This immediately makes the value suspect. I am on the 'scientist' end of the bidding spectrum, but even I know that there are lots of auctions in which concealment will be effective, allowing us to make a contract that would be easy to beat if the opps had more info. And I think that double-dummy analysis will usually reward the defence more than the offence, if only because the most important trick for most contracts, where success hangs in the balance, is trick one.. the opening lead.

I agree. I know there has been a fair bit of study that shows DD results are pretty darn close to actual result. But on more complicated questions if you can show that some action is 66% right or 80% right with DD play then the margin of difference provides some confidence. Whereas if DD play says some action is a 50.5% action and another is a 49.5% action then I might be much more worried that we were outside the precision of the analysis. Also, sometimes, depending on the question, I might not do the simulation DD but might instead know my line ahead of time and count the outcome. I.e., in high contracts I might know that given the way I played it in real life I needed the K of trump onside or a 3-3 split in some side suit given one of the opponents opened a weak 2 in some other suit, even if some other squeeze or deep finesse might make on DD analysis I might instead count the number of times my condition is true in the opponents hands and count the number of times I'd make the contract that way. There a large number of simulations can be useful without any DD analysis.

Quote

2. I may be wrong, but I have the impression that this process may incorporate the results of hands on which the posed auction simply couldn't happen. This is especially true when we are simulating an uncontested auction.. because we may not be constraining the opps' hands. Say we constrain only N-S.... and we give ourselves a combined 5 card heart holding. Once in a while, an opp will hold 6 or 7 or even 8 of them, and would surely enter the auction. Yet our simulator, while using these hands in the double-dummy analysis of the play, will not 'know' to make that opp bid. So we are including in our analysis of our auction the results from hands that could never occur in real life.

You are wrong. At least if the person doing the simulation is reasonably diligent about this. I have ways of evaluating hands for first bids in many of my systems (from when I wanted to simulate frequency of different openers and what frequencies of shapes and point ranges would be held with each bid), including high level preempts and gambling 3nt and what not, and can easily put restrictions that opponents hold hands that would pass or a hand that would open a vulnerable weak 2♠ or what not.

Quote

3. In a similar vein, I suspect that most of the small-sample simulators who use BridgeMaster or similar programmes are familiar with the tedious weeding out of hands that are inappropriate. No matter how well you program your constraints, you will inevitably generate some hands where the bidding simply doesn't fit the hands. We may specify that W holds 5+ spades, no other suit longer than 3 cards, and 7-16 hcp because we want to generate hands involving a 1♠ overcall. Then we see a hand such as 87654 KQx Qxx xx... well, maybe you overcall 1♠ but most wouldn't. That was an extreme example, and it is possible to be more restrictive in your constraints, but the point is that, no matter how much time you spend on your constraints, some generated hands won't mesh. I don't see how that is accomplished when I read that someone generated 100,000 hands and the results showed how many tricks NS could take in hearts, notrump, etc.

I am familiar with that process and do it too. I weed through hands in early samples of the runs before I do a larger run, examining every hand and if I see one that wouldn't work I add the conditions to remove it from the sample to the script. In your example if I had originally only done what you suggested but then saw the 87654 hand I'd do something like say for 7-12 point hands I need 2 of the top 3 or 3 of the top 5 spades, or whatever the reasonable suit quality was, and add that to the script. It is useful to do this both to make sure the conditions on the bidding is correct and also to make sure the math is correct of whatever you are tracking. I usually look at 10 deals in a set (because of screen size and program default) and generally look at somewhere between 3 and 10 sets while debugging the script or adding the right counting and statistical variables to it, and then generally look at another 3 to 5 sets once I think I've eliminated the last bug or made the last change.

I'm sure that often there are judgment hands that might be the 1 in 100 or 1 in 500 type hands that go each way. For instance if you have a cut off that says someone needs one of the top 3 cards in a suit to take some action, maybe they wouldn't really do it with Q32 but would do it with JT9. But again if your initial small samples were properly weeded through then these judgment hands should cancel to some degree and should effect the margin of determining the answer to only some small degree. So again something that is a 80% winner might only be a 79 or 81 or 82% winner. But that is eating around the margins.

[TimG]

I think double dummy has value.

As you say, the biggest loss relative to double dummy result comes from the opening lead and overall the defense performs a bit worse than DD while declarer performs a bit better than double dummy. If I find that a certain combination results in 9+ tricks in NT ~50% of the time, I am pretty confident that bidding the game is worthwhile. If I find that the same opening hand plus a slightly different responding hand makes 9+ tricks in NT ~56% of the time, I can be pretty confident that the second hand is worth more than the first. Recognizing all the while that declarer will likely make more often than the DD simulation suggests.

I recently did some work with 4432 hands facing 4432 or 4333 hands. The 4432 or 4333 hands had 15 HCP (strong NT) and the 4432 hands were responding hands with 10 points. Using DD I could find that 4=4=3=2 facing 4=3=3=3 produced x tricks in a spade contract and that 4=4=3=2 facing 4=3=2=4 produced y tricks in a spade contract. If I tell you that 4=4=3=2 facing 4=3=3=3 produces 9.7 tricks on a DD basis, you might reasonably say: "So what? We don't play double dummy." But, If I tell you 4=4=3=2 facing 4=3=3=3 produces 0.4 fewer tricks* than 4=4=3=2 facing 4=3=2=4, that might be of some value to you because even though we don't play double dummy, it seems reasonable to expect that each simulation will be affected the same way by being DD. That is, while the absolute number of tricks may not have great value, the relative number of tricks should have value.

* I don't remember the actual numbers and am not going to look them up for this post. I will note, however, that someone I shared the results with suggested I post to BBO to see what other people thought. I declined for a couple of reasons, one of which is the disdain that some have for DD simulations.

Along similar lines, you might hold QTxx K9xx xx Axx and want to know which suit is more attractive to lead against a 1N-3N auction. DD over 1000s of hands is probably going to give you a better answer than whatever by-hand method you use over 100-150 hands.

In my opinion, DD has its place if you understand the limitations.

[cherdano]

Once I get a new laptop, I will buy GIB and use it for single dummy simulation. I expect it to be very useful for some specific problems (e.g. "best lead against 1N-3N"), not so useful in many other situations.

[hrothgar]

For what its worth, this is an area where I have (fairly) strong opinions.

Moreover, I like to think that I have some credibility in this area. I've done a lot of work with computer simulations in my day. I am currently working at the MathWorks as product manager for the company's Statistics system. I spend a lot of time working with different types of Monte Carlo simulations. (MATLAB rocks, btw)

I think that a number of other posters have raised some very good points. In particular, Stephen's comment that folks have used single dummy generators to test the validity of double dummy generators is very important. As I understand matters, single dummy generators and double dummy generators are pretty much equivalent when measuring the mean number of tricks that can be taken. I recall seeing some results that suggested a statistically significant difference in the variance. However, most of the analysis that I have seen focuses on studying the mean so this doesn't end up as a big concern. What is a big concern is sample size... Double Dummy simulators run much faster than single dummy systems. Folks are often time constrained when they run their analysis. As a result, double dummy is often preferred. In a similar vein, Frances mentions running a sample of 100 hands and inspecting these by hand. While I have great faith in Frances as an analyst, I'd prefer a much larger sample.

I'd like to present an analogy that folks might find useful:

I'm not sure if folks know how most digital scales work. Digital scales are quite inaccurate. A traditional "balanced scale" where folks move physical weights back and forth are significantly more accurate than a digital scale. There's a lot of variance in a single measurement on a digital scale. If I take a single measurement of a weight using a digital scale and balance scale, the balance scale will almost win.

What's nice about digital scales is that they are very fast. A digital scale can measure the weight of an item 10,000 during the same period it would take me to do one measurement on a physical scale. In fact, digital scales are designed to take advantage of this fact.

If I weigh a tablespoon of garlic on my (digital) kitchen scale, the scale will actually weigh the garlic a couple thousands times and then given me the mean of all those observations. While any one individual observation is relatively inaccurate, the average of 1,000 independent observations is VERY accurate. In Statistics (more formally Machine Learning) this is referred to as "Boosting": combining a set of weak learning algorithms to create a strong learning algorithm.

In my mind, the single most important thing to understand about any kind of Monte Carlo simulation is "GIGO". Garbage In / Garbage Out. If you don't do a good job designing your simulation (worse yet, if you're using a flawed random number generator) all of your results need to be questions. Many of the critiques that Mike raises don't appear to be able Monte Carlo simulations in theory, but rather with specific limitations on the implementation. For example, consider the following critique

Quote

I may be wrong, but I have the impression that this process may incorporate the results of hands on which the posed auction simply couldn't happen. This is especially true when we are simulating an uncontested auction.. because we may not be constraining the opps' hands.

This is a very valid concern. I echo earlier comments that hand generators different significantly in their ability to generate useful constraints. Part of the reason that I like Hans van Stavern's Dealer is the extent to which I can generate complex constraints.

On last comment (once again quote Stephen)

Quote

I think most of the people who run simulations are capable of recognizing when a simulation is likely to be useful in answering a problem, and when it is clearly not.

I am far less sanguine about the capabilities of "most people" running simulations. I see a lot of really scary stuff come by my desks, most of which seems to involve inappropriate use of bootstraps... This stuff is being done by professional engineers as part of their paid jobs.

Lord knows what the hobbiests are doing

[Mbodell]

I guess I should mention that I use Thomas Andrew's deal program for my simulations. And then program in the constraints on the hand. I haven't used Tcl other than with this but it is a pretty straightforward language as far as these things go. And the program has hooks to integrate with GIB for DD analysis.

[Tcyk]

I am a BridgeBrowser veteran. By that, I mean that I have been using it for a number of years. I was one of the early testers. I am not an expert. However, I can search on just about any criteria you can imagine. Hand shape, strength, opening bid, overcalls, doubles, vulnerability. The interesting thing is that I can perform this search for many millions of hands. I can limit the hands to only those played by the stars (for those that are worried about the average caliber of play). The only real limitation is how much time you want to spend doing these searches and the availability of hands to search. The advantage is that the results are those played by humans, not some double dummy analysis program.

Sometimes, the BridgeBrowser results are not what you might suspect. I did an extensive seach on 1NT openings with no shape criteria but with HCP in 1-point increments from 8 to 20. I searched some 20 million hands. Yes it took several days. What amazed me was that the lower the no trump strength the better the results at both IMPs and MPs. I could attribute this to two things, the declarer advantage and lack of experience or defensive tools against weak no trump.

I suspect that no simulator could verify or refute these findings about 1NT opening bids. The poor results of the super strong openings is probably because someone miscounted the strength of their hand or was an absolute beginner and saw nothing wrong with being too strong for the bid. Partner had no way of knowing of the extra strength.

I once ran extensive searches of psychic opening bids and their results. I just looked for 1S opening bids with less than 4 spades and less than 10 HCP. There may be a systemic meaning to these opening bids but they are rare. I manually examined a sampling of the hands that were found and I saw none that were alerted. The average result for the hands found was not very good for the psychic opening but there were large variations in the resulting scores. (... and I had a list of people that psyched.) You could never find this information using a computer simulation.

I think there are times when a computer simulation program does have advantages. When trying to learn a new convention or system, I can deal appropriate hands to practice bidding. I usually deal hands in groups of 16, either random or constrained. If I find a hand I don't know how to bid, I search the documentation until I find the answer (or sometimes no answer).

I also have a program that strips, player, auction, play, and results from hands written in pbn format Using this program, I can search for hands using BridgeBrowser with specified criteria, strip the extraneous information, and bid the hands myself. Then, I can go back to the BridgeBrowser hands and see how the hands were bid in the real world and the results. By doing this, I can evaluate the effectiveness of my own bidding. You can't do this with a computer simulation.

I am interested in the effectiveness of some of the newer bidding systems. If I want to see how Revision Club or Diamond-Major does in IMP games, I can search for hands played by the author of the system. I can do the same thing with other systems if I happen to know the user name of those that play the system. You can't do that with a computer simulator.

- end-of-rant

[inquiry]

I will address the issue of the program Mikeh asked about, and Frances finds useless. The simulation issue, I will leave to another post. I will refer to that program as DB for "database".

DB is not a simulation program, as such. It does not do what simulations do (hold one hand constant, vary others within set parameters). It wasn't designed to do that, so this is not a surprise.

There are a lot of ways you can use the DB, all as just that a database. The ways I use if most often is to study the bidding of an individual pair or a single player. For instance, I have over 17,000 hands bid by Jimmy Cayne and his partner. <ore than 700 by Fantoni and nunes, etc. Some one whats to know how versace and Lauria respond after 1NT, they were referred in this forum to notes on their bidding, I can call up all the hands they have opened 1NT and look how they bid and see the actual hands. This stuff is invaluable.

Another really useful advantage of the DB program is to find hands that fit some odd requirment you want. I used it to work on MisIry convention after Mishovnbg introduced it to me and wanted to play it. I seached 200 million hands (not deals, a lot less than 200 unique deals...btw, the database has 300 million hands now) to find hands that were at least 5-5 and very strong, to see how well the strong variants of the convention would work, and found "holes" in the method that I worked on to fix. I found more than 25 thousand hands that fit the requirements, no I didn't use all of them.

I also use the DB to answer some odd questions from time to time. Like after 3 passes, should you strickly adhere to Pearson Points in 4th seat. I think the overwhelming evidence is that the answer is no. I will not repeat the data here, but the evidence seems clear to me.

I will say that two influential teaching pro's use DB to find hands that fit certain requirements and then pick among those hands to give to their customers. For instance, one might find "flawed" 1NT opening hands and give them to their clients (maybe with the real matching hand, and if interesting enough, with other hands they generate themselves). Of course, they don't care (or even probably look) at what happened at the table, this is just an easy way to find a lot of hands that meet certain requirements.... similar to simulations, but they can let the program search through ones that seem to cause everyone problems, and then just look through those hands..simplifying the search compared to simulations.


Finally, in passing, let me address those who might want to use the DB as a substitute for simulations. Peter Cheung did an extensive double dummy analysis of hands played by OKBRIDGE players (comparing the number of tricks they took in their contracts versus the double dummy result for play in that same contract). He referred to this as "single/double dummy and actual play". The source of the OKBRIDGE hands was the very same DB program being discussed, plus his own software to analyze play. The analysis was ofbridge hands using OkBridge database (30 million plays) (the largest of the DB files).

His analysis showed:

Quote

The most important general finding is that double dummy analysis is very accurate as compared to actual play from OKBridge. The overall total number of tricks taken by the declarer is 9.21 (9.22 for imp and 9.20 for mp).  The double dummy analysis of the same deal produce 9.11 (9.12 for imp and 9.11 for mp).  So actual play by OKBridge player takes 0.1 tricks more then the double dummy analysis result. This is from 383,000 deals and over 25 million plays.

He has a whole lot of stuff on simulations, and the like. The page discussing the above results is. Double dummy/Actual play

His home page has a lot of simulation stuff and discussion, including 1NT bids with and without five card major, etc. Dig around, it can be fun...

Peter Cheung's webpage

Now, don't get too excited about the finding that double dummy and declarer play give the same result. After looking at millions of plays, the standard deviation gets low, and on any hand, the defense at one table may chunk two tricks and at another the declearer may do the same, and the end result is the double dummy one. On the other hand, this finding is very common (lots of hands end up with about the double dummy result for the given contract)... if you end up believing that is true (and it seems to be if you find 1200 or so hands or more that meet the requirement) then there maybe something you dig out of the DB that is useful if you choose to try to use it for simulaitions.

Finally, the DB allows double dummy of any individual hand, and since it has hands YOU played (if you play on Bridge Base), you can review your own results in play. And you can review your results for any bidding sequence. Like, how well do your preempts work? How good do you do when you overcall 2 of a minor, how do you do when you don't make a preempt that many players would make? By looking at thousands of hands you have played, you can begin to see if you are too conservative or too aggressive, at least against the current level of opposition. One don't want to learn bad habits in a poor game if one has aspirations to higher things.

BUT I CAN'T IMAGINE HOW STUDYING HANDS BID BY HELGMO, VERSACE, FRED, RODWELL AND THE LIKE, USING A DATABASE PROGRAM CAN POSSIBLE BE A BAD THING MANY PEOPLE SAY. Anyone who reads Bridge World for the event reports should salivate over the DB program, just to be able to recall, sort, play double dummy, study, analyze, and watch each card played one card at a time by players like these (forget the stuff i listed above). But then maybe it is just me... I am no longer pushing this program on you guys (i was even pushing "free access" to all gold stars and almost free (one dollar) to the rest of you, but you guys just don't seem to get it or want it, and I no longer care if you ever do... I have continous access to it so i am happy. (IF I sound grumpy about it, I am not. Because I am over it, really, I am over how you guys treat a database program. Seriously, it doesn't bother me. No kidding. All that is behind me. I don't feel mad or disappointed few see the light, and attack those who use it. It doesn't bother me one iota. Well,,,, maybe not. )

[cherdano]

Could someone please point me to the evidence that double dummy analysis is good enough?
And no, it is not sufficient to show me a study of 10,000 hands where on average, declarers took almost as many tricks as they would have double dummy.

Let me give an example. Han recently posted a double dummy lead analysis where leading from xx was exactly as good as leading from Kxxx. My immediate objection was that the lead from xx assumes that partner will always find the right shift later, which is much harder when you led from a short suit than when you lead from one of your good suits. So maybe single dummy leading from xx is not such a great idea, after all.
I would be very surprised if the studies that Stephen and Richard cite addressed my objection.

[hotShot]

cherdano, on May 17 2008, 09:15 AM, said:

Could someone please point me to the evidence that double dummy analysis is good enough?
And no, it is not sufficient to show me a study of 10,000 hands where on average, declarers took almost as many tricks as they would have double dummy.

Let me give an example. Han recently posted a double dummy lead analysis where leading from xx was exactly as good as leading from Kxxx. My immediate objection was that the lead from xx assumes that partner will always find the right shift later, which is much harder when you led from a short suit than when you lead from one of your good suits. So maybe single dummy leading from xx is not such a great idea, after all.
I would be very surprised if the studies that Stephen and Richard cite addressed my objection.

It's said that there are math gods, human number cruncher that can really calculate odds in bridge. Mere mortals like me use simulations to estimate the odds. Knowing that it is a very good estimate, is good enough for me.

[Ant590]

The wider issue here is the qualitative / quantitative spectrum in any research. On one side we have double-dummy simulations based on a few thousand hands, which can't claim to take into account all the variables in bridge (e.g. feelings, table presence, tilt - let alone inferences from the bidding etc) On the other we have the responses we obtain from local experts and BBO forum posters when faced with a particular scenario that is usually not generalizable to other hands (what vul, scoring, standard of opps are usual considerations).

In an ideal world these two methods would 'triangulate', and give an accepted answer. However, if only allowed one technique I know which I would prefer – and because of this I feel simulations should be taken with a huge pinch of salt. Assuming the validity of the stats presented, (I don't think I have even seen a significance figure from a T-test or similar), they are still not representing bridge, but only a mathematical representation of bridge.