[helene_t]

It's been a while since we have discussed AI-generated bidding systems.

I am currently toying with a fairly simple idea:

Assume that opps aren't going to interfere (ok, we will relax that assumption at some point but walk before you run blah blah. Also, for optimizing sequences that take place after both opps have passed it may be a reasonable simplification).

We know that the optimal relay system allocates F(i) hand types to the i'th most expensive bid where F is the Fibonacci sequence. But then you have to think about relay breaks and captaincy appointment so I prefer to optimize the system for 2-way information exchange. So the proportion of hand types allocated to the bids are 1/2, 1/4, 1/8 etc.

This is nice because we can then just organize the hand types in a balanced binary tree using hierarchical clustering. So in 1st seat, for example, it could be
Pass = Left daughter
1♣ = Right daughter of right daughter
1♦ = Right daughter of left daughter of right daughter
etc.

We have some decisions to make:
- When to branch left and when to branch right? In a non-GF situation maybe we should allocate bids so as to allow partner to pass as often as possible, especially when we are in danger of coming too high. But this is difficult to formalize in a way that can be implemented efficiently so I thought that a simple approximation would be to go low with the subtree that minimizes the expectation of PAR^p, where p might be 0.5 or thereabouts
- Criteria for passing In general we should pass on some hands where the last bid may not be the optimal contract (when in a hole, stop digging). I am not sure how to formalize it. For now, I think I will just tell it to pass whenever the expected loss relative to PAR is less than say 1 IMP. Maybe the expectation of LostIMP^q where q is 2 or thereabouts?
- Distance metric for clustering Here I will use the same metric that I used in my previous attempts (where I used an induction algorithm, i.e. top-down rather than the bottom-up approach of hierarchical clustering): the distance between two hands is the expected IMP loss if partner has to place the contract without being able to tell the two hands apart.
- Aggregation of distances to produce subtree-distances There's mean and median and minimax etc. I always just use whatever the software gives me by default, but if there's someone here who is into machine learning maybe they can suggest something more intelligent? Extending the "partner-can't-tell-them-apart" definition to subtrees would be ideal but probably too computationally demanding.
- Presentation of the system I have generated 1800 clusters of hands a priori which means that in 1st seat, pass will be defined as (either of 900 hand types), 1♣ as (either of 450 hand types) etc. It would be nice to try to find natural-language summaries of those hand type clusters, such as "unbalanced without a 6-card major, 10-16 points" or whatever. Anybody knows of a package (preferably for R or C++) that can do something like that?
- Benchmarking Csaba has a script that can bid and play hands using GIB so that would be a reasonable comparator.

Any thoughts?

[thepossum]

 helene_t, on 2023-April-20, 03:49, said:

It's been a while since we have discussed AI-generated bidding systems.

......<lots of stuff I no longer feel able to comment on>

Any thoughts?

I have many thoughts on the matter which are constrained by rustiness of knowledge to critique anything you said, restricted by my politeness, and informed by experience and cynicism (or do I mean skepticism)

Maybe one day I will be made to look foolish and arrogant

I will wait and read any intelligent comments

PS You may think I am somewhat flippant and disrespectful. When I have fewer whiskies inside me I will try to understand what you wrote

[helene_t]

One additional thought: basing the distance metric on partner placing the safety level, rather than the contract, could be interesting as it might lead to multi bids: for example, if partner has shown a 6-card in an unknown major and you have support for both, you can set the safety level to 3♥. Also, if playing a weak NT system and partner opens a suit, you will sometimes know that 3m is either lawful, or safe because partner will have the strength for 3nt.

[DavidKok]

 helene_t, on 2023-April-20, 03:49, said:

We know that the optimal relay system allocates F(i) hand types to the i'th most expensive bid where F is the Fibonacci sequence. But then you have to think about relay breaks and captaincy appointment so I prefer to optimize the system for 2-way information exchange. So the proportion of hand types allocated to the bids are 1/2, 1/4, 1/8 etc.

I think this is not correct. Showing hand type is a means to an end, the end being reaching the optimal contract. It is not clear what it means for a hand to belong to a type (the standard take is that this should just mean distribution, i.e. the four suit lengths, along with a general description of strength. But why not specific stoppers, situational stoppers, conditional strength descriptions and other?). More importantly, more space should be assigned to situations where it is unclear what our likely best contract is, while we are relatively safe in bidding higher when we have a clearer picture of possible best contracts. In particular, each subsequent bid should not cut hand types in two, but cut the uncertainty in final contracts in two, i.e. halve the entropy of the conditional distribution of best final contracts.

 helene_t, on 2023-April-20, 03:49, said:

- In a non-GF situation maybe we should allocate bids so as to allow partner to pass as often as possible, especially when we are in danger of coming too high.

Hands that want to bid to a specific contract but are too weak to bid any higher are among the least complex ones and provide the greatest certainty. As a result these are among the best candidates for jump bids. As a rule of thumb, any bid that suggests itself as a final contract eliminates most possible bidding sequences that go through that bid, and significantly reduces the amount of information that can be communicated by the system.

 helene_t, on 2023-April-20, 03:49, said:

Any thoughts?

The rest looks really good. I like the 'width' of a cluster as the average IMP cost if partner cannot tell individual hands in a cluster apart.

 thepossum, on 2023-April-20, 04:18, said:

I will wait and read any intelligent comments

If only I had read this first, I could have spared us both my comment.

[DavidKok]

 DavidKok, on 2023-April-20, 09:21, said:

In particular, each subsequent bid should not cut hand types in two, but cut the uncertainty in final contracts in two, i.e. halve the entropy of the conditional distribution of best final contracts.

On reflection this too was naive. We don't care about 'best contacts', only about IMPs/matchpoints/VIPs/other. An ideal system chooses at each turn the bid that will maximise the expected number of these. To make this more than a tautology I think the notion of 'value of information' can help. Helping declarer decide between (say) 6♠ and 6NT, conditional on both being near certain to make, is not very valuable at IMPs. Any information that helps distinguish distributions of possible best final contracts primarily along the 6♠-versus-6NT axis, conditional on both being a clear favourite to make, is low value. Meanwhile (e.g.) 3NT vs 6NT, or a significant risk of 4♥-1 versus 3NT=, is high value because of the number of IMPs riding on that decision.
Trying to reduce the entropy of possible best final contracts by log(2) on each step of the bidding is a good strategy for maximising information density, but it does not coincide with maximising our bridge score.

[helene_t]

Thanks for sharing your thoughts, David.

I agree it's a bit problematic to say that we should just split left with 50% of hands (or some 39% in a relay system) as it begs the question whether it means 50% of hands or 50% of the probability mass (taking into account that some hands are more likely than others given opps' silence) or 50% of hand "types" whatever that means. Ideally you should make cheap bids (little immediate information exchange but lots of space saved for future information exchange) with hands that can't know now what info partner wants, and more expensive bids with hands that have an unusual story to tell which partner surely would like to hear.

So for example, with a solid 7-card suit and nothing else you open 3NT, while with some 5431-hand you start low, not only because the safety level is lower but also because we don't know yet whether it is the 3-card spades or the 5-card clubs that is more interesting to partner.

My approach doesn't rely on such considerations, at least not explicitly so it might well fail. But maybe knowing-which-feature-partner-wants-to-know-about correlates well enough with safety level that it won't be too bad just to split left or right based on safety level.

With respect to the question of whether entropy of the optimal contract is a useful metric, it is maybe a question that would be interesting to research in isolation. Last time we discussed AI-generated bidding systems (https://www.bridgeba...ed-by-computer/), Tysen used entropy of the optimal contract while I used IMPs lost. There were too many other differences between our approaches to tell if entropy vs IMPs was a reason for the different results we got, though.

[thepossum]

I spent a few hours reading all this today (including the other thread)
Apart from my ignorance of relay systems (irrelevant to my concerns) I am still skeptical

I have hopefully a few reasonably intelligent questions
1. It seems that all these things, whether bidding systems or anything complex, require a huge amount of human intelligence to start them off and keep them on track
2. Helene. Do you have the criteria you used to cluster your hands in the first place - I know a little about clustering and "natural language" labelling but it usually requires human selected attributes to start dividing up or clustering your variance - I know there are tools that can analyse all kinds of things, pull out patterns and group overlapping variance and maybe come up with a descriptor that means something to a Bridge player - sounds complicated to me - I imagine a very innovative system could come up with descriptors that mean nothing. Maybe they would be forced to break with the WBF and other major bodies and start a rival form of the game with their own directors. One of them tries unsuccessfully to be elected to the rules committee etc
3. Sorry. Do you have a public GitHub project or anything like that?
oops 4. While ignorant of relay systems I am a bit less ignorant after today but they still would do my limited brain capacity in
sorry 5. As a person occasionally involved in academic modelling - practical and theoretical - I do not mean to be disrespectful at all when I say how we always have to reduce things to some extremely simplified scenario and then eventually (never) make things more complicated and real. At least hopefully it improves knowledge and informs in some way
sorry 6. Final question. Have any mainly computer/AI generated systems ever been used at top level competition

What will really impress me is the day someone gives a robot the rules of bridge, maybe a few million expert hands, it thinks for a few minutes then wins the world championship with their friend. My final skeptical observation and prediction is that any intelligent robot would observe who wins things and how they bid and play

[helene_t]

 thepossum, on 2023-April-21, 00:35, said:

2. Helene. Do you have the criteria you used to cluster your hands in the first place

So far I have just clustered them by exact number of hearts and spades, number of clubs and diamonds in 2-points brackets (0-1, 2-3 etc) and number of HCPs in two-point brackets. It would be nice to have texture information also, but the problem is that even with the current crude clustering criteria, I get some 1800 clusters which is about the limit of what I can manage effectively. When both partners can have either of 1800 hand types, there are some 3 million deal types so I generated 100 million deals to make sure that most deal types are represented by a reasonable sample size. I can't do much more than this without breaking my computer.
What could easily be done is replacing HCPs and suit length with some estimate of the hand's playing strength in each of the denominations, that wouldn't necesarilly be higher dimensional than what I do now.
What I could also do is to switch to a different model once a GF has been established and a fit found. The we would have more parameters but fewer hands possible.

[DavidKok]

I'm still reading the old thread, so perhaps the rest of this comment will prove to be dated. I think my previous comments in this thread already have. Oh well.

Trying to reduce the entropy of best final contract by log(2) each step sounded nice to me when I wrote it, but should be impossible. Not counting sacrifices there are only 21 possible optimal contracts (lowest partscore, game, small and grand slam in each denomination, and passout). If we could really reduce entropy by log(2) each step we should be able to be reasonably certain about the final contract after 5 steps or so, i.e. get our small versus grand slam decisions right at the 1NT level. Even if we allow some 'free' bidding space by claiming that the lowest playable partscores are 1NT, 2♥, 2♠, 3♣ and 3♦ the same logic holds.

The old thread's comments on multi-meaning bids are important and relevant. In my mind a final working product would be something along the lines of "on the auction thus far, we have shown one of the following hand types <list>, while partner has shown one of the following hand types <other list>. Based on simulations of my actual hand facing partner's possible hands, our next bids should clarify trick-taking potential in a certain strain". A reasonable example would be asking partner for extra shape (i.e. show increased trick-taking potential in their longest suit), clarify a multi-meaning bid or show min/max range information (i.e. show increased trick-taking potential across the board).

One huge brute force option I was thinking of is setting a (global?) maximum of possible meanings per bid, i.e. 4 or so, and requiring that each subsequent bid is a subset of the previous ones. This would give each bid on each auction around 40 weights (min and max length of each suit and in HCP means 10 values per possible meaning, times the maximum allowed number of meanings). I'd also require that these are all mutually exclusive with the other options at each round - i.e. each hand only has one system bid, and each of the categories in each bid are cleanly split (note that standard bidding systems usually don't do this, but we have to start somewhere). Then a cost function of reaching good/bad contracts could help optimise these values. Hopefully the system would gravitate to something like "it is important to clarify suit length/offer extra length when you have it". The main issue is the explosion of the system size, so maybe a good start would be to start this approach with a mini-bridge equivalent. Some restrictions that I think are interesting (and mentioned in that other thread) include:

• No interference, to simplify.
  
• Only score for getting to the right denomination, not caring about level (but include the fact that majors pay better somehow?).
  
• Only allow two full rounds of bidding, to reduce complexity.
  
• Force a certain start to the bidding, e.g. North opens a strong (15-17) notrump, as a proof of concept.
  
• Simplify even further, and use the above approach (force 15-17 NT and give responder something in the 10-14 range) but only allow 3NT, 4♥ or 4♠ as final contracts and see how well that can be optimised with this approach.

[hrothgar]

Sorry to flog a dead horse, but I think that you're much better off starting with auction termination

[helene_t]

Haha that was a predictable comment:) but you may have noted that I addressed that in two of my points in the OP

[Gilithin]

I think your basic axiom is simply untrue. The Fibonacci distribution of hands in relay systems is only valid because these auctions are game-forcing and thus we know that the available space runs to 3NT. In a more general bidding situation we have to have the ability to stop in a part-score when that seems advantageous. So you first have to deal with those hands, and only once you have gotten down to the pure game-forcing sequences can you switch to the suggested hand division. In addition, while less important under the assumption of no interference, there are certain principles that make bidding systems more resilient, such as homogeneity. I also strongly disagree with your first addendum, that we should make bids designed to allow partner to pass as often as possible. It should be noted that mechanisms like transfers, relays and puppets, that have massively increased in their usage since the days when bidding systems actually did have more non-forcing calls, are explicitly non-passable. Many of these allow us to pass the response but that is quite a different thing from allowing partner to pass directly. In general, too many non-forcing calls make a bidding system less efficient. So I would suggest you relax this stipulation - we need to be able to pass when we reach a sensible contract and do not want to go to a higher level (PS-game-slam). There are different ways to achieve that and you should probably allow your (presumed) iterative process to decide for itself the best one.

[helene_t]

 thepossum, on 2023-April-21, 00:35, said:

3. Do you have a public GitHub project or anything like that?

I have a GitHub account and I also make ShinyApps but I am really not good at software engineering (my last IT job was 21 years ago) so I wouldn't encourage anyone to try to use any of my code. Once I get something that is worth sharing I will do so, but it will be in the form of ShinyApps or bare-bone algorithm implementations. Not anything like a software engineering project.

Quote

6. Have any mainly computer/AI generated systems ever been used at top level competition

I don't think such a system has ever been used by humans at all. There was one reasonably playable AI system published a few years ago but the problem is that even if such a system would be technically great, it might be impossible to memorize, and it may be illegal.

[helene_t]

 Gilithin, on 2023-April-21, 20:03, said:

I also strongly disagree with your first addendum, that we should make bids designed to allow partner to pass as often as possible. It should be noted that mechanisms like transfers, relays and puppets, that have massively increased in their usage since the days when bidding systems actually did have more non-forcing calls, are explicitly non-passable. Many of these allow us to pass the response but that is quite a different thing from allowing partner to pass directly. In general, too many non-forcing calls make a bidding system less efficient. So I would suggest you relax this stipulation - we need to be able to pass when we reach a sensible contract and do not want to go to a higher level (PS-game-slam). There are different ways to achieve that and you should probably allow your (presumed) iterative process to decide for itself the best one.

Good points.

Maybe the award for allowing partner to pass could be made dependent on captaincy?

Also, it is possible that this exercise could lead to a crude draft which could subsequently be refined by a genetic algorithm or such.

[Gilithin]

 helene_t, on 2023-April-22, 08:52, said:

Good points.

Maybe the award for allowing partner to pass could be made dependent on captaincy?

Also, it is possible that this exercise could lead to a crude draft which could subsequently be refined by a genetic algorithm or such.

As I mentioned, there are various solutions. One as you suggest is to have a pure captaincy system. A simpler one is to borrow from modern approach-forcing and simply say that a bid is only not forcing if the hand making it has limited itself in some way. As opposed to the original approach-forcing where a bid is by default non-forcing. Coming from an Acol background, it occurs to me you might find that method comforting, even though theory has determined it to be inefficient. Finally, a popular solution on these forums is to use the first round of bidding to determine the range of the hands and take it from there. If you look at the systems from Adam and Zelandakh, they both essentially start by defining a range as partscore-inv; inv-game; or game-slam and then refine the rest of the auction to optimise for that range. No doubt there are a few other solutions too. Sadly I cannot tell you which is best though; ideally a true AI-generated bidding system would answer that question for us!

[helene_t]

With respect to auction termination, does anyone (hi Richard) have some ideas?

I would like the players to decide whether to pass based on:
- How much will passing on average lose relative to the best contract above?
- How much will each higher contract gain relative to pass, and what is the SD across partner's hands? Generally, higher contracts weight more as we will be more likely to be able to find them, but generally if any higher contract has a high gain/SD then we know what contract is better than pass so we shouldn't pass.
- How much does partner already know about our hand? If they know a lot then we have more freedom to seek contract improvement and should pass less often

Maybe I am missing something important.

I was thinking of starting with a fairly restrictive auction termination criterion, which will probably lead to many contracts that are too high. And then post-hoc generate a dataset of opportunities to improve the contract by passing earlier, and fit a regression model that can identify those opportunities.

[hrothgar]

So, I'd start by implementing a (known) auction termination system.

Make sure that you can replicate a set of existing logic.

if / when you're able to do do this then it makes sense to branch out and consider how to dynamically generate a system.

FWIW, I think that being able to evaluate the relative efficiency of different competing termination systems is both

1. Extremely valuable in and of itself
2. A necessary first step towards what you're eventually trying to accomplish

[helene_t]

I have now made the opening bids. It is a quite chaotic picture with all the lower bids being multi-bids with an awful lot of meanings, so I am not sure how to sumarize it. But generally:
Pass: Less than 14 points, club length
1c: 8-17 points
1d: 10+ points, spade length
1h: 14-21 points, diamond length
1s: 14-17 points, short clubs
1N: 18-21 points, both majors
2c: 20+ points, short clubs
2d: 22+ points, short clubs
2h: 22+ points, [4261]
2s: 22+ points, [2461]
2N: 22+ points, [3361]

It is a bit weird with all those bids showing club shortage, while the strong hands with long clubs being scatered randomly. Hopefully a system optimized to auctions where opps are going to interfere will be more inteligeble as you then can't rely on plenty of bidding space to resolve your shape later.

I must admit my enthusiasm for this approach is cooled a bit. In the old thread, since I optimized the openings to partner giving a punt immediately, the openings tended to be quite descriptive, but here it's like each opening bid is just a random pile of stuff that will be resolved later.

[straube]

I so deserve a chance to play against this.

Helene, I'm a little confused with your description of your pass. Is it 0-8 any or up to 14 with clubs?

[helene_t]

 straube, on 2023-April-27, 13:42, said:

I so deserve a chance to play against this.

Helene, I'm a little confused with your description of your pass. Is it 0-8 any or up to 14 with clubs?

It is a bit loose, most of the hands that pass will have 0-13 points and 3+ clubs. But all the 1suit openings have some weird variants also.