BBO Discussion Forums: January's Robot Simulations - New version of ACBL-Ben vs many games - BBO Discussion Forums

Jump to content

Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

January's Robot Simulations - New version of ACBL-Ben vs many games 5 types of bots in 7 types of games

#1 User is offline   diana_eva 

  • PipPipPipPipPipPipPipPip
  • Group: Admin
  • Posts: 5,008
  • Joined: 2009-July-26
  • Gender:Female
  • Location:bucharest / romania

Posted 2024-January-07, 22:46

In preparation for our Innovation Week event (Jan 8-14, 2024), Lorand ran a new set of simulations, using an improved variation of the "ACBL Ben" robot and comparing how various bridge robots are scoring in diverse types of games.


Meet the digital bridge brains starring in these simulations:
  • GIB Advanced and Basic: These are BBO's signature robots, familiar to most of our players. They are the mainstay of BBO, with players interacting with these two robots in almost every game across the platform.
  • Argine: A guest robot known for its ability to play more exotic systems such as Acol, SEF, and Basic SAYC in our special daylongs. Argine is also adept at the 2/1 system, allowing for a comprehensive comparison with other robots.
  • ACBL Ben: This experimental AI bridge engine represents a sample of our ongoing AI reseach and development. Trained on a massive dataset of 100 million hands from ACBL pair games on BBO, this new iteration of ACBL Ben performs better than the one we tested last month, in this simulation.


3-Day Stars & Platinum Robot Individual - Click for Results
48 boards total, 16 each day.
No bidding, all boards pre-bid by GIB.
For star players and those with at least 5 ACBL Platinum masterpoints.

1. Argine: 55.29% (Top 15%)
2. Advanced GIB: 54.61% (Top 20%)
3. Basic GIB: 54.16% (Top 20%)
4. Simplified Argine: 53.35% (Top 25%)
5. ACBL-Ben: 48.62% (Top 50%)

Robots replayed 960 boards.
There were 618 players in the tournament.


Zenith Daylong Reward - Click for Results
16 boards, not-best-hand
Awards BBO Points and BB$ prizes.

1. Advanced GIB: 54.08% (Top 30%)
2. Argine: 53.46% (Top 35%)
3. Basic GIB: 50.95% (Top 50%)
4. ACBL-Ben: 50.62% (Top 50%)
5. Simplified Argine: 50.46% (Top 55%)

Robots replayed 1,920 boards.
There were 1,691 players in the tournament.


ACBL Lifemaster Daylong - Click for Results
12 boards, best-hand
Awards BBO Points and ACBL Masterpoints.
Only for ACBL Life Masters.

1. Advanced GIB: 63.24% (Top 10%)
2. Basic GIB: 60.46% (Top 20%)
3. ACBL-Ben: 56.50% (Top 30%)
4. Argine: 56.22% (Top 30%)
5. Simplified Argine: 54.46% (Top 40%)

Robots replayed 120 boards.
There were 146 players in the tournament.


ACBL Daylong - Click for Results
12 boards, best-hand
Awards BBO Points and ACBL Masterpoints.

1. Argine: 61.17% (Top 15%)
2. Advanced GIB: 60.93% (Top 15%)
3. Simplified Argine: 59.06% (Top 20%)
4. ACBL-Ben: 58.39% (Top 20%)
5. Basic GIB: 55.52% (Top 30%)

Robots replayed 240 boards.
There were 1,229 players in the tournament.


BBO Premium Daylong Just Declare - Click for Results
8 boards, no bidding, all hands pre-bid with GIB.
Awards BBO Points.

1. Argine: 60.33% (Top 20%)
2. Advanced GIB: 59.77% (Top 20%)
3. Basic GIB: 58.91% (Top 25%)
4. Simplified Argine: 57.05% (Top 25%)
5. ACBL-Ben: 54.32% (Top 40%)

Robots replayed 320 boards.
There were 781 players in the tournament.


BBO Premium Daylong (MP) - Click for Results
8 boards, best-hand.
Awards BBO Points.

1. Argine: 57.36% (Top 30%)
2. Advanced GIB: 57.00% (Top 30%)
3. Simplified Argine: 54.99% (Top 35%)
4. ACBL-Ben: 53.22% (Top 40%)
5. Basic GIB: 52.98% (Top 45%)

Robots replayed 320 boards.
There were 834 players in the tournament.


Free BBO Daylong (MP) - Click for Results
8 boards, best-hand.
Free, open to all.

1. Advanced GIB: 61.05% (Top 20%)
2. Argine: 59.42% (Top 25%)
3. ACBL-Ben: 57.29% (Top 30%)
4. Basic GIB: 56.91% (Top 30%)
5. Simplified Argine: 54.86% (Top 35%)

Robots replayed 1,600 boards.
There were 19,135 players in the tournament.


More details here.

#2 User is offline   pilowsky 

  • PipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 3,775
  • Joined: 2019-October-04
  • Gender:Male
  • Location:Poland

Posted 2024-January-08, 01:15

Looks like one could conclude that the field is strongest in non-ACBL tournaments.

Is it possible to isolate hands with unusual shapes/strength to see how the bidding systems compare in auctions that might be more difficult?
Fortuna Fortis Felix
0

#3 User is offline   pescetom 

  • PipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 7,932
  • Joined: 2014-February-18
  • Gender:Male
  • Location:Italy

Posted 2024-January-12, 16:24

Looks like one could conclude that "Advanced" GiB is slightly better than Argine and way better than Ben.
Which begs the questions of whether there is a better way to train Ben and (above all) why nobody has fixed the known bugs of Gib for 5 years now.
0

#4 User is offline   0 carbon 

  • PipPipPipPipPip
  • Group: Full Members
  • Posts: 537
  • Joined: 2009-January-19
  • Gender:Male

Posted 2024-January-14, 20:11

While Ben should learn to be a good player, if the training is right, I cannot see how it could be a good bidder given all the conflicting conventions those 100m hands reflect. I'm surprised its play is so bad! Did you reward properly?
tOM, Manager,GlobalClub ♣ My convention charts ♣ Ottawa RADBC webmaster
Posted Image
0
Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

3 User(s) are reading this topic
0 members, 3 guests, 0 anonymous users