I will match the specs of what they are using to play the game, hardware wise. So it should replicate exactly the conditions and winrates in real times
2 Likes
Leela wins!
2 Likes
Leela wins again!
3 Likes
LZ only used half the hardware horsepower of its opponent.
5 gtx 1080ti
vs golaxy 10 gtx 1080 ti
1 Like
So apparently candidate network 126 (59bb7337) is not actually ladder proof.
I was playing some test games against Leela 11 and was suprised to find that in one instance Leela Zero using 15 block 192 filters on network (59bb7337) lost to Leela 11 due to misreading and mistakenly playing out a ladder. So I rewind the game and played again, this time giving Leela Zero network 126 (59bb7337) far more playouts in attempt to see if it would spot the mistake and not play out a doomed ladder. I was very surprised to see that it still would play out the ladder even when I set playout to 1600, 3200, 6400, and 12800.
Here is the sgf, can duplicate by moving to move 78, whereby setting Leela 11 as black, and Leela Zero (59bb7337) as white. Leela 11 for move 79 will play A17, whereby regardless of how many playouts are used, the Leela Zero (59bb7337) as white will either immediately play out the doomed ladder and/or it will play some other moves first, only then to fall for the ladder that doesnāt work.
Maybe I missed something or there is a good explaination or a bug with how Sabaki connects to Leela Zero or something, but this just seems very odd.
2 Likes
Andy Liu plays Leela Zero, network 127
1 Like
Is anyone going to stream finals or at least Leelaās match for 3rd place?
1 Like
For curiosity sake today I tested CrazyStone Deep Learning edition 1.01 against Leela Zero network #128
1 Like
Facebook open sourced their networks weight and support for it has just added to Leela Zero. Expect a big jump in performance nowā¦
1 Like
Should have linked this as well. Puts Google/Deepmind in a less than favourable light for not doing the sameā¦
2 Likes
Apples and oranges.
You canāt compare what Google do with their leading edge R&D with what FB does with their copy code.
1 Like
My comments on gh:
Soā¦ overall ELF is at least >500+ Elo stronger than current best LZ network and possibily up to 800 elo stronker. But it appears to have some latent issues such as being even more prone to ladder fallability and other certain fragilities at less than superhigh playouts, which ironically are things that arenāt as prelevant in the current LZ net arch. By using the approach of hybrid mix 50% LZ/ 50% ELF self-games and continuing to train the LZ net but using ELF as a part-time strength-gainer games-generator (at least until which point LZ catches up to ELF or even surpasses it) may patch the current weak spots in both net archs and set up a framework that is repeatable into the future, such as if AGZ weights get released or facebook teams releases a (new) and even stronger ELF OpenGo network sometime in the future, etc this can be repeated again. And it gives the LZ project the chance to figure out how to resolve two of the biggest issues of superhuman Go AI, that is ladders, large group deaths, and things like high handicap and variable komi and whatnots. Since the current 15b still has enough capacity left, the adding of ELF games will allow LZ to catch up enough to give the devs a way to figure out how to lower gating or remove gating etc. For sure the continuation of āthings as usualā for the LZ project is no longer an option, if for no other reason than the strength gap being too great for volunteer clients to still be willing to go the business as usual path, and as gcp stated, its not an option to train the ELF network directly either. (not to mention that switching directly to the ELF network would be in essence killing off the Leela Zero itself) So looks like the roadmap going forward is to train LZ hybridized with ELF, so it seems like there is a way to make good on the fb post about ELF helping project like Leela Zero after all!
1000 games of Facebook OpenGo vs Leela Zero on Nvidia Tesla V100 at 80,000 playouts per move.
(acuatlly its 998 games and Leela Zero had a 18:998 winrate, which means the ELF is ~700 elo stronger)
What is not in debate is this, in terms of open source Go, we have well passed without a shadow of any doubt into āsuperhumanā territory. ELF is estimated to be 4400 real elo on max hardware while top players are about 3700.
2 Likes
If LZ had a 18:998 winrate, that means they played 1016 games.
1 Like
It was 18 out of 998 games, or 18:980. Two games apparently were not recorded.
3 Likes
Thanks, I was wondering if that was maybe the case but couldnāt be sure.
LZ-ELF (using only 4800 v) just obliterated LZ using a whopping 80,000 v
Pure domination!
2 Likes
Quick trip back in time, merely about two years ago:
(many people thought including dywrin and haylee and nick sibicky that this first edition (which ironically is also the last edition given the progress of LZ and now LZ-ELF) was at best 5d, and couldnāt complete against even a 7d Dywrin.tv or 4p Haylee etc0 )
and two months ago:
(the first place was short lived, soon after we saw a bunch of US President names bots on the top rankings and it turns out these were actually the facebook bots which have now been incorporated into LZ-ELF version)
and two days ago:
and now today:
we have free, open source, open weights, multi-platform, LeelaZero-ELF that ranks higher than Ke Jie on a modest GTX 780 gpuā¦ anyone can download it from zero.sjeng.org or even use the Lizzie GUI with it directly.
The only thing crazy is how fast Go AI is progressing for the open publicā¦
Not only has this killed commerial Go but I think its gonna put go schools and go instructors out of jobs too.
1 Like
Dude, no.
CS was only on top of the unreliable incremental elo chart with insufficient games. By Bayes elo, CS was fifth after two versions of cronus, Pegasus, and Zen. Also, this was a āzeroā version of CS. Your post implies it was the deep learning first edition version.
Citation needed.
Perhaps my students would have something to say about thisā¦ No matter how strong the engine, it canāt explain its moves and wonāt tell you what to study for improvement.
4 Likes