Rengo update: large games can start now

There’s been a bug, since the beginning it seems, that large rengos can’t start.

It’s slightly odd that no-one has complained about this - maybe the huge rengo organisers are all trying to grow their participant lists.

I was recently able to fix this, so I think we’re good for at least a few hundred participants, which is the biggest I’ve tested.

I will mention, while I’m here, that moderators can and will start old rengo challenges that appear abandonned and unloved.

The best way for rengo organisers to avoid this is to communicate their thoughts and plans in the rengo challenge chat.


As a curiosity, ChatGPT provided the bugfix for this...

Me:

In this code, the last loops that save the players is very slow. Is there a way to speed this up?

self.game.rengo_black_team = [] 
self.game.rengo_white_team = [] 
for p in black_team: 
   self.game.rengo_black_team.append(p.id) 
for p in white_team: 
   self.game.rengo_white_team.append(p.id) 
logger.debug("start_rengo adding game to players...") 
for p in black_team: 
   p.games.add(self.game) 
for p in white_team: 
   p.games.add(self.game) 
logger.debug("start_rengo saving players...")
for p in black_team: 
    p.save() 
for p in white_team: 
   p.save()

ChatGPT:

If the saving process for each player is slow, it might be because each save() call is initiating a database transaction. In many database systems, the overhead of initiating and committing a transaction can be significant, particularly if you’re doing it many times in a loop. One way to speed this up is to use bulk operations which allow you to perform the operation on many objects at once.

Without knowing the specifics of your database system and your ORM (Object-Relational Mapping), I can’t provide an exact solution. However, if you’re using Django, the ORM provides a bulk_update() method that you might be able to use.

In the case of Django, you could potentially change your code to something like this:

self.game.rengo_black_team = [p.id for p in black_team]
self.game.rengo_white_team = [p.id for p in white_team]

logger.debug("start_rengo adding game to players...")
for team in [black_team, white_team]:
    for player in team:
        player.games.add(self.game)

logger.debug("start_rengo saving players...")
Player.objects.bulk_update(black_team + white_team, ['games'])

This code assumes Player is the class of the objects in black_team and white_team, and that games is the field you wish to update.

Note that bulk_update() can’t be used to update many-to-many relationships (which games might be), and you may need to make further adjustments depending on the specifics of your situation.

Me:

games is a many to many relationship - this is django on postrgres

ChatGPT:

Thanks for the additional context. Because games is a ManyToMany field, the bulk_update method will not work here. When dealing with ManyToMany relationships in Django, every addition or removal is indeed its own database transaction, which can lead to slow operations when performed in a loop.

One approach to optimize this is to use the add function with multiple instances. The add function of Django can take an arbitrary number of object instances, and will perform the operation in a single database query:

self.game.rengo_black_team = [p.id for p in black_team]
self.game.rengo_white_team = [p.id for p in white_team]

logger.debug("start_rengo adding game to players...")
self.game.players.add(*black_team)
self.game.players.add(*white_team)

# Saving game
self.game.save()

In the code above, I’ve assumed that the game has a ManyToMany field named players that refers to all players (both black and white). The * operator in Python is used to unpack the list of players into individual arguments for the add function.

This approach significantly reduces the number of database transactions by adding all the players to the game in two operations (one for each team), and then saving the game. Note that you don’t need to call save() on individual players as Django takes care of updating the ManyToMany relationship when you call add.

Remember to test this thoroughly to ensure it works as expected in your specific context.

me:

Are you assuming that game.players is the reverse name for the many to many?

ChatGPT:

Yes, in my previous response I did assume that game.players is the reverse relationship for the many-to-many field from the Player model to the Game model. If your naming is different, you should adjust the code accordingly.

In Django, when you create a ManyToManyField, Django automatically creates a “reverse” relation on the related model. For example, if your Player model has a ManyToManyField to a Game model, Django would allow you to access related Game instances from a Player instance, and vice versa.

The name of this reverse relation is, by default, the lowercased name of the model that defined the ManyToManyField, but you can customize it using the related_name argument when defining the ManyToManyField.

Here’s an example:

class Game(models.Model):
    ...

class Player(models.Model):
    games = models.ManyToManyField(Game, related_name='players')

In this case, you can access the games a player is part of with player.games.all(), and the players in a game with game.players.all().

If your field names are different, please replace players with the actual name of the reverse relation.

Me:

Your assumption was correct :slight_smile:

ChatGPT:

Great! Then the revised code should work fine in your context.

7 Likes

From all the shenanigans I’ve seen, I had forgotten it can actually be used for stuff that isn’t “write my report for me”!

I think you left out a word: “will start deleting old…”

I have always thought it was weird that organizers don’t use the challenge chat to say when the game will start. Also weird that players join a challenge with an unknown start.

No, I meant what I said.

Okay, I get it now.