Jul 02, 2022 | Ruby
Say you want to loop through a collection of 100,000 users. You can grab all users with User.all, but that is very inefficient since it will try to instantiate all the objects at once.
When you have a situation like this, batch processing is your best friend. It will allow you to work with your user records in batches, without memory consumption going through the roof.
This method will let you grab your user records in batches, with default batch size of 1,000. This batch_size value can easily be changed if needed.
So, instead of doing this:
User.all.each do |u|
# do stuff here...
end
You can do this:
User.find_each do |u|
# do stuff here...
end
If you want to override the batch size, you can do it like this:
User.find_each(batch_size: 100) do |u|
# do stuff here...
end
There are many available options you can use with .find_each method. You can specify a different batch_size as in example above, you can specify start and finish values, where start is the primary key value (ex: your user ID) which tells the method from where to start (including the start value) and where to finish (including the finish value). You can also specify the order of your users (can be :asc or :desc, and default value is :asc).