Use Cursor When Retrieveing Large Dataset (Laravel Optimization)

Learn how to optimize the retrieval of large datasets in Laravel by making use cursor. Use this method to efficiently retrieve big datasets to enhance performance and speed up queries.

Created on Sep 01, 2021

•

1067 views

One of the ways to optimize queries while retrieving large datasets is to make use of Laravel cursor. Making use of the cursor allows large datasets to be loaded into the memory only when necessary which for this case during the data accessing/loop is made.

Optimise Query using Laravel Cursor

For this scenario, imagine you are querying thousands or millions of "posts" table. Imagine you have 100,000 records, having the code below executed will make the memory full and may even crash the web page.

<?php

// when using eloquent
$posts = Post::all();

// when using query builder
$posts = DB::table('posts')->get();

foreach($posts as $post){
    // do something with $post;
}

To query the "posts" table with cursor, you can call the "cursor()" method available to both the eloquent and query builder. In this case it will be as follows.

<?php

// when using eloquent
foreach(Post::cursor() as $post){
   // do something with $post;
}
 
// when using query builder
foreach(DB::table('posts')->cursor() as $post){
   // do something with $post;
}

The code example above will retrieve all of the records from the table and hydrate the Eloquent models one by one. This approach only make one database query to retrieve all of the posts but uses the PHP Generator to optimize the memory usage.

Use Collection each Method to Loop

One of the ways to loop the eloquent instance is to use Laravel collection "each()" method and in generate I would have considered this my choice of writing loops.

<?php

Post::cursor()->each(function ($post) {
    // do something with post
});

Do make sure to check out more of our snippets for Laravel optimization, tips, and tricks to better enhance your Laravel application. Cheers and happy coding!

Laravel Snippets

new

If you like our tutorial, do make sure to support us by being our Patreon or buy us some coffee ☕️

Patreon