REST API Development

API Pagination Strategies

18 min Lesson 12 of 35

Introduction to API Pagination

Pagination is essential for any API that returns collections of data. Without pagination, returning large datasets can overwhelm your server, consume excessive bandwidth, and create poor user experiences. Laravel provides three powerful pagination methods, each suited for different use cases. In this comprehensive lesson, we'll explore offset pagination, cursor-based pagination, and simple pagination, learning when and how to use each effectively.

Why Pagination Matters

Before implementing pagination, let's understand why it's critical for production APIs:

Benefits of API Pagination:

Performance: Reduces database query execution time and memory usage
Bandwidth: Minimizes data transfer and API response size
User Experience: Enables faster initial page loads and progressive data loading
Scalability: Allows APIs to handle datasets with millions of records
Resource Management: Prevents server resource exhaustion
Mobile Optimization: Reduces data consumption for mobile users

Offset-Based Pagination (Traditional)

Offset pagination is the most common pagination method. Laravel's paginate() method provides this out of the box:

<?php
// app/Http/Controllers/Api/PostController.php

namespace App\Http\Controllers\Api;

use App\Models\Post;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;

class PostController extends Controller
{
    public function index(Request $request): JsonResponse
    {
        // Basic pagination: 15 items per page
        $posts = Post::with('author:id,name')
            ->latest()
            ->paginate(15);

        return response()->json($posts);
    }
}
</pre>

Laravel's pagination response structure includes metadata for easy navigation:

{
    "current_page": 1,
    "data": [
        {"id": 1, "title": "First Post", "author": {"id": 1, "name": "John"}},
        {"id": 2, "title": "Second Post", "author": {"id": 2, "name": "Jane"}},
        // ... 13 more items
    ],
    "first_page_url": "http://api.example.com/posts?page=1",
    "from": 1,
    "last_page": 10,
    "last_page_url": "http://api.example.com/posts?page=10",
    "links": [
        {"url": null, "label": "&laquo; Previous", "active": false},
        {"url": "http://api.example.com/posts?page=1", "label": "1", "active": true},
        {"url": "http://api.example.com/posts?page=2", "label": "2", "active": false},
        {"url": "http://api.example.com/posts?page=2", "label": "Next &raquo;", "active": false}
    ],
    "next_page_url": "http://api.example.com/posts?page=2",
    "path": "http://api.example.com/posts",
    "per_page": 15,
    "prev_page_url": null,
    "to": 15,
    "total": 150
}

Custom Per-Page Values

Allow clients to specify how many items they want per page:

<?php
public function index(Request $request): JsonResponse
{
    $request->validate([
        'per_page' => 'integer|min:1|max:100',
    ]);

    $perPage = $request->input('per_page', 15); // Default 15
    $maxPerPage = 100; // Prevent abuse

    $posts = Post::with('author:id,name')
        ->latest()
        ->paginate(min($perPage, $maxPerPage));

    return response()->json($posts);
}

// Usage:
// GET /api/posts?per_page=25
// GET /api/posts?page=2&per_page=50
</pre>

Pro Tip: Always enforce a maximum per_page value (typically 100) to prevent clients from requesting thousands of records in a single request, which could overload your server.

Simple Pagination

When you don't need to display total page counts or allow direct page jumps, use simplePaginate(). It's more efficient because it doesn't execute a COUNT query:

<?php
public function index(): JsonResponse
{
    // More efficient - no COUNT query
    $posts = Post::with('author:id,name')
        ->latest()
        ->simplePaginate(15);

    return response()->json($posts);
}
</pre>

Simple pagination response structure (notice the absence of total and last_page):

{
    "current_page": 1,
    "data": [/* ... */],
    "first_page_url": "http://api.example.com/posts?page=1",
    "from": 1,
    "next_page_url": "http://api.example.com/posts?page=2",
    "path": "http://api.example.com/posts",
    "per_page": 15,
    "prev_page_url": null,
    "to": 15
}

Cursor-Based Pagination (Advanced)

Cursor pagination is the most efficient method for large datasets and real-time feeds. It uses encoded cursors instead of page numbers, making it perfect for infinite scroll interfaces:

<?php
public function index(): JsonResponse
{
    // Cursor pagination - most efficient for large datasets
    $posts = Post::with('author:id,name')
        ->latest()
        ->cursorPaginate(15);

    return response()->json($posts);
}
</pre>

Cursor pagination response structure:

{
    "data": [/* ... */],
    "path": "http://api.example.com/posts",
    "per_page": 15,
    "next_cursor": "eyJpZCI6MTUsIl9wb2ludHNUb05leHRJdGVtcyI6dHJ1ZX0",
    "next_page_url": "http://api.example.com/posts?cursor=eyJpZCI6MTUsIl9wb2ludHNUb05leHRJdGVtcyI6dHJ1ZX0",
    "prev_cursor": null,
    "prev_page_url": null
}

Key Differences:

Offset Pagination: Uses page numbers (page=2). Can jump to any page, but inefficient for large datasets.
Cursor Pagination: Uses encoded cursors. Cannot jump to arbitrary pages, but extremely efficient and consistent even when data changes.
Simple Pagination: Like offset but without total count. Good for "Next/Previous" only interfaces.

Comparison: When to Use Each Method

// Offset Pagination - Use when:
// ✓ Users need to jump to specific pages
// ✓ Displaying page numbers (1, 2, 3...)
// ✓ Dataset is relatively small (< 100K records)
// ✗ Avoid for real-time feeds (inconsistent with new data)
$users = User::paginate(20);

// Cursor Pagination - Use when:
// ✓ Infinite scroll / "Load More" UIs
// ✓ Large datasets (millions of records)
// ✓ Real-time feeds (social media, notifications)
// ✓ Performance is critical
// ✗ Cannot jump to arbitrary pages
$posts = Post::latest()->cursorPaginate(20);

// Simple Pagination - Use when:
// ✓ Only need "Next/Previous" navigation
// ✓ Want better performance than offset pagination
// ✓ Don't need total count
$comments = Comment::simplePaginate(20);

Custom Pagination Response Format

Transform Laravel's default pagination structure to match your API conventions:

<?php
// app/Http/Resources/PostCollection.php

namespace App\Http\Resources;

use Illuminate\Http\Resources\Json\ResourceCollection;

class PostCollection extends ResourceCollection
{
    public function toArray($request): array
    {
        return [
            'items' => PostResource::collection($this->collection),
            'pagination' => [
                'total' => $this->total(),
                'count' => $this->count(),
                'per_page' => $this->perPage(),
                'current_page' => $this->currentPage(),
                'total_pages' => $this->lastPage(),
            ],
            'links' => [
                'first' => $this->url(1),
                'last' => $this->url($this->lastPage()),
                'prev' => $this->previousPageUrl(),
                'next' => $this->nextPageUrl(),
            ],
        ];
    }
}

// Controller
public function index(): JsonResponse
{
    $posts = Post::with('author')->paginate(15);
    return response()->json(new PostCollection($posts));
}
</pre>

Custom response structure:

{
    "items": [
        {"id": 1, "title": "First Post"},
        {"id": 2, "title": "Second Post"}
    ],
    "pagination": {
        "total": 150,
        "count": 15,
        "per_page": 15,
        "current_page": 1,
        "total_pages": 10
    },
    "links": {
        "first": "http://api.example.com/posts?page=1",
        "last": "http://api.example.com/posts?page=10",
        "prev": null,
        "next": "http://api.example.com/posts?page=2"
    }
}

Pagination with Filtering and Sorting

Combine pagination with query parameters for powerful data retrieval:

<?php
public function index(Request $request): JsonResponse
{
    $request->validate([
        'per_page' => 'integer|min:1|max:100',
        'sort_by' => 'in:created_at,title,views',
        'sort_direction' => 'in:asc,desc',
        'status' => 'in:draft,published,archived',
        'search' => 'string|max:255',
    ]);

    $query = Post::with('author:id,name');

    // Filter by status
    if ($request->has('status')) {
        $query->where('status', $request->status);
    }

    // Search
    if ($request->filled('search')) {
        $query->where('title', 'like', '%' . $request->search . '%');
    }

    // Sorting
    $sortBy = $request->input('sort_by', 'created_at');
    $sortDirection = $request->input('sort_direction', 'desc');
    $query->orderBy($sortBy, $sortDirection);

    // Paginate
    $perPage = min($request->input('per_page', 15), 100);
    $posts = $query->paginate($perPage);

    return response()->json($posts);
}

// Usage:
// GET /api/posts?status=published&sort_by=views&sort_direction=desc&per_page=25
// GET /api/posts?search=Laravel&page=2
</pre>

Pagination Performance Optimization

1. Avoid N+1 Queries with Eager Loading

<?php
// Bad: N+1 query problem
$posts = Post::paginate(15); // Each post triggers author query

// Good: Eager loading
$posts = Post::with('author:id,name', 'tags:id,name')
    ->paginate(15);
</pre>

2. Use Select to Limit Columns

<?php
// Bad: Selecting all columns including large text fields
$posts = Post::paginate(15);

// Good: Select only needed columns
$posts = Post::select(['id', 'title', 'excerpt', 'author_id', 'created_at'])
    ->with('author:id,name')
    ->paginate(15);
</pre>

3. Add Database Indexes

<?php
// Migration
Schema::table('posts', function (Blueprint $table) {
    // Index for pagination queries
    $table->index(['created_at', 'id']); // Compound index

    // Index for filtered pagination
    $table->index('status');
    $table->index(['status', 'created_at']);

    // Index for cursor pagination
    $table->index(['id', 'created_at']);
});
</pre>

Warning: For offset pagination on large datasets (page=1000), MySQL must skip all previous rows, making deep pagination extremely slow. Use cursor pagination for better performance on large datasets.

Cursor Pagination Implementation Details

Understanding how cursor pagination works under the hood:

<?php
// Basic cursor pagination
$posts = Post::orderBy('id')->cursorPaginate(15);

// Cursor with multiple order columns
$posts = Post::orderBy('created_at', 'desc')
    ->orderBy('id', 'desc') // Tiebreaker for unique sorting
    ->cursorPaginate(15);

// Cursor pagination with filtering (safe from inconsistencies)
$posts = Post::where('status', 'published')
    ->orderBy('created_at', 'desc')
    ->orderBy('id', 'desc')
    ->cursorPaginate(15);

// The generated SQL uses WHERE clauses instead of OFFSET:
// SELECT * FROM posts
// WHERE (created_at < '2024-01-15' OR (created_at = '2024-01-15' AND id < 123))
// ORDER BY created_at DESC, id DESC
// LIMIT 15
</pre>

Cursor Pagination Best Practice: Always include a unique column (usually id) as a tiebreaker in your orderBy() clause to ensure consistent results.

Appending Query Parameters to Pagination Links

Preserve query parameters across pagination requests:

<?php
public function index(Request $request): JsonResponse
{
    $posts = Post::where('status', $request->status)
        ->paginate(15)
        ->appends($request->except('page')); // Preserve all params except 'page'

    return response()->json($posts);
}

// Alternative: Append specific parameters
$posts = Post::paginate(15)->appends([
    'status' => $request->status,
    'sort_by' => $request->sort_by,
]);

// Result: Links include query parameters
// http://api.example.com/posts?status=published&page=2
</pre>

Pagination Metadata Methods

Laravel's paginator provides many helpful methods:

<?php
$posts = Post::paginate(15);

// Get pagination information
$posts->total();           // Total number of items
$posts->count();           // Number of items on current page
$posts->perPage();         // Items per page
$posts->currentPage();     // Current page number
$posts->lastPage();        // Last page number
$posts->hasPages();        // Check if pagination needed
$posts->hasMorePages();    // Check if more pages exist
$posts->onFirstPage();     // Check if on first page
$posts->onLastPage();      // Check if on last page
$posts->firstItem();       // Index of first item (e.g., 16 on page 2)
$posts->lastItem();        // Index of last item (e.g., 30 on page 2)

// URLs
$posts->url(5);            // URL for page 5
$posts->previousPageUrl(); // Previous page URL (null if on first)
$posts->nextPageUrl();     // Next page URL (null if on last)

// Get items
$posts->items();           // Get items as array
</pre>

Manual Pagination

Create pagination from arrays or custom queries:

<?php
use Illuminate\Pagination\LengthAwarePaginator;
use Illuminate\Pagination\Paginator;

// Manual paginator with total count
$items = collect(range(1, 100)); // Your data source
$perPage = 15;
$currentPage = LengthAwarePaginator::resolveCurrentPage();
$currentItems = $items->slice(($currentPage - 1) * $perPage, $perPage)->values();

$paginator = new LengthAwarePaginator(
    $currentItems,
    $items->count(),
    $perPage,
    $currentPage,
    ['path' => LengthAwarePaginator::resolveCurrentPath()]
);

// Manual simple paginator (no total count)
$simplePaginator = new Paginator(
    $currentItems,
    $perPage,
    $currentPage,
    ['path' => Paginator::resolveCurrentPath()]
);
</pre>

Testing Pagination

Write comprehensive tests for your paginated endpoints:

<?php
// tests/Feature/PostPaginationTest.php

namespace Tests\Feature;

use App\Models\Post;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Tests\TestCase;

class PostPaginationTest extends TestCase
{
    use RefreshDatabase;

    public function test_pagination_returns_correct_number_of_items()
    {
        Post::factory()->count(50)->create();

        $response = $this->getJson('/api/posts?per_page=15');

        $response->assertOk()
            ->assertJsonCount(15, 'data')
            ->assertJsonStructure([
                'current_page',
                'data',
                'first_page_url',
                'from',
                'last_page',
                'last_page_url',
                'next_page_url',
                'per_page',
                'prev_page_url',
                'to',
                'total',
            ]);

        $this->assertEquals(50, $response->json('total'));
        $this->assertEquals(4, $response->json('last_page'));
    }

    public function test_pagination_respects_max_per_page()
    {
        Post::factory()->count(200)->create();

        $response = $this->getJson('/api/posts?per_page=999');

        $response->assertOk();
        $this->assertLessThanOrEqual(100, $response->json('per_page'));
    }

    public function test_cursor_pagination_works()
    {
        Post::factory()->count(30)->create();

        $response = $this->getJson('/api/posts/cursor');

        $response->assertOk()
            ->assertJsonStructure([
                'data',
                'next_cursor',
                'next_page_url',
                'per_page',
            ]);

        // Follow cursor to next page
        $nextCursor = $response->json('next_cursor');
        $response2 = $this->getJson('/api/posts/cursor?cursor=' . $nextCursor);
        $response2->assertOk();
    }
}
</pre>

Practice Exercise:

Implement all three pagination methods (offset, cursor, simple) for a User API endpoint with filtering by role and search by name
Create a custom API response format that matches the JSON:API specification for pagination
Build a performance comparison test that measures query execution time for offset vs cursor pagination at different page depths (page 1, 100, 1000)
Implement a "keyset pagination" system using multiple columns (created_at + id) for consistent ordering even when new records are inserted
Create a React or Vue component that implements infinite scroll using cursor pagination with your Laravel API

Best Practices

Pagination Best Practices:

Choose the right method: Use cursor for large datasets and real-time feeds, offset for traditional pagination
Set reasonable defaults: 15-25 items per page is usually ideal
Enforce maximum limits: Prevent abuse by capping per_page at 100
Add proper indexes: Index columns used in ORDER BY and WHERE clauses
Use eager loading: Avoid N+1 queries with with()
Preserve query parameters: Use appends() to maintain filters across pages
Document pagination: Clearly explain pagination in your API documentation
Test edge cases: Test empty results, single page, last page, invalid page numbers

Summary

In this lesson, you've mastered API pagination in Laravel. You now understand the differences between offset pagination, cursor pagination, and simple pagination, and when to use each method. You've learned how to customize pagination responses, combine pagination with filtering and sorting, optimize performance with indexes and eager loading, and test your paginated endpoints thoroughly. Proper pagination is fundamental to building scalable APIs that can efficiently handle large datasets while providing excellent user experiences.