Fix Notion Pagination Missing Records: Get Complete Results (Not Missing) for Developers & Automation Builders

hq720 48
  • Main keyword (keyword focus): notion pagination missing records
  • Predicate (main action): Fix / Get / Retrieve
  • Relations Lexical used: Antonym (“complete results” vs “missing records”)

If you’re seeing Notion pagination missing records, the fastest fix is to treat every Notion list response as partial until you loop through it using has_more + next_cursor and pass that value back as start_cursor on the next request. Notion’s API is cursor-paginated and can return fewer than the maximum results per call, so a “missing records” symptom is often just an unfinished retrieval. (developers.notion.com)

Many teams still lose items even after adding cursors, because the root problem isn’t pagination at all—it’s filters/sorts, permissions, or calling the wrong endpoint/version (database vs data source). Those issues can make the dataset look “incomplete” even when pagination is correct. (developers.notion.com)

Next, you’ll learn a safe, repeatable cursor loop (including guardrails for duplicates and rate limits) that works in scripts and automations, so your sync doesn’t silently stop at 100 rows. (developers.notion.com)

Introduce a new idea: once you can reliably retrieve everything, you can validate coverage (100% fetched), detect false “missing” caused by hidden states, and harden workflows so the problem doesn’t come back during scaling—especially in Notion Troubleshooting situations that involve retries and bad requests.

Notion logo

What does “Notion pagination missing records” mean in database queries?

“Notion pagination missing records” means your integration is reading only a partial page of results from a Notion list endpoint (often capped at 100 per call), then stopping before it follows the cursor to fetch the remaining pages. (developers.notion.com)

To better understand the symptom, separate it into two different realities:

  1. Records exist in the database but your API client didn’t retrieve them yet (classic pagination bug).
  2. Records are genuinely not returned due to filters, sorts, permissions, or endpoint/version mismatches (not a pagination problem).

Here’s a quick “symptom → likely cause” table to anchor the rest of the guide (this table lists common signals you can observe and the most likely root cause category):

Symptom you see Most likely cause category
You always get exactly 100 (or 10) items Pagination not implemented or page_size left default
You get 100, then 100, then stop early Cursor loop bug (start_cursor not updated, loop condition wrong)
Counts vary between runs Sorting not stable or data changed mid-sync
Some users see fewer items than others Permissions / sharing not configured for the integration
Items missing only when filters/sorts used Filter/sort excludes them or uses a property that’s empty/changed

Now let’s pin down the two most common causes first.

Why do Notion API queries return only the first page?

Because Notion list responses are cursor-paginated: the response includes has_more and (when has_more is true) a next_cursor you must pass back as start_cursor to continue. (developers.notion.com)

Even if you set page_size to 100, you should still assume the API can return fewer than 100 results and still have more pages. The only reliable “keep going” signal is has_more === true. (developers.notion.com)

In practice, the “only first page” issue happens when developers:

  • Forget to implement the loop entirely (they call query once and believe the dataset is complete).
  • Implement the loop but never update start_cursor with the newest next_cursor.
  • Break early when next_cursor is present but they mis-handle null/undefined.
  • Re-send the entire previous response body back to Notion (bloated request body), triggering errors like payload too large—then they swallow the exception and silently stop. (stackoverflow.com)

This is also where Notion Troubleshooting habits matter: if your code catches errors and continues without alerting, you’ll “finish” with missing data and never know it.

Cursor-based pagination illustrated

How can you confirm records are missing vs filtered or hidden?

You can confirm “missing vs filtered” by comparing three views of truth:

  1. Notion UI view with no filters (baseline count and presence).
  2. API query with no filter/sort (should match UI much more closely, subject to pagination).
  3. API query with your real filter/sort (the one used in production).

More specifically, do this checklist:

  • Remove filters and sorts in your API request temporarily. If “missing” disappears, the problem is not pagination—it’s logic. (developers.notion.com)
  • Verify the integration has access: a database must be shared with the integration; otherwise you’ll see access errors or incomplete visibility. (developers.notion.com)
  • Check if you’re querying the right thing: Notion’s API has evolved (databases vs data sources in newer versions). If you’re mixing concepts, you can “miss” items simply because you’re not querying the intended container. (developers.notion.com)

Once you’ve isolated that pagination is truly the culprit, implementation quality becomes everything.

How do you implement cursor-based pagination correctly to avoid missing records?

Implement cursor pagination by looping until has_more is false, and on each iteration passing the previous response’s next_cursor back as start_cursor—with a stable sort to minimize duplicates or skips when data changes mid-run. (developers.notion.com)

Next, let’s break the implementation into the three parts most teams get wrong: (1) cursor handling, (2) loop safety, and (3) page sizing.

What are next_cursor and has_more, and how should you use them?

has_more tells you whether there are additional pages, and next_cursor is an opaque token you pass as start_cursor to fetch the next page. Notion only includes next_cursor when has_more is true. (developers.notion.com)

Rules that prevent subtle bugs:

  • Treat next_cursor as opaque: don’t parse it, don’t modify it. (developers.notion.com)
  • Never assume you’re done because you received fewer than 100 items—use has_more. (developers.notion.com)
  • Always store the latest next_cursor returned (don’t keep the first one).

If you’re debugging a workflow that sometimes stops early, log this trio on every page:

  • page_index
  • results.length
  • has_more
  • next_cursor (or a hashed version if you don’t want to log full tokens)

This makes Notion Troubleshooting drastically faster, because you can see exactly which page ended the run.

What is the safest pagination loop pattern for scripts and automations?

The safest loop pattern is: request → append → update cursor → stop when has_more is false—and keep your request body minimal after the first page. (developers.notion.com)

A robust pattern (conceptually) looks like this:

  • Initialize:
    • cursor = undefined
    • allResults = []
  • Repeat:
    • Build request with:
      • page_size (up to 100)
      • start_cursor only if cursor exists
      • same filter/sort every time
    • Call API
    • Append resp.results to allResults
    • If resp.has_more is true:
      • cursor = resp.next_cursor
    • Else stop

Why “minimal request body” matters: one common failure mode is accidentally sending back the entire accumulated payload (including previous results) as the next request body, which can trigger request-size errors and make the script stop after a few pages. (stackoverflow.com)

If you also see notion webhook 400 bad request in logs during paginated runs, it’s often because the automation platform or your code is sending malformed JSON, wrong content-type headers, or an unexpectedly large body after concatenation. Treat that as a reliability bug, not a one-off. (stackoverflow.com)

What page_size should you choose, and what are the trade-offs?

Use page_size = 100 for efficiency (it’s the maximum), but design as if the response can still return fewer than your requested size. (developers.notion.com)

Trade-offs to consider:

  • Bigger page size (100)
    • Pros: fewer requests, faster total runtime, lower overhead.
    • Cons: larger payloads, heavier memory usage, higher chance a single request bumps into platform limits.
  • Smaller page size (25–50)
    • Pros: smoother for automations with tight memory/time limits; easier retries.
    • Cons: more requests; can amplify rate-limit pressure.

If your automation sometimes experiences notion tasks delayed queue backlog, smaller page sizes combined with resumable cursors can reduce retry storms, because each unit of work becomes lighter and less likely to time out before completion.

Is pagination the only reason Notion records go “missing”?

No—Notion pagination is a common cause of missing records, but filters/sorts, permissions, and using the wrong endpoint/version are equally frequent reasons, and they can look identical from the outside. (developers.notion.com)

Then, instead of guessing, validate each category systematically.

How do filters, sorts, and archived pages affect “missing” results?

Filters and sorts can exclude items in ways that are easy to overlook, especially when properties are empty, renamed, or calculated.

Common “false missing” scenarios:

  • Archived/trashed pages: depending on how your database is managed, items may not appear in certain queries or views.
  • Sorts using unstable properties: if you sort by a property that changes during the run (like “Last edited time”), your pagination window can shift and produce duplicates or apparent gaps.
  • Filters on computed properties: rollups/formulas may not behave as you expect at scale or in edge cases (especially with relation-heavy schemas).

If the missing set is consistent (the same IDs missing every time), filters/sorts are more likely than pagination. If the missing set changes run-to-run, unstable sorting or mid-run edits become prime suspects.

How do Notion permissions and sharing settings cause incomplete datasets?

If the integration token cannot “see” a page, it cannot retrieve it—period. Notion requires that the database be shared with the integration, and insufficient capabilities can produce access errors (403) or not-found responses (404) depending on the situation. (developers.notion.com)

This is why two users can compare counts and believe pagination is broken, when the reality is:

  • User A shared the database with the integration (full visibility).
  • User B did not (partial or zero visibility).
  • Or the integration has read scope but the content was moved into a location it can’t access.

In Notion Troubleshooting, always confirm access first, because you can perfectly paginate an empty or partial permission view and still be wrong.

What endpoint or Notion API version mistakes commonly lead to missing data?

A frequent source of confusion is using deprecated vs newer concepts (databases vs data sources) or calling a different endpoint than your mental model assumes. Notion’s docs explicitly note API version changes and the split between databases and data sources in newer versions. (developers.notion.com)

If you migrate code or copy examples from different eras, you can end up with:

  • A query that technically succeeds,
  • but returns a different scope than intended,
  • which you then interpret as “missing records.”

When you suspect version mismatch, confirm the endpoint you’re using matches the API version and object model you’re targeting, then re-check pagination.

How do you prevent pagination gaps and duplicates during incremental syncs?

Prevent gaps and duplicates by combining cursor pagination with a stable sort, an idempotent store, and a checkpoint strategy that can resume from a known cursor or timestamp safely. (developers.notion.com)

Next, we’ll translate that into practical rules you can apply immediately.

What is the best sorting strategy for consistent pagination?

The best strategy is to sort by a field that is:

  • Consistent (doesn’t change during the sync), and
  • Monotonic or at least stable enough to reduce reshuffling.

In many systems, “created time” is more stable than “last edited time.” In Notion, choose the most stable available sort for your use case, and avoid volatile sorts during full backfills.

Why this matters (big picture): when lists are large and dynamic, retrieving data in a stable order reduces the chance that the item order changes between page 1 and page 2, which is a classic way to create duplicates or make items appear missing.

According to a study by Aalto University from the Computer Science programme, in 2023, performance-oriented data retrieval work comparing list retrieval strategies discusses pagination approaches (including paging-related methods) as part of improving large-result fetching in web environments. (aaltodoc.aalto.fi)

When should you do a full export vs incremental sync?

Do a full export when you need a clean baseline (first-time sync, schema changes, historical backfill). Switch to incremental sync when you can trust your baseline and only need changes since a checkpoint.

A practical decision rule:

  • Full export if:
    • you have never validated “100% fetched”
    • you changed filters/sorts/properties used by your integration
    • you suspect historical gaps
  • Incremental sync if:
    • you store IDs and can dedupe
    • you track a checkpoint (cursor or timestamp)
    • your workflow can retry without reprocessing everything

This is also where operational issues show up: if your platform frequently hits timeouts and you see notion tasks delayed queue backlog, it’s often safer to run incremental batches (more frequent, smaller windows) than to repeatedly attempt giant backfills.

Contextual Border (macro → micro): Up to this point, you’ve focused on correctly retrieving all records (macro correctness). Next, we shift into micro-level reliability: validation, automation limits, hidden failure modes, and advanced Notion Troubleshooting patterns that prevent silent data loss.

How can you validate that you retrieved 100% of records from Notion?

You can validate 100% retrieval by combining pagination completeness checks (has_more loop ends naturally), count reconciliation against a trusted baseline, and ID-based deduplication to prove coverage without relying on “looks right” sampling. (developers.notion.com)

Then, treat validation as part of your pipeline—not an afterthought—so you catch gaps immediately.

What are the best integrity checks after pagination?

Use at least three integrity checks:

  1. Loop completeness check
  2. Distinct ID count
    • Put every page ID into a set; compare:
      • total_fetched
      • unique_ids
      • duplicates_detected
  3. Spot-check by deterministic sampling
    • Sample pages by a stable criterion:
      • earliest created pages
      • latest created pages
      • specific known IDs from the UI
    • Verify those IDs appear in your fetched set.

If these checks disagree, stop the pipeline and log a diagnostic bundle (cursor values, request parameters, last successful page index). This is the difference between “we sometimes miss records” and a workflow you can trust.

How do automation platforms cap results, and how does that mimic missing records?

Many automation tools apply hidden caps: maximum runtime, maximum memory, or maximum items returned per step. That can mimic pagination issues because:

  • The platform stops the run mid-loop.
  • The workflow retries from the beginning.
  • You end up with duplicates, partial coverage, or both.

This is also where you’ll see confusing secondary errors like notion webhook 400 bad request (bad retry payloads) or notion trigger not firing (the next stage never runs because the previous stage timed out). When debugging, don’t only examine Notion—examine the platform’s execution limits and retry semantics.

If you want a practical demonstration of a working multi-page query pattern, Thomas Frank’s Notion API pagination example shows a loop that continues while more pages exist and passes start_cursor each time. (thomasfrank.notion.site)

How do you handle duplicates and avoid “gaps” when re-running syncs?

Handle duplicates and avoid gaps with idempotency:

  • Store each page by its unique ID as the primary key.
  • Upsert pages (insert if new, update if exists).
  • Track progress:
    • last successful cursor (for a backfill run), or
    • a “last synced at” timestamp plus a safe overlap window.

A simple, resilient pattern is:

  • Always re-fetch a small overlap window (e.g., last 5–15 minutes of changes),
  • Deduplicate by ID,
  • Accept that overlaps cost a bit more compute but buy you reliability.

If you skip idempotency, any rerun can create inconsistent results that feel like “missing records” because the dataset you’re inspecting is not the same dataset your pipeline is storing.

What’s the difference between missing records and incomplete property data?

“Missing records” means the page itself is absent from results; “incomplete property data” means the page is returned, but some properties are empty, truncated, or not expanded the way you expect.

Common causes of incomplete property data include:

  • You filtered properties out (intentionally or accidentally).
  • Some property types require extra calls (e.g., retrieving specific property items depending on your approach).
  • Rollups/formulas may have limitations in certain relation-heavy designs. (developers.notion.com)

In Notion Troubleshooting, it’s crucial not to mix these two, because the fix is different:

  • Missing page → pagination/permissions/filters/endpoints.
  • Missing property values → schema, property retrieval strategy, or formatting/logic issues.

Offset-based pagination illustrated

Leave a Reply

Your email address will not be published. Required fields are marked *