Best Practices — Colocated Permissions Index

This guide covers best practices for integrating with the FGA Colocated Permissions Index (Search With Permissions). It is organized into five sections:

0. Security
1. Idempotency and Deduplication Patterns
2. Leveraging Freshness for Authorization Decisions
3. Migrating Between Authorization Models
4. Querying Permissions in Data Providers

0. Security

When you use a Colocated Permissions Index, you are storing authorization data in your own environment. That means the local permissions_index table, cache, warehouse table, or derived dataset should be treated as sensitive application data and secured accordingly.

In particular, avoid using PII in your IDs. Values such as subject_id, object_id, or related business identifiers are often copied into analytics systems, logs, exports, dashboards, and debugging workflows. If those IDs contain email addresses, employee numbers, customer names, or other directly identifying data, you are spreading PII into every system that consumes the index.

Prefer opaque, stable identifiers such as internal UUIDs or synthetic IDs, and keep any mapping back to user-facing or sensitive data in a separate, securely protected system.

At minimum, secure your permissions table with the same care you apply to other sensitive operational data:

• restrict read and write access using least-privilege roles,
• encrypt data at rest and in transit,
• avoid broad analyst or developer access unless it is required,
• audit access to the table and any downstream derived datasets,
• apply retention and deletion policies appropriate for your environment,
• review logs, exports, and BI tooling to ensure that permission data is not unintentionally exposed.

If you version by index_id or copy permissions into multiple destinations, apply the same controls everywhere the data is replicated. A colocated index improves query performance, but it also creates additional copies of authorization data that you are responsible for protecting.

1. Idempotency and Deduplication Patterns

Delivery Guarantees

The Permissions Index Read Expansions API provides at-least-once, ordered delivery. This means:

• At-least-once: Every expansion event will be delivered. No permission change is silently dropped. However, the same event may be delivered more than once, particularly after a consumer reconnects and a previous from continuation token was sent.
• Ordered: Events are delivered in the order they were computed. Within a single stream, you will never see a delete for an expansion that was never added, and you will never see events arrive out of order they were processed.

Because delivery is at-least-once (not exactly-once), your consumer must be prepared to read duplicate events safely.

Why Duplicates Happen

Duplicates occur when a consumer disconnects and reconnects to the Expansions stream. The consumer resumes from its last saved from continuation token, and there is a window during which events delivered after the token was saved but before the disconnect may be re-delivered.

Timeline:
  ┌─ Event A (from: c1) ──── consumer processes, saves token c1
  ├─ Event B (from: c2) ──── consumer processes, but crashes before saving c2
  ├─ Event C (from: c3) ──── never received
  │
  └─ Consumer reconnects with ?from=c1
       ├─ Event B (from: c2) ──── DUPLICATE (already processed)
       └─ Event C (from: c3) ──── new event

Continuation Token Management

The from continuation token is your resume point for the Expansions stream. Proper token management determines where a consumer restarts after disconnects, crashes, or deploys.

Safe Replay

The safest approach is to save the from continuation token in the same transaction as the expansion update. If your process crashes after receiving an event but before persisting its effects, restarting from the last committed token causes that event to be delivered again. Because your consumer is idempotent, this is what makes replay safe.

The inverse is dangerous: if you save a newer token before the corresponding event is durably processed, a crash can cause that event to be skipped permanently on restart.

Safe pattern:
    receive event (from=c2)
    apply event durably (idempotent upsert)
    save last_continuation_token = c2

Unsafe pattern:
    receive event (from=c2)
    save last_continuation_token = c2
    crash before applying event

Save Only the Most Recent Processed from Continuation Token

In practice, that means:

• Do not save a from continuation token as soon as you receive an event.
• Instead, save the most recent from continuation token only after the event has been written to your database, queue or internal stream.
• If you process events in batches, save the from continuation token from the last successfully committed event in the batch.

On Restart

On restart, read the continuation token and pass it as the ?from= parameter:

GET /stores/{storeID}/indexes/{indexID}/expansions?from=token_xyz

When the Stream Closes

After about 5 minutes of streaming, your consumer may receive a close event like this:

{"result":{"closed":{"reason":"STREAM_CLOSED_REASON_CONNECTION_LIFETIME_EXCEEDED"}}}

This is expected behavior, not an error. It means the current stream connection reached its maximum lifetime and should be replaced with a new connection.

When this happens:

1. Treat the close event as a normal reconnect signal.
2. Read the most recent durably saved from continuation token.
3. Open a new Read Expansions request using that token.
4. Continue processing events as normal.

on_closed(event):
    if event.reason == "STREAM_CLOSED_REASON_CONNECTION_LIFETIME_EXCEEDED":
        token = load_last_continuation_token()
        reconnect_with(token)

Do not clear local state, rebuild the index, or treat this as data loss. If you have been saving the most recent processed from continuation token correctly, the new connection will resume from the right point. You may see some replayed events after reconnect, which is expected and safe because the consumer is idempotent.

If reconnecting fails, retry with backoff. But the normal path should be: close event → reconnect with the saved from continuation token → continue streaming.

What Happens if You Lose the Continuation Token?

If you lose or do not have a continuation token, omit the ?from= parameter. The stream will replay all expansion events from the beginning of the index. This is safe because the consumer is idempotent — replaying all events will rebuild the same final state.

This is effectively a "full reindex" operation. For large indexes, it may take time, but the result will be correct.

Designing an Idempotent Consumer

Goal: replaying the same expansion event twice should produce the same result as applying it once.

Note: Idempotency must be enforced by any consumer of an at-least-once stream that manages materialized state.

• If you run a single intermediary consumer that reads the Expansions stream and republishes events to your own internal event stream, idempotency must be enforced in that intermediary and in any downstream consumer that materializes its own state.
• If you run multiple independent consumers that each read the Expansions stream directly, each consumer must independently enforce idempotency (while also maintaining its own from continuation token).

Idempotency works together with continuation token management: idempotency makes replay safe, and continuation token management determines where replay resumes.

Expansion Event Fields Are Naturally Idempotent

Each expansion event represents a single permission: a subject–relation–object tuple. The fields (subject_type, subject_id, subject_relation, relation, object_type, object_id) together form a natural key that uniquely identifies that permission. This makes idempotency straightforward:

• EXPANSION_OPERATION_INSERT: UPSERT on the natural key. If the row already exists, the write overwrites it with the same data — same result either way.
• EXPANSION_OPERATION_DELETE: DELETE WHERE on the natural key. If the row doesn't exist, zero rows are affected — no error.

That's all your consumer needs to do to keep your colocated Permissions Index state up to date. Replaying the same event twice produces the same result as applying it once. Even a full replay from the beginning of the stream rebuilds in the same final state.

Database Pattern: UPSERT on the Natural Key

Define a permissions_index table with the expansion's natural key as the primary key:

CREATE TABLE permissions_index (
    subject_type      TEXT      NOT NULL,   -- e.g., 'user'
    subject_id        TEXT      NOT NULL,   -- e.g., 'alice'
    subject_relation  TEXT,                 -- e.g., 'member' (null for direct users)
    relation          TEXT      NOT NULL,   -- e.g., 'can_view'
    object_type       TEXT      NOT NULL,   -- e.g., 'document'
    object_id         TEXT      NOT NULL,   -- e.g., '3-1'
    tuple_written_at  TIMESTAMP,            -- when the original tuple was written to FGA
    updated_at        TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,

    PRIMARY KEY (subject_type, subject_id, subject_relation, relation, object_type, object_id)
);

Why is subject_relation in the key: When a userset like group:engineering#member is assigned can_view on a document, each resulting expansion includes subject_relation = "member". A direct user assignment to the same document has subject_relation = "". These are distinct permissions and must be separate rows. Without subject_relation in the key, one would silently overwrite the other.

Then process each event with two operations:

• EXPANSION_OPERATION_INSERT → upsert with conflict resolution
• EXPANSION_OPERATION_DELETE → conditional delete

This is idempotent because:

• Duplicate INSERT: The SQL ON CONFLICT OR ON DUPLICATE clause overwrites the existing row with the same data. The result is identical.
• Duplicate DELETE: Deleting a row that does not exist is a no-op (0 rows affected). No error.

Replaying all events from the beginning rebuilds the same final state, because every INSERT is an upsert and every DELETE is conditional.

The safest approach is to save the from continuation token in the same transaction as the expansion update. This ensures that if the process crashes, the token and the data remain consistent.

BEGIN;

-- Apply the expansion (idempotent upsert)
-- Save the continuation token 
COMMIT;

On restart, read the continuation token and pass it as the ?from= parameter:

GET /stores/{storeID}/indexes/{indexID}/expansions?from=token_xyz

In-Memory Pattern: Sets

If you are building an in-memory permissions index (for a caching layer, a real-time search proxy, etc.), use set-based data structures where add/remove operations are naturally idempotent.

// Pseudocode
permissions = Map<String, Set<String>>
// key: "user:alice" or "team:product#member"
// value: {"document:1", "document:2"}

on_expansion_event(event):
    subject = event.subject_type + ":" + event.subject_id
    if event.subject_relation != "":
        subject += "#" + event.subject_relation
    object = event.object_type + ":" + event.object_id

    if event.operation == "EXPANSION_OPERATION_INSERT":
        permissions[subject].add(object)
    else:
        permissions[subject].remove(object)
        if permissions[subject].is_empty():
            permissions.delete(subject)

Adding an element to a set that already contains it is a no-op. Removing an element from a set that doesn't contain it is a no-op. Idempotency comes for free.

Deduplication in an Intermediary Consumer

When you run a single intermediary consumer — a process that reads the Expansions stream and republishes events to your own internal queue, Kafka topic, or event bus — you introduce a second delivery boundary where duplicates can appear. Both the intermediary and any downstream consumer that materializes state must enforce idempotency independently.

The Two Boundaries

FGA Expansions stream
       │  (at-least-once)
       ▼
Intermediary consumer        ← owns: from continuation token, deduplication before publish
       │  (your internal stream — also at-least-once unless you guarantee exactly-once)
       ▼
Downstream consumer(s)       ← owns: idempotent writes to the permissions table

Each boundary can produce its own duplicates:

1. FGA → intermediary: The Expansions stream is at-least-once. The intermediary may receive the same event more than once after a reconnect or restart.
2. Intermediary → downstream: Unless your internal queue provides exactly-once delivery, the same event can be delivered to the downstream consumer more than once.

How to Deduplicate in the Intermediary

Use the full event identity as the deduplication key — the natural key fields plus operation and tuple_written_at:

(subject_type, subject_id, subject_relation, relation, object_type, object_id, operation, tuple_written_at)

Why include tuple_written_at?

FGA stream replays emit the exact same event bytes, including the same tuple_written_at, so a match on the full event is a true duplicate. Without it, the dedup key alone can cause false suppression: if a permission is revoked and immediately re-granted on the same tuple, both events share the same natural key + operation. Including tuple_written_at distinguishes them because the re-grant produces a newer timestamp.

This is only applicable for handling duplicate events in the intermediary consumer. Do not include tuple_written_at in the final, downstream permissions table primary key.

Because duplicates from a reconnect arrive seconds apart — not minutes — a short TTL is sufficient. A 10–30 second sliding window keeps memory bounded without needing to track every event the intermediary has ever seen.

Always advance the from continuation token, even when the event is a duplicate and you choose not to republish it. Skipping the token update can cause the stream to replay a large range of events on the next restart.

Token Ownership

• The intermediary owns the from continuation token for the FGA Expansions stream. Save it in the same transaction or write as any state update (see Continuation Token Management).
• Each downstream consumer maintains its own offset or cursor within your internal queue. They do not interact with the FGA from continuation token directly.

Downstream Consumers Still Need Idempotency

Even with deduplication in the intermediary, downstream consumers should still apply idempotent writes. Your internal queue may redeliver events due to its own at-least-once guarantees, and the intermediary's deduplication window cannot eliminate every duplicate across all failure scenarios. The UPSERT-on-natural-key and DELETE-WHERE patterns described above remain the correct approach for any consumer that materializes state.

Maintaining Event Order in High-Throughput Scenarios

When processing expansion events in rapid succession, it's critical that your consumer maintains the exact order in which events were received from the stream. Standard timestamp precision (milliseconds) may be insufficient to differentiate events that arrive within the same millisecond.

Problem: Timestamp Collisions in Rapid Event Streams

When multiple events arrive within the same millisecond, assigning timestamps using new Date() or Date.now() produces identical values. This can lead to:

1. Display order issues: Events may appear out of order in UI feeds or logs
2. Sort instability: Sorting by timestamp alone cannot guarantee the receive order
3. Batching artifacts: Framework-level state batching (e.g., React) may reorder events with identical timestamps

Pattern: Monotonic Timestamps with Sub-Millisecond Precision

Ensure each event receives a unique, monotonically increasing timestamp by tracking the last assigned timestamp and incrementing by a small delta when necessary:

// JavaScript/TypeScript example
let eventCounter = 0;
let lastTimestamp = 0;

function getMonotonicTimestamp() {
  const now = Date.now();
  const timestamp = now > lastTimestamp ? now : lastTimestamp + 0.001;
  lastTimestamp = timestamp;
  return new Date(timestamp);
}

function processExpansionEvent(event) {
  const parsedEvent = {
    id: `evt-${++eventCounter}`,
    timestamp: getMonotonicTimestamp(),  // Guaranteed unique and ordered
    operation: event.operation === "EXPANSION_OPERATION_DELETE" ? "remove" : "add",
    user: `${event.subject_type}:${event.subject_id}`,
    relation: event.relation,
    object: `${event.object_type}:${event.object_id}`,
    tuple_written_at: event.tuple_written_at,
  };

  // Apply to your index...
}

This pattern ensures:

• Uniqueness: No two events receive the same timestamp value
• Monotonicity: Timestamps strictly increase in the order events are received
• Sub-millisecond precision: Events within the same millisecond are differentiated by fractional milliseconds (0.001ms = 1 microsecond)

Some platforms provide high-resolution timers with microsecond precision. Use these when available.

---

2. Leveraging Freshness for Authorization Decisions

A permissions index is eventually consistent with FGA. After a tuple is written to FGA, there is a brief period (typically seconds) where your colocated permissions index has not yet received the corresponding expansion events.

This means FGA is always the source of truth. Your colocated permissions index is an optimized read replica for search and filtering use cases. You can calculate the freshness of your index at any point in time.

What is Freshness?

Freshness indicates how up-to-date your colocated permissions index is relative to FGA. It answers the question: "How fresh is my index right now?"

Freshness Formula

Each expansion event includes a tuple_written_at field — the time the original tuple was written to FGA, for example:

{
    "result": { 
        "event": {
            "from":"FmhrVnpxNHBsUWgyNHc3NXh0R2NIX2eEAb8whDm7Dxliq5tblJNxfn9cud3OFdTAxweqAUtHJj2udjfufHZY4SkvkzJS7kEydaOxkDfhcmd1p0P4-AlHfMksAGaGjJvrxPzV8fLPgJmzrftqn3twPKlZKjDhhQyEg230CO2CR0l_947bOngyJqe1ZLUh3XErvOwHp81GlONH1eAFQg==",
            "subject_type":"team",
            "subject_id":"engineering",
            "subject_relation":"member",
            "object_type":"document",
            "object_id":"3-1",
            "relation":"can_view",
            "operation":"EXPANSION_OPERATION_INSERT",
            "tuple_written_at":"2026-04-03T19:57:51Z"
        }
    }
}

The stream also emits a freshness event every 2 seconds of expansion inactivity with an as_fresh_as timestamp, for example:

{
    "result": {
        "freshness": {
            "as_fresh_as":"2026-04-15T14:12:09.029627Z"
        }
    }
}

To compute freshness, use the most recent of these two timestamps:

freshness = now() - max(latest_tuple_written_at, latest_as_fresh_as)

Your consumer should track both timestamps and compute and save freshness as an operational metric for making authorization decisions with your colocated permissions index.

Monitoring Freshness

To monitor whether the index is fresh, treat freshness as a consumer-side operational metric. In practice, your consumer should keep track of the most recent tuple_written_at and the most recent as_fresh_as timestamps. Each time either value advances, recompute freshness as now() - max(latest_tuple_written_at, latest_as_fresh_as) and publish that result as a metric. This gives you a continuously updated view of how current the index is, including during quiet periods when no expansion events are flowing.

Important: Do not determine index freshness by comparing tuple_written_at to NOW() in a database query. A tuple_written_at from 3 days ago does not mean the expansion is stale — it means the permission was granted 3 days ago, and your index has it. The tuple_written_at column is useful for audit purposes ("when was this permission originally granted?") but not for measuring whether your index is current.

Evaluating Freshness on Every Request

If freshness is used to determine whether the permissions index is usable, then every request that relies on the index should check the latest freshness state before trusting index results. The consumer should continuously maintain the latest freshness state, and each request should compare that current value to an application threshold, such as “use the index only if freshness is less than 10 seconds”.

With this model, the process typically looks like this:

1. Define MAX_ACCEPTABLE_FRESHNESS for your application. e.g. 300 seconds (5 minutes)
2. Receive either tuple_written_at or as_fresh_as timestamp.
3. Update permissions_index_freshness_state. This can live in-memory, in a shared key-value store, or in a single database table row.
   • latest_tuple_written_at=tuple_written_at
   • latest_as_fresh_as=as_fresh_as
   • freshness_seconds=now() - max(latest_tuple_written_at, latest_as_fresh_as)
   • updated_at=now()
4. Read the current freshness state.
5. Compare it to the application threshold. (freshness_state.freshness_seconds > MAX_ACCEPTABLE_FRESHNESS)
   • If freshness is acceptable, use the permissions index.
   • If freshness is above the threshold or the freshness state is unavailable, fall back to a higher-consistency path such as FGA Check/ListObjects, or fail closed.

Heartbeats vs Freshness

Under normal conditions, your consumer will receive expansion or freshness events, but should not expect heartbeats. Heartbeats are only emitted when the server is mid-computation on an expansion that takes longer than 30 seconds — the server cannot yet report a freshness timestamp and no new expansion events exist yet, so it sends a heartbeat instead. Your consumer should treat heartbeats as signals of connection health.

{
    "result": {
        "heartbeat": {}
    }
}

When to Use the Colocated Permissions Index vs. FGA

Use Case	Recommended Approach	Why
Search results filtering	Permissions Index	Filtering thousands of results requires low-latency lookups. The index is optimized for this. A brief delay (seconds) before a newly-permitted document appears in search is acceptable.
Paginated list views	Permissions Index	Same as search: you need to JOIN or filter large sets efficiently.
Analytics / reporting	Permissions Index	Batch queries over permissions data for BI dashboards benefit from colocated storage.
Single-resource access gate	FGA Check API	When a user clicks on a specific document, verify access against FGA directly. This gives you real-time correctness at the moment of access.
Sensitive operations (delete, share, export)	FGA Check API	Security-critical actions should always be authorized against the source of truth.

General rule: Use the Permissions Index for read-heavy, list-based operations where eventual consistency is acceptable within your freshness threshold. Use the FGA Check API for write/action checks where real-time correctness is required.

Combining the Permissions Index with FGA Check

A common pattern is to use the Permissions Index for initial filtering and FGA Check for confirmation:

-- Step 1: Query using the Permissions Index and business data together
results = SELECT d.* FROM documents d
                    JOIN permissions_index p ON d.id = p.object_id
                    WHERE p.subject_id = 'alice'
                        AND p.relation = 'can_view'
                        AND p.object_type = 'document'
                        AND d.title LIKE '%quarterly%'
          ORDER BY d.updated_at DESC
          LIMIT 25

// Step 2: When user clicks a result, verify with FGA Check
on_document_open(doc_id):
    allowed = fga.check(user="user:alice", relation="can_view", object="document:{doc_id}")
    if not allowed:
        show_error("Access has been revoked. Please refresh your search.")

This gives you the best of both worlds: the performance of the colocated index for search, and the real-time correctness of FGA Check for access checks.

Fallback Strategy When Freshness is Too High

If freshness exceeds your acceptable MAX_ACCEPTABLE_FRESHNESS threshold — whether due to a consumer disconnection, a large backpressure of expansions, or infrastructure issues — you need a fallback plan to ensure authorization decisions remain correct.

Before falling back, monitor three signals:

1. Event freshness: If incoming expansion events have a tuple_written_at that is significantly behind, the pipeline is lagging under active writes. This typically occurs after index creation or re-index events. The index must process all historical tuples written to FGA, and if the first tuple write for that store occurred 5 years ago, then the tuple_written_at timestamp will be 5 years ago making freshness_seconds be 157,784,760 seconds.
2. Index freshness: If the latest freshness event shows high latency (e.g., > 20 seconds), the index is lagging.
3. Connection health: If both freshness events and heartbeats have stopped, the consumer may be disconnected from the stream entirely.

In any scenario, if freshness exceeds your acceptable MAX_ACCEPTABLE_FRESHNESS threshold, trigger your fallback plan.

Your fallback plan could be:

• Show a staleness warning to the user (e.g., "Results may not reflect recent permission changes")
• Cancel the request and surface an error if freshness exceeds a critical threshold
• Route authorization decisions through FGA Check or ListObjects (if available)
• Using FGA BatchCheck on the top N results only, as a compromise between correctness and performance
• Route to a backup internal authorization system with less granularity
• Or something else

Returning to Normal

Once the consumer processes all historical events, or finishes computing high expansion events, or reconnects, freshness will return to acceptable levels. At that point, it is safe to resume using the colocated permissions index for authorization decisions.

Handling the "New Enemy" Problem

The "new enemy" problem occurs when access is revoked but the colocated index hasn't processed the revocation yet. During this window, the index incorrectly says the user still has access.

Mitigation strategies:

1. Use FGA Check for sensitive operations: Always gate sensitive actions through FGA Check. The Check API reflects revocations in near real-time.
2. Monitor freshness and alert on lag: If your index falls behind by more than your SLA (e.g., 20 seconds), alert your operations team and consider triggering the fallback strategy.
3. Accept the tradeoff for search: For search result filtering, showing non-sensitive details like Title, Date, or Owner for recently-revoked document in results for a few extra seconds is a reasonable tradeoff. When the user clicks the document, the FGA Check will deny access.

---

3. Migrating Between Authorization Models

When a new model is created in FGA, it needs to be validated for compatibility with existing permissions indexes.

Migrating your authorization model is a two-part process:

1. Create a new authorization model
2. Decide whether that new model can be attached to the existing index.

The compatibility rule is the Indexable Path. If the new model preserves the same indexable path, it is compatible with the existing index. If the indexable path changes, the model is incompatible and you must create a new index and migrate to it.

The Core Model-Change Flow

At a high level, the process is:

1. Create a new authorization model
2. Compare its Indexable Path to the path used by the current index
3. If the path is unchanged, attach the new model ID to the existing index
4. If the path changed, create a new index and migrate application traffic to it
5. Use the appropriate model ID when calling the /expansions endpoint

Compatible Model Changes

A new model is compatible when its Indexable Path is unchanged.

In that case:

• The existing index remains valid
• The existing permissions_index data remains valid
• The flattened expansions do not change shape
• No re-index is needed
• Your application can continue to use the existing table

Operationally, the main step is to attach the new model ID to the existing index and then use that model ID when opening the /expansions stream. Once attached, passing that model ID to /expansions guarantees that the stream is producing expansions that are compatible with that model.

Note: For Developer Preview, this step is manual. Contact your Auth0 FGA Account Executive (AE) or Technical Account Manager (TAM) for assistance. In the future, we may automate this step for compatible changes.

on_new_model_created(new_model_id):
    if indexable_path(new_model_id) == indexable_path(current_index):
        attach_model_to_index(index_id=current_index.id, model_id=new_model_id)
        current_model_id = new_model_id

open_expansions_stream(index_id=current_index.id, model_id=current_model_id)

For compatible changes, your consumer logic does not need to change. Keep processing insert/delete events normally and keep using the same continuation-token, idempotency, and freshness rules.

Incompatible Model Changes

A new model is incompatible when its Indexable Path changes.

Typical examples include:

• Removing or renaming an indexed relation
• Changing the chain of relations that determines the indexed permission
• Changing the index definition so the current flattened rows are no longer the right materialization

In that case:

• The existing index is no longer a valid materialization for the new model
• The existing permissions_index table cannot simply be reinterpreted under the new model
• A new index must be created
• Creating that new index triggers a re-index
• The re-index may take a long time
• The re-index will emit new expansions, and those expansions are billable to the customer

Consumers should treat unattached or incompatible model IDs as invalid and expect the /expansions streaming request to fail.

What If a permissions_index Table Already Exists?

If you already have a populated permissions_index table, the safest approach depends on whether the model change is compatible.

Option 1: Reuse the Existing Table for Compatible Changes

If the model is compatible, keep the existing table and keep the existing data. Attach the new model ID to the existing index and continue streaming into the same table.

This is the simplest path because:

• The stored expansions remain valid
• No table rewrite is required
• No application cutover is required

Option 2: Shadow Table for Incompatible Changes

If the model is incompatible, the best practice is usually to build a new table for the new index, for example:

• permissions_index_v1 for the old index
• permissions_index_v2 for the new index

or equivalent names based on index purpose.

This lets you:

• Keep the old application path running
• Build the new index in parallel
• Validate the new shape before cutover
• Switch reads atomically later

This is usually safer than mutating one shared table in place while a re-index is underway.

Option 3: Shared Table Versioned by index_id

If you prefer a single physical table, version rows by index_id and include index_id in the primary key and query filters.

For example:

CREATE TABLE permissions_index (
    index_id           TEXT      NOT NULL,
    subject_type       TEXT      NOT NULL,
    subject_id         TEXT      NOT NULL,
    subject_relation   TEXT,
    relation           TEXT      NOT NULL,
    object_type        TEXT      NOT NULL,
    object_id          TEXT      NOT NULL,
    tuple_written_at   TIMESTAMP,
    updated_at         TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (index_id, subject_type, subject_id, subject_relation, relation, object_type, object_id)
);

This can work well if your platform already expects multiple active index versions. The tradeoff is that every query and every maintenance operation must now be index-aware. For most application cutovers, separate shadow tables are easier to reason about.

Recommended Cutover Process

For incompatible changes, treat cutover as a controlled migration.

1. Create the new authorization model
2. Confirm the Indexable Path changed
3. Create a new index
4. Create a new destination table or table namespace for that index
5. Start a new consumer for the new index
6. Wait for the re-index to complete and for freshness to reach your acceptable threshold
7. Validate that application queries against the new table return the expected results
8. Switch application reads from the old index/table to the new index/table
9. Monitor for a stabilization period
10. Retire the old consumer, old index, and old table when safe

Application Cutover Patterns

There are two common cutover patterns:

• Configuration cutover: the application reads from whatever table or index ID is marked active in configuration. Once validation is complete, update that configuration in one step.
• View or alias cutover: the application always reads from a stable database view or alias, and cutover updates that view to point at the new underlying table.

Both patterns avoid changing application query logic during the migration.

Practical Rule

Use this rule of thumb:

• If the Indexable Path is unchanged, attach the new model to the existing index and keep using the current table,
• If the Indexable Path changed, create a new index, build it separately, and cut over application reads only after validation.

Passing the wrong model to /expansions should fail, and that is a feature, not a bug.

---

4. Querying Permissions inside Managed Data Platforms

Overview

A common integration pattern is to maintain a colocated permissions index table in a data warehouse (like Snowflake, Databricks, BigQuery) so that analytics queries, dashboards, reporting, or even AI agents can respect access control. The idea is simple: your consumer reads the Read Expansions stream, inserts/deletes rows in a Snowflake table, and your analysts can JOIN that table with any other Snowflake table to filter results by permission.

Joining Permissions with Business Data

This is where the colocated Permissions Index is valuable. Instead of calling the FGA Check API for every row in a query (infeasible or impractical at scale), you JOIN against the colocated permissions table.

Basic: Filter Documents by User Access

-- "Search for documents matching 'quarterly report' that alice can view"
SELECT d.id, d.title, d.author, d.updated_at
FROM documents d
JOIN permissions_index p
    ON  p.object_type  = 'document'
    AND p.object_id    = d.id
WHERE p.subject_type = 'user'
  AND p.subject_id   = 'alice'
  AND p.relation     = 'can_view'
  AND d.title ILIKE '%quarterly report%'
ORDER BY d.updated_at DESC
LIMIT 25;

This replaces what would otherwise require thousands of FGA Check calls (one per document) with a single SQL JOIN.

Analytics: Count Documents per User

-- "How many documents can each user view?" (for capacity planning / audit)
SELECT
    p.subject_id   AS user_id,
    COUNT(*)        AS accessible_document_count
FROM permissions_index p
WHERE p.subject_type = 'user'
  AND p.relation     = 'can_view'
  AND p.object_type  = 'document'
GROUP BY p.subject_id
ORDER BY accessible_document_count DESC;

Access Audit: Who Can See a Specific Document?

-- "Who has can_view access to document 'budget-2026'?"
SELECT
    p.subject_type,
    p.subject_id,
    p.tuple_written_at   AS access_granted_at
FROM permissions_index p
WHERE p.object_type = 'document'
  AND p.object_id   = 'budget-2026'
  AND p.relation    = 'can_view'
ORDER BY p.tuple_written_at DESC;

Cross-Table Analytics: Permissions + Activity

-- "Which documents that alice can view have not been accessed in 90 days?"
-- Useful for data governance and cleanup
SELECT d.id, d.title, d.last_accessed_at
FROM documents d
JOIN permissions_index p
    ON  p.object_type  = 'document'
    AND p.object_id    = d.id
WHERE p.subject_type = 'user'
  AND p.subject_id   = 'alice'
  AND p.relation     = 'can_view'
  AND d.last_accessed_at < DATEADD('day', -90, CURRENT_TIMESTAMP())
ORDER BY d.last_accessed_at ASC;

Row-Level Security with the Colocated Permissions Index

If your Snowflake environment supports row access policies, you can enforce permissions transparently so that analysts never see unauthorized data — without modifying their queries.

-- Create a row access policy that filters based on the permissions index
CREATE OR REPLACE ROW ACCESS POLICY document_access_policy
AS (object_id VARCHAR) RETURNS BOOLEAN ->
    EXISTS (
        SELECT 1 FROM permissions_index p
        WHERE p.object_type  = 'document'
          AND p.object_id    = object_id
          AND p.subject_type = 'user'
          AND p.subject_id   = CURRENT_USER()
          AND p.relation     = 'can_view'
    );

-- Apply the policy to the documents table
ALTER TABLE documents ADD ROW ACCESS POLICY document_access_policy ON (id);

Now any query against documents is automatically filtered by the user's permissions:

-- This query automatically respects access control via the row access policy
SELECT * FROM documents WHERE title ILIKE '%budget%';
-- Only returns documents the current Snowflake user has can_view access to

Note: This approach requires that Snowflake user names match the subject_id values in the permissions index. You may need a mapping table if your FGA user IDs differ from Snowflake user names.

---

FAQ

Can I run multiple consumers for the same index?

Yes. Each consumer tracks its own continuation token independently. You can have one consumer writing to PostgreSQL and another writing to Snowflake, each progressing at their own rate.

What happens if my consumer goes down for an extended period?

When the consumer reconnects with its last saved from continuation token, the stream will deliver all events from that token forward. The consumer will catch up to the current state. If the token is very old, catching up may take time depending on the volume of events that occurred during the outage.

If you have lost or do not have a continuation token, omit the ?from= parameter to replay from the beginning. Because the consumer is idempotent, this will rebuild the correct state.

Should I store the full expansion event or just the key fields?

At minimum, store the natural key fields (subject_type, subject_id, subject_relation, relation, object_type, object_id). These are sufficient for JOIN queries. Additionally, we recommend storing:

• tuple_written_at — useful for freshness monitoring and audit trails

If you need a full event audit trail, store raw events in a separate permissions_index_events table.

What indexes should I create on the permissions table?

At minimum:

-- For "what can this user see?" queries
CREATE INDEX idx_perm_subject ON permissions_index
    (subject_type, subject_id, subject_relation, relation);

-- For "who can see this object?" queries
CREATE INDEX idx_perm_object ON permissions_index
    (object_type, object_id, relation);

The right indexes depend on your query patterns. If you primarily filter by user, prioritize the subject index. If you primarily audit by object, prioritize the object index.

Relation vs. Subject Relation

An expansion event has two relation fields that serve different purposes:

• relation — the permission on the object. It answers: what can the subject do to this object?
• subject_relation — the relation on the subject. It answers: through which userset was this subject resolved?

The distinction arises because FGA allows you to assign usersets — not just individual users — to objects. Consider this model:

type user

type group
  relations
    define member: [user]

type document
  relations
    define can_view: [user, group#member]

The type restriction [user, group#member] means can_view can be assigned to either a direct user or to the member userset of a group. When you write a tuple that assigns a userset:

user:   group:engineering#member
relation: can_view
object:   document:report

FGA resolves every user who is a member of group:engineering and produces flattened expansion events. If user:alice and user:bob are members, the expansions are:

subject_type	subject_id	subject_relation	relation	object_type	object_id
user	alice	member	can_view	document	report
user	bob	member	can_view	document	report

Notice:

• relation is can_view — the permission granted on the document.
• subject_relation is member — the relation on group that was used to resolve the userset.

If instead a direct tuple had been written (user:alice can_view document:report), the expansion would have no subject_relation because no userset was involved.