Skip to main content

Eventual Consistency when Performing Checks, Reads and Expand

Background​

The Okta FGA service has been optimized for lower latency and high availability on all read requests rather than strong consistency. That means that results from the check, expand and read requests may not reflect the latest state and what was just written in the database, and might take a few seconds to converge.

Some of the choices that Okta FGA has made include:

Eventually consistent, multi-region active-active database​

The Okta FGA service uses an eventually consistent database in its implementation. This means that if a read request is performed on a region different than the region where the original write request was made, the data returned by the read might not reflect the data written by the write.

For example, assuming the following authorization model

model
  schema 1.1

type user

type document
  relations
    define viewer: [user]
    define can_view: viewer

Immediately after writing a tuple saying that Bob is a viewer of document:meeting_notes.doc, calling Check to see whether Bob can view document:meeting_notes.doc may return false.

Initialize the SDK
// Checkout the "How to Setup the SDK Client" page for more details.
const { CredentialsMethod, OpenFgaClient } = require('@openfga/sdk'); // OR import { CredentialsMethod, OpenFgaClient } from '@openfga/sdk';

// Ensure the environment variables are set
// FGA_API_URL = 'https://api.us1.fga.dev' // 'https://api.eu1.fga.dev' for EU and 'https://api.au1.fga.dev' for AU
// FGA_STORE_ID = 'YOUR_STORE_ID' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page
// FGA_MODEL_ID = 'YOUR_MODEL_ID' - optional, can be overridden per request, helps reduce latency
// FGA_API_TOKEN_ISSUER = 'fga.us.auth0.com'
// FGA_API_AUDIENCE = 'https://api.us1.fga.dev/' // 'https://api.eu1.fga.dev/' for EU and 'https://api.au1.fga.dev/' for AU
// FGA_CLIENT_ID = 'YOUR_CLIENT_ID' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page
// FGA_CLIENT_SECRET = 'YOUR_CLIENT_SECRET' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page

const fgaClient = new OpenFgaClient({
    apiUrl: process.env.FGA_API_URL,
    storeId: process.env.FGA_STORE_ID,
    authorizationModelId: process.env.FGA_MODEL_ID,
    credentials: { // Credentials are not needed if connecting to the Playground API
      method: CredentialsMethod.ClientCredentials,
      config: {
        apiTokenIssuer: process.env.FGA_API_TOKEN_ISSUER,
        apiAudience: process.env.FGA_API_AUDIENCE,
        clientId: process.env.FGA_CLIENT_ID,
        clientSecret: process.env.FGA_CLIENT_SECRET,
      },
    },
});

await fgaClient.write({
  writes: [
      {"user":"user:bob","relation":"viewer","object":"document:meeting_notes.doc"}
  ],
}, {
  authorization_model_id: "01HVMMBCMGZNT3SED4Z17ECXCA" 
});
Initialize the SDK
// Checkout the "How to Setup the SDK Client" page for more details.
const { CredentialsMethod, OpenFgaClient } = require('@openfga/sdk'); // OR import { CredentialsMethod, OpenFgaClient } from '@openfga/sdk';

// Ensure the environment variables are set
// FGA_API_URL = 'https://api.us1.fga.dev' // 'https://api.eu1.fga.dev' for EU and 'https://api.au1.fga.dev' for AU
// FGA_STORE_ID = 'YOUR_STORE_ID' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page
// FGA_MODEL_ID = 'YOUR_MODEL_ID' - optional, can be overridden per request, helps reduce latency
// FGA_API_TOKEN_ISSUER = 'fga.us.auth0.com'
// FGA_API_AUDIENCE = 'https://api.us1.fga.dev/' // 'https://api.eu1.fga.dev/' for EU and 'https://api.au1.fga.dev/' for AU
// FGA_CLIENT_ID = 'YOUR_CLIENT_ID' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page
// FGA_CLIENT_SECRET = 'YOUR_CLIENT_SECRET' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page

const fgaClient = new OpenFgaClient({
    apiUrl: process.env.FGA_API_URL,
    storeId: process.env.FGA_STORE_ID,
    authorizationModelId: process.env.FGA_MODEL_ID,
    credentials: { // Credentials are not needed if connecting to the Playground API
      method: CredentialsMethod.ClientCredentials,
      config: {
        apiTokenIssuer: process.env.FGA_API_TOKEN_ISSUER,
        apiAudience: process.env.FGA_API_AUDIENCE,
        clientId: process.env.FGA_CLIENT_ID,
        clientSecret: process.env.FGA_CLIENT_SECRET,
      },
    },
});

// Run a check
const { allowed } = await fgaClient.check({
    user: 'user:bob',
    relation: 'can_view',
    object: 'document:meeting_notes.doc',
  }, {
  authorization_model_id: '01HVMMBCMGZNT3SED4Z17ECXCA',
});

// allowed = false

Image showing call flow

Call flow for check and write. Notice that before cross-region replication happens, check will return false even though the tuple has already been written.

Caching DB requests & evaluations​

To further reduce latency, Okta FGA is configured to cache certain evaluations for a brief period of time.

That means if a certain tuple is read (externally or internally) from the DB, is then modified by a write and then subsequently read - the result of the first read will be returned for up to 10 seconds after the first call.

So issuing the following Check request

Initialize the SDK
// Checkout the "How to Setup the SDK Client" page for more details.
const { CredentialsMethod, OpenFgaClient } = require('@openfga/sdk'); // OR import { CredentialsMethod, OpenFgaClient } from '@openfga/sdk';

// Ensure the environment variables are set
// FGA_API_URL = 'https://api.us1.fga.dev' // 'https://api.eu1.fga.dev' for EU and 'https://api.au1.fga.dev' for AU
// FGA_STORE_ID = 'YOUR_STORE_ID' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page
// FGA_MODEL_ID = 'YOUR_MODEL_ID' - optional, can be overridden per request, helps reduce latency
// FGA_API_TOKEN_ISSUER = 'fga.us.auth0.com'
// FGA_API_AUDIENCE = 'https://api.us1.fga.dev/' // 'https://api.eu1.fga.dev/' for EU and 'https://api.au1.fga.dev/' for AU
// FGA_CLIENT_ID = 'YOUR_CLIENT_ID' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page
// FGA_CLIENT_SECRET = 'YOUR_CLIENT_SECRET' - Get this from your store settings in the dashboard, refer to the "How to get your API Keys" page

const fgaClient = new OpenFgaClient({
    apiUrl: process.env.FGA_API_URL,
    storeId: process.env.FGA_STORE_ID,
    authorizationModelId: process.env.FGA_MODEL_ID,
    credentials: { // Credentials are not needed if connecting to the Playground API
      method: CredentialsMethod.ClientCredentials,
      config: {
        apiTokenIssuer: process.env.FGA_API_TOKEN_ISSUER,
        apiAudience: process.env.FGA_API_AUDIENCE,
        clientId: process.env.FGA_CLIENT_ID,
        clientSecret: process.env.FGA_CLIENT_SECRET,
      },
    },
});

// Run a check
const { allowed } = await fgaClient.check({
    user: 'user:bob',
    relation: 'can_view',
    object: 'document:meeting_notes.doc',
  }, {
  authorization_model_id: '01HVMMBCMGZNT3SED4Z17ECXCA',
});

// allowed = false

Will result in Okta FGA returning false to that same subsequent check for up to 10 seconds, even if the tuple (user:bob, reader, document:meeting_notes.doc) was written and replicated during that time. Additionally, any further read requests will not return the (user:bob, reader, document:meeting_notes.doc) tuple till 10 seconds have passed from the original check request.

Implication​

We have talked to a few customers and others in the industry regarding the need for strong consistency and concepts in the Zanzibar paper alluding to it (e.g. Zookies), and found that while strong consistency is important to some customers, availability and low latency are the essential requirements that users need.

You must take into consideration the eventually consistent design of Okta FGA when designing your services. Do not expect to write and then read that same data shortly after.

In most cases, when users share a resource, by the time they have sent the link and the recipient has clicked on it, the stale cache would have cleared.

Also watch out that the same applies when revoking access. What we found is that for many customers, they are fine with a user having access for a few seconds after access was revoked, but that may not be true for your use-case.

In case this is an issue for you, please reach out to us and we can discuss alternative solutions.

Check the following sections for more on how to check, read and expand.
Check, Read and Expand

Comparison Between Check, Read And Expand API Calls.

Have Feedback?

You can use any of our support channels for any questions or suggestions you may have.