Metadata Indexer

An open source and self-hostable service that indexes all the embed urls mentioned in Farcaster casts, and their metadata.

Why should you use this, and not build your own, or load the data on pageload?

We've put 100+ hours of work into this tool; is this a core differentiator for your app? If not, perhaps use this infra rather than building it yourself
You can and are free to use this even if you aren't integrating Mod.
Fetching metadata on user's pageload is slow (1s+) and you can't always be sure of the dimensions of the image(s), so you may accidentally introduce Cumulative Layout Shift (opens in a new tab)
It supports special handlers for NFT-extended open graph spec (opens in a new tab), CAIP-19 urls, Opensea, Zora, Images and Arweave files.
This indexer live indexes every new Farcaster cast as soon as it's propagated to our hub.
It supports JSON Linked Data (opens in a new tab) and gives you additional metadata about the page
We're already indexing and caching the metadata of over 500,000 urls shared on Farcaster.
It's MIT Licensed and if you don't want to rely on our APIs or are worried about it shutting down, you can always self host (opens in a new tab) it easier than you can build it yourself.

How to use

Endpoint to get the Metadata for a list of cast hashes

POST https://api.modprotocol.org/api/cast-embeds-metadata

curl --request POST \
--url https://api.modprotocol.org/api/cast-embeds-metadata \
--header 'Content-Type: application/json' \
--data '["0x15f0c75fd1735789b52c708f858816d149910c1d","0x1af019b5ad82f159efa3f2c51b4d4653604180d4"]'

This will return a JSON object with the following structure:

{
  "0x1234...abcd": [
    {
      "castHash": "0x1234...abcd",
      "url": "...",
      "normalizedUrl": "...",
      "index": 0,
      "metadata": {
        "title": "Example Title",
        "description": "Example Description",
        "image": {
          "url": "https://example.com/image.png"
        }
        // ...
      }
    }
  ]
}

Returned metadata objects conform to the UrlMetadata type. This can then be used to render embeds in a cast.

import { UrlMetadata } from "@mod-protocol/core";
 
cast.embeds.map((embed) => {
  const metadata: UrlMetadata = metadataResponse[cast.hash][embed.url];
  return <RichEmbed metadata={metadata} />;
});

Endpoint to get the Metadata for a list of urls

/api/cast-embeds-metadata/by-url

Fetching metadata from the cache by url is as simple as making a POST request to the following endpoint with a list of urls in the body.

curl --request POST \
--url https://api.modprotocol.org/api/cast-embeds-metadata/by-url \
--header 'Content-Type: application/json' \
--data '["https://google.com","https://twitter.com"]'

This will return a JSON object with the following structure:

{
  "https://google.com": {
    "image": {
      "url": "https://www.google.com/images/branding/googleg/1x/googleg_standard_color_128dp.png",
      "height": 128,
      "width": 128
    },
    "description": "Celebrate Native American Heritage Month with Google",
    "title": "Google",
    "publisher": "google.com",
    "logo": {
      "url": "https://www.google.com/favicon.ico"
    },
    "mimeType": "text/html; charset=UTF-8"
  },
  "https://twitter.com": {
    "description": "From breaking news and entertainment to sports and politics, get the full story with all the live commentary.",
    "title": "@ on Twitter",
    "publisher": "Twitter",
    "logo": {
      "url": "https://abs.twimg.com/favicons/twitter.3.ico"
    },
    "mimeType": "text/html; charset=utf-8"
  }
}

Returned metadata objects conform to the UrlMetadata type. This can then be used to render embeds in a cast.

import { UrlMetadata } from "@mod-protocol/core";
 
const metadata: UrlMetadata = metadataResponse[embed.url];
return <RichEmbed metadata={metadata} />;

Typescript type of Metadata

type UrlMetadata = {
    image?: {
        url?: string;
        width?: number;
        height?: number;
    };
    "json-ld"?: Record<string, object[]>;
    description?: string;
    alt?: string;
    title?: string;
    publisher?: string;
    logo?: {
        url: string;
    };
    nft?: {
      owner?: FarcasterUser;
      tokenId?: string;
      mediaUrl?: string;
      mintUrl?: string;
      mimeType?: string;
      collection: {
        id: string;
        name: string;
        chain: string;
        contractAddress: string;
        creatorAddress: string;
        itemCount: number;
        ownerCount: number;
        mintUrl?: string;
        description?: string;
        imageUrl?: string;
        openSeaUrl?: string;
        creator?: FarcasterUser;
      };
    };
};

Tailwind + Radix Farcaster Editor