How Mintable is Revolutionising NFT Discovery with AWS
Mintable is a unified platform where you can create, trade, and interact with NFTs. Here is how they are re-imagining NFT search with Generative AI on AWS.
Writom Guha Roy
Amazon Employee
Published Oct 3, 2024
Co-authored by Mintable and Amazon Web Services (AWS)
- S A Sureshkumar, Solutions Architect, Mintable
- Writom Guha Roy, Sr. Solutions Architect, AWS
In the rapidly evolving world of Non-Fungible Tokens (NFTs), finding the perfect digital asset can feel like searching for a needle in a haystack. At Mintable, they have always been committed to making the NFT experience as seamless and intuitive as possible. Mintable is excited to announce a groundbreaking advancement in NFT search technology that's set to transform how creators and collectors interact with digital assets.
Traditional NFT marketplaces rely heavily on keyword-based searches, which often fall short in capturing the nuanced desires of users. Imagine wanting to find a rare, animated NFT of a space cat wearing a cowboy hat – how would you even begin to formulate that search query?
The limitations of keyword searches have long been a pain point in the NFT space:
- Users struggle to articulate complex visual concepts in simple keywords.
- Relevant NFTs are often missed due to inconsistent tagging or descriptions.
- The intent behind a search (browsing, buying, or researching) is lost in translation.
These challenges have created a significant gap between user intent and search results, often leading to frustration and missed opportunities for both creators and collectors.
Mintable envisioned a future where finding the perfect NFT is as simple as describing it to a friend. To turn this vision into reality, they've harnessed the power of Amazon Bedrock, specifically the Claude 3.5 Sonnet model, in combination with Amazon OpenSearch and AWS AppSync.
Their new search system understands natural language queries, interprets user intent, and delivers highly relevant results with actionable next steps. Here's how it works:
- Natural Language Understanding: Users can now search using conversational language. A query like "Find some pudgy penguins that I can buy" is perfectly valid and understood.
- Intent Recognition: The system identifies whether you're looking to buy, sell, research, or just browse, tailoring the results accordingly.
- Contextual Awareness: Previous searches, user preferences, and market trends are factored into the results, providing a personalized experience.
Actionable Results: If your intent is to buy, the search results will include direct links to purchase pages, streamlining the acquisition process.
Mintable's advanced search capability is powered by a sophisticated integration of AWS services:
- Amazon Bedrock (Claude 3.5 Sonnet): This large language model forms the brain of their search system, interpreting natural language queries and understanding user intent.
- Amazon OpenSearch: Handles the heavy lifting of indexing and retrieving NFT metadata, ensuring lightning-fast and accurate results.
- AWS AppSync: Manages the API layer, allowing them to create flexible and efficient resolvers that orchestrate the search process.
- AWS Amplify (Gen 2): Provides the framework for their backend and frontend implementations, seamlessly integrating various AWS services.
- Amazon DynamoDB: Serves as the caching layer, improving performance for repeated queries.
To give you a behind-the-scenes look at how Mintable's search system works, let's walk through the journey of a search query:
- Query Reception and Cache Check When a user inputs a search query, their first AppSync resolver checks DynamoDB for a cached result. If found, they can quickly return the previously transformed OpenSearch query, significantly reducing response time for common searches.
- Context Preparation If there's no cache hit, their second resolver springs into action. It fetches the current OpenSearch mapping from their OpenSearch instance. This mapping is crucial as it defines the structure of their NFT data. Then they augment this with additional notes and instructions, crafting a comprehensive system message for Amazon Bedrock.
- Natural Language Processing with Amazon Bedrock Their third resolver sends this enriched prompt to Amazon Bedrock. Here, the Claude 3.5 Sonnet model works its magic, processing the natural language query and returning two key pieces of information:
- A structured OpenSearch query that will fetch relevant NFTs
- An identified intent, helping us understand exactly what the user wants to do
- Intent Recognition The system identifies the user's intent from a comprehensive list including actions like checking floor prices, listing NFTs for sale, transferring NFTs, and more. This intent is returned in a structured JSON format, including details like the NFT name, collection, ownership context, and price information.
- Query Enhancement Based on the identified intent and user context, they further refine the OpenSearch query. This might involve adding filters for the user's wallet addresses (for searches of owned NFTs), visibility settings, or other relevant parameters.
- Result Retrieval and Presentation Finally, they execute the enhanced OpenSearch query, retrieving a set of NFTs that precisely match the user's intent and context. These results are then presented to the user, often with actionable next steps based on the identified intent.
This entire process happens in milliseconds, providing users with a seamless, context-aware search experience that feels almost magical in its ability to understand and respond to natural language queries.
Please refer to the sample code base here.
One of the key innovations in Mintable's search system is its ability to recognize a wide range of user intents. Here's a glimpse into the types of intents their system can identify:
- Checking floor prices
- Listing NFTs for sale
- Transferring NFTs
- Buying or selling NFTs
- Creating or updating NFT listings
- Exploring transaction history
- Investigating ownership details
- Gathering collection information
- Managing favourites and watchlists
- Placing bids and managing offers
- Searching NFTs by price
By accurately identifying these intents, they can tailor the entire user experience, from the search results displayed, to the actions they enable users to take, directly from the search interface.
To further improve the accuracy and relevance of their search results, they've implemented Retrieval Augmented Generation (RAG). This technique allows their system to pull in real-time data about NFT collections, recent sales, and market trends, ensuring that the information provided is always up-to-date and contextually relevant.
Let's walk through a real example to showcase the power of their new search system:
User Query: "Find some pudgy penguins that I can buy"
- Intent Recognition: The system identifies this as a purchase-oriented query for the Pudgy Penguins collection.
- Contextual Search: It looks for Pudgy Penguin NFTs currently listed for sale, considering factors like the user's price range based on previous interactions.
- Result Presentation: The user is presented with a curated list of Pudgy Penguin NFTs available for purchase, complete with key details like price, rarity, and unique attributes.
- Actionable Links: Each result includes a direct "Buy Now" link, streamlining the purchase process.
- Additional Context: The system might also provide recent sales data for Pudgy Penguins or notify the user of any ongoing promotions related to the collection.
This entire process happens in a matter of seconds, providing a seamless and intuitive search experience.
Mintable's advanced search system offers numerous benefits for both creators and collectors:
- Enhanced Discoverability: Unique and niche NFTs have a better chance of being found by interested collectors.
- Intuitive Navigation: Users can explore the NFT space more naturally, without needing to learn complex search techniques.
- Efficient Transactions: By understanding user intent, they can provide direct paths to purchase, list, or learn more about NFTs.
Market Insights: The system can provide contextual information about collections and market trends, helping users make informed decisions.
While Mintable is excited about this leap forward in NFT search technology, they're already looking to the future. Some enhancements on their roadmap include:
- Visual Search: Allowing users to search by uploading images or sketches.
- Multi-Modal Queries: Combining text, image, and even audio inputs for more precise searches.
- Voice-Activated Search: Enabling users to search for NFTs using voice commands, making the process even more intuitive.
At Mintable, they believe that technology should empower creativity and connection in the NFT space. Their new search system, powered by Amazon Bedrock and Amazon OpenSearch, represents a significant step towards making the vast and exciting world of NFTs more accessible to everyone.
Mintable invites you to experience the future of NFT search firsthand. Visit Mintable today and try out their new search feature – your perfect NFT is just a conversation away!
Disclaimer: Pudgy Penguins is a registered trademark of Pudgy Penguins, and its inclusion here is for illustrative purposes only. This blog is not affiliated with or endorsed by Pudgy Penguins.
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.